Control system of autonomous vehicle and control method of autonomous vehicle

Abstract

The present invention relates to a control system of an autonomous vehicle and a control method of an autonomous vehicle. The control system of the autonomous vehicle is for performing efficient computation control by parallelizing operations between a plurality of modules of the autonomous vehicle, the control system comprising a plurality of modules arranged in the autonomous vehicle and configured to perform connection operations, and a module scheduling system configured to monitor operations of the plurality of modules, parallelize the plurality of modules with additional threads, and control the plurality of modules to operate aperiodically with a pipelining method. Thus, the hourly processing capacity of the plurality of modules can be optimized by controlling parallel processing of the plurality of modules, and when the entire logical computation period is longer than the sensor input period, errors due to delays occurring in the data processing process between the modules are overcome.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2020-0169677, filed on December 7, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to a control system and a control method for an autonomous vehicle, and more specifically, to a control system and a control method configured to perform control for efficient computation by parallelizing the operations among the constituent modules of the autonomous vehicle. Background Technology

[0004] An autonomous vehicle is a vehicle that can drive itself without a driver or passenger, and an autonomous driving system is a system used to monitor and control the autonomous driving of an autonomous vehicle.

[0005] Autonomous vehicles typically operate based on information about their driving environment, with little or no input from the driver. An autonomous vehicle includes a sensor system comprising one or more sensors configured to detect information relevant to the vehicle's driving environment. Autonomous vehicles operate in manual mode, fully autonomous mode, or partially autonomous mode.

[0006] Autonomous driving systems can utilize multiple LiDAR, radar, and cameras to perform complex calculations in determining driving conditions in complex environments such as cities. Each sensor has an input period, and as the input period decreases, nearby objects are used more frequently in the determination process, thus requiring sufficiently short input periods.

[0007] Figure 1 (Prior art) is a schematic diagram illustrating an example of the operational relationships between components in a conventional autonomous vehicle. For example... Figure 1 As shown, the signal detected by the first sensor Sensor 1, driven by the first clock CLK 1, is sent and processed by the first module, and processed by the second module using the second clock CLK 2. Then, the signal detected by the second sensor Sensor 2, driven by the third clock CLK 3, is sent and processed by the first module, and processed by the second module using the fourth clock CLK 4.

[0008] In this scenario, when the autonomous driving logic calculation period is longer than the sensor operation period, if the next sensor's sensing value is input before the calculations required to determine driving are processed, the sensing value may not be processed. When delays occur between modules, errors in autonomous driving due to discrepancies between the received object's position information and its actual position can lead to potential accidents. Summary of the Invention

[0009] The purpose of this invention is to provide a control system and method for autonomous vehicles to overcome errors caused by delays that occur during data processing between multiple modules.

[0010] Another object of the present invention is to provide a control system and method for an autonomous vehicle, for improving the hourly processing capacity of multiple modules.

[0011] Another object of the present invention is to provide a control system and method for an autonomous vehicle to overcome the problem that occurs when the entire logic calculation period is longer than the sensor input period.

[0012] To achieve the objectives of this invention, the control system and method for autonomous vehicles can precisely schedule the operation sequence of multiple modules by parallelizing the modules arranged in the autonomous vehicle.

[0013] In one aspect, the control system and method for an autonomous vehicle according to the present invention include a plurality of modules and a module scheduling system, the plurality of modules being arranged in the autonomous vehicle and configured to perform connectivity operations; the module scheduling system is configured to monitor the operations of the plurality of modules, to parallelize the plurality of modules using additional threads, and to control the aperiodic operation of the plurality of modules using a pipelined method.

[0014] The control system of an autonomous vehicle may include a sensing module unit, a processing module unit, and a control module unit. The sensing module unit includes multiple sensor modules configured to output various sensing data. The processing module unit is configured to process the sensing data provided by the multiple sensor modules of the sensing module unit. The control module unit is configured to control the operation of autonomous driving using the processing results of the processing module unit.

[0015] Each of the multiple modules may include: a module receiving thread, a receiving buffer, a module main thread, a sending buffer, and a module sending thread. The module receiving thread is configured to receive data from the previous module non-periodically. The receiving buffer is configured to temporarily store the data received from the module receiving thread. The module main thread is configured to process the data stored in the receiving buffer non-periodically. The sending buffer is configured to temporarily store data based on the operation result of the module main thread. The module sending thread is configured to send the data stored in the sending buffer to another module non-periodically.

[0016] The module scheduling system may include a receive management module and a send management module. The receive management module is configured to receive information about the data receiving status from the module receive thread inside the module, process information about the module processing status from the module main thread inside the module, and provide processing instructions to the module main thread inside the module. The send management module is configured to receive information about the module processing status from the module main thread inside the module, receive information about the data sending status from the module send thread inside the module, and provide send instructions to the module send thread inside the module.

[0017] The module scheduling system can monitor the module receive thread, module main thread, and module send thread within each module, and can execute control to parallelize the send, receive, and processing operations of each thread.

[0018] The module scheduling system can control the module's receiving thread to receive the second frame of data while the module's main thread is processing the first frame of data stored in the receive buffer.

[0019] The module scheduling system can control the module's sending thread to send the processed first frame of data stored in the sending buffer while the module's main thread is processing the second frame of data stored in the receiving buffer.

[0020] The module scheduling system can control the module receiving thread to receive the third frame of data while the module sending thread is sending the processed first frame of data stored in the sending buffer.

[0021] The receive management module can receive information about the data reception status from the receive thread within each module performing the connection operation, and information about the module processing status from the main thread within each module performing the connection operation. It can also control the processing operations of the main thread within each module. Similarly, the send management module can receive information about the module processing status from the main thread within each module performing the connection operation, and information about the data sending status from the send thread within each module performing the connection operation. It can also control the processing operations of the send thread within each module.

[0022] On the other hand, the present invention provides a control method for an autonomous vehicle using a module scheduling system. The module scheduling system is used to control multiple modules arranged in the autonomous vehicle and configured to perform connection operations. The control method includes: the receiving management module of the module scheduling system receiving information about the data receiving status from the module receiving thread inside the module and receiving information about the module processing status from the module main thread inside the module; the receiving management module of the module scheduling system providing processing instructions to the module main thread; the sending management module of the module scheduling system receiving information about the module processing status from the module main thread and receiving information about the data sending status from the module sending thread inside the module; and the sending management module of the module scheduling system providing sending instructions to the module sending thread non-periodically using a pipeline method.

[0023] The module scheduling system can monitor the module receive thread, module main thread, and module send thread within each module, and can execute control to parallelize the send, receive, and processing operations of each thread.

[0024] The module scheduling system can control the module's receiving thread to receive the second frame of data while the module's main thread is processing the first frame of data stored in the receive buffer.

[0025] The module scheduling system can control the module's sending thread to send the processed first frame of data stored in the sending buffer while the module's main thread is processing the second frame of data stored in the receiving buffer.

[0026] The module scheduling system can control the module receiving thread to receive the third frame of data while the module sending thread is sending the processed first frame of data stored in the sending buffer.

[0027] On the other hand, the present invention provides a control method for an autonomous vehicle utilizing multiple modules. Each module includes a module receiving thread, a module main thread, and a module sending thread, and performs connection operations. The control method includes: the module receiving thread receiving data from the previous module non-periodically; after receiving data, the module receiving thread immediately stores the received data in a receiving buffer; after the data in the receiving buffer is updated, the module main thread immediately reads and processes the data stored in the receiving buffer non-periodically; after the module main thread finishes processing the data, the module main thread immediately stores the processed data in a sending buffer; and after the data in the sending buffer is updated, the module sending thread immediately sends the data stored in the sending buffer non-periodically to the next module, wherein at least two of the data sending and receiving processes and the data transmission process occur simultaneously.

[0028] When the receiving thread of the module receives a data packet and stores the data packet in the receiving buffer and the receiving flag is in the enabled state, and when the main thread of the module finishes processing the current frame and the data processing flag is in the enabled state, the processing instruction flag of the receiving management module can be enabled.

[0029] When the module's main thread finishes processing the current frame and the data processing flag is enabled, and when the module's sending thread does not send data and the data sending flag is disabled, the sending management module's processing instruction flag can be enabled. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and constitute a part of this application, are included to provide a further understanding of the invention and illustrate embodiments thereof, and together with the description, explain the principles of the invention. In the drawings:

[0031] Figure 1 (Prior art) is a schematic diagram illustrating an example of the operational relationships between components of a conventional autonomous vehicle;

[0032] Figure 2 This is a schematic diagram illustrating an example of the operational relationships between component modules in the control system of an autonomous vehicle according to the present invention;

[0033] Figure 3 This is a block diagram illustrating the configuration of the structure of the control system for an autonomous vehicle according to the present invention;

[0034] Figure 4 This is a block diagram illustrating the configuration of each module of the control system for an autonomous vehicle according to the present invention;

[0035] Figure 5 This is a schematic diagram illustrating an example of each module of the control system for an autonomous vehicle according to the present invention;

[0036] Figure 6a This is a schematic diagram illustrating the operation of the control system of an autonomous vehicle according to the present invention under the control of the receiving management module;

[0037] Figure 6b This is a schematic diagram illustrating the operation of the control system of an autonomous vehicle according to the present invention under the control of the transmission management module;

[0038] Figure 7 This is a schematic diagram illustrating an example of a delay occurring during the scheduling of each module in the control system of an autonomous vehicle according to the present invention;

[0039] Figure 8This is a schematic diagram illustrating an example of the operation of each module in the control system of an autonomous vehicle according to the present invention during each time period. Detailed Implementation

[0040] It should be understood that the term "vehicle" or "of a vehicle" or other similar terms as used herein generally include motor vehicles, such as passenger vehicles including sport utility vehicles (SUVs), buses, trucks, and various commercial vehicles, vessels including various boats and ships, aircraft, etc., and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., vehicles derived from non-petroleum fuels). As mentioned herein, a hybrid vehicle is a vehicle with two or more power sources, such as both gasoline power and electric power.

[0041] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the stated features, values, steps, operations, elements, and / or components, but do not exclude the presence or inclusion of one or more other features, values, steps, operations, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more related enumerated items. Throughout the specification, unless explicitly stated otherwise, the word “comprising” and variations such as “including” or “including” should be understood to imply the inclusion of the stated elements but not exclude any other elements. Furthermore, the terms “unit,” “device,” “component,” and “module” described in the specification mean a unit for performing at least one function and operation, and can be implemented by hardware components or software components and combinations thereof.

[0042] Furthermore, the control logic of this invention can be embodied in a non-volatile computer-readable medium containing executable program instructions that are executed by a processor, controller, etc. Examples of computer-readable media include, but are not limited to, ROM, RAM, optical disc (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer-readable medium can also be distributed across a network-connected computer system, allowing it to be stored and executed in a distributed manner, for example, via a telematics server or a controller area network (CAN).

[0043] In the exemplary embodiments of the present invention disclosed in the specification, the specific structural and functional descriptions are only for the purpose of illustrating the embodiments of the present invention, and the exemplary embodiments of the present invention can be embodied in various forms and are not limited to the embodiments described herein.

[0044] Exemplary embodiments of the present invention can be varied and embodied in various forms, wherein exemplary embodiments of the invention are shown. However, the exemplary embodiments of the present invention should not be construed as limiting to the embodiments described herein, and any variations, equivalents, or alternatives within the spirit and scope of the invention should be understood to fall within the scope of the invention.

[0045] It should be understood that although the terms first, second, third, etc., may be used herein to describe different elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and a second element may be referred to as a first element, without departing from the teachings of the invention.

[0046] It should be understood that when an element (such as a layer, region, or substrate) is referred to as "on another element," "connected to," or "attached to" another element, it can be directly on, directly connected to, or attached to the other element, or there may be intermediate elements. In contrast, when an element is referred to as "directly on another element or layer," "directly connected to," or "directly attached to" another element or layer, there are no intermediate elements or layers. Other terms used to describe relationships between elements or layers should be interpreted in a similar manner; for example, "between" is relative to "directly between," "adjacent" is relative to "directly adjacent," etc.

[0047] The terminology used in this specification is for explaining specific exemplary embodiments and is not intended to limit the concept of the invention. Therefore, unless the context clearly indicates otherwise, singular expressions in this specification include plural expressions.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the inventive concept pertains. It will be further understood that terms such as those defined in general dictionaries should be interpreted as having the same meaning as in the context of the relevant technical field, and not as having an ideal or overly formal meaning, unless expressly defined herein.

[0049] When exemplary embodiments are embodied in various ways, the functions or operations described in a specific block may be performed in a manner different from the order shown in the flowchart. For example, two consecutive blocks may be executed substantially simultaneously, and may be executed in reverse order depending on the associated functions or operations.

[0050] The structure and operation of the control system for an autonomous vehicle will be described below with reference to the accompanying drawings.

[0051] Figure 2 This is a schematic diagram illustrating an example of the operational relationships between component modules in a control system for an autonomous vehicle according to the present invention. Figure 2 As shown, the signal detected by the first sensor Sensor 1, driven by the first clock CLK 1, can be sent and processed by the first module. Data processed by the first module using the second clock CLK 2 can be processed by the second module, and simultaneously, the signal detected by the second sensor Sensor 2 can be sent and processed by the first module. Then, the data processed by the first module can be processed by the second module using the third clock CLK 3.

[0052] That is, the control system of the autonomous vehicle according to the present invention can reduce the processing time of all modules by controlling the vehicle so that the next module can immediately and non-periodically perform operations after the operation of the previous module is terminated.

[0053] Figure 3 This is a block diagram illustrating the configuration of the structure of a control system for an autonomous vehicle according to the present invention. The control system may include multiple modules 100 and a module scheduling system 200. The multiple modules 100 are arranged in the autonomous vehicle and perform connection operations. The module scheduling system 200 monitors the multiple modules and uses additional threads to parallelize the multiple modules and uses a pipeline method to control the non-periodic operation of the multiple modules.

[0054] like Figure 3 As shown, multiple modules 100 can be configured in a manner that does not change their order. That is, the result data of module 1 110 can be sent to module 2 120, and then the processing result of module 2 120 can be sent to module 3 130 and processed by module 3 130.

[0055] Based on the functional characteristics of the multiple modules 100, the multiple modules 100 can be classified into sensing module units, processing module units, and control module units.

[0056] The sensing module unit may include LiDAR, RaDAR, cameras, ultrasonic sensors, etc. LiDAR measures the distance, height, and orientation of nearby objects by emitting laser pulses and receiving the light reflected back by nearby objects. RaDAR measures the distance, height, and orientation of objects by emitting radio signals and receiving the radio signals reflected by nearby structures. Cameras generate image data by capturing images of the front, rear, and right / left sides of the vehicle. Ultrasonic sensors measure the distance, height, and orientation of objects by emitting ultrasonic waves and receiving the reflected signals when the emitted waves are reflected by nearby structures. The control system may include various sensors, such as on-board sensors, multiple infrared sensors, or sensors for sensing the vehicle's surrounding environment. On-board sensors are located inside the vehicle and monitor the driver's state. Multiple infrared sensors are mounted at the front, rear, and right / left sides of the vehicle to detect the approach of objects. Sensors for sensing the vehicle's surrounding environment include impact sensors, illuminance sensors, or humidity sensors. Impact sensors sense impacts applied to the vehicle and provide the controller with the information needed by the driver to control the vehicle. Illuminance or humidity sensors sense the vehicle's illuminance or humidity and provide the controller with the information needed by the driver to control the vehicle. Furthermore, the control system can use sensor values ​​measured by gyroscope sensors, speed sensors, acceleration sensors, etc., to calculate the vehicle's travel distance and direction.

[0057] The processing module unit can process sensing data provided by multiple sensor modules of the sensing module unit. In this case, the processing module unit can be configured according to a one-to-one correspondence with each sensor module of the sensing module unit, and the information from multiple sensor modules can be shared and provided by a single processing module unit as needed. The control module unit can use the processing results from the processing module unit to control the operation of autonomous driving.

[0058] The processing module unit and control module unit can include any type of processing device. For example, one or both of the processing module unit and control module unit can include: a microprocessor, a preprocessor (e.g., an image preprocessor), a graphics processor, a central processing unit (CPU), auxiliary circuitry, a digital signal processing device, an integrated circuit, or a suitable type of device for executing applications and processing and analyzing images. In some embodiments, the processing module unit and / or control module unit can include a single-core or multi-core processor, a microcontroller, and a type of CPU.

[0059] For example, when module 1 110 is a camera, module 2 120 can be an object recognizer for receiving images acquired by the camera and identifying objects. For example, the object recognizer can perform operations to identify whether an object is mobile or fixed. Information about the objects identified by module 2 120 can be sent to module 3 130, and module 3 130 can control the vehicle using various methods such as lane changing, acceleration changing, speed changing, or braking.

[0060] The module scheduling system 200 may include a receiving management module 210 and a sending management module 220. The receiving management module 210 is used to monitor the operation of each module and execute parallel control related to the data receiving and data processing of each module. The sending management module 220 is used to monitor the operation of each module and execute parallel control related to the data processing and data sending of each module.

[0061] Figure 4 This is a block diagram illustrating the configuration of each module of the control system for an autonomous vehicle according to the present invention. In this example, although module 1 110 is shown, all modules of the plurality of modules 100 may include the same components.

[0062] Module 1 110 may include: a module receiving thread 111, a receiving buffer 112, a module main thread 113, a sending buffer 114, and a module sending thread 115. The module receiving thread 111 is used to receive data from the previous module non-periodically using the protocol methods of Transmission Control Protocol (TCP) or User Datagram Protocol (UDP). The receiving buffer 112 is used to temporarily store the data received from the module receiving thread 111. The module main thread 113 is used to process the data stored in the receiving buffer 112 non-periodically. The sending buffer 114 is used to temporarily store data based on the operation result of the module main thread 113. The module sending thread 115 is used to send the data stored in the sending buffer 114 non-periodically to another module.

[0063] Figure 5This is a schematic diagram illustrating examples of each module of the control system for an autonomous vehicle according to the present invention. For example, the module receiving thread 111 of module 1 110 can immediately perform the operation of storing data in the receiving buffer 112 after receiving data non-periodically. The module main thread 113 of module 1 110 can immediately read and process the data stored in the receiving buffer 112 after updating the receiving buffer 112 non-periodically. Then, the operation of storing the processed data in the transmitting buffer 114 can be performed immediately after the data has been fully processed. The module transmitting thread 115 of module 1 110 can immediately read the data stored in the transmitting buffer 114 non-periodically after the transmitting buffer 114 is updated, and can perform the operation of sending the data to module 2 120, which is the next module.

[0064] Figure 6a This is a schematic diagram illustrating the operation of the control system of an autonomous vehicle according to the present invention under the control of a receiving management module. The receiving management module 210 can receive data receiving status from the module receiving thread 111 within module 1110, receive module processing status from the module main thread 113 within module 1110, and can provide processing instructions to the module main thread 113 within module 1110. The module main thread 113 can store the processed data in the transmitting buffer 114 according to the control of the receiving management module 210.

[0065] The data reception status indicates that when module receiving thread 111 receives a data packet and inputs it into the receive buffer, the "data reception flag" indicates the ON-state.

[0066] The module processing status indicates that the "data processing flag" is enabled when the module's main thread 113 finishes processing the current frame.

[0067] The processing instruction condition of the receiving management module 210 indicates that when the "data receiving flag" of the module receiving thread 111 is in the open state and the "data processing flag" of the module main thread 113 is in the open state, the "processing instruction flag" indicates the open state.

[0068] When corresponding flags exist within the same application, the flags can share variables and be represented independently, allowing the use of global variables and mutexes to avoid overlap. When corresponding flags exist in different applications, the flags can also be sent and represented through shared variables using TCP / UDP socket programming or any other type of communication method. The receive management module 210 can operate as a separate application or thread to minimize the impact of other modules, but the invention is not limited thereto.

[0069] Figure 6b This is a schematic diagram illustrating the operation of the control system of an autonomous vehicle according to the present invention under the control of the transmission management module.

[0070] The transmission management module 220 can receive the module processing status from the module main thread 113 inside module 1 110, receive the data transmission status from the module transmission thread 115 inside module 1 110, and provide transmission commands to the module transmission thread 115 inside module 1 110. Under the control of the transmission management module 220, the module transmission thread 115 can send the data stored in the transmission buffer 114 to module 2 120.

[0071] The module processing status indicates that the "data processing flag" is enabled when the module's main thread 113 finishes processing the current frame.

[0072] The data transmission status indicates that when module transmission thread 115 is not currently in the transmission state, the "data processing flag" indicates an OFF state. The processing instruction condition of transmission management module 220 indicates that the "data processing flag" of module main thread 113 is ON and the "data transmission flag" of module transmission thread 115 is OFF. When the corresponding flags exist in the same application, the flags can also share variables and can be represented independently to utilize global variables and mutexes without overlapping. When the corresponding flags exist in different applications, the flags can also be transmitted and can be represented by shared variables using TCP / UDP socket programming or any other type of communication method. Transmission management module 220 can operate as an independent application or thread to minimize the influence of other modules, but the invention is not limited to this.

[0073] Figure 7 This is a schematic diagram illustrating an example of a delay occurring during the scheduling of each module in the control system of an autonomous vehicle according to the present invention. In conventional control systems for autonomous vehicles, delays can occur because threads within each module have the same operating time period, and receive, process, and send operations are also performed based on the next operating time period after the operation of the previous module is completed. If the calculation time period of the previous module is long, and the input value of the next sensor is input before the calculation required to determine driving is processed, the input value may not be processed. According to the present invention, the module scheduling system 200 can monitor the operation of the module receive thread, module main thread, and module send thread within each of the multiple modules 100, and can control the threads to operate in an organized manner at appropriate times to execute a pipeline.

[0074] In the following description, the operation of module 1 110 receiving and processing sensing information in 100ms sensor time intervals and then sending the sensing information to module 2120, as well as the process of processing fourth frame data at any time in each module operation state, will be illustrated with reference to the exemplary accompanying drawings.

[0075] For example, when module 1 110 finishes processing the fourth frame of data, the module scheduling system 200, which is monitoring this process, can check the status of module 2 120. Module 2 120 can then finish processing the third frame of data, and the check by the module scheduling system 200 will send the third frame of data to module 3 130 (Tx). (3) The sending management module 220 can provide sending instructions to the module sending thread 115 of module 1 110. Therefore, the module sending thread 115 of module 1 110 can send the fourth frame data stored in the sending buffer 114 to module 2 120 (T). x(4) Then, the receive management module 210 of the module scheduling system 200 can send processing instructions to the module receive thread of module 2 120, and the module receive thread of module 2 120 can receive the processed fourth frame data (Rx) from module 1110. (4) ).

[0076] Module 1 110 processes the fourth frame of data and sends it to Module 2 120. Therefore, the receive management module 210 of the module scheduling system 200, which is monitoring this process, can send a receive instruction to the module receive thread 111. The module receive thread 111 of Module 1 110 can start receiving the fifth frame of data in the next operation period (i.e., 100ms), and the module main thread 113 can process the fifth frame of data.

[0077] The module scheduling system 200, which monitors the processing of the fourth frame data by the main thread of module 2 120, can check the status of module 3 130 at appropriate times. The module scheduling system 200, which monitors the processing of the third frame data by module 3 130, can control the module sending thread of module 3 130 to send the processed third frame data to module 4 140 (Tx). (3) Then, the module scheduling system 200 can control the module sending thread of module 2 120 to send the processed fourth frame data stored in the sending buffer to module 3 130 (Tx). (4) The module receiving thread of module 3 130 can receive the processed fourth frame data (Rx) from module 2 120 according to the control signals provided by the module scheduling system 200. (4) ).

[0078] The module scheduling system 200, which monitors the processing of the fourth frame data by the main thread of module 3 130, can check the status of module 4 140 at appropriate times. The module scheduling system 200, which checks the processing of the third frame data by module 4 140, can control the module sending thread of module 3 130 to send the processed fourth frame data to module 4 140 (Tx). (4) The module receiving thread of module 4 140 can receive the processed fourth frame data (Rx) from module 3 130 according to the control signals provided by the module scheduling system 200. (4) ).

[0079] As described above, any module of the module scheduling system 200 can monitor the data processing of the module receiving thread, the module main thread, and the module sending thread, and can control the operation of the module receiving thread of the next module after the corresponding module sending thread based on the data processing status of the next module at the corresponding time.

[0080] Figure 8 This is a schematic diagram illustrating an example of the operation of each module in the control system of an autonomous vehicle according to the present invention during each time period.

[0081] In the following text, "time period" refers to an example of the sensing time period of the sensor module, but the invention is not limited thereto. To help understand the operation, the following scenario will be illustrated by example: the first module is a "camera" that is one of the sensor modules such as a camera, LiDAR, laser, ultrasonic sensor or GPS; the second module is an "image preprocessor"; the third module is a "driving controller"; and the fourth module is a "steering device".

[0082] When the first time period begins, module 1 110 can process the sensing data provided by the camera. The module scheduling system 200, which monitors the status of module 1, can provide control signals to module 1's module sending thread 115 for sending the processed first frame data, and can provide control signals to module 2 120's module receiving thread for receiving the processed first frame data. The processing of module 1 110 can be terminated before the start of the second time period, so module 2 can even receive the processed first frame data before the start of the second time period.

[0083] While module 1 110 receives the second frame of data and terminates its processing at the start of the second time period, module scheduling system 200 can check the status of module 2 120 at the appropriate time. Module scheduling system 200 can check whether module 2 120 is still processing the first frame of data and the time when module 2 120 terminated processing the first frame of data. Module scheduling system 200, in checking whether module 2 120 terminated processing the first frame of data, can provide control signals to module 2 120 and module 1 110 for sending the processed second frame of data, can provide control signals to module 3 130 for sending the processed first frame of data, and can provide control signals to module 2 120 for receiving the processed second frame of data, and can also provide control signals to module 3 130 for receiving the processed first frame of data.

[0084] During the third time period, the module scheduling system 200, after checking module 2 120 has finished processing the second frame of data, can check the status of module 3 130 at the corresponding time. When module 1 110 receives the third frame of data and terminates its processing, the module scheduling system 200 can check the status of modules 2 120, 3 130, and 4 140 at the corresponding time. The module scheduling system 200, after checking module 2 120 has finished processing the second frame of data, can provide module 2 120 with control signals for sending the processed second frame of data; it can provide module 3 130, after processing the first frame of data, with control signals for sending the processed first frame of data to module 4 140 and for receiving the processed second frame of data from module 2 120; and it can provide module 4 140 with control signals for receiving the processed first frame of data from module 3 130.

[0085] Then, the module scheduling system 200 can check the operating status of modules 2120 and 3130 when module 1 110 completes processing the third frame of data. The module scheduling system 200 can provide module 1 110 with control signals for sending the processed third frame of data, and can provide module 2 120 (which has finished processing the second frame of data) with control signals for sending the processed second frame of data to module 3 130 and for receiving the third frame of data processed by module 1 110. It can also provide module 3 130 (which has finished processing the first frame of data) with control signals for receiving the second frame of data processed by module 2 120.

[0086] During the fourth time period, the module scheduling system 200 can check the operating status of modules 2120 and 3130 when module 1110 finishes processing the fourth frame of data. Module 3130 is still processing the second frame of data; therefore, the module scheduling system 200, upon detecting that module 2120 has finished processing the third frame of data, can provide module 2120 with a control signal to receive the fourth frame of data processed by module 1110, but may not provide module 3130 with a control signal for receiving the data. After a predetermined time period, the module scheduling system 200, upon detecting that module 3130 has finished processing the second frame of data, can provide control signals to module 3130 and module 2120 to send the processed third frame of data, can provide control signals to module 3130 to send the processed second frame of data to module 4140 and to receive the third frame of data processed by module 2120, and can provide control signals to module 4140 to receive the second frame of data processed by module 3130.

[0087] The module scheduling system 200, which monitors the current operating status of multiple modules in real time, can provide processing instructions, sending instructions, and receiving instructions based on the predetermined operating status of the receiving thread, main thread, and sending thread of the module. It can check the operating status of the next module when the processing of the previous module is completed, and can immediately provide sending and receiving control signals to the previous module and the next module respectively when it is able to process the next module.

[0088] As mentioned above, by controlling the parallel processing of multiple modules of the control system of an autonomous vehicle, the hourly processing capacity of multiple modules can be improved. Therefore, when the entire logic calculation period is longer than the sensor input period, errors caused by delays that occur during data processing between modules can be overcome.

[0089] The control system for the autonomous vehicle according to the present invention can have the following effects.

[0090] First, it can overcome errors caused by delays that occur during data processing between multiple modules.

[0091] Secondly, it can improve the hourly processing capacity of multiple modules.

[0092] Third, it can overcome the problem that occurs when the entire logic calculation period is longer than the sensor input period.

[0093] It will be apparent to those skilled in the art that various modifications and alterations can be made to the present invention without departing from its spirit or scope. Therefore, the present invention is intended to cover all modifications and alterations of the invention as long as they fall within the scope of the appended claims and their equivalents.

Claims

1. A control system for an autonomous vehicle, comprising: Multiple modules are arranged in the autonomous vehicle and configured to perform connectivity operations; and The module scheduling system is configured to monitor the operations of multiple modules, use additional threads to parallelize the multiple modules, and use a pipeline method to control the non-periodic operation of the multiple modules. The module scheduling system includes: The receiving management module is configured to: receive information about the data receiving status from the module's receiving thread within the module; process information about the module's processing status from the module's main thread within the module; and provide processing instructions to the module's main thread within the module; and The sending management module is configured to: receive information about the module's processing status from the module's main thread, receive information about the data sending status from the module's sending thread, and provide sending instructions to the module's sending thread.

2. The control system for an autonomous vehicle according to claim 1, wherein, The plurality of modules includes: A sensing module unit, comprising multiple sensor modules configured to output various sensing data; A processing module unit configured to process sensing data provided by a plurality of sensor modules from a sensing module unit; and The control module unit is configured to control the operation of autonomous driving using the processing results of the processing module unit.

3. The control system for an autonomous vehicle according to claim 2, wherein, Each of the plurality of modules includes: The module receiving thread is configured to receive data from the previous module non-periodically; A receive buffer, configured to temporarily store data received from the module's receive thread; The module's main thread is configured to process data stored in the receive buffer non-periodically; A send buffer, configured to temporarily store data based on the operation results of the module's main thread; A module sending thread, configured to periodically send data stored in a send buffer to another module.

4. The control system for an autonomous vehicle according to claim 3, wherein, The module scheduling system monitors the module receiving thread, module main thread, and module sending thread within each module, and performs control to parallelize the sending, receiving, and processing operations of each thread.

5. The control system for an autonomous vehicle according to claim 4, wherein, The module scheduling system controls the module receiving thread to receive the second frame of data while the module's main thread processes the first frame of data stored in the receive buffer.

6. The control system for an autonomous vehicle according to claim 4, wherein, While the module's main thread is processing the second frame of data stored in the receive buffer, the module scheduling system controls the module's sending thread to send the processed first frame of data stored in the send buffer.

7. The control system for an autonomous vehicle according to claim 4, wherein, The module scheduling system controls the module receiving thread to receive the third frame of data while the module sending thread is sending the processed first frame of data stored in the sending buffer.

8. The control system for an autonomous vehicle according to claim 3, wherein: The receiving management module receives information about the data receiving status from the module receiving thread inside each module that performs the connection operation, and receives information about the module processing status from the module main thread inside each module that performs the connection operation, and controls the processing operation of the module main thread inside each module. The sending management module receives information about the module processing status from the main thread of each module performing the connection operation, and receives information about the data sending status from the sending thread of each module performing the connection operation, and controls the sending operation of the sending thread of each module.

9. A control method for an autonomous vehicle utilizing a module scheduling system, the module scheduling system being used to control multiple modules arranged in the autonomous vehicle and configured to perform connectivity operations, the control method comprising: The receiving management module of the module scheduling system receives information about the data receiving status from the module receiving thread inside the module and information about the module processing status from the module main thread inside the module. The receiving and management module of the module scheduling system provides processing instructions to the module's main thread. The sending management module of the module scheduling system receives information about the module processing status from the module main thread and information about the data sending status from the module sending thread inside the module. The sending management module of the module scheduling system uses a pipelined method to provide sending instructions to the module sending thread non-periodically.

10. The control method for an autonomous vehicle utilizing a module scheduling system according to claim 9, wherein, The module scheduling system monitors the module receiving thread, module main thread, and module sending thread within each module, and performs control to parallelize the sending, receiving, and processing operations of each thread.

11. The control method for an autonomous vehicle utilizing a module scheduling system according to claim 10, wherein, The module scheduling system controls the module receiving thread to receive the second frame of data while the module's main thread processes the first frame of data stored in the receive buffer.

12. The control method for an autonomous vehicle utilizing a module scheduling system according to claim 10, wherein, While the module's main thread is processing the second frame of data stored in the receive buffer, the module scheduling system controls the module's sending thread to send the processed first frame of data stored in the send buffer.

13. The control method for an autonomous vehicle utilizing a module scheduling system according to claim 10, wherein, The module scheduling system controls the module receiving thread to receive the third frame of data while the module sending thread is sending the processed first frame of data stored in the sending buffer.

14. The control method for an automated vehicle using a module scheduling system according to claim 9, wherein, The plurality of modules includes: A sensing module unit, comprising multiple sensor modules configured to output various sensing data; A processing module unit configured to process sensing data provided by a plurality of sensor modules from a sensing module unit; and The control module unit is configured to control the operation of autonomous driving using the processing results of the processing module unit.

15. The control method for an autonomous vehicle utilizing a module scheduling system according to claim 9, wherein, The receive management module and / or send management module operate as a separate application.

16. The control method for an autonomous vehicle utilizing a module scheduling system according to claim 9, wherein, The module scheduling system checks the operation status of the next module when the previous module finishes processing, and immediately provides send and receive control signals to the previous module and the next module respectively when it is able to process the next module.

17. A control method for an autonomous vehicle utilizing multiple modules, each module including a module receiving thread, a module main thread, and a module sending thread, and performing connection operations, the control method comprising: The module receiving thread receives data from the previous module non-periodically; After the module's receiving thread receives the data, it immediately stores the received data in the receive buffer; After the data in the receive buffer is updated, the module's main thread immediately reads and processes the data stored in the receive buffer non-periodically. After the module's main thread finishes processing the data, the processed data is immediately stored in the send buffer; After the data in the transmit buffer is updated, the module's transmit thread immediately sends the data stored in the transmit buffer to the next module aperiodically. Among them, at least two of the data sending and receiving processes and the data transmission process occur simultaneously; When the receiving thread of the module receives a data packet and stores the data packet in the receiving buffer and the receiving flag is in the enabled state, and when the main thread of the module finishes processing the current frame and the data processing flag is in the enabled state, the processing instruction flag of the receiving management module is enabled.

18. The control method for an autonomous vehicle utilizing multiple modules according to claim 17, wherein, When the module's main thread finishes processing the current frame and the data processing flag is enabled, and when the module's sending thread does not send data and the data sending flag is disabled, the sending management module's processing instruction flag is enabled.

Images