Control unit for processing data related to a working module

Abstract

The invention relates to a control unit for processing data relating to a work module. The control unit 10 receives data from the work module 12 and collects the received data in a predefined format in a data warehouse 16. The control unit (10) then passes the collected data through a predefined set of perceptron layers (18) for processing the collected data. The control unit 10 enhances features extracted from the collected data in at least one starting layer 18 (al) of the predefined set of perceptron layers 18 and compresses the collected data in at least one ending layer 18 (b) of the predefined set of perceptron layers 18. The control unit 10 then passes the enhanced and compressed features of the data to a neural network backbone architecture module 20 for application in a perceptual application 22.

Description

Technical Field

[0001] The present invention relates to a control unit for processing data related to a working module. Background Art

[0002] Deploying a neural network model on edge devices with limited resources is a huge challenge and requires reducing the model size, which in turn reduces its performance in perception applications. Currently in the market, there are various available technologies that deal with the feature enhancement performance of neural networks in perception applications. However, they do not increase the requirements in terms of hardware resources because this reduces the possibility of deploying the model on edge devices. Sending raw features or unenhanced compressed features to the neural network will have a negative impact on the model performance of the neural network in perception applications.

[0003] Patent US10582205 discloses a method for enhancing at least one segment of lower-quality visual data using a hierarchical algorithm, the method comprising receiving at least one segment of lower-quality visual data; and extracting a subset of features from at least one segment of the lower-quality visual data. Forming and enhancing a plurality of dimensionality-reduced visual data layers from the extracted features to form at least one segment of higher-quality visual data. The at least one segment of higher-quality visual data corresponds to the at least one segment of the received lower-quality visual data. Brief Description of the Drawings

[0004] Figure 1 Illustrates a control unit for processing data related to a working module according to an embodiment of the present invention;

[0005] Figure 2 Illustrates a flowchart of a method for processing data related to a working module according to the present invention. Detailed Description

[0006] Figure 1 Illustrates a control unit for processing data related to a working module according to an embodiment of the present invention. The control unit 10 receives data from the working module 12 and collects the received data in a data warehouse 16 in a predefined format. Then, the control unit 10 passes the collected data through a predefined set of perceptron layers 18 for processing the collected data. The control unit 10 enhances the features extracted from the collected data in at least one starting layer 18(a) of the predefined set of perceptron layers 18, and compresses the collected data in at least one ending layer 18(b) of the predefined set of perceptron layers 18. Then, the control unit 10 transmits the enhanced and compressed features of the data to a neural network backbone architecture module 20 for application in a perception application 22.

[0007] Further details the construction of the control unit 10 connected to the working module 12. The control unit 10 is a logic circuit and software program, implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any component that operates on signals based on operation instructions. The working module 12 is selected from a group of working modules including radio radar, image sensors, LIDAR, etc. However, those skilled in the art should understand that the working module is not limited to the modules mentioned above, but can be any other module used for collecting data. The control unit 10 includes an enhancement and compression module 24 for processing data. Processes the data in the point cloud repository 16 during the preprocessing stage.

[0008] The data repository 16 is a point cloud repository, and the predefined format for collecting data is in the form of M points and N features, that is, the point cloud data is collected in the format of M points with N features, and the point cloud data includes any one of the following: a single time frame, a combination / cumulation of multiple time frames. For example, if the working module 12 is an image sensor, the image sensor 12 captures multiple / single images based on requirements. Each image is called a frame.

[0009] The predefined set of perceptron layers 18 includes an odd number of perceptron layers 18, which are distributed in the enhancement module and the compression module 24 of the control unit 10. The minimum number of perceptron layers 18 is three, and the first layer and the middle layer 18(a) will perform feature enhancement for better understanding, and the last layer 18(b) will perform compression, thus maintaining the hardware size without damage. The control unit 10 shares the weights of each layer of the perceptron layer 18 existing in the point cloud repository 16. Each layer 18 is given a vector weight, and the value of the weight is maintained such that when performing feature enhancement, the first layer 18(a) will have a weight smaller than or equal to that of the consecutive perceptron layers 18. In addition, when performing feature compression, the weight of the last perceptron layer 18(b) is less than that of the previous perceptron layer 18.

[0010] The enhanced and compressed data is transmitted to the neural network module 20 existing in the control unit 10, so that the formed data can be used for any one of the perceptron applications 22, such as autonomous driving, healthcare, etc. Since the collected data is enhanced and compressed during the processing stage, the size of the network model is reduced.

[0011] Figure 2The figure illustrates a flowchart of a method for processing data related to a working module according to the present invention. In step S1, data is received from the working module 12. In step S2, the received data is collected in a data warehouse 16 in a predefined format. In step S3, the collected data is passed through a predefined set of perceptron layers 18 for processing the collected data. In step S4, the features extracted from the collected data are enhanced in at least one starting layer 18(a) of the predefined set of perceptron layers 18, and the collected data is compressed in at least one ending layer 18(b) of the predefined set of perceptron layers 18. In step S5, the enhanced and compressed features of the data are transmitted to a neural network backbone architecture module 20 for application in a perception application 22.

[0012] The working method of the control unit 10 is explained in detail. The working module 12 is a sensor module according to an exemplary embodiment, and for better understanding of the present invention, the sensor module 12 can be a radio radar sensor or an image sensor, and the perception application is autonomous driving in a vehicle. The radio radar / image sensor 12 is mounted outside the vehicle and is made to transmit and receive electromagnetic waves. The radio radar 12 detects the objects present in the environment by transmitting and receiving electromagnetic waves. These waves will hit the objects present in the surrounding environment and will return to reach the radio radar. In the case of the image sensor 12, multiple images of the surrounding environment are captured, and each image is considered as a single frame for detecting objects. These are considered as the data that needs to be processed in the control unit 10.

[0013] The data warehouse 16 is also referred to as a point cloud warehouse, which receives object data from the radio radar module 12. According to an embodiment of the present invention, the data warehouse 16 is made as an integral part of the control unit 10, i.e., the data is stored in the memory of the control unit 10. According to another embodiment, the data warehouse 16 is a cloud warehouse connected to the control unit 10 via any one of the communication means known in the prior art

[0014] The data received from the radio radar module 12 is collected in the format of M*N, where M refers to the points involved in the collected data, and N refers to the features involved in the collected data. The point cloud data is collected in the format of M points with N features, and the point cloud data includes any one of the following: a single time frame, a combination / cumulation of multiple time frames. For ease of understanding of the present invention, the M*N format is also referred to as a predefined format. Thus, the collected data is passed through an enhancement and compression module 24 of the control unit 10 to obtain an output, which is also point cloud data of M points with Q features.

[0015] The compression and enhancement module 24 consists of L shared multi-layer perceptron layers 18, and the minimum value of L is maintained as an odd number. For example, the minimum value of L is maintained as three. The weights of the layers 18 are shared across the points in the point cloud repository 16. The predefined set of perceptron layers 18 includes three perceptron layers. In these three layers, the first layer 18(a) processes N input features to give enhanced features of size O, where O is greater than or equal to N. The middle layer 18(a) performs additional enhancement and outputs features of size P, where P is greater than O. The last layer 18(b) of the predefined set compresses the enhanced features from the middle layer and outputs features of size Q, where Q is less than P. This allows the module 24 to compress the enhanced features, thus reducing the hardware resource requirements in the neural network backbone module 20 and also reducing the size of the network module.

[0016] However, it is to be understood that any number of layers 18 can be added between the three layers 18 by maintaining the total number of layers as an odd number based on requirements. For example, if the enhancement and compression module 24 includes five layers, then the first, second, and third layers 18(a) will perform the enhancement of the features extracted from the collected data. These features are related to the objects detected in the vehicle's surrounding environment. And the last two layers 18(b), i.e., the fourth and fifth layers, are used to compress the data with the enhanced features, thus maintaining the size of the network module 20.

[0017] In addition, if the first layer of the neural network backbone module 20 outputs features of size R, then Q is made less than or equal to R. Making R as close as possible to Q will result in a significant reduction in the hardware resource requirements of the neural network. The output of the neural network backbone module 20 is transmitted to / used for various perception applications. The perception applications 22 using this feature compression and enhancement module 24 run with better performance than those not using this feature compression and enhancement module 24. These applications 22 also consume fewer hardware resources compared to running with the prior art enhancement methods.

[0018] The present invention discloses a method for point cloud feature compression and enhancement. The method collects point cloud data having M points and N features and passes the point cloud data through the feature compression and enhancement module 24. By doing so, the control unit 10 obtains new features of the point cloud data. By enhancing the features in multiple layers, this helps to understand and train the module in a better way. Such a point cloud with new features can be passed through the neural network backbone structure module 20, and the output of the neural network backbone structure module 20 can be used by several perception applications 22. The presence of our feature compression and enhancement module 24 significantly reduces the hardware resource requirements of the neural network model 20 while improving the performance of the perception applications 22 due to feature enhancement.

[0019] It should be understood that the embodiments explained above in the description are merely illustrative and do not limit the scope of the present invention. Many such embodiments and other modifications and variations of the embodiments explained in the description are contemplated. The scope of the present invention is defined only by the scope of the claims.

Claims

1. A control unit (10) for processing data related to a working module (12), the control unit (10) being adapted to: - Receive data from the working module (12); - Collect the received data in a data warehouse (16) in a predefined format; - Pass the collected data through a predefined set of perceptron layers (18) for processing the collected data; - Enhance features extracted from the collected data in at least one starting layer (18(a)) of the predefined set of perceptron layers (18), and compress the collected data in at least one ending layer (18(b)) of the predefined set of perceptron layers (18); - Transmit the enhanced and compressed features of the data to a neural network backbone architecture module (20) for application in a perception application (22).

2. The control unit (10) according to claim 1, wherein the working module (12) is a sensor selected from a group of sensors including a radio radar, a camera, and a lidar.

3. The control unit (10) according to claim 1, wherein the data warehouse (16) is a point cloud warehouse, and the predefined format for collecting the data is in the form of M points and N features.

4. The control unit (10) according to claim 3, wherein the point cloud data is collected in the format of M points with N features, and the point cloud data includes any one of the following: a single time frame, a combination / cumulation of multiple time frames.

5. The control unit (10) according to claim 1, wherein the predefined set of perceptron layers (18) includes an odd number of perceptron layers, and the odd number of perceptron layers are distributed in an enhancement and compression module (24) of the control unit (10).

6. The control unit (10) according to claim 5, wherein the weights of each perceptron layer (18) are shared across the points present in the point cloud warehouse (16).

7. The control unit (10) according to claim 6, wherein when performing the enhancement of the features, the weights of the first perceptron layer (18(a)) are less than or equal to the weights of the consecutive perceptron layers (18).

8. The control unit (10) according to claim 7, wherein when performing the compression of the features, the weights of the last perceptron layer (18(b)) are less than the previous perceptron layer (18).

9. The control unit (10) according to claim 1, wherein due to the enhancement and compression of the collected data in the preprocessing stage, the network model size is reduced.

10. A method for processing data related to a working module (12) by a control unit (10), the method comprising: - Receive data from the working module (12); - Collect the received data in a data warehouse (14) in a predefined format; - Pass the collected data through a predefined set of perceptron layers (18) for processing the collected data; - Enhance the features extracted from the collected data in at least one starting layer (18(a)) of the predefined set of perceptron layers (18), and compress the collected data in at least one ending layer (18(b)) of the predefined set of perceptron layers (18); - Transmit the enhanced and compressed features of the data to the neural network backbone architecture module (20) for application in a perception application (22).

Images