Sensor processing method, processing device and motor vehicle sensor system

Abstract

The present disclosure relates to a sensor processing method, processing device and motor vehicle sensor system. A processing method for processing data from a sensor system, the method comprising the steps of receiving sensor data acquired from the sensor system, the sensor data comprising a set of data points associated with a field of view (1) of at least one sensor in an environment. Selecting data points located within one or more regions of interest (2), the one or more regions of interest being selected based on a set of criteria. The selected data points are then processed to detect objects or perform semantic segmentation within the one or more regions of interest. The one or more regions of interest can be selected based on a scene determination of a vehicle (10) in the environment.

Description

Technical Field

[0001] This disclosure relates to a sensor processing method, processing apparatus, and non-transitory computer-readable medium. This disclosure also relates to a motor vehicle sensor system. Specifically, this disclosure relates to a motor vehicle mesh processing system, and more specifically, to a method and system for scene-aware object detection and semantic segmentation. Background Technology

[0002] Object detection and semantic segmentation processing methods (e.g., using convolutional neural networks (CNNs)) operate on a discrete grid of fixed size. Image data from the receiver range of a sensor system is provided as an input grid or matrix, having a defined number of cells representing a sensor image. During processing, convolutional filters or kernels slide across the input data grid in multiple strides to produce output data in the form of feature maps. Thus, the position of each filter moves independently relative to the grid, and the convolution operation is locally applied to the individual blocks of cells selected by that filter during any given stride step. Therefore, as will be understood, the larger the input data grid, the more expensive the image processing becomes, because processing the entire grid requires more sensor strides.

[0003] As mentioned above, depending on the scenario, only a portion of the input data grid is actually of interest for subsequent detection / segmentation processing. That is, the input data grid will typically be set large enough to cover all regions of interest for all potential driving scenarios. For example, in applications involving motor vehicle RADAR, LIDAR, or camera image processing, the input data grid will often be set large enough to include long-range object detection exceeding 100m, such as for high-speed driving on highways. Therefore, the system will use a large grid to cover that entire area. However, such a system also needs to be trained and operational for urban driving and parking scenarios, where shorter-range object detection is required. However, the processing overhead remains high in these scenarios. Furthermore, a large training dataset must be provided to train the CNN, as various scenarios can occur within a large sampling area. This presents challenges in terms of both processing costs and optimizing the size of the dataset required to train the CNN model.

[0004] Therefore, it is still necessary to address the aforementioned shortcomings of traditional object detection and segmentation methods and systems (such as CNN image processing systems). Summary of the Invention

[0005] According to a first aspect, a method for processing data from a sensor system is provided, the method comprising the steps of: receiving sensor data acquired from the sensor system, the sensor data including a set of data points associated with the field of view of at least one sensor in an environment; selecting data points located within one or more regions of interest, the one or more regions of interest being selected based on a set of criteria; and processing the selected data points to detect objects or perform semantic segmentation within the one or more regions of interest.

[0006] In this way, the position and scale of one or more regions of interest to be processed can be adjusted based on the main scene using a standard set. For example, in a vehicle sensor system, a standard set can be used to identify the main scene in which the vehicle is located. This minimizes processing requirements by limiting processing operations to those regions of interest for a specific vehicle scene within the original image frame. In addition to minimizing processing requirements, algorithm coverage can be improved by leveraging knowledge about the vehicle context. Therefore, the problem of high computational and training costs associated with processing large regions for object detection / segmentation operations can be addressed by dynamically selecting relevant regions from the original image frame through shifting and scaling based on the context. Thus, it is not necessary to process the entire input dataset in use. Simultaneously, training can be limited to a subset of the complete region to constrain the resources required for training.

[0007] In this implementation, sensor data represents an input grid that depicts the field of view of an image frame.

[0008] In one implementation, the step of selecting data points located within one or more regions of interest includes selecting a processing grid as a subset of the input grid.

[0009] In this implementation, the size and position of the processing grid within the input grid are selected based on the standard set. This allows for the selection of a larger input dataset or one or more regions of interest within the input grid. Data units within the selected matrix are then extracted from surrounding data and fed into an object detection and segmentation system, such as a convolutional neural network (CNN), for subsequent processing.

[0010] In this implementation, the one or more regions of interest are selected from a predetermined set of one or more regions of interest, each of which is associated with a different set of criteria. These different sets of criteria can be derived from one or more vehicle operation inputs. Such vehicle operation inputs can be initiated by the driver, such as driver-controlled vehicle speed, selected driving mode, gear selection, yaw / turn rate, steering wheel position, acceleration level, deceleration level, and / or other user inputs. Thus, different scenarios identified by the criteria sets can have different regions of interest associated with them, each region of interest having a predefined shape and position for optimizing object detection in that scenario.

[0011] In this implementation, the size and location of one or more regions of interest are dynamically selected based on the set of criteria. This allows the size and location of the processed regions of interest to be dynamically adjusted based on the primary situation. This also allows for changes in selection based on real-time processing. Furthermore, in this implementation, a neural network for vehicle state estimation can be used to dynamically select regions of interest based on all available data, including multiple sensor inputs, to determine the driving scenario. For example, detected high vehicle speeds, along with guardrails detected by a RADAR or camera system, can be used as criteria to indicate a highway scenario. Conversely, detected low vehicle speeds, along with parked vehicles detected by a RADAR or camera system, can be used as criteria to indicate a parking scenario.

[0012] In this implementation, the sensor system is a vehicle sensor system, and the method further includes the step of determining a vehicle scenario in the environment based on one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input. Thus, one or more vehicle inputs can be used to determine the vehicle scenario. For example, high-speed driving above a speed threshold can be used as a criterion to trigger the adoption of one or more regions of interest associated with highway driving. Conversely, speeds below a threshold can be used as a criterion to trigger the adoption of one or more regions of interest associated with parking operations, wherein the positioning of the processed region of interest in front of or behind the vehicle is determined based on gear selection. Similarly, steering wheel rotation can trigger movement of the region of interest in the corresponding direction for the associated rotation operation.

[0013] In an implementation, the step of processing the selected data points includes processing them through one of a convolutional neural network, a shift window transformer, a pointwise machine learning processor, or other locally connected processing layers.

[0014] In this implementation, one or more areas of interest are selected based on the scene determination of the vehicle in the environment.

[0015] According to another aspect, a processing apparatus for processing data from a sensor system is provided, the processing apparatus comprising: an input terminal for receiving sensor data acquired from the sensor system, the sensor data including a set of data points associated with the field of view of at least one sensor of a vehicle in the environment; a selection module for selecting data points located within one or more regions of interest, the one or more regions of interest being selected based on a set of criteria; and a processor for processing the selected data points to detect objects or perform semantic segmentation within the one or more regions of interest. Thus, an apparatus for implementing the above-described method is provided.

[0016] In one implementation, the sensor data represents an input grid of an image frame that depicts the field of view.

[0017] In one implementation, the selection module selects data points located within one or more regions of interest by selecting a processing grid as a subset of the input grid.

[0018] In various embodiments, the selection module is configured to select the size and location of the processing grid within the input grid based on the criteria set. This allows the selection module to specify one or more regions of interest within a larger input dataset / input grid based on the primary vehicle scenario indicated by the criteria set. For example, a processing grid covering the regions of interest can be configured with a length and width twice that of the largest object to be identified. This provides tolerance for detecting objects of varying sizes, as well as sufficient context for effective object detection and semantic segmentation. For example, for a highway driving scenario, the width of the processing grid can be set to at least the width of the road.

[0019] In one implementation, the selection module is configured to select one or more regions of interest from a predetermined set of predetermined regions of interest, wherein each predetermined region of interest is associated with a different set of criteria. This allows for optimization of object detection, for example, for specific vehicle scenarios such as highway driving or parking.

[0020] In this implementation, the selection module is configured to dynamically select the size and location of one or more regions of interest based on the set of criteria.

[0021] In one implementation, the selection module includes a determination input terminal for receiving scene determination based on the standard set used to specify the region of interest. Thus, the selection module can receive one or more inputs from a controller in the vehicle to determine the primary scene.

[0022] In this implementation, the sensor system is a vehicle sensor system, and the device further includes a determination module for generating the scene determination based on one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input. Thus, one or more vehicle operation inputs can be used to select one or more optimized regions of interest based on an understanding of the primary scenario.

[0023] In one implementation, the processor includes one of a convolutional neural network, a shift window transformer, a pointwise machine learning processor, or other locally connected processing layers.

[0024] In one implementation, one or more areas of interest are selected based on a scene determination of the vehicle in the environment.

[0025] According to another aspect, a computer program product including instructions is provided, which, when executed by a computer, cause the computer to perform the above-described method steps.

[0026] According to another aspect, a non-transitory computer-readable medium is provided, comprising instructions that, when executed by a processor, cause the processor to perform the methods described above.

[0027] According to another aspect, a method for training a machine learning system for a sensor system is provided, the method comprising the steps of: (i) selecting one or more regions of interest for a predetermined scene, wherein the regions of interest are associated with a subset of data points within sensor data representing the field of view of at least one sensor in the environment; (ii) generating a training dataset from multiple sample frames associated with the field of view of the at least one sensor during the predetermined scene by selecting one or more regions of interest for each sample frame; and (iii) training the machine learning system using the training dataset. In this way, training can be limited to a subset of the complete input sensor data to minimize the processing resources required to develop an optimized model. Simultaneously, the reduced training processing area allows for a more focused object detection training dataset. For example, objects in convenient regions of image frames can be used for training, and the resulting object detections can be moved to hit different regions of interest or segments. This effectively allows for a better-trained model that can work effectively in a wider range of scenes.

[0028] In this implementation, the step of selecting the training regions of interest includes selecting one or more regions of interest within the field of view of at least one sensor, based on a predetermined scene, and specifying their size and location. This allows for the selection of the configuration of the training regions of interest to optimize object detection while minimizing computational overhead. For example, the processing grid covering one or more regions of interest can be adjusted according to the needs of the relevant task.

[0029] In this implementation, the step of selecting the training region of interest includes selecting a processing grid as a subset of the input grid representing image frames associated with a predetermined scene. This allows the training processing grid to be configured based on identifying portions of the input image frames that are most likely to contain relevant information during a specified scene (e.g., a vehicle driving scene).

[0030] In this implementation, the method further includes repeating steps (i) to (iii) for other predetermined scenarios. In this way, the model can be trained for multiple different scenarios.

[0031] In the implementation, one or more predetermined vehicle scenarios include one or more of the following: highway driving, parking, reversing, forward parking, parallel parking, city driving, and cross-traffic turning.

[0032] According to another aspect, a vehicle sensor system for a vehicle is provided, the system comprising: at least one sensor for generating sensor data including a set of data points associated with the field of view of the vehicle in the environment; an input for receiving the sensor data; a selection module for selecting data points located within one or more regions of interest, the one or more regions of interest being selected based on a set of criteria; and a processor for processing the selected data points to detect objects within the one or more regions of interest. This provides an improved vehicle sensor system with less computational overhead associated with its processing steps (e.g., image processing operations). Therefore, this allows implementation, for example, on an embedded device, rather than requiring high-power general-purpose processing capabilities.

[0033] In this implementation, the vehicle sensor system further includes a determination module for generating a scene determination based on a set of standards, which includes one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input. Thus, the system can determine the current vehicle scene based on one or more vehicle control inputs.

[0034] In various embodiments, the determination module is connected to one or more vehicle electronic control units to receive scene determination input data to generate scene determinations. This allows the system to receive control inputs from other controllers within the vehicle.

[0035] In an implementation, the at least one sensor includes a RADAR, LIDAR sensor, or a camera. Attached Figure Description

[0036] An illustrative embodiment will now be described with reference to the accompanying drawings, in which:

[0037] Figure 1 This is a schematic diagram of the input mesh and two processing mesh variants;

[0038] Figure 2 This is a schematic diagram of an image processing apparatus for a motor vehicle sensor system according to an embodiment;

[0039] Figure 3 A vehicle including a vehicle sensor system is shown in a first parking scenario;

[0040] Figure 4 The vehicle, including a vehicle sensor system, is shown in a second parking scenario; and

[0041] Figure 5 A vehicle including a vehicle sensor system is shown in a highway scenario. Detailed Implementation

[0042] Figure 1 This is a schematic diagram of an input grid 1 for representing an image frame of the field of view of at least one sensor of a vehicle depicting an environment. For example, the vehicle's RADAR sensor system may include a RADAR sensor array, which together can provide sensor data including a set of data points for the field of view around the vehicle. This can be used, for example, to generate a 360-degree radar image of the environment around the vehicle. The resulting image frame can be output as a Cartesian grid for subsequent processing. In conventional image processing systems, the entire input grid 1 would be processed, for example, through one or more convolutional layers of a convolutional neural network (CNN). However, in this implementation, a subset of cells can be selected from the input grid 1 as regions of interest specified by processing grids 2a and 2b. Then, only the selected subset is processed by the CNN. This reduces computational overhead by minimizing the size of the grid matrix that needs to be processed. Importantly, this benefit is taken advantage of by selecting the size, shape, and position within the input grid of the processing grid based on the regions of interest within the image frame for a specific scene. That is, in Figure 1The diagram shows two processing mesh variants, 2a and 2b, and one or the other of these variants can be selected for processing depending on the specific driving scenario in which the vehicle is located.

[0043] Regarding this point, Figure 2 This is a schematic diagram of a processing apparatus for a motor vehicle sensor system according to an embodiment. A selection module 4 for selecting a processing grid is provided, a determination module 8 for generating scene determination based on a standard set is provided, and a CNN 6 for processing the selected processing grid is provided. It should be understood that the selection module 4, the determination module 8, and the CNN 6 can be implemented using one or more microprocessors, for example, as part of a motor vehicle controller unit or an embedded device.

[0044] The selection module 4 has an input terminal 3 through which it receives sensor data. For example, in one embodiment, the input terminal 3 may receive RADAR, LIDAR, or camera sensor data, which can be processed to generate 360-degree view image frames. Simultaneously, the selection module 4 includes a determination input terminal 9 for receiving scene determination from the determination module 8.

[0045] The determination module 8 generates a scene determination based on state input received from a vehicle controller located elsewhere in the vehicle. In an implementation, the input may include one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input.

[0046] In use, the scene determination from the determination module 8 is input to the selection module 4 and used to select processing grids 2a and 2b as a subset of input grid 1. Upon selection, data 5 for the selected processing grid is extracted from the surrounding input grid data and fed into the input of CNN 6, where it is passed through at least one convolutional layer. CNN 6 then outputs output data 7, which can be used for subsequent operations, such as advanced driver assistance systems (ADAS), automatic parking, or autonomous driving capabilities.

[0047] Therefore, depending on the usage scenario, the processing grid can be shifted and scaled to where it is needed, without calculating the rest of the input grid.

[0048] Regarding this, there are generally three different scenarios or situations related to parking: (i) finding parking space, (ii) parking forward, and (iii) parking backward.

[0049] Figure 3The illustration shows a vehicle 10 including a vehicle sensor system in a first parking scenario where the vehicle 10 is searching for a parking space 15. In this scenario, a large viewing distance to the front and sides is required to identify an empty parking space 15 by detecting other vehicles 16 or free space. Thus, a processing grid 2 is selected to cover these areas, rather than the entire field of view of the RADAR signal 11. For example, in various embodiments, the scene determination module 8 can detect that the vehicle speed is below a lower threshold, such as below 20 kph, and determine that this indicates a parking operation is in progress. Therefore, processing grid 2 is selected accordingly. Thus, subsequent processing of CNN 6 is limited to the region of interest defined by processing grid 2, thereby minimizing computational overhead while still allowing object detection for identifying other vehicles 16, walls 14, and bushes 12, etc. It should also be understood that while the above applies to forward parking operations, if reverse gear is selected, processing grid 2 can be moved backward to focus on these areas.

[0050] Figure 4 The image shows vehicle 10 in a second parking scenario, where parking space 15 is selected. In this scenario, vehicle 10 is no longer needed. Figure 3 The larger processing grid shown is used instead of the smaller one. Instead, much smaller distances to the front / rear / side of vehicle 10 need to be processed. Therefore, a smaller, more centered processing grid 2 is used. Similarly, in some embodiments, the determination module 8 can receive scene data indicating whether parking space 15 is on the left or right, for example, by receiving indicator data. In this case, processing grid 2 can be limited to the corresponding side, thus avoiding the need to process data associated with the non-parking side.

[0051] In another parking scenario example, the processing grid can be selected to be + / - 10m in each direction starting from the sensor origin. To find parking space, the processing grid can be shifted to x∈[0, 20], y∈[-10, 10], and when reversing or having passed the parking space, the grid can be shifted to x∈[-20, 0], y∈[-10, 10]. When moving into the parking space, it can then be shifted back to x∈[-10, 10], y∈[-10, 10] to cover both the front and rear. As a result, for the same computational resources, the vehicle parking function can identify twice the distance of available parking space in front compared to a fixed grid scheme. Additionally, rear object detection during reversing parking can also be doubled, again without additional computational cost.

[0052] Figure 5A vehicle 10 is shown in a highway scene. In this case, due to the high vehicle speed, the processing grid 2 needs a large longitudinal range. Meanwhile, because space is limited by the width of the road guardrail 17, the selected processing grid 2 can be relatively narrow in lateral size. Therefore, in various embodiments, the scene determination module 8 can detect when the vehicle speed exceeds a higher threshold, such as a speed exceeding 60 kph, and determine that this indicates the vehicle is in a highway driving scene. This can then trigger the application. Figure 5 The long and narrow processing grids shown.

[0053] In urban driving scenarios, such as at speeds between 20 and 60 kph, a high longitudinal range is less important. This is because, for example, the lower vehicle speed reduces the need to identify distant objects. Simultaneously, the unobstructed view is typically limited to shorter distances due to the proximity of surrounding objects. However, if the vehicle is turning, as indicated by the yaw rate, a much larger lateral range is required. Therefore, the scene determination module 8 can receive input for identifying these different scenes and dynamically change the processing mesh variant based on key requirements.

[0054] In this implementation, a selected subset of the input grid is shifted and scaled to focus on the region of interest for any particular scene. Therefore, it is not necessary to process a large input grid covering all events. Furthermore, training can be limited to a subset of the full input grid, thus limiting the resources required for training. Therefore, this implementation provides increased algorithmic coverage and reduced computational power requirements by leveraging knowledge about the scenario and adjusting the position and scale of the processing grid accordingly.

[0055] Regarding this, it should be understood that when working in a grid coordinate system, the size of the detected objects or structures does not change based on their distance or position within the grid matrix. For example, As an example of a Cartesian grid input to a convolution function F. Typically, x is padded, such that for... This produces an output y = F(x) of equal size. Since convolution is shift-invariant, it performs the same operation regardless of the position within the grid being processed. Therefore:

[0056] F(x)[100:H–100,0:W-10]≈F(x[100:H–100,0:W-10])

[0057] Due to the fill, the only differences will be at the edges of the mesh. The fill can be ignored, or the mesh can be filled and cut later, where the fill is outside the region of interest. Therefore:

[0058] F(x[100-padding:H–100+padding,0:W-10+padding])[padding:H-200+

[0059] padding,0:W-10]==F(x)[100:H-100,0:W-10]

[0060] Due to the pooling and stacking of multiple layers within a CNN, the information contained within an image frame can travel further than the width of a single filter. Thus, object detection is generally improved when a larger context is present. However, processing grid 2 can be selected in a way that preserves the context of the specific scene for subsequent processing. Therefore, the processing grid can be shifted and scaled to reduce computational overhead without sacrificing object detection / segmentation performance. For example, in the highway driving scene discussed above, processing grid 2 could be scaled and positioned to have a road width and length at least twice the size of the largest expected object to be detected. This allows sufficient data to be retained from the original input grid to avoid compromising object detection / segmentation performance.

[0061] Based on the above, it is advantageous to train the CNN using a training processing grid of size that only covers the relevant objects / structures and adjacent context for a specific scene. Therefore, the training processing grid can be selected such that there are no blind spots or other problematic areas in the training region. Furthermore, the training processing grid can typically be much smaller than a conventional input grid that covers all relevant areas. When selecting the training processing grid, a training dataset can be generated from multiple sample image frames depicting the sensor's field of view during a predetermined vehicle scene by selecting the grid for each sample image frame. These training processing grids can then be processed to train the CNN. This thereby saves computational power and memory consumption during training. Simultaneously, because the size of the objects or structures does not change within the Cartesian grid, training performed on one focused scene can actually be used for other focused scenes. For example, training for parking space detection on the left side of the vehicle will be relevant to the parking scene on the right side. Therefore, this implementation provides a more efficient training process.

[0062] Therefore, by moving and scaling the processing grid to focus on the needs of a specific scene and the different object detection / semantic segmentation subtasks associated with that scene, more efficient CNN image processing and training can be achieved. This improves performance and alleviates the known limitations of fixed-grid schemes. Furthermore, because the processing grid is selected before being input into the CNN, the CNN structure itself remains unchanged. Therefore, this implementation can be carried out using general-purpose hardware accelerators.

[0063] It should be understood that the above embodiments are illustrative of applications only. In practice, the embodiments can be applied to many different configurations, and detailed embodiments are readily implemented by those skilled in the art.

[0064] For example, although a single processing grid was selected from the input grid in the illustrative example above, it should be understood that this selection can also be achieved by selecting data points located within one or more regions of interest that are within the field of view provided by the sensor data.

Claims

1. A method for processing data from a sensor system, the method comprising the steps of: Receive sensor data acquired from the sensor system, the sensor data including a set of data points associated with the field of view of at least one sensor in the environment; The scenario is determined based on the driver's input of one or more vehicle operations. Based on the determined scene, select the position and size of one or more regions of interest within the field of view; as well as Process data points from the data point set within one or more regions of interest to detect objects or perform semantic segmentation within those regions. The detection object or the semantic segmentation depends on one or more vehicle operation inputs initiated by the driver and on whether the scenario determined based on one or more vehicle operation inputs initiated by the driver is highway driving, parking, reversing parking, forward parking, parallel parking, city driving, or turning at intersections.

2. The method according to claim 1, wherein, The sensor data represents an input grid depicting the field of view of an image frame, wherein the step of selecting the location and size of the one or more regions of interest includes selecting a processing grid as a subset of the input grid, and wherein the size and location of the processing grid within the input grid are selected according to the determined scene.

3. The method according to claim 1, wherein, The sensor system is a motor vehicle sensor system, and the one or more vehicle operation inputs initiated by the driver include one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input.

4. A processing apparatus for processing data from a sensor system, the processing apparatus comprising: An input terminal is configured to receive sensor data acquired from the sensor system, the sensor data including a set of data points associated with the field of view of at least one sensor in the environment; The determining module is used to determine a scenario based on one or more vehicle operation inputs initiated by the driver; The selection module is used to select the position and size of one or more regions of interest in the field of view based on the determined scene; as well as A processor, configured to process data points of a set of data points within one or more regions of interest to detect objects or perform semantic segmentation within those regions. The detection object or the semantic segmentation depends on one or more vehicle operation inputs initiated by the driver and on whether the scenario determined based on one or more vehicle operation inputs initiated by the driver is highway driving, parking, reversing parking, forward parking, parallel parking, city driving, or turning at intersections.

5. The processing apparatus according to claim 4, wherein, The sensor data represents an input grid depicting the field of view of an image frame, wherein the selection module selects the position and size of one or more regions of interest by selecting a processing grid as a subset of the input grid, and wherein the selection module is configured to select the size and position of the processing grid within the input grid according to a determined scene.

6. The processing apparatus according to claim 4, wherein, The sensor system is a motor vehicle sensor system, and the one or more vehicle operation inputs initiated by the driver include one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input.

7. The processing apparatus according to claim 4, wherein, The processor includes one of a convolutional neural network, a shift window transformer, a pointwise machine learning processor, or other locally connected processing layers.

8. A non-transitory computer-readable medium storing a computer program comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 3.

9. A vehicle sensor system for a vehicle, the vehicle sensor system comprising: At least one sensor, said at least one sensor being used to generate sensor data including a set of data points associated with the vehicle’s field of view in the environment; The input terminal is used to receive the sensor data; The determining module is used to determine a scenario based on one or more vehicle operation inputs initiated by the driver; The selection module is used to select the position and size of one or more regions of interest in the field of view based on the determined scene; as well as A processor, configured to process data points of a set of data points within one or more regions of interest to detect objects or perform semantic segmentation within those regions. The detection object or the semantic segmentation depends on one or more vehicle operation inputs initiated by the driver and on whether the scenario determined based on one or more vehicle operation inputs initiated by the driver is highway driving, parking, reversing parking, forward parking, parallel parking, city driving, or turning at intersections.

10. The motor vehicle sensor system according to claim 9, wherein, The vehicle operation inputs initiated by the driver include one or more of the following: vehicle speed, selected driving mode, gear selection, yaw rate, steering wheel position, acceleration level, deceleration level, and user input.

Images