Method and device for detecting parking space in surroundings of vehicle

Abstract

The present application relates to a method for detecting a parking space in a surrounding environment of a vehicle, comprising: obtaining a plurality of types of sensing data; for each grid in the grid window, multiple types of features, corresponding to the multiple types of sensing data, of the grid are determined based on the multiple types of sensing data, and the multiple types of features, corresponding to the multiple types of sensing data, of the grid are integrated into integrated features of the grid, the grid window corresponds to a space area of a first size, and each grid in the grid window corresponds to a space area of a second size in the space area of the first size; and outputting a detection result of the parking space in the grid window based on the integrated feature of each grid in the grid window.

Description

Technical Field

[0001] This application relates to automatic vehicle control, and more specifically, to a method and apparatus for determining the location of obstacles in the surrounding environment of a vehicle based on ultrasonic signals. Background Technology

[0002] Parking space detection is a crucial step in enabling vehicle parking assistance functions. This can be achieved by using sensor signals to detect obstacles and available spaces in the vehicle's surroundings. More accurate obstacle detection allows for more precise determination of parking space locations, thereby improving the performance of the vehicle's parking assistance function.

[0003] Ultrasonic sensors (USS) offer a low-cost method for obstacle detection. However, conventional detection methods using ultrasonic signals are usually rule-based and cannot be effectively applied to more complex scenarios. Furthermore, the accuracy of obstacle location detection based on ultrasonic signals needs to be improved to help enhance the performance of vehicle parking assistance functions.

[0004] The fusion of multiple sensor types can help improve detection accuracy in autonomous driving. However, general multi-sensor fusion methods are usually accompanied by extremely high computational complexity. Therefore, it would be beneficial to utilize multi-sensor fusion with lower computational cost for parking assistance scenarios while improving obstacle location detection accuracy. Summary of the Invention

[0005] The following brief introduction is provided to present some of the selected concepts in a simplified manner, which will be further described in the detailed description that follows. This brief introduction is not intended to highlight the key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] According to one aspect of this application, a method for detecting parking spaces in the surrounding environment of a vehicle is provided, comprising: acquiring multiple types of sensing data; for each grid cell in a grid window, determining multiple types of features corresponding to the multiple types of sensing data for each grid cell based on the multiple types of sensing data; integrating the multiple types of features corresponding to the multiple types of sensing data for each grid cell into an integrated feature of the grid cell, wherein the grid window corresponds to a spatial region of a first size, and each grid cell in the grid window corresponds to a spatial region of a second size within the spatial region of the first size; and outputting a detection result of a parking space in the grid window based on the integrated feature of each grid cell in the grid window.

[0007] According to one aspect of this application, a method for assisted parking is provided, comprising: detecting a parking space by performing the method described herein; and planning a route for driving a vehicle into the parking space based on the detected parking space.

[0008] According to one aspect of this application, an apparatus for detecting parking spaces in the surrounding environment of a vehicle is provided, comprising: a feature integration module, which, for each grid cell in a grid window, determines multiple types of features corresponding to the multiple types of sensing data for each grid cell based on multiple types of sensing data, and integrates the multiple types of features corresponding to the multiple types of sensing data for each grid cell into an integrated feature of the grid cell, wherein the grid window corresponds to a spatial region of a first size, and each grid cell in the grid window corresponds to a spatial region of a second size within the spatial region of the first size; and a detection module, which outputs a detection result of a parking space in the grid window based on the integrated feature of each grid cell in the grid window.

[0009] According to one aspect of this application, a control system for a vehicle is provided, comprising: one or more processing units configured, when executing program instructions, to perform the method described herein for detecting a parking space or for assisting parking.

[0010] According to one aspect of this application, a machine-readable storage medium is provided that stores executable instructions, which, when executed, cause one or more processors to perform the methods described herein for detecting parking spaces or for assisting parking.

[0011] According to one aspect of this application, a computer program product is provided, comprising executable instructions that, when executed, cause one or more processors to perform the methods described herein for detecting parking spaces or for assisting parking.

[0012] The fusion of multiple sensor types according to this application can help improve the accuracy of obstacle location detection during assisted parking, and can also achieve the integration or fusion of multiple types of sensing data at a lower computational cost for specific parking assistance applications. Furthermore, by combining ultrasonic sensing signals (USS) with other auxiliary sensing signals, the accuracy of obstacle detection during assisted parking can be improved based on other sensors configured in the vehicle at a low cost using USS signals. This allows for the complementary and accurate detection of obstacle locations based on multiple sensor types in various complex scenarios, thereby achieving accurate detection of available parking spaces and contributing to improved performance of assisted parking functions. Attached Figure Description

[0013] A further understanding of the nature and advantages of this disclosure can be achieved by referring to the accompanying drawings. In the drawings, similar components or features may have the same reference numerals.

[0014] Figure 1 A schematic diagram of a vehicle according to one embodiment is shown.

[0015] Figure 2 A schematic diagram of a control system in a vehicle according to one embodiment is shown.

[0016] Figure 3 A schematic diagram of sensing data according to one embodiment is shown.

[0017] Figure 4 A schematic diagram of a method for determining parking spaces around a vehicle, according to one embodiment, is shown.

[0018] Figures 5 to 9 Schematic diagrams of an apparatus for detecting parking spaces in the environment surrounding a vehicle, according to one embodiment, are shown.

[0019] Figure 10 A method for detecting parking spaces in the environment surrounding a vehicle, according to one embodiment, is shown.

[0020] Figure 11 A method for assisted parking according to one embodiment is shown.

[0021] Figure 12 An apparatus for detecting parking spaces in the environment surrounding a vehicle is shown according to one embodiment.

[0022] Figure 13 A block diagram of a control system for a vehicle according to one embodiment is shown. Detailed Implementation

[0023] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0024] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Unless explicitly indicated by the context, the definition of a term remains consistent throughout the specification.

[0025] Figure 1 A schematic diagram of a vehicle according to one embodiment is shown.

[0026] Vehicles typically equip themselves with various types of environmental perception sensors. Commonly used environmental perception sensors generally fall into two categories: visual image sensors and radar-based ranging sensors. Image sensors can be simply referred to as cameras, such as... Figure 1 The camera shown is 120. Commonly used radar-type ranging sensors include LiDAR (Light Detection and Ranging), millimeter-wave radar (Radio Detection and Ranging), and ultrasonic radar (USS, UltraSonic Sensor), such as... Figure 1 The lidar 140, millimeter-wave radar 130, and ultrasonic radar 110 shown are illustrated.

[0027] exist Figure 1 In the example shown, a camera 120, an ultrasonic sensor 110, and a millimeter-wave radar 130 are respectively installed on the front, rear, left, and right sides of vehicle 100, and a lidar 140 is installed on the front of the vehicle. The front, rear, left, and right sides of the vehicle correspond to the front, rear, left side, and right side of the vehicle, respectively. It is understood that the number and installation positions of the various types of sensors 110 to 140 are not limited to... Figure 1 As shown, in different implementations, vehicle 100 may include more or fewer sensors 110 to 140, and the mounting locations of the sensors may also vary. In different implementations, vehicle 100 may not necessarily install all of the types of sensors 110-140 shown, but may install only a portion of the types of sensors shown. For example, vehicle 100 may not install LiDAR 140, but only install camera 120, ultrasonic sensor 110, and millimeter-wave radar 130; or vehicle 100 may only install camera 120, ultrasonic sensor 110, and LiDAR 140; and so on.

[0028] exist Figure 1In the example shown, the working principle of the ranging sensor is introduced using ultrasonic sensor 110 as an example. Those skilled in the art will understand that ultrasonic sensor 110, millimeter-wave radar 130 and lidar 140 are all commonly used ranging sensors in the art, and the corresponding sensing data are also known in the art. Therefore, the details of millimeter-wave radar 130 and lidar 140 will not be described in detail.

[0029] like Figure 1 As shown, the ultrasonic sensor 110 can emit an ultrasonic signal 112. When the ultrasonic signal 112 encounters an obstacle and is reflected, the ultrasonic sensor 110 can receive the reflected ultrasonic echo signal. Based on the emitted ultrasonic signal and the received echo, various ultrasonic data related to the ultrasonic signal can be obtained. In one embodiment, the ultrasonic sensor module may include the ultrasonic sensor 110 and a processing unit. The ultrasonic sensor 110 includes a transmitter for emitting ultrasonic signals and a receiver for receiving ultrasonic signals such as echoes. The corresponding processing unit can obtain various ultrasonic data based on the emitted ultrasonic signal and the received echo signal, which includes various features related to the ultrasonic signal.

[0030] For example, Figure 1 The two ultrasonic sensors 110 on the right front side of the vehicle shown can constitute an ultrasonic sensor module, which includes two ultrasonic sensors for transmitting and receiving ultrasonic signals and a processing unit. This processing unit can obtain various ultrasonic data based on the ultrasonic signals emitted by the two corresponding sensor units and the corresponding received echo signals; similarly, Figure 1 The ultrasonic sensor 110 on the right rear side, the ultrasonic sensor 110 on the left front side, the ultrasonic sensor 110 on the left rear side, the ultrasonic sensor 110 on the front side, and the ultrasonic sensor 110 on the rear side of the vehicle shown can all be combined with their corresponding processing units to form an ultrasonic sensor module.

[0031] For example, Figure 1 The four ultrasonic sensors 110 on the right side of the vehicle and their corresponding processing units can form an ultrasonic sensor module. The processing unit can obtain various ultrasonic data based on the ultrasonic signals emitted by the four sensors and the corresponding echo signals received. Similarly, the ultrasonic sensors 110 on the left side, the front ultrasonic sensors 110, and the rear ultrasonic sensors can all form an ultrasonic sensor module with their corresponding processing units.

[0032] For example, Figure 1 All the ultrasonic sensors 110 shown can be combined with their corresponding processing units to form an ultrasonic sensor module. The processing unit can obtain various ultrasonic data based on the ultrasonic signals emitted by the corresponding sensors and the corresponding echo signals received.

[0033] In the examples above, the processing unit can control the transmission and reception of signals from various ultrasonic sensors, and obtain various ultrasonic data based on the ultrasonic signals emitted and the corresponding echo signals received by the respective sensors. In this document, depending on the context, "ultrasonic sensor" can refer to an ultrasonic sensing unit used for transmitting and receiving ultrasonic signals and echoes, or it can refer to the ultrasonic sensor module in the examples above. Those skilled in the art can distinguish the meaning of "ultrasonic sensor" in a specific context. Similarly, millimeter-wave radar and lidar can refer to sensing units used for transmitting and receiving millimeter-wave signals, laser signals, and echoes, or they can refer to corresponding sensor modules including sensing units and corresponding processing units. A camera can refer to a sensing unit used for capturing images, or it can refer to a sensor module including sensing units and corresponding processing units. Those skilled in the art can distinguish the meaning of millimeter-wave radar, lidar, and cameras in a specific context.

[0034] The processing unit of the ultrasonic sensor module can obtain various ultrasonic data based on the ultrasonic signals emitted by the corresponding sensor 110 and the corresponding received echo signals. In one embodiment, the ultrasonic data may include echo data, which may include echo amplitude, echo significance, echo distance, echo coordinates, sensor coordinates, timestamps, etc. In one embodiment, the centerline method can be used to obtain the echo data. For example, as the vehicle moves along direction 150, the ultrasonic sensor 110 on the right front side of the vehicle sends ultrasonic signals and receives ultrasonic echo signals under the control of its processing unit. In the centerline method, it is assumed that the reflection point on the detected obstacle is on the centerline 114 of the ultrasonic arc. Based on this, the processing unit can detect the reflection point of the corresponding ultrasonic signal as the position of the echo, which can be represented, for example, as echo coordinates.

[0035] In one embodiment, the ultrasonic data may include intersection data, which may include intersection coordinates, intersection distances, nearest neighbor distances, sensor coordinates, and nearest neighbor coordinates. For example, as the vehicle moves along direction 150, the ultrasonic sensor 110 on the right front side of the vehicle sends ultrasonic signals and receives ultrasonic echo signals under the control of its processing unit. Figure 1In the example shown, two sensors 110 emit two ultrasonic signals 112. Based on the known positional relationship between the two sensors 110, the transmission time of the two ultrasonic signals 112, and the reception time of the corresponding echoes, the processing unit can calculate the intersection point 116 of the ultrasonic arcs of the two ultrasonic signals 112. This intersection point 116 can represent a reflection point on the detected obstacle, and the data regarding this intersection point 116 is used as the aforementioned intersection point data. In one embodiment, the intersection point is not limited to the intersection point of two ultrasonic arcs of ultrasonic signals emitted by two sensors, but can also be the intersection point of two ultrasonic arcs of ultrasonic signals emitted by the same sensor at different times. Those skilled in the art will understand that the intersection point data is calculated based on echo data, and corresponding intersection point data may be obtained based on the ultrasonic arcs of any two echo data within a certain range.

[0036] In one embodiment, the ultrasound data may include the aforementioned echo data and / or crossover data, and optionally may also include other ultrasound signal-related data, such as ultrasound signal-related data obtained based on methods known in the art or possible future methods.

[0037] Based on a similar process, the processing unit of the lidar module 140 can obtain corresponding lidar data based on the signal emitted by the transmitter and the corresponding received echo or reflected signal. For example, the lidar data can be point cloud data, where each point or sample includes information such as distance, coordinates, intensity, and reflectivity. The processing unit of the millimeter-wave radar module 130 can obtain corresponding millimeter-wave radar data based on the signal emitted by the transmitter and the corresponding received echo or reflected signal. For example, the millimeter-wave radar data can be point cloud data, where each point includes information such as distance, velocity, angle, coordinates, intensity, and reflectivity. The processing unit of the camera module 120 can obtain point cloud data based on the image captured by the image sensor. Each point in the point cloud can be a pixel value of the image, including information such as coordinates, brightness, and color.

[0038] The processing units of sensor modules 110 to 140 can provide the obtained sensing data to the vehicle's control unit 160. In one embodiment, the vehicle's control unit 160 may be an electronic control unit (ECU) or a central processing unit. In one embodiment, some or all of the operations performed by the processing units of sensor modules 110 to 140 may also be performed by the vehicle's control unit 160. For example, the operation of obtaining corresponding sensing data, such as point cloud data, based on the signals captured by the respective sensor modules 110 to 140 may optionally be performed by the control unit 160 or other processing units besides the sensor modules.

[0039] Figure 2A schematic diagram of a control system in a vehicle according to one embodiment is shown.

[0040] exist Figure 2 In the control system shown, with Figure 1 The same components use the same reference numerals, which will be understood by those skilled in the art. Figure 2 Only components related to the technical solutions of this disclosure are shown.

[0041] like Figure 2 As shown, the ultrasonic sensor module 110-n, image sensor module 120-n, millimeter-wave radar module 130-n, and lidar module 140-n each include one or more corresponding sensors 110 to 140 and processing units 115 to 145. (As described above...) Figure 1 As described, the vehicle may include only one ultrasonic sensor module 110-n, one image sensor module 120-n, one millimeter-wave radar module 130-n, and one lidar module 140-n, each of which includes, for example... Figure 1 The diagram shows one or more sensors of a corresponding type and a corresponding processing unit mounted on a vehicle. Those skilled in the art will understand that in different implementations, the vehicle may include one or more sensor modules of a specific type, each of which may include one or more sensors.

[0042] The vehicle control unit 160 can control the operation of sensor modules 110-n to 140-n, receive sensing data from sensor modules 110-n to 140-n, and perform further operations based on the sensing data. As described above, some or all of the operations performed by the processing units 115 to 145 of sensor modules 110-n to 140-n can also be performed alternatively by the vehicle control unit 160 or other processing units besides sensor modules 110-n to 140-n.

[0043] The vehicle control system may also include a human-machine interface 180, through which the control unit 160 can output information understandable by a user, such as a driver, and can receive information input by the user. In one embodiment, the user can input a selection to enter an assisted parking mode through the human-machine interface 180. Upon receiving the instruction to enter the assisted parking mode, the control unit 160 can control the vehicle to operate in the assisted parking mode, for example, controlling the vehicle to automatically find a parking space and automatically park. In another embodiment, the control unit 160 can automatically control the vehicle to enter either the assisted parking mode or the automatic parking mode.

[0044] Figure 3 A schematic diagram of sensing data according to one embodiment is shown.

[0045] exist Figure 3 The illustrated embodiment uses ultrasonic data as an example to describe the sensing data. For example, in a parking lot, while a vehicle is traveling in direction 310, the ultrasonic sensor module 110 acquires ultrasonic data 320. This ultrasonic data 320 can be point cloud data, which includes the aforementioned echo data and / or intersection data. Figure 3 Each ultrasound data sample point 320 shown represents an echo data sample point or a cross data sample point. Each echo data sample point may include the echo amplitude, echo intensity, echo distance, echo coordinates, sensor coordinates, timestamp, etc., of the echo. Each cross data sample point may include the cross point coordinates, cross point distance, cross point nearest neighbor distance, sensor coordinates, cross point nearest neighbor coordinates, etc. Those skilled in the art will understand that, in addition to the echo data and cross point data characteristics exemplified above, echo data and cross point data may also include other ultrasound signal-related characteristics. Features of echo data and cross point data known in the art and those that may be adopted in the future can all be applied to the technical solutions of this disclosure. Those skilled in the art will understand that, in addition to the illustrated echo data and cross point data, ultrasound data 320 may also include other ultrasound signal-related data, such as ultrasound signal-related data obtained based on methods known in the art or possible future methods.

[0046] Similarly, for example in a parking lot, while the vehicle is moving in direction 310, other types of sensor modules can also acquire corresponding types of sensing data. For example, camera module 120 can acquire image data, millimeter-wave radar module 130 can acquire millimeter-wave radar data, and lidar module 140 can acquire lidar data. Millimeter-wave radar data and lidar data can also be point cloud data. Millimeter-wave radar data sample points can include information such as distance, speed, angle, coordinates, intensity, and reflectivity, while lidar data sample points can include information such as distance, coordinates, intensity, and reflectivity. Those skilled in the art will understand that, in addition to the characteristics of millimeter-wave radar data and lidar data exemplified above, millimeter-wave radar data and lidar data can also include other characteristics. All known and potentially future characteristics of millimeter-wave radar data and lidar data in the art can be applied to the technical solutions of this disclosure.

[0047] In one embodiment, for example Figure 3The ultrasonic data 320 and other types of sensing data shown can be cached in the buffer or memory of the vehicle's processing system. The vehicle's control unit 160 or other processing unit can further process the cached ultrasonic data 320 and other types of sensing data. In one embodiment, the vehicle's control unit 160 or other processing unit can further process the cached multiple types of sensing data to determine the location of obstacles and vacant spaces. In one embodiment, the vehicle's control unit 160 or other processing unit can determine whether there are vacant parking spaces based on the determined locations of obstacles and vacant spaces. In one embodiment, the vehicle's control unit 160 or other processing unit can plan an automatic parking route based on the detection of vacant parking spaces.

[0048] Figure 4 A schematic diagram of a method for determining parking spaces around a vehicle, according to one embodiment, is shown.

[0049] exist Figure 4 In the illustrated embodiment, a grid window 330 is provided, which includes a plurality of grids 335. The grid window 330 corresponds to a spatial area of ​​a first size, and each grid 335 in the grid window corresponds to a spatial area of ​​a second size within the spatial area of ​​the first size. For example, the grid window 330 may be configured to correspond to a spatial area of ​​5 meters by 3 meters, which can approximately cover the area of ​​a parking space, and each grid 335 may be configured to correspond to a spatial area of ​​0.1 meters by 0.1 meters. In this example, the grid window 330 may include 50 by 30 grids. The size of the grid window 330 is not limited to a specific value, nor is the size of each grid 335 limited to a specific value. In one embodiment, the size of the grid window 330 may be set based on the size of the buffer. In one embodiment, the size of the grid 335 may be set based on the requirements for position detection accuracy.

[0050] Sensing data 320 in the buffer can be extracted based on grid window 330 or based on each grid 335 in grid window 330. Figure 4 The example shown only illustrates ultrasound data as sensing data 320, but sensing data 320 also includes other types of sensing data 320, such as one or more of the aforementioned image data, millimeter-wave radar data, and lidar data. It is understood that the different types of sensing data 320 can be transformed to the same coordinate system, so that the sample points of the different types of sensing data are spatially aligned.

[0051] In one embodiment, features related to ultrasound signals of a grid can be determined based on ultrasound data within each grid 335 of the grid window 330; features related to image signals of a grid can be determined based on image data within each grid 335 of the grid window 330; features related to millimeter-wave radar signals of a grid can be determined based on millimeter-wave radar data within each grid 335 of the grid window 330; and features related to lidar signals of a grid can be determined based on lidar data within each grid 335 of the grid window 330.

[0052] For example, grid 335 includes multiple ultrasound data samples, which may include echo data sample points and / or intersection data sample points. Each echo data sample point may include ultrasound signal-related features such as echo amplitude, echo intensity, echo distance, echo coordinates, sensor coordinates, and timestamp. Each intersection data sample point may include ultrasound signal-related features such as intersection coordinates, intersection distance, intersection nearest neighbor distance, sensor coordinates, and intersection nearest neighbor coordinates. In one embodiment, the average value of the corresponding features of each ultrasound data sample point in grid 335 can be calculated as the ultrasound signal-related feature of grid 335. For example, grid 335 includes multiple echo data samples (e.g., 10 echo data samples) and multiple cross data samples (e.g., 10 cross data samples). Each echo data sample includes multiple features related to the ultrasound signal (e.g., 8 echo features 1-8), and each cross data sample includes multiple features related to the ultrasound signal (e.g., 8 cross features 9-16). Then, for each ultrasound data sample, a feature vector including multi-dimensional features can be obtained. For example, in this case, a feature vector including 16 features 1-16 can be obtained for each ultrasound data sample, and the average feature vector of the 20 ultrasound data samples in the grid is calculated as the ultrasound signal-related feature vector of the grid. For example, in the above example, the average feature vector of the echo data (e.g., containing features 1-8) can be calculated for multiple (e.g., 10) echo data samples, and the average feature vector of the intersection data (e.g., containing features 9-16) can be calculated for multiple (e.g., 10) intersection data samples. The average feature vector of the echo data and the average feature vector of the intersection data are combined to form the average feature vector of the ultrasound data samples within the grid 335 (e.g., containing features 1-16), which serves as the ultrasound signal-related feature of the grid 335.

[0053] The operation described above, which calculates the average value of the corresponding features of each ultrasound data sample point in grid 335, can be called average pooling. Other suitable synthesis operations can also be used to calculate the ultrasound signal-related feature vector of the grid. For example, max pooling and random pooling methods can be used to calculate the ultrasound signal-related feature vector of the grid. In the max pooling method, for each feature, the maximum value is selected from the corresponding feature values ​​of each ultrasound data sample point in grid 335 as the feature value and thus the ultrasound signal-related feature of grid 335. In the random pooling method, for each feature, probability values ​​are assigned to these feature values ​​based on the corresponding feature values ​​of each ultrasound data sample point in grid 335, and one feature value is randomly selected based on these probability values ​​as the ultrasound signal-related feature of grid 335. Those skilled in the art will understand that appropriate methods can be used to determine the ultrasound signal-related features of the grid based on the ultrasound data within the grid, and the technical solutions disclosed herein are not limited to the specific implementation methods described in the above examples.

[0054] The above combination Figure 4 This illustrates how features related to the ultrasound signal of a grid can be determined based on ultrasound data within each grid 335 of the grid window 330. Similarly, features related to each type of sensing data can be determined based on each type of sensing data within each grid 335 of the grid window 330.

[0055] For image data, raster 335 includes multiple image data samples, such as multiple pixels. Each image data sample may include features related to the image signal, such as brightness, chroma, and depth. Similarly, the image data-related features of the raster 335 can be determined based on the multiple image data samples in the raster 335 using methods such as average pooling, max pooling, and random pooling.

[0056] For millimeter-wave radar data, grid 335 includes multiple millimeter-wave radar data samples. Each millimeter-wave radar data sample can include features related to the millimeter-wave radar signal, such as range, velocity, angle, phase, coordinates, intensity, reflectivity, and timestamp. Similarly, the millimeter-wave radar data-related features of the grid can be determined based on the multiple millimeter-wave radar data samples in grid 335 using methods such as average pooling, max pooling, and random pooling.

[0057] For LiDAR data, grid 335 includes multiple LiDAR data samples. Each LiDAR data sample can include features related to the LiDAR signal, such as distance, velocity, angle, phase, coordinates, intensity, reflectivity, and timestamp. Similarly, the features related to the LiDAR data of the grid can be determined based on the multiple LiDAR data samples in grid 335 using the methods described above, such as average pooling, max pooling, and random pooling.

[0058] In one embodiment, features related to ultrasonic signals, image data, millimeter-wave radar data, and lidar data of the grid 335 can be integrated together as integrated features of the grid 335 related to multiple types of sensor data. For example, the aforementioned features related to ultrasonic signals, image data, millimeter-wave radar data, and lidar data of the grid 335 can be concatenated to form a feature vector of the grid 335 as its integrated feature. In this example, for a grid window 330 comprising 50 by 30 grids, a feature map IFM containing 1500 of the aforementioned integrated feature vectors is formed. Assuming the dimension of the aforementioned integrated feature vectors is N, the feature map of the grid window 330 can be represented as a tensor of shape N×50×30.

[0059] It is understood that in the above embodiments, the integrated vector of grid 335 contains features of four types of sense data, but in other embodiments, the integrated vector of grid 335 may contain more or fewer types of sense data features.

[0060] Figure 5 A schematic diagram of a device for detecting parking spaces around a vehicle, according to one embodiment, is shown.

[0061] The device 500 includes a feature integration module 510 and a detection module 520. (Continuing with the integration...) Figure 4 In the example described, the feature integration module 510 receives multiple types of sensing data, such as ultrasound data S1, image data S2, millimeter-wave radar data S3, and lidar data S4, and generates an integrated feature map (IFM) based on the received multiple types of sensing data S1 to S4. For example, in Figure 4 In the example described, the generated integrated feature map IFM is a tensor of shape N×50×30.

[0062] The detection module 520 detects parking spaces around the vehicle based on an integrated feature map (IFM). For example, the detection result of a parking space can be a bounding box indicating vacant space within a grid window; alternatively, it can be a grid map indicating obstacles and vacant space. It is understood that the detection result of a parking space is not limited to a specific form; the detection result can also represent an obstacle region, thus implicitly indicating a vacant area. The detection module 520 can be implemented using a neural network model, where the integrated feature map (IFM) of the grid window 330 is used as input to the neural network model 520, and the neural network model 520 outputs a detection result P based on the integrated feature map (IFM). The neural network model 520 can be referred to as a perceptual neural network module 520. In one embodiment, the output detection result P can be a segmentation map representing obstacles and vacant space, where the areas of obstacles and vacant space are represented as occupied and vacant states, respectively. In one embodiment, the output detection result P can be an identification map representing obstacles, where obstacles are identified by their boundary points or bounding boxes, and corresponding other areas identify vacant space. In one embodiment, the output detection result P can also be a bounding box indicating vacant space.

[0063] In one embodiment, a convolutional neural network can be used to implement the perceptual neural network module 520. In another embodiment, a transformer neural network can be used to implement the perceptual neural network module 520. It is understood that any suitable neural network structure can be used to implement the perceptual neural network module 520. Any suitable method can be used to train the perceptual neural network module 520, where the input and output training data can be the aforementioned integrated feature map (IFM) and corresponding output labels, such as the segmentation map labels representing obstacles and free spaces, the boundary points or bounding box labels representing obstacles, and the bounding box labels representing free parking spaces. Furthermore, the neural network model 520 can be trained based on the output predicted values ​​and output label values ​​of the neural network module 520 based on the integrated feature map (IFM). For example, a loss value, including, for example, l2 norm, cross-entropy, can be determined based on the output predicted values ​​and output label values, and the neural network model 520 can be updated based on the loss value using an optimizer such as Adam.

[0064] By employing the integrated feature maps described above to detect parking spaces, or more specifically, obstacles and vacant spaces, the different characteristics of various types of sensor data can be utilized, thereby improving detection accuracy and reliability under various conditions. For example, USS signals are susceptible to weather conditions and have inherently low accuracy. Furthermore, due to cost and installation limitations, the field of view (FOV) of LiDAR signals is limited in practical applications, often resulting in incomplete coverage of parking areas. Image signals may also fail to function properly in poor lighting conditions. Additionally, millimeter-wave radar cannot detect non-reflective objects (such as trees and bushes). Therefore, in many cases, some types of sensing data may be missing or unreliable, while others may be available. Thus, by utilizing the different characteristics of various types of sensor data and complementing each other under different conditions, detection accuracy and reliability under various circumstances can be improved.

[0065] Figure 6 A schematic diagram of a device for detecting parking spaces around a vehicle, according to one embodiment, is shown. Figure 6 and Figure 5 The same or corresponding modules are represented by the same or corresponding reference numerals.

[0066] like Figure 6 As shown, the feature integration module 510 includes an anomaly detection submodule 5110 and a feature integration submodule 5120. The anomaly detection submodule 5110 detects whether one or more types of sensing data from multiple types of sensing data S1 to S4 are abnormal. In one embodiment, the anomaly detection submodule 5110 determines whether one or more types of sensing data are missing. For example, the anomaly detection submodule 5110 determines that a certain type of sensing data is missing based on the absence of such data. For example, when the anomaly detection module determines that no valid LiDAR data has been obtained, it can determine that LiDAR data is missing. In one embodiment, predefined feature values ​​can be used to replace LiDAR data; for example, zero values ​​can be used to replace missing LiDAR data. Accordingly, in the integrated feature map (IFM), the value corresponding to the feature portion of the LiDAR data is zero.

[0067] In one embodiment, the anomaly detection submodule 5110 can determine whether one or more types of sensing data are abnormal based on time. For example, it can determine whether one or more types of sensing data are abnormal based on whether the time interval between the timestamps of multiple types of sensing data S1 to S4 is greater than a threshold. For example, a base time can be determined based on the timestamps of multiple types of sensing data S1 to S4, and it can be determined whether the time difference between the timestamps of one or more types of sensing data and the base time is greater than a threshold. If it is greater than the threshold, it is determined that the type of sensing data is abnormal. Accordingly, a predefined feature value can be used to replace the type of sensing data, such as using zero value to replace the type of sensing data. The above-mentioned base time can be determined in any suitable way. For example, the earliest time among the timestamps of multiple types of sensing data S1 to S4 can be determined as the base time, or the average time of the timestamps of multiple types of sensing data S1 to S4 can be determined as the base time.

[0068] Figure 7 A schematic diagram of a device for detecting parking spaces around a vehicle, according to one embodiment, is shown. Figure 7 and Figures 5 to 6 The same or corresponding modules are represented by the same or corresponding reference numerals.

[0069] like Figure 7 As shown, the anomaly detection submodule 5110 includes outlier detection submodules 51110 to 51140. Outlier detection submodules 51110 to 51140 can be implemented using a neural network model, for example, an out-of-distribution (OOD) detection model. The OOD model is a commonly used neural network model in machine learning; it learns the data distribution of normal data samples to detect outliers outside the normal data distribution. For example, outlier detection submodules 51110 to 51140 can be trained using normal ultrasound data, image data, millimeter-wave radar data, and lidar data, respectively. Accordingly, outlier detection submodules 51110 to 51140 are used to detect whether various types of sensing data S1 to S4 are abnormal. For example, when outlier detection submodule 51140 detects an anomaly in lidar data, it can replace the lidar data with a predefined feature value, for example, using zero to replace the abnormal lidar data.

[0070] like Figure 7 As shown, the anomaly detection submodule 5110 includes a rule-based detection submodule 51150, which can detect according to... Figure 6The illustrated embodiments describe determining whether a certain type of sensing data is abnormal based on its availability and / or time. It is understood that the anomaly detection submodule 5110 may also include only anomaly detection submodules 51110 to 51140.

[0071] Figure 8 A schematic diagram of a device for detecting parking spaces around a vehicle, according to one embodiment, is shown. Figure 8 and Figures 5 to 7 The same or corresponding modules are represented by the same or corresponding reference numerals.

[0072] like Figure 8 As shown, the anomaly detection submodule 5110 includes parking space detection submodules 51110D to 51140D and a voting submodule 51160. The parking space detection submodules 51110D to 51140D can be implemented using neural network models, such as convolutional neural networks or transformer neural networks. The parking space detection submodules 51110D to 51140D detect parking spaces in the vehicle's surrounding environment based on a single type of sensing data S1, S2, S3, or S4, respectively. It is understood that the detected parking space can be a vacant space between obstacles; this vacant space is not necessarily a complete vacant parking space, but rather its availability can be determined based on further evaluation (e.g., based on the size of the vacant space). In one embodiment, the inputs to the parking space detection submodules 51110D to 51140D can be ultrasonic data, image data, millimeter-wave radar data, and lidar data, respectively, and the outputs can be the detected parking spaces. In one embodiment, the inputs to the parking space detection submodules 51110D to 51140D can be respectively the combination described above. Figure 4 The description of the grid window 330 includes features related to ultrasonic signals, features related to image data, features related to millimeter-wave radar data, and features related to lidar data for each grid cell. The input can be the detected parking space.

[0073] The voting submodule 51160 determines whether one or more types of sensing data are abnormal based on the detection results from the parking space detection submodules 51110D to 51140D. For example, it performs statistical analysis on multiple types of detection results. When the difference between the parking space areas indicated by at least a predetermined number of types of detection results is less than a threshold, the sensing data of that at least predetermined number of types is determined to be normal. Conversely, when the difference between a type of parking space detection result and the parking space area indicated by the aforementioned at least predetermined number of types of parking space detection results is greater than a threshold, the sensing data of that type is determined to be abnormal. For example, when the outlier detection submodule 51140 detects abnormal LiDAR data, it can use predefined feature values ​​to replace the LiDAR data, for example, using zero values ​​to replace abnormal LiDAR data.

[0074] On the other hand, if the voting submodule 51160 cannot determine that the difference between parking space areas indicated by at least a predetermined number of types of detection results is less than a threshold, it outputs an indication that it cannot determine. Accordingly, the feature integration module 5120 can generate an integrated feature map IFM based on all types of sensing data, or the feature integration module 5120 can assume that all other types of data besides ultrasound data S1 are abnormal data to generate an integrated feature map IFM.

[0075] Understandable. Figure 8 The anomaly detection submodule 5110 shown may also include Figure 7 The rule-based detection submodule 51150 shown is illustrated.

[0076] Figure 9 A schematic diagram of a device for detecting parking spaces around a vehicle, according to one embodiment, is shown. Figure 9 and Figures 5 to 8 The same or corresponding modules are represented by the same or corresponding reference numerals.

[0077] like Figure 9 As shown, the feature integration module 510 also includes a compensation submodule 5130. When the anomaly detection submodule 5110 determines that a certain type of sensing data is abnormal, the compensation submodule 5130 can generate sensing data of that type that is detected as abnormal based on other types of sensing data. This generated sensing data can be called simulated sensing data. Accordingly, the feature integration submodule 5120 generates an integrated feature map (IFM) based on this type of simulated sensing data and other types of normal sensing data. Continuing with... Figure 4In the aforementioned example, for instance, the anomaly detection submodule 5110 determines that the lidar data S4 is abnormal, and the compensation submodule 5130 can generate simulated lidar data S4 based on the image data S2. Accordingly, the feature integration submodule 5120 generates an integrated feature map (IFM) based on the normal ultrasound data S1, image data S2, millimeter-wave radar data S3, and simulated lidar data S4.

[0078] In one embodiment, the compensation submodule 5130 can be implemented using a neural network model, for example, a diffusion model to generate another type of simulated sensing data based on one type of sensing data. The structure of diffusion models is known in the art, and it is understood that any suitable neural network structure can be used to implement the compensation submodule 5130. The training of the compensation submodule 5130 can be implemented using any suitable method, where the input and output training data can be one or more types of collected sensing data and another type of sensing data, such as image data and simultaneously collected LiDAR data in the example above. In one embodiment, since the probability of missing LiDAR data is highest in assisted parking scenarios, the compensation submodule 5130 includes a diffusion model for generating simulated LiDAR data based on one or more types of sensing data, for example, a diffusion model that generates simulated LiDAR data based on ultrasonic data and / or image data. In another embodiment, the compensation submodule 5130 may also include a diffusion model for generating other types of simulated sensing data, such as a diffusion model for generating image data based on ultrasonic data, a diffusion model for generating millimeter-wave radar data based on ultrasonic data, etc. The compensation submodule 5130 can determine the activation of the corresponding diffusion model to generate simulated sensing data of the type of anomaly based on the sensing data type of the anomaly determined by the anomaly detection submodule 5110.

[0079] When a certain type of sensing data is abnormal, the characteristics of the input data of the perceptual neural network model 520 can be effectively enhanced by generating simulated sensing data of that type using the compensation submodule 5130, thereby helping to improve the detection accuracy and reliability of the perceptual neural network model 520.

[0080] Figure 10 A method for detecting parking spaces in the environment surrounding a vehicle, according to one embodiment, is shown.

[0081] In step 1010, multiple types of sensing data are obtained. In one embodiment, the multiple types of sensing data include at least two of the following: ultrasonic sensing data, image sensing data, millimeter-wave radar sensing data, and lidar sensing data. In another embodiment, the multiple types of sensing data include ultrasonic sensing data and at least one of the following: image sensing data, millimeter-wave radar sensing data, and lidar sensing data. Although in Figures 4 to 9 In the exemplified embodiments, ultrasonic sensing data, image sensing data, millimeter-wave radar sensing data, and lidar sensing data are used as examples of various types of sensing data to describe the technical solutions of the embodiments of this disclosure. However, it is understood that in specific implementations, more or fewer types of sensors and correspondingly more or fewer types of sensing data may be used.

[0082] In step 1020, for each grid cell in the grid window, based on the multiple types of sensing data, the multiple types of features corresponding to the multiple types of sensing data are determined for the grid cell, and the multiple types of features corresponding to the multiple types of sensing data are integrated into the integrated features of the grid cell, wherein the grid window corresponds to a spatial region of a first size, and each grid cell in the grid window corresponds to a spatial region of a second size in the spatial region of the first size.

[0083] In step 1030, based on the integrated features of each grid cell in the grid window, the detection result of the parking space in the grid window is output. For example, the detection result of the parking space can be a bounding box indicating the free space in the grid window, or a grid map indicating obstacles and free space. It is understood that the detection result of the parking space is not limited to a specific form.

[0084] According to one embodiment, the grid window slides in steps. Steps 1010 to 1030 are repeated for each slide of the grid window. The step size is smaller than the length of the grid window along the sliding direction. For example, the step size can be 0.5 meters, 1 meter, 1.5 meters, etc.

[0085] According to one embodiment, step 1020 further includes: for each type of sensing data among the multiple types of sensing data within the grid, determining the features of the grid corresponding to that type of sensing data by synthesizing the corresponding features of each sensing data point of that type within the grid; and concatenating the features of the grid corresponding to the multiple types of sensing data to obtain the integrated features of the grid. In one embodiment, the synthesis includes one of average pooling, max pooling, and random pooling.

[0086] According to one embodiment, step 1020 further includes: when some types of sensing data are missing from the multiple types of sensing data, determining the features of the raster corresponding to the missing partial types of sensing data as predefined features. For example, the predefined features may be predefined values, such as zero values.

[0087] According to one embodiment, the method further includes: determining a first type of sensing data anomaly among the plurality of types of sensing data. For example, the anomaly includes missing values ​​or other abnormal conditions. The method further includes: generating simulated sensing data of the first type based on at least a portion of the other types of sensing data among the plurality of types of sensing data. The generated simulated sensing data of the first type is used as or replaces the first type of sensing data. According to one embodiment, the first type of sensing data anomaly is determined by determining that the first type of sensing data cannot be obtained. According to one embodiment, the first type of sensing data anomaly is determined by determining that the time difference between the timestamp of the first type of sensing data and a reference time is greater than a threshold. According to one embodiment, the first type of sensing data anomaly is determined by performing outlier detection on each of the plurality of types of sensing data. According to one embodiment, the first type of sensing data anomaly is determined by performing a voting detection on the plurality of types of sensing data.

[0088] According to one embodiment, step 1020 further includes: determining a first type of feature of the grid corresponding to the first type of sensing data based on the generated first type of simulated sensing data.

[0089] Figure 11 A method for assisted parking according to one embodiment is shown.

[0090] In step 1110, parking spaces around the vehicle are detected. This can be achieved using a combination of methods described in this paper. Figure 1-10 Various embodiments are described to determine parking spaces around a vehicle based on ultrasonic signals.

[0091] In step 1120, a route for driving the vehicle into the parking space is planned based on the detected parking space. Various suitable methods in the art can be used to implement the parking route planning in step 1120.

[0092] Figure 12 An apparatus for detecting parking spaces in the environment surrounding a vehicle is shown according to one embodiment.

[0093] The device 1200 includes a feature integration module 1210, which, for each grid in a grid window, determines multiple types of features corresponding to the multiple types of sensing data based on multiple types of sensing data, and integrates the multiple types of features corresponding to the multiple types of sensing data into an integrated feature of the grid, wherein the grid window corresponds to a spatial region of a first size, and each grid in the grid window corresponds to a spatial region of a second size in the spatial region of the first size.

[0094] The device 1200 includes a detection module 1220, which outputs the detection results of parking spaces in the grid window based on the integrated features of each grid in the grid window.

[0095] According to one embodiment, the feature integration module 1210 determines the feature of the grid corresponding to each type of sensing data in the grid by synthesizing the corresponding features of each sensing data point of that type in the grid. The synthesis includes one of average pooling, max pooling, and random pooling. According to another embodiment, the feature integration module 1210 concatenates the features of the grid corresponding to the various types of sensing data to obtain the integrated feature of the grid.

[0096] According to one embodiment, when some types of sensing data in the multiple types of sensing data are missing, the feature integration module 1210 determines the features of the grid corresponding to the missing types of sensing data as predefined features.

[0097] According to one embodiment, the apparatus 1200 further includes an anomaly detection module and a compensation module. The anomaly detection module determines that a first type of sensing data is anomaly among the multiple types of sensing data. The compensation module generates simulated sensing data of the first type based on at least a portion of the other types of sensing data among the multiple types of sensing data. The generated simulated sensing data of the first type is used as the first type of sensing data.

[0098] According to one embodiment, the feature integration module 1210 further determines the first type of features of the grid corresponding to the first type of sensing data based on the generated first type of analog sensing data.

[0099] According to one embodiment, the anomaly detection module determines that the first type of sensing data is abnormal by determining that the first type of sensing data cannot be obtained. According to one embodiment, the anomaly detection module determines that the first type of sensing data is abnormal by determining that the time difference between the timestamp of the first type of sensing data and a reference time is greater than a threshold. According to one embodiment, the anomaly detection module determines the first type of sensing data is abnormal by performing outlier detection on each of the multiple types of sensing data. According to one embodiment, the anomaly detection module determines the first type of sensing data is abnormal by performing a voting detection on the multiple types of sensing data.

[0100] Figure 13 A block diagram of a control system for a vehicle according to one embodiment is shown.

[0101] According to one embodiment, the control system or processing system 1300 may include one or more control units or processing units 1310, which execute one or more machine-readable instructions stored or encoded in a machine-readable storage medium (i.e., memory 1320). Although not in Figure 13 As shown in the figure, but those skilled in the art will understand that the control system 1300 may include various other components, such as various communication modules, bus modules, and possibly user interface modules.

[0102] In one embodiment, the control system 1300 may include Figure 1 and 2 The control unit 160 and / or processing units 115, 125, 135, 145 shown are configured to execute the above-described combination of program instructions when executing program instructions. Figure 1-12 The description includes various operations and functions.

[0103] According to one embodiment, a machine-readable medium is provided. This machine-readable medium may have instructions that, when executed by a machine such as control unit 1310, cause a device such as control unit 1310, a vehicle, etc., to perform the above-described embodiments of this application. Figure 1-12 The description includes various operations and functions.

[0104] According to one embodiment, a computer program product is provided. This computer program product may include instructions that, when executed by a machine such as control unit 1310, cause a device such as control unit 1310, a vehicle, etc., to perform the above-described embodiments of this application. Figure 1-12 The description includes various operations and functions.

[0105] According to one embodiment, a vehicle is provided. The vehicle has, as in... Figure 1The various types of sensors shown are used to sense the surrounding environment to help obtain various types of sensing data, and have Figure 13 The control system is 1300.

[0106] The specific embodiments described above with reference to the accompanying drawings are exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of the claims. The term "example" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" compared to other embodiments. Specific details are included to provide an understanding of the described technology. However, these technologies can be practiced without these specific details. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0107] The foregoing description of this disclosure is provided to enable any person skilled in the art to implement or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for detecting parking spaces in the environment surrounding a vehicle, comprising: Obtain multiple types of sensing data; For each grid cell in the grid window, based on the multiple types of sensing data, determine the multiple types of features of the grid cell that correspond to the multiple types of sensing data respectively, and integrate the multiple types of features of the grid cell that correspond to the multiple types of sensing data respectively into the integrated features of the grid cell, wherein the grid window corresponds to a spatial region of a first size, and each grid cell in the grid window corresponds to a spatial region of a second size in the spatial region of the first size; Based on the integrated features of each grid cell in the grid window, the detection results of parking spaces in the grid window are output.

2. The method as described in claim 1, wherein, The grid window slides in steps, and for each sliding grid window, the various types of features of the determined grid, the integrated features of the integrated grid, and the detection result of the output parking space are executed.

3. The method as described in claim 1, wherein, The step of determining the multiple types of features corresponding to each grid cell in the grid window, respectively, for each type of sensing data within the grid cell, includes: for each type of sensing data within the grid cell, determining the feature corresponding to that type of sensing data by synthesizing the corresponding features of each sensing data point of that type within the grid cell. Specifically, for each grid cell in the grid window, integrating the various types of features corresponding to the various types of sensing data into an integrated feature of the grid cell includes: concatenating the various types of features corresponding to the various types of sensing data to obtain the integrated feature of the grid cell.

4. The method of claim 3, wherein the synthesis comprises one of average pooling, max pooling, and random pooling.

5. The method of claim 3, wherein, The step of determining the features of each grid cell in the grid window that correspond to the various types of sensing data includes: when some types of sensing data are missing, determining the features of the grid cell that correspond to the missing types of sensing data as predefined features.

6. The method of claim 1 or 3, further comprising: Identify the first type of sensing data anomaly among the multiple types of sensing data; The first type of simulated sensing data is generated based on at least a portion of the other types of sensing data among the multiple types of sensing data, wherein the generated first type of simulated sensing data is used as the first type of sensing data.

7. The method of claim 6, wherein, The step of determining the multiple types of features corresponding to the multiple types of sensing data for each grid cell in the grid window includes: determining the first type of features corresponding to the first type of sensing data for the grid cell based on the generated first type of simulated sensing data.

8. The method of claim 6, wherein, The determination of a first type of sensing data anomaly among the multiple types of sensing data includes: The first type of sensing data anomaly is determined by identifying the inability to obtain sensing data of the first type; or The first type of sensing data is determined to be abnormal by determining that the time difference between the timestamp of the first type of sensing data and the reference time is greater than a threshold; or Anomalies in the first type of sensing data are determined by performing outlier detection on each of the various types of sensing data; or The first type of sensing data anomaly is determined by voting detection on the various types of sensing data.

9. The method of claim 1, wherein, The various types of sensing data include at least two of the following: ultrasonic sensing data, image sensing data, millimeter-wave radar sensing data, and lidar sensing data; or The various types of sensing data include ultrasonic sensing data and at least one of the following: image sensing data, millimeter-wave radar sensing data, and lidar sensing data.

10. A method for assisting parking, comprising: Detecting parking spaces using the method described in any one of claims 1 to 9; Based on the detected parking spaces, a route is planned to drive the vehicle into the parking space.

11. An apparatus for detecting parking spaces in the environment surrounding a vehicle, comprising: The feature integration module, for each grid cell in the grid window, determines multiple types of features corresponding to the multiple types of sensing data for each grid cell based on multiple types of sensing data, and integrates the multiple types of features corresponding to the multiple types of sensing data for each grid cell into an integrated feature of the grid cell, wherein the grid window corresponds to a spatial region of a first size, and each grid cell in the grid window corresponds to a spatial region of a second size within the spatial region of the first size; The detection module outputs the detection results of parking spaces in the grid window based on the integrated features of each grid cell in the grid window.

12. The apparatus of claim 11, wherein, The feature integration module determines the feature corresponding to that type of sensing data in the grid by synthesizing the corresponding features of each sensing data point of that type within the grid for each type of sensing data among the various types of sensing data. The feature integration module concatenates the features of the grid with the features corresponding to the various types of sensing data to obtain the integrated features of the grid.

13. The apparatus of claim 11 or 12 further includes an anomaly detection module and a compensation module. The anomaly detection module determines that the first type of sensing data is abnormal among the multiple types of sensing data; The compensation module generates simulated sensing data of the first type based on at least a subset of other types of sensing data from the plurality of sensing data types, wherein, The generated first type of simulated sensing data is used as the first type of sensing data.

14. A control system for a vehicle, comprising: One or more processing units, which are configured to perform the method as described in any one of claims 1 to 10 when executing program instructions.

15. A vehicle comprising: Multiple types of sensors are used to sense the environment around the vehicle to help obtain various types of sensing data; The control system as described in claim 14.

16. A machine-readable storage medium storing executable instructions that, when executed, cause one or more processors to perform the method as described in any one of claims 1 to 10.

17. A computer program product comprising executable instructions that, when executed, cause one or more processors to perform the method as described in any one of claims 1 to 10.

Images