Obstacle detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose an obstacle detection method and device, electronic equipment and storage medium, the method comprising: determining at least one target pixel point in a target image to be processed, and target position information of the at least one target pixel point; determining at least one historical image to be processed associated with the target image to be processed, and calling historical position information of each historical image to be processed; determining a target grid corresponding to a target vehicle and target grid attribute information of each sub-grid region in the target grid according to vehicle position information of the target vehicle, the target position information and each historical position information; and determining a target obstacle according to each target grid attribute information. The technical scheme of the embodiments of the present application realizes the technical effect of accurately and comprehensively detecting various types of obstacles on the road through a pixel-level processing process.

Description

Technical Field

[0001] The present invention relates to the field of unmanned vehicle technology, and in particular to an obstacle detection method, device, electronic device and storage medium. Background Technology

[0002] Currently, the perception algorithms for unmanned delivery vehicles mostly use deep neural networks to detect common obstacles. These common obstacles can be vehicles, pedestrians, non-motorized vehicles, etc. Alternatively, lidar can be installed on the unmanned delivery vehicle to identify the corresponding obstacles by collecting point cloud data in the space.

[0003] When implementing this technical solution based on the above method, the inventors discovered the following problems:

[0004] Deep neural networks can only detect common obstacles. For uncommon obstacles, such as bricks, steel pipes, and plastic boxes on the ground, deep neural networks cannot determine the object type and therefore cannot output corresponding detection results. When using LiDAR to detect obstacles, the collected point cloud data is relatively sparse, which means that autonomous vehicles cannot detect obstacles that are close to the road surface, low in height, and small in size. Therefore, both of these detection methods are prone to false detections or missed detections, preventing autonomous vehicles from accurately and comprehensively detecting obstacles on the road and resulting in poor driving performance. Summary of the Invention

[0005] This invention provides an obstacle detection method, device, electronic device, and storage medium to achieve the technical effect of accurately and comprehensively detecting various types of obstacles on roads.

[0006] In a first aspect, embodiments of the present invention provide an obstacle detection method applied to unmanned vehicles, the method comprising:

[0007] Identify at least one target pixel in the target image to be processed, and the target location information of the at least one target pixel;

[0008] Identify at least one historical image to be processed associated with the target image to be processed, and retrieve the historical location information of each historical image to be processed;

[0009] Based on the vehicle location information of the target vehicle, the target location information, and each of the historical location information, determine the target grid corresponding to the target vehicle, and the target grid attribute information of each sub-grid area in the target grid;

[0010] Based on the attribute information of each target grid, the target obstacles are determined.

[0011] Secondly, embodiments of the present invention also provide an obstacle detection device, the device comprising:

[0012] The target location information determination module is used to determine at least one target pixel in the target image to be processed, and the target location information of the at least one target pixel;

[0013] The historical location information determination module is used to determine at least one historical image to be processed associated with the target image to be processed, and to retrieve the historical location information of each historical image to be processed;

[0014] The target grid attribute information determination module is used to determine the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid area in the target grid based on the vehicle location information of the target vehicle, the target location information and each of the historical location information.

[0015] The target obstacle determination module is used to determine target obstacles based on the attribute information of each target grid.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0017] One or more processors;

[0018] Storage device for storing one or more programs.

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the obstacle detection method as described in any embodiment of the present invention.

[0020] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the obstacle detection method as described in any of the embodiments of the present invention.

[0021] The technical solution of this invention first determines at least one target pixel in the target image to be processed, and the target position information of the at least one target pixel. Then, it determines at least one historical image to be processed associated with the target image to be processed, and retrieves the historical position information of each historical image to be processed, thus determining the position information of the pixel at multiple times. Furthermore, based on the vehicle position information, target position information, and historical position information of the target vehicle, it determines the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid region in the target grid. Based on the target grid attribute information, it determines the target obstacle. This provides a convenient obstacle detection method for unmanned vehicles, and achieves non-discriminatory detection of various types of obstacles through pixel-level processing, thereby making the obstacle detection results more accurate and comprehensive, and avoiding the problem of false detection or missed detection of obstacles in existing detection methods. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0023] Figure 1 This is a flowchart (I) illustrating an obstacle detection method provided in an embodiment of the present invention;

[0024] Figure 2 This is a schematic flowchart (II) of an obstacle detection method provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram (III) of an obstacle detection method provided in an embodiment of the present invention;

[0026] Figure 4 A structural block diagram (IV) of an obstacle detection device provided in an embodiment of the present invention;

[0027] Figure 5 A schematic diagram (V) of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0029] Figure 1This is a flowchart (I) of an obstacle detection method provided by an embodiment of the present invention. This embodiment is applicable to situations where unmanned vehicles detect various types of obstacles on the road. The method can be executed by an obstacle detection device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server.

[0030] like Figure 1 As shown, the method specifically includes the following steps:

[0031] S110. Determine at least one target pixel in the target image to be processed, and the target position information of the at least one target pixel.

[0032] This involves installing cameras on the autonomous vehicle, and using images captured directly by these cameras as images to be processed. For example, during the autonomous vehicle's operation, the cameras can record information about the driving environment, and each recorded video frame can be used as an image to be processed. Specifically, the images to be processed contain information about various elements, primarily including road information such as the road surface, road markings, and obstacles. They can also include pedestrians, vehicles, the sky, and buildings. It can be understood that the images to be processed serve as the data basis for the autonomous vehicle to determine obstacles on the road. Although they include various types of elements, only those elements that are present or about to be present on the autonomous vehicle's road and may collide with it are considered obstacles to be detected.

[0033] In this embodiment, when the autonomous vehicle needs to identify obstacles on the current road, it can use the image to be processed captured by the camera at the current moment as the target image to be processed, or it can use the image to be processed captured by the camera when the autonomous vehicle is at its current position as the target image to be processed. For example, when the autonomous vehicle needs to identify obstacles on the road at a certain moment, it can use the image of the road ahead captured by the camera at that moment as the target image to be processed; when the autonomous vehicle is at a certain position and needs to identify obstacles on the road, it can use the image captured by the camera at that position as the target image to be processed.

[0034] Since there may be various obstacles on the road with different shapes, colors, sizes and other characteristics, traditional deep learning models or LiDAR cannot accurately and comprehensively detect these obstacles. Therefore, in this embodiment, it is necessary to process the target image at the pixel level first.

[0035] Specifically, it is necessary to identify at least one target pixel in the target image to be processed, and to determine the target location information of at least one target pixel. The target pixel can be one or more pixels in the target image to be processed that correspond to an obstacle. For example, if the captured target image shows a cardboard box placed in the middle of the road ahead, and the autonomous vehicle continues to move forward, it may collide with the cardboard box. In this case, the multiple pixels in the target image to be processed that correspond to the cardboard box as an obstacle are the target pixels.

[0036] Furthermore, after identifying the target pixels, their coordinates in a two-dimensional coordinate system can be used as the target location information. Continuing with the example above, once multiple target pixels corresponding to the cardboard box are identified, a two-dimensional coordinate system associated with the target image to be processed can be determined, and the coordinates of these pixels in the two-dimensional coordinate system can be used as the target location information.

[0037] S120. Determine at least one historical image to be processed associated with the target image to be processed, and retrieve the historical location information of each historical image to be processed.

[0038] In this embodiment, due to the limitations of the shooting range of the imaging device and the two-dimensional image, a single target image cannot be used to determine three-dimensional obstacles. Furthermore, a single target image cannot accurately and comprehensively describe the various types of obstacles present on the current road. Therefore, after determining the target image, it is necessary to identify at least one historical image from the multiple images captured by the imaging device.

[0039] There are several ways to determine the historical images to be processed. One way is to select multiple images within a preset time period before the target time corresponding to the target image as historical images to be processed after determining the target image. Another way is to further determine the video segment to which the image belongs after determining the target image, and then select multiple video frames in the video segment that are before the target image as historical images to be processed.

[0040] Furthermore, in order to achieve comprehensive detection of obstacles on the road, it is also necessary to determine the corresponding pixels of each obstacle in each historical image to be processed in the same way as determining the target pixels. Further, the coordinate values ​​of these pixels in the corresponding two-dimensional coordinate system associated with the image are determined, and these coordinate values ​​are used as historical position information. This disclosure will not be elaborated further here.

[0041] S130. Based on the vehicle location information, target location information, and historical location information of the target vehicle, determine the target grid corresponding to the target vehicle, as well as the target grid attribute information of each sub-grid area in the target grid.

[0042] The target vehicle refers to the vehicle that is performing obstacle detection. Correspondingly, the vehicle position information is the information associated with the current position of the target vehicle. For example, it can be the coordinate value in the three-dimensional spatial coordinate system where the target vehicle is currently located.

[0043] Those skilled in the art should understand that the process of an autonomous vehicle detecting obstacles on the road is essentially a process of determining the position of these obstacles relative to the vehicle itself. Therefore, in this embodiment, after determining the vehicle's position information, the target position information, and various historical position information, it is necessary to first convert this information (such as specific coordinate values) into the same image.

[0044] Specifically, the above information can be converted into the same raster image. First, the target raster corresponding to the target vehicle is determined. The target raster can be a raster image, that is, an image that has been discretized in both space and brightness, consisting of at least multiple sub-rasteres. When the target raster is a matrix, any element can correspond to a point in the image, and the value of that element corresponds to the gray level of that point in the image.

[0045] Simultaneously, each sub-grid is also associated with corresponding target grid attribute information. This can be understood as follows: when converting vehicle location information, target location information, and historical location information into the same target grid, the sub-grid can be assigned corresponding attribute values ​​as target grid attribute information, thereby reflecting the corresponding entities in the image to be processed. For example, when using the values ​​reflecting the grayscale levels of pixels in the image as the target grid attribute information of the sub-grid, the shape of obstacles on the road can be constructed using the grayscale values ​​of multiple images in the target grid.

[0046] S140. Determine the target obstacles based on the attribute information of each target grid.

[0047] In this embodiment, after determining the attribute information of each target grid in the target grid, the target obstacles on the current road can be determined based on this information. For example, when the value reflecting the gray level of a pixel in the image is used as the attribute information of the target grid, multiple gray values ​​of the same sub-grid can be statistically analyzed. Based on the statistical results, it can be determined whether there is an obstacle at the location corresponding to the sub-grid. For example, based on one target image to be processed and nine historical images to be processed, if ten gray values ​​are determined for the same sub-grid, all ten gray values ​​are 255. Based on this, it can be determined that there is an obstacle at the location corresponding to the sub-grid on the road.

[0048] Furthermore, to determine the overall shape of an obstacle within the target grid, it is necessary to construct a model based on the detection results of each sub-grid. For example, multiple adjacent sub-grids representing the presence of obstacles are integrated, and the resulting two-dimensional integrated result represents the shape of the obstacle, thus achieving overall obstacle detection on the road. It can be understood that the accuracy of the constructed obstacle shape is related to the size of the sub-grids in the target grid; the smaller the sub-grids and the higher the density of the target grid, the higher the accuracy of the constructed obstacle shape.

[0049] The technical solution of this embodiment first determines at least one target pixel in the target image to be processed, and the target position information of at least one target pixel. Then, it determines at least one historical image to be processed associated with the target image to be processed, and retrieves the historical position information of each historical image to be processed, that is, it determines the position information of the pixel at multiple times. Furthermore, based on the vehicle position information, target position information, and historical position information of the target vehicle, it determines the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid area in the target grid. Based on the target grid attribute information, it determines the target obstacle. This provides a convenient obstacle detection method for unmanned vehicles, and achieves non-discriminatory detection of various types of obstacles through pixel-level processing, thereby making the obstacle detection results more accurate and comprehensive, and avoiding the obstacle false detection or missed detection problems that occur in existing detection methods.

[0050] Figure 2 This is a flowchart (II) illustrating an obstacle detection method provided in an embodiment of the present invention. Based on the aforementioned embodiments, the position information of each target pixel is converted to the world coordinate system. Simultaneously, a mapping table representing the relationship between the two-dimensional and three-dimensional coordinate values ​​of each target pixel is constructed based on the conversion result. This allows for real-time retrieval of the three-dimensional coordinate values ​​of the target pixels when needed, via a lookup table. After determining the target grid and each sub-grid region, the target grid attribute information of each sub-grid region is determined. Meanwhile, the attribute information of the sub-grid regions within the blind zone remains unchanged. By continuously outputting obstacle attributes within these sub-grids, the situation where the autonomous vehicle mistakenly identifies undetected blind zones as drivable roads is avoided. Finally, based on the target grid attribute information, clustering processing is performed on the sub-grid regions to be aggregated, thereby determining the area occupied by the target obstacle within the target grid. Specific implementation details can be found in the technical solution of this embodiment. Technical terms identical or corresponding to those in the above embodiments will not be repeated here.

[0051] like Figure 2 As shown, the method specifically includes the following steps:

[0052] S210. Determine at least one target pixel in the target image to be processed, and the target position information of the at least one target pixel.

[0053] In this embodiment, in the process of determining at least one target pixel in the target image to be processed, optionally, each pixel in the target image to be processed is classified and labeled based on an image semantic segmentation algorithm to obtain a category label for each pixel; based on each category label, at least one target pixel in the target image to be processed is determined.

[0054] In this process, the various elements of the road or environment in which the autonomous vehicle travels can be pre-classified. Optional elements include the sky, roads, buildings, pedestrians, etc. In this embodiment, the category of each element can also be pre-set. Based on this, the image semantic segmentation algorithm can determine the category label corresponding to each pixel in the target image to be processed according to the pre-labeled element categories.

[0055] In this embodiment, to detect obstacles that may collide with the autonomous vehicle, among the determined multiple categories of markers, other categories besides road category markers can be identified first. Furthermore, the pixels corresponding to these other category markers are used as target pixels in the target image to be processed. In practical applications, to reflect the pixels corresponding to non-road category markers in a more concise form, the attributes of each pixel can be binarized after semantic segmentation of the image. That is, the pixel values ​​of pixels carrying road category markers that the autonomous vehicle can drive on are set to 0, and the pixel values ​​of pixels without road category markers are set to 1.

[0056] It is understandable that this method does not directly identify and detect obstacle images in the traditional way, but directly determines the pixels that do not belong to the roads that autonomous vehicles can drive on. In other words, the area reflected by the determined pixels is the area where obstacles exist.

[0057] In the process of determining the target position information of at least one target pixel, optionally, the target position information of each target pixel can be determined based on the pre-established mapping relationship between pixel coordinates and world coordinates.

[0058] The target location information refers to the three-dimensional coordinates of the target pixel in the world coordinate system. The mapping relationship is determined based on a pre-defined transformation matrix. This transformation matrix is ​​based on the intrinsic and extrinsic calibration matrices of the camera device mounted on the target vehicle, the vertical distance between the camera device and the horizontal plane, and the coordinate system distance. The coordinate system distance can be the distance between the camera device's coordinate system and the world coordinate system. This distance can be a constant or a variable value; in practical applications, this value can be determined in real time.

[0059] It should be noted that the installation position of the camera device on the unmanned vehicle is fixed, and correspondingly, the vertical distance between the camera device and the horizontal plane is also fixed. Furthermore, the intrinsic and extrinsic parameter calibration matrices of the camera device can also be predetermined. Optionally, these matrices can be the factory parameters of the camera device, or they can be parameters determined after calibration by the personnel.

[0060] For example, the coordinates of a target pixel in a two-dimensional coordinate system are known to be P. i = (u, v), the process of determining the target position information of a point based on the pre-established mapping relationship between pixel coordinates and world coordinates, that is, determining the three-dimensional coordinates P of the point in the world coordinate system. w =(x w y w , z w The process of z. w Given the height coordinates of the camera device in a three-dimensional coordinate system, i.e., the vertical distance of the camera device from the horizontal plane. P i and P w The following equations must be satisfied.

[0061]

[0062] Among them, M I M is the intrinsic parameter calibration matrix of the camera device (known). E Z is the extrinsic calibration matrix (known) between the camera device and the three-dimensional coordinate system (world coordinate system). c Let be the distance between the camera device's coordinate system and the three-dimensional coordinate system. Substituting the above parameters, we can obtain the following formula:

[0063]

[0064] Through the above calculations, the three-dimensional coordinates of each target pixel in the world coordinate system can be obtained, i.e., the target position information.

[0065] It should also be noted that since the coordinate transformation process reflected by the above formula is mainly related to the intrinsic and extrinsic calibration matrices of the camera device, in practical applications, in order to reduce the time consumed by running the above algorithm online, when the coordinates of each target pixel in the target image to be processed are obtained, the corresponding three-dimensional coordinate values ​​can be directly calculated. At the same time, based on the transformation results, a mapping table representing the relationship between the two-dimensional and three-dimensional coordinate values ​​of each target pixel is constructed. Based on this, when the three-dimensional coordinate values ​​of the target pixel are needed in the future, these three-dimensional coordinate values ​​can be called in real time by looking up the table.

[0066] S220. Based on the generation time of the target image to be processed and the preset time interval, determine at least one historical image to be processed associated with the target image to be processed; retrieve the historical location information corresponding to the historical image to be processed.

[0067] In this embodiment, since the historical images to be processed and the target images to be processed are related in terms of time, when a target image to be processed is determined, in order to further determine the historical images to be processed, it is also necessary to determine the generation time of the target image to be processed. This can be understood as obtaining the timestamp information generated when the camera device captures the target image to be processed.

[0068] Furthermore, based on the pre-set time intervals, historical images to be processed can be determined. Specifically, after determining the generation time t of the target image to be processed, multiple timestamps within a pre-set time interval δt prior to that time can be identified, and the multiple frames corresponding to these timestamps are then used as historical images to be processed. In other words, by determining the timestamp information within the interval [t-δt, t], the images corresponding to each timestamp within the interval are used as a set of images to be processed. It can be understood that the set of images to be processed includes at least one historical image to be processed and one target image to be processed.

[0069] In this embodiment, after obtaining at least one historical image to be processed, historical location information corresponding to the historical image to be processed can be retrieved. Specifically, each pixel in the historical image to be processed can be determined in the same way as the target pixel. The pixel values ​​of pixels that the autonomous vehicle can drive on and that carry road category markers are set to 0, and the pixel values ​​of pixels that do not carry road category markers are set to 1. Through binarization, pixels other than those with road category markers are identified in the historical image to be processed as historical target pixels. Further, the two-dimensional coordinates of the historical target pixels in their corresponding historical images to be processed are transformed to obtain their three-dimensional coordinate values ​​in the world coordinate system, and each three-dimensional coordinate value is used as the historical location information in this embodiment. Finally, based on all historical location information (i.e., three-dimensional coordinate values) carrying non-road attributes in the historical images to be processed and all target location information (i.e., three-dimensional coordinate values) carrying non-road attributes in the target images to be processed, a set T is constructed to represent the set of obstacle points on the road at time t.

[0070] It should be noted that, in order to ensure that the two-dimensional coordinates of pixels in multiple images can be transformed into a unified world coordinate system, for any historical image to be processed and the unique target image to be processed, the historical position information and target position information of each pixel are determined based on the same coordinate system.

[0071] Because camera devices may have blind spots, and image semantic segmentation models inevitably produce false detections, a single frame image cannot fully reflect obstacles on the road. Therefore, after identifying the target image to be processed, historical images associated with the target image to be processed are further identified. It can be understood that this method not only compensates for the blind spot information in the image captured by the autonomous vehicle at the current time t, but also alleviates the false detection phenomenon caused by the semantic segmentation model.

[0072] S230. Based on the vehicle location information, the side length information of the sub-grid area, and the coverage length and coverage width of the target vehicle, determine the target grid and each sub-grid area in the target grid.

[0073] The vehicle location information can be the coordinates of the autonomous vehicle's current position in the world coordinate system. Once the vehicle location information is determined, a two-dimensional target grid can be constructed centered on that location. In practical applications, the vehicle's coordinates within the constructed target grid can be defined as (x...). center y center ).

[0074] In this embodiment, since the target grid is composed of multiple sub-grids, while determining the target grid, the regions of each sub-grid can also be determined. Specifically, each sub-grid region can be determined using the following formula:

[0075]

[0076] Where `grid_length` is the length of the target grid, `grid_width` is the width of the target grid, and the coverage areas of the target grid in the x-axis and y-axis directions are `roi_x` and `roi_y`, respectively. It can be understood that the ratio of `roi_x` to the length of the target grid is the length of the sub-grid, and the ratio of `roi_y` to the width of the target grid is the width of the sub-grid. In practical applications, since each sub-grid in the target grid can be square, the side length of the determined sub-grid area is represented by `grid_size`.

[0077] S240. Based on the historical location information of each historical image to be processed and the target location information of the target image to be processed, determine the target raster attribute information of each sub-raster region.

[0078] In this embodiment, after constructing the target grid based on the vehicle location information and determining each sub-grid region, in order for the autonomous vehicle to detect obstacles on the road, it is also necessary to integrate the location information corresponding to each pixel in the image to be processed into the target grid. This can be understood as mapping the historical location information (two-dimensional coordinate values) of the historical images to be processed and the target location information (two-dimensional coordinate values) in the target image to each sub-grid in the target grid.

[0079] Optionally, for each image to be processed, the target sub-grid region to which each location information belongs is determined based on the horizontal and vertical coordinates of each location information in the current image to be processed, the side length information of the sub-grid region, and the horizontal and vertical coordinates of the vehicle location information.

[0080] Specifically, when retrieving historical location information, in order to represent the set of obstacle points on the road at time t, a set T has already been constructed based on all historical location information (i.e., 3D coordinate values) carrying non-road attributes in the historical images to be processed, and all target location information (i.e., 3D coordinate values) carrying non-road attributes in the target image to be processed. Therefore, it can be understood that the image to be processed includes the target image to be processed and each historical image to be processed, and the location information includes target location information and historical location information. Furthermore, after determining the set of location information in the image to be processed, the target sub-grid region corresponding to each location information can be determined in the target grid based on the following formula:

[0081]

[0082] Where grid_size is the side length of each sub-grid cell of the square; (x center y center (x) represents the vehicle's coordinates within the constructed target grid, corresponding to (x) grid y grid (x, y) represents the coordinates of each target pixel in the constructed target raster after the transformation; (x, y) represents the coordinates of each target pixel in its coordinate system before the coordinate transformation.

[0083] In practical applications, to integrate a set of obstacle points within a continuous time period, (x, y) can be the coordinate values ​​obtained by integrating the location information of each point into the same local coordinate system (i.e., the local coordinate system). The local coordinate system can be a coordinate system established with the starting point on the vehicle's travel route as the origin. It can be understood that after integrating the location information of each pixel into the local coordinate system, the position of static obstacles on the road will not change as the vehicle travels. Simultaneously, a pre-set transformation matrix can be used to transform the vehicle's location information (the vehicle's three-dimensional coordinates in the world coordinate system) into the aforementioned local coordinate system. This will not be elaborated further in the embodiments disclosed herein.

[0084] After identifying the target sub-raster regions corresponding to each location information within the target raster, since each sub-raster is associated with corresponding attribute information, it is necessary to label the target sub-raster regions corresponding to each image to be processed as obstacle attributes and the non-target sub-raster regions as free attributes, thus obtaining a raster attribute label sequence corresponding to each sub-raster region. Further, based on each raster attribute label sequence, the target raster attribute information is determined.

[0085] Specifically, since each location information represents the coordinate values ​​of the pixels reflecting obstacles, associating this location information with sub-grids allows for the labeling of the sub-grid's attribute information. For example, the sub-grid can be labeled as an obstacle in its attribute information. Simultaneously, the attribute information of other sub-grids that cannot be associated with location information is labeled as free attributes, indicating that no obstacle exists at the location corresponding to that sub-grid. It can be understood that since each location information originates from the target image to be processed and at least one historical image to be processed, the same sub-grid can be associated with multiple location information sets, thereby obtaining multiple corresponding attribute information sets. For a single sub-grid, its raster attribute label sequence can be constructed based on these multiple attribute information sets.

[0086] In this embodiment, after obtaining the grid attribute label sequence of each sub-grid, its target grid attribute information can be determined. Optionally, the target grid attribute information of the corresponding sub-grid area can be determined based on the frequency of obstacle attributes and free attributes in each grid attribute label sequence.

[0087] This can be understood as follows: each grid cell is associated with a label sequence of length δt, recording the label information at each time point within the interval [t-δt, t]. By integrating this label information, the final attribute of the sub-grid can be determined. Based on the strategy of finding the mode, for any sub-grid cell, the frequency of occurrence of two attributes (obstacle attribute and free attribute) among multiple label information can be determined, thereby determining whether there is an obstacle at the corresponding location of the sub-grid cell.

[0088] For example, for a sub-grid in a target grid, if all ten labels in its attribute label sequence are obstacle attributes, it can be determined that there is an obstacle at the location corresponding to the sub-grid. If eight of the ten labels in the attribute label sequence are obstacle attributes and the other two are free attributes, it means that there is a high probability that there is an obstacle at the location corresponding to the sub-grid. If all ten labels in the attribute label sequence are free attributes, it means that there is no obstacle at the location corresponding to the sub-grid.

[0089] It's important to note that if the location information is determined to be within the blind spot of the target vehicle based on its horizontal and vertical coordinates, the attribute information of the sub-grid region to which the location information belongs remains unchanged. This can be understood as follows: since the camera on the autonomous vehicle may have blind spots during recording, after obtaining the two-dimensional coordinates of a pixel, if it's determined through these coordinates that the pixel is within the camera's blind spot, then during the construction of the sub-grid attribute label sequence, its existing attributes are maintained (e.g., the attribute corresponding to the blind spot is marked as an obstacle attribute and kept there at all times). Multi-frame accumulation is avoided, thus ensuring that the sub-grid can continuously output obstacle attributes. This method prevents the autonomous vehicle from mistakenly identifying undetected blind spots as drivable roads.

[0090] S250. Based on the attribute information of each target grid, determine at least one sub-grid region to be aggregated for obstacle attributes; determine the target obstacle by clustering the at least one sub-grid region to be aggregated.

[0091] Specifically, after obtaining the target raster attribute information of each sub-raster in the target raster, each row (or each column) of sub-raster can be traversed to determine the sub-raster regions to be aggregated, which are marked as obstacle attributes. Further, a set B is constructed based on the sub-raster regions to be aggregated, and nearest neighbor clustering is performed on the points in set B to generate at least one clustering result. The convex polygon constructed based on each clustering result represents the obstacles on the road; that is, the shape of the obstacles is represented in the target raster.

[0092] It should be noted that, in order to improve the accuracy of the detection results, in the above multiple coordinate transformations, only points that are on or close to the ground plane in the world coordinate system (i.e., three-dimensional space) can be selected. Based on this, each cluster result actually reflects the top view of the obstacle. The autonomous vehicle can plan the subsequent driving path based on the top view of the obstacle in the target grid.

[0093] The technical solution of this embodiment transforms the position information of each target pixel into the world coordinate system. Simultaneously, based on the transformation result, a mapping table representing the relationship between the two-dimensional and three-dimensional coordinate values ​​of each target pixel is constructed. This allows for real-time retrieval of the three-dimensional coordinate values ​​of the target pixels when needed, via a lookup table. After determining the target grid and its sub-grid regions, the target grid attribute information of each sub-grid region is determined. Meanwhile, the attribute information of the sub-grid regions within the blind zone remains unchanged. By continuously outputting obstacle attributes within these sub-grids, the situation where the autonomous vehicle mistakenly treats undetected blind zones as drivable roads is avoided. Finally, based on the target grid attribute information, clustering processing is performed on the sub-grid regions to be aggregated, thereby determining the area occupied by the target obstacle within the target grid.

[0094] As an optional embodiment of the above embodiments, Figure 3 This is a schematic diagram (III) illustrating the principle of an obstacle detection method provided in an embodiment of the present invention. To clearly illustrate the technical solution of this embodiment, we can take the scenario of an unmanned vehicle detecting various types of obstacles on a road as an example. However, this method is not limited to the above scenario and can be applied to various scenarios requiring obstacle detection on roads.

[0095] See Figure 3 To achieve pixel-level processing in obstacle detection for autonomous vehicles, eliminating the need for ground truth labeling of obstacle features or obstacle identification in images, it is first necessary to extract pixels from the non-drivable areas of the autonomous vehicle. Specifically, image semantic segmentation technology can be used to process the raw images captured by the camera device to determine the semantic label of each pixel in the image, and these semantic labels are used as attributes of the pixel, such as road labels and non-road labels. Further, the semantic segmentation results are binarized, setting the pixel values ​​of pixels with drivable road attributes to 0, and the pixel values ​​of pixels with other attributes to 1.

[0096] See also Figure 3 Since pixels in a two-dimensional image cannot be used for obstacle detection in autonomous vehicles, it is necessary to transform the coordinates of the pixels in the two-dimensional image to a three-dimensional coordinate system. This three-dimensional coordinate system can be a coordinate system established based on the autonomous vehicle. Specifically, the height z of the camera device on the autonomous vehicle relative to the ground in the three-dimensional coordinate system can be determined first. w And the pixel coordinate transformation is achieved based on the following formula:

[0097]

[0098] Among them, M I M is the intrinsic parameter calibration matrix of the camera device (known). E Z is the extrinsic calibration matrix (known) between the camera device and the three-dimensional coordinate system (world coordinate system). c Let P be the distance between the camera device's coordinate system and the three-dimensional coordinate system; simultaneously, let P be the coordinates of the target pixel in the two-dimensional coordinate system. i = (u, v), with coordinates P in three-dimensional space. w =(x w y w , z w Based on this, substituting the above parameters, we can obtain the following formula:

[0099]

[0100] The above calculations can be used to obtain the coordinates of each pixel in the two-dimensional image in three-dimensional space.

[0101] See also Figure 3 Since the points of obstacles reflected in a single frame image are not complete, when an autonomous vehicle detects an obstacle on the road at a certain moment, it is necessary to fuse images from multiple frames before that moment for processing. This can be understood as determining the pixels in the historical images collected by the camera device based on the above method, and transforming the two-dimensional coordinates of each pixel in the image to obtain its coordinates in three-dimensional space.

[0102] See also Figure 3 After obtaining the three-dimensional coordinates of pixels in multiple frames of images, in order to fuse the set of obstacle points in a continuous time, it is also necessary to integrate these three-dimensional coordinates into the same local coordinate system. The local coordinate system can be a three-dimensional coordinate system established with the starting point of the autonomous vehicle's movement as the origin and through a pre-set transformation matrix. It can be understood that after the coordinate values ​​of pixels in the three-dimensional space coordinate system in the image are uniformly integrated into the local coordinate system, the position information of static obstacles will not change as the vehicle moves.

[0103] See also Figure 3 After converting the vehicle location information and all pixels to a local coordinate system, a raster map can be created with the vehicle as the center. Each sub-grid in the raster map stores the attribute information of that grid. The formulas involved in creating the raster map are as follows:

[0104]

[0105] Where grid_size is the side length of each sub-grid cell of the square; (x center y center Let (x) be the coordinates of the autonomous vehicle in the constructed grid map, and correspondingly, (x) grid y grid (x, y) represents the coordinates of each pixel in the constructed raster image after the transformation; (x, y) represents the coordinates of each pixel in its local coordinate system before the coordinate transformation.

[0106] Meanwhile, the other parameters of the raster map also satisfy:

[0107]

[0108] Where grid_length is the length of the raster image, grid_width is the width of the raster image, and the coverage areas of the raster image in the x-axis direction and y-axis direction are roi_x and roi_y, respectively.

[0109] See also Figure 3After converting the vehicle and pixel coordinates to a raster image, the attributes of sub-rasteres can be labeled. For example, when a pixel's value is 0, it indicates that the pixel corresponds to a road area in the image; therefore, the attribute label of the sub-raster corresponding to this pixel can also be labeled as a free attribute. When a pixel's value is 1, it indicates that the pixel corresponds to an obstacle in the image; therefore, the attribute label of the sub-raster corresponding to this pixel can also be labeled as an obstacle attribute. Since sub-rasteres in the raster image are associated with pixels in multiple images, there are multiple attribute labels for each sub-raster.

[0110] See also Figure 3 After determining multiple attribute labels for a sub-raster, a label sequence corresponding to that sub-raster can be constructed. By integrating the label sequence, the final attribute of the sub-raster can be obtained. For example, when the frequency of free attributes in the label sequence is high, the road in the image corresponding to the sub-raster can be identified. When the frequency of obstacle attributes in the label sequence is high, the obstacle in the image corresponding to the sub-raster can be identified. Those skilled in the art should understand that after obtaining the label sequence of the sub-raster, the rules for determining its final attribute can be set according to the actual situation. This embodiment of the present disclosure does not make specific limitations here.

[0111] It should be noted that, since this embodiment involves the calculation of a large number of pixels in the process of calculating the grid map and the label sequence of each sub-grid, in order to improve the calculation efficiency, a Compute Unified Device Architecture (CUDA) can be used for parallel acceleration. This approach can optimize the overall calculation time and CPU resource usage, thereby enabling real-time detection of obstacles.

[0112] See also Figure 3 After determining the final attributes of each sub-grid in the raster map, each row (or column) of sub-grids can be traversed to identify boundary sub-grids between obstacle attribute sub-grids and free attribute sub-grids. Further, a set B is constructed based on these boundary sub-grids. Nearest neighbor clustering is then performed on the grids in set B to obtain at least one clustering result. Finally, a corresponding convex polygon is constructed based on each clustering result. It can be understood that the resulting convex polygons represent the actual obstacles detected on the road.

[0113] The beneficial effects of the above technical solution are: it provides a convenient obstacle detection method for unmanned vehicles, and achieves indiscriminate detection of various types of obstacles through pixel-level processing, thereby making the obstacle detection results more accurate and comprehensive, and avoiding the problems of false detection or missed detection of obstacles in existing detection methods.

[0114] Figure 4 This is a structural block diagram (fourth) of an obstacle detection device provided in an embodiment of the present invention. It can execute the obstacle detection method provided in any embodiment of the present invention, and possesses the corresponding functional modules and beneficial effects of the method. For example... Figure 4 As shown, the device specifically includes: a target location information determination module 310, a historical location information determination module 320, a target grid attribute information determination module 330, and a target obstacle determination module 340.

[0115] The target location information determination module 310 is used to determine at least one target pixel in the target image to be processed, and the target location information of the at least one target pixel.

[0116] The historical location information determination module 320 is used to determine at least one historical image to be processed associated with the target image to be processed, and to retrieve the historical location information of each historical image to be processed.

[0117] The target grid attribute information determination module 330 is used to determine the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid area in the target grid based on the vehicle location information of the target vehicle, the target location information and each of the historical location information.

[0118] The target obstacle determination module 340 is used to determine target obstacles based on the attribute information of each target grid.

[0119] Based on the above technical solutions, the target location information determination module 310 includes a classification and marking unit, a target pixel point determination unit, and a target location information determination unit.

[0120] The classification and labeling unit is used to classify and label each pixel in the target image to be processed based on the image semantic segmentation algorithm, so as to obtain the category label of each pixel.

[0121] The target pixel determination unit is used to determine at least one target pixel in the target image to be processed based on each category label.

[0122] The target location information determination unit is used to determine the target location information of each target pixel based on a pre-established mapping relationship between pixel coordinates and world coordinates; wherein the mapping relationship is determined based on a pre-determined transformation matrix, which is determined based on the intrinsic parameter calibration matrix and extrinsic parameter calibration matrix of the camera device installed on the target vehicle, the vertical distance between the camera device and the horizontal plane, and the coordinate system distance.

[0123] Based on the above technical solutions, the historical location information determination module 320 includes a historical image to be processed determination unit and a historical location information determination unit.

[0124] The historical image to be processed determination unit is used to determine at least one historical image to be processed associated with the target image to be processed based on the generation time of the target image to be processed and a preset time interval.

[0125] A historical location information determination unit is used to retrieve historical location information corresponding to the historical image to be processed; wherein the historical location information and the target location information are determined based on the same coordinate system.

[0126] Based on the above technical solutions, the target raster attribute information determination module 330 includes a target raster determination unit and a target raster attribute information determination unit.

[0127] The target grid determination unit is used to determine the target grid and each sub-grid area in the target grid based on the vehicle location information, the side length information of the sub-grid area, and the coverage length and coverage width covering the target vehicle.

[0128] The target raster attribute information determination unit is used to determine the target raster attribute information of each sub-raster region based on the historical position information of each historical image to be processed and the target position information of the target image to be processed.

[0129] Optionally, the target raster attribute information determination unit is further configured to, for each image to be processed, determine the target sub-raster region to which each location information belongs based on the horizontal and vertical coordinates of each location information in the current image to be processed, the side length information of the sub-raster region, and the horizontal and vertical coordinates of the vehicle location information; wherein, the image to be processed includes the target image to be processed and each historical image to be processed, and the location information includes the target location information and the historical location information; the target sub-raster region corresponding to each image to be processed is marked as an obstacle attribute, and the non-target sub-raster region is marked as a free attribute, to obtain a raster attribute label sequence corresponding to each sub-raster region; and the target raster attribute information is determined based on each raster attribute label sequence.

[0130] Optionally, the target raster attribute information determination unit is also used to determine the target raster attribute information of the corresponding sub-raster area based on the frequency of obstacle attributes and free attributes in each raster attribute label sequence.

[0131] Based on the above technical solutions, the obstacle detection device also includes an attribute information retention module.

[0132] The attribute information preservation module is used to preserve the attribute information of the sub-grid region to which the location information belongs if the location information is determined to be within the blind spot of the target vehicle based on the horizontal and vertical coordinates of the location information.

[0133] Optionally, the target obstacle determination module 340 is further configured to determine at least one sub-grid region of obstacle attributes to be aggregated based on the attribute information of each target grid; and to determine the target obstacle by clustering the at least one sub-grid region to be aggregated.

[0134] The technical solution provided in this embodiment first determines at least one target pixel in the target image to be processed, and the target position information of at least one target pixel. Then, it determines at least one historical image to be processed associated with the target image to be processed, and retrieves the historical position information of each historical image to be processed, thus determining the position information of the pixel at multiple times. Furthermore, based on the vehicle position information, target position information, and historical position information of the target vehicle, it determines the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid area in the target grid. Based on the target grid attribute information, it determines the target obstacle. This provides a convenient obstacle detection method for unmanned vehicles, and achieves indiscriminate detection of various types of obstacles through pixel-level processing, thereby making the obstacle detection results more accurate and comprehensive, and avoiding the problem of false detection or missed detection of obstacles in existing detection methods.

[0135] The obstacle detection device provided in the embodiments of the present invention can execute the obstacle detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0136] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0137] Figure 5 A schematic diagram (V) of an electronic device provided in an embodiment of the present invention. Figure 5 A block diagram is shown of an exemplary electronic device 40 suitable for implementing embodiments of the present invention. Figure 5 The electronic device 40 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0138] like Figure 5As shown, electronic device 40 is represented in the form of a general-purpose computing device. The components of electronic device 40 may include, but are not limited to: one or more processors or processing units 401, system memory 402, and bus 403 connecting different system components (including system memory 402 and processing unit 401).

[0139] Bus 403 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0140] Electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0141] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media (… Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0142] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 407 typically perform the functions and / or methods described in the embodiments of the present invention.

[0143] Electronic device 40 can also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and with one or more devices that enable a user to interact with the electronic device 40, and / or with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 411. Furthermore, electronic device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 412. As shown, network adapter 412 communicates with other modules of electronic device 40 via bus 403. It should be understood that, although... Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0144] The processing unit 401 executes various functional applications and data processing by running programs stored in the system memory 402, such as implementing the obstacle detection method provided in the embodiments of the present invention.

[0145] This invention also provides a storage medium (six) containing computer-executable instructions, which, when executed by a computer processor, are used to perform an obstacle detection method.

[0146] The method includes:

[0147] Identify at least one target pixel in the target image to be processed, and the target location information of the at least one target pixel;

[0148] Identify at least one historical image to be processed associated with the target image to be processed, and retrieve the historical location information of each historical image to be processed;

[0149] Based on the vehicle location information of the target vehicle, the target location information, and each of the historical location information, determine the target grid corresponding to the target vehicle, and the target grid attribute information of each sub-grid area in the target grid;

[0150] Based on the attribute information of each target grid, the target obstacles are determined.

[0151] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0152] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable project code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device.

[0153] The project code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0154] Computer project code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The project code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0155] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. An obstacle detection method, characterized in that, Applications in autonomous vehicles include: Identify at least one target pixel in the target image to be processed, and the target location information of the at least one target pixel; Identify at least one historical image to be processed associated with the target image to be processed, and retrieve the historical location information of each historical image to be processed; Based on the vehicle location information of the target vehicle, the target location information, and each of the historical location information, determine the target grid corresponding to the target vehicle, and the target grid attribute information of each sub-grid area in the target grid; Based on the attribute information of each target grid, the target obstacles are determined; The step of determining the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid region in the target grid based on the vehicle location information of the target vehicle, the target location information, and the historical location information includes: Based on the vehicle location information, the side length information of the sub-grid area, and the coverage length and coverage width covering the target vehicle, the target grid and each sub-grid area in the target grid are determined; wherein, the vehicle location information is the coordinate value corresponding to the current position of the unmanned vehicle in the world coordinate system; For each image to be processed, the target sub-grid region to which each location information belongs is determined based on the horizontal and vertical coordinates of each location information in the current image to be processed, the side length information of the sub-grid region, and the horizontal and vertical coordinates of the vehicle location information; wherein, the image to be processed includes the target image to be processed and each historical image to be processed, and the location information includes the target location information and the historical location information; The target sub-raster region corresponding to each image to be processed is labeled as an obstacle attribute, and the non-target sub-raster region is labeled as a free attribute, thus obtaining a raster attribute label sequence corresponding to each sub-raster region; The target raster attribute information is determined based on the sequence of raster attribute labels.

2. The method according to claim 1, characterized in that, Determining at least one target pixel in the target image to be processed includes: Based on the image semantic segmentation algorithm, each pixel in the target image to be processed is classified and labeled to obtain the category label of each pixel; Based on the category labels, at least one target pixel in the target image to be processed is determined.

3. The method according to claim 1, characterized in that, Determining the target location information of the at least one target pixel includes: Based on the pre-established mapping relationship between pixel coordinates and world coordinates, the target position information of each target pixel is determined; The mapping relationship is determined based on a predetermined transformation matrix, which is based on the intrinsic parameter calibration matrix and extrinsic parameter calibration matrix of the camera device installed on the target vehicle, the vertical distance between the camera device and the horizontal plane, and the coordinate system distance.

4. The method according to claim 1, characterized in that, The step of determining at least one historical image to be processed associated with the target image to be processed, and retrieving the historical location information of each of the historical images to be processed, includes: Based on the generation time of the target image to be processed and a preset time interval, determine at least one historical image to be processed that is associated with the target image to be processed; Retrieve historical location information corresponding to the historical image to be processed; wherein the historical location information and the target location information are determined based on the same coordinate system.

5. The method according to claim 1, characterized in that, The step of determining the target raster attribute information based on the sequence of raster attribute labels includes: Based on the frequency of obstacle attributes and free attributes in each grid attribute label sequence, the target grid attribute information of the corresponding sub-grid area is determined.

6. The method according to claim 1, characterized in that, Also includes: If, based on the horizontal and vertical coordinates of the location information, it is determined that the location information is within the blind spot of the target vehicle, then the attribute information of the sub-grid region to which the location information belongs remains unchanged.

7. The method according to claim 1, characterized in that, The step of determining target obstacles based on the attribute information of each target grid includes: Based on the attribute information of each target grid, determine at least one sub-grid region of obstacle attributes to be aggregated; The target obstacle is determined by clustering the at least one sub-grid region to be aggregated.

8. An obstacle detection device, characterized in that, include: The target location information determination module is used to determine at least one target pixel in the target image to be processed, and the target location information of the at least one target pixel; The historical location information determination module is used to determine at least one historical image to be processed associated with the target image to be processed, and to retrieve the historical location information of each historical image to be processed; The target grid attribute information determination module is used to determine the target grid corresponding to the target vehicle and the target grid attribute information of each sub-grid area in the target grid based on the vehicle location information of the target vehicle, the target location information and each of the historical location information. The target obstacle determination module is used to determine target obstacles based on the attribute information of each target grid. The target raster attribute information determination module includes a target raster determination unit and a target raster attribute information determination unit. The target grid determination unit determines the target grid and each sub-grid region within the target grid based on the vehicle location information, the side length information of the sub-grid region, and the coverage length and coverage width covering the target vehicle; wherein, the vehicle location information is the coordinate value corresponding to the current position of the unmanned vehicle in the world coordinate system; The target raster attribute information determination unit is used to determine the target sub-raster region corresponding to each location information for each image to be processed, based on the horizontal and vertical coordinates of each location information in the current image to be processed, the side length information of the sub-raster region, and the horizontal and vertical coordinates of the vehicle location information; wherein, the image to be processed includes the target image to be processed and each historical image to be processed, and the location information includes the target location information and the historical location information; the target sub-raster region corresponding to each image to be processed is marked as an obstacle attribute, and the non-target sub-raster region is marked as a free attribute, to obtain a raster attribute label sequence corresponding to each sub-raster region; and the target raster attribute information is determined based on each raster attribute label sequence.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the obstacle detection method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the obstacle detection method as described in any one of claims 1-7.

Images