Obstacle detection method and device, electronic equipment and storage medium
By performing obstacle detection and depth analysis on the vehicle perimeter image of the autonomous driving car, determining the coordinates of candidate obstacles and determining the obstacle information in combination with reference coordinates, the problem of difficulty in accurately detecting long-distance obstacles during driving is solved, and the accuracy and stability of obstacle positions are improved.
Patent Information
- Application Number
- CN202510241477.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-28
- Publication Date
- 2025-06-06
AI Technical Summary
It is difficult for autonomous vehicles to accurately detect the depth information of long-distance obstacles during driving, resulting in jumping problems in obstacle position prediction, affecting driving safety.
By detecting obstacles on the vehicle perimeter image of the target vehicle, the position information and depth of the predicted obstacle are determined, and an obstacle whose depth is less than the depth threshold is determined as a candidate obstacle. According to the position information and depth of the candidate obstacle, its coordinates are determined under the vehicle coordinate system, and the target obstacle information is determined from the vehicle perimeter image in combination with the reference coordinates.
The position accuracy of close-range obstacles is improved and the position of long-range obstacles is determined stably, avoiding the jumping problem of obstacle position prediction and enhancing the driving safety of autonomous driving cars.
Smart Images
Figure CN120107327A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to an obstacle detection method, device, electronic device and storage medium. Background Art
[0002] In the autonomous driving system, obstacle detection is a key link to ensure driving safety. Autonomous vehicles need to identify and locate various obstacles on the road in real time during driving, including static obstacles (such as buildings, telephone poles, roadblocks, etc.) and dynamic obstacles (such as moving vehicles, pedestrians, bicycles, etc.). These obstacles may block the vehicle's driving path or pose a potential threat to the vehicle. Therefore, accurately detecting obstacles around the vehicle is crucial for autonomous vehicles to make correct decisions. Summary of the invention
[0003] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.
[0004] The first embodiment of the present disclosure provides an obstacle detection method, including:
[0005] Perform obstacle detection on the current image around the target vehicle to determine the first position information and depth corresponding to the predicted obstacle;
[0006] Determine the predicted obstacles whose depth is less than the depth threshold as candidate obstacles;
[0007] Determining, according to the first position information and the depth corresponding to the candidate obstacle, a first coordinate corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle;
[0008] Target obstacle information around the target vehicle is determined according to the reference coordinates, the first coordinates and the vehicle surrounding image.
[0009] The second aspect of the present disclosure provides an obstacle detection device, including:
[0010] A first determination module is used to perform obstacle detection on the current vehicle surrounding image of the target vehicle, and determine the first position information and depth corresponding to the predicted obstacle;
[0011] A second determination module is used to determine predicted obstacles with a depth less than a depth threshold as candidate obstacles;
[0012] A third determination module, configured to determine a first coordinate corresponding to the candidate obstacle in a vehicle coordinate system of the target vehicle according to the first position information and the depth corresponding to the candidate obstacle;
[0013] The fourth determination module is used to determine target obstacle information around the target vehicle according to the reference coordinates, the first coordinates and the vehicle surrounding image.
[0014] The third aspect embodiment of the present disclosure proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the obstacle detection method proposed in the first aspect embodiment of the present disclosure is implemented.
[0015] The fourth aspect embodiment of the present disclosure proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the obstacle detection method proposed in the first aspect embodiment of the present disclosure is implemented.
[0016] The fifth aspect of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the obstacle detection method proposed in the first aspect of the present disclosure.
[0017] The sixth aspect embodiment of the present disclosure provides a chip, including a processing unit and an interface circuit, wherein the processing unit obtains program instructions through the interface circuit, and the program instructions are executed by the processing unit, and the processing unit is used to execute the obstacle detection method proposed in the first aspect embodiment of the present disclosure.
[0018] The obstacle detection method, device, electronic device and storage medium provided by the present disclosure have the following beneficial effects:
[0019] In the disclosed embodiment, obstacle detection is first performed on the current vehicle surrounding image of the target vehicle to determine the first position information and depth corresponding to the predicted obstacle, and then the predicted obstacle with a depth less than the depth threshold is determined as a candidate obstacle, and the first coordinate corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle is determined according to the first position information and depth corresponding to the candidate obstacle, and finally the target obstacle information around the target vehicle is determined according to the reference coordinate, the first coordinate and the vehicle surrounding image. Thus, for the area close to the vehicle, the target obstacle information is determined from the vehicle surrounding image in combination with the first coordinate and the reference coordinate within the depth threshold range, and for the area far from the vehicle, the target obstacle information is determined from the vehicle surrounding image only according to the reference coordinate outside the depth threshold range, thereby avoiding the problem of inaccurate depth information of the predicted obstacle at a long distance, which causes the final predicted long-distance obstacle to jump, and improves the position accuracy of the determined short-distance obstacle and the position stability of the long-distance obstacle.
[0020] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0022] Figure 1 A flowchart of an obstacle detection method provided by an embodiment of the present disclosure;
[0023] Figure 2 A flowchart of an obstacle detection method provided by another embodiment of the present disclosure;
[0024] Figure 3 A schematic diagram of the structure of an obstacle detection device provided by another embodiment of the present disclosure;
[0025] Figure 4 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present disclosure is shown;
[0026] Figure 5 It is a schematic diagram of the structure of the chip proposed in the embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] Some embodiments of the present disclosure will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but can be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, for clarity and brevity, descriptions of features known in the art may be omitted.
[0028] The embodiments described in some embodiments of the present disclosure below do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0029] The obstacle detection method, device, electronic device, and storage medium according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.
[0030] Figure 1 A flowchart of an obstacle detection method provided by an embodiment of the present disclosure.
[0031] The embodiment of the present disclosure takes the obstacle detection method configured in an obstacle detection device as an example. The obstacle detection device can be applied to any electronic device or chip so that the electronic device or chip can perform an obstacle detection function.
[0032] like Figure 1 As shown, the obstacle detection method may include the following steps:
[0033] Step 101 : performing obstacle detection on the current image around the target vehicle to determine first position information and depth corresponding to the predicted obstacle.
[0034] The target vehicle may be a vehicle currently undergoing obstacle detection.
[0035] The vehicle image may be an image captured by an image sensor in the target vehicle. In some embodiments, the image sensor may include a camera, a fisheye camera, a depth camera, etc. This disclosure does not limit this.
[0036] In some embodiments, the current vehicle surrounding image may be one or more images, which are not limited in the present disclosure. For example, vehicle images captured by different image sensors at the same time are obtained.
[0037] The predicted obstacle is the obstacle predicted after obstacle detection is performed on the vehicle image.
[0038] The first position information corresponding to the predicted obstacle may be the predicted position of the obstacle in the image around the vehicle.
[0039] The depth is the distance between the predicted obstacle and the image sensor.
[0040] In some embodiments, the Mono3D detection solution may be used to perform obstacle detection on the image around the vehicle to obtain the first position information and depth corresponding to the predicted obstacle.
[0041] In some embodiments, a pre-trained obstacle detection model can be used to perform obstacle detection on the image around the vehicle to obtain a detection frame corresponding to the predicted obstacle, that is, the first position information. Then, features corresponding to the detection frame are extracted from the feature map of the image around the vehicle, and the features corresponding to the detection frame are input into the cosine trained deep regression network to obtain depth information.
[0042] In some embodiments, the obstacle detection model may be a faster region-based convolutional neural network (Faster R-CNN) or a single shot multibox detector (Single Shot MultiBox Detector, SSD), which is not limited in the present disclosure.
[0043] In some embodiments, the deep regression network may be a fully connected layer network, and an appropriate loss function may be used to train the deep regression network. Commonly used loss functions include mean square error function and mean absolute error function. These loss functions compare the predicted depth value with the true depth value and calculate the error to train the deep regression model.
[0044] Step 102: Determine the predicted obstacles whose depth is less than the depth threshold as candidate obstacles.
[0045] In some embodiments, the depth threshold may be a preset value, such as 80 meters, 100 meters, etc. This disclosure does not limit this.
[0046] It should be noted that after the depth is greater than or equal to the depth threshold, the predicted depth information may be inaccurate because the obstacle is smaller in the image. If the first coordinate is determined based on the predicted obstacle with a depth greater than or equal to the depth threshold, and then the obstacle is predicted in combination with the first coordinate, the obstacle position may jump during the obstacle prediction process, affecting the accuracy of obstacle detection. Therefore, the present disclosure determines the predicted obstacle with a depth less than the depth threshold as a candidate obstacle, and then determines the first coordinate corresponding to the candidate obstacle.
[0047] Step 103 : determining a first coordinate corresponding to the candidate obstacle in a vehicle coordinate system of the target vehicle according to the first position information and the depth corresponding to the candidate obstacle.
[0048] In some embodiments, the first position information and the depth may be converted to a vehicle coordinate system of the target vehicle based on parameters of a camera that captures vehicle images.
[0049] In some embodiments, the initial coordinates of the center point of the candidate obstacle in the vehicle surrounding image can be determined based on the first position information, and the initial coordinates and the corresponding depth can be converted to the vehicle coordinate system of the target vehicle based on the camera parameters to obtain the first coordinates corresponding to the candidate obstacle.
[0050] Among them, the vehicle coordinate system is a special moving coordinate system that describes the movement of the vehicle, and the origin of the coordinate system coincides with the center of mass of the vehicle.
[0051] Step 104, determining target obstacle information around the target vehicle according to the reference coordinates, the first coordinates and the vehicle surrounding image.
[0052] The reference coordinates may be coordinates of obstacles that are likely to appear in the vehicle coordinate system of the target vehicle.
[0053] In some embodiments, the reference coordinates may be coordinates that are manually set based on experience and are likely to contain obstacles in the vehicle coordinate system.
[0054] In some embodiments, the probability of an obstacle appearing at each sample coordinate can also be determined based on sample obstacle information corresponding to the sample vehicle at at least one sample time, and then the sample coordinate with a probability greater than a probability threshold is determined as the reference coordinate.
[0055] The sample vehicle may be a vehicle of the same type as the target vehicle, or may be a vehicle of a different type from the target vehicle, which is not limited in the present disclosure.
[0056] The sample coordinates are the coordinate points in the vehicle coordinate system corresponding to the sample vehicle.
[0057] The sample obstacle information includes the sample obstacle at the sample time and the corresponding second position information in the vehicle coordinate system of the sample vehicle.
[0058] In some embodiments, the second position information may be manually annotated based on the relationship between the obstacle and the vehicle at the sample time. Alternatively, the second position information may be obtained by detecting the vehicle surrounding image of the sample vehicle at the sample time using an obstacle detection method. This disclosure does not limit this.
[0059] The probability threshold may be pre-set, for example, the probability threshold may be 80%, 60%, etc. This disclosure does not limit this.
[0060] It should be noted that the higher the probability of an obstacle appearing at each sample coordinate, the greater the probability of an obstacle appearing at the position corresponding to the sample coordinate in the current vehicle surrounding image. Therefore, the present disclosure determines the sample coordinates with a probability greater than the probability threshold as the reference coordinates.
[0061] In some embodiments, based on the second position information corresponding to the obstacle at each sample moment in the vehicle coordinate system of the sample vehicle, the target sample moment at which the obstacle exists at each sample coordinate is determined, and the ratio between the number of target sample moments corresponding to each sample coordinate and the total number of sample moments is determined as the probability of the obstacle appearing at each sample coordinate.
[0062] For example, there are 100 sample moments in total, and there are 50 target sample moments at which an obstacle exists at sample coordinate 1, so the probability of an obstacle appearing at sample coordinate 1 is 50%; there are 80 target sample moments at which an obstacle exists at sample coordinate 2, so the probability of an obstacle appearing at sample coordinate 2 is 80%.
[0063] In some embodiments, the distance between the sample coordinates and the origin of the vehicle coordinate system is less than a depth threshold.
[0064] It should be noted that, within the range where the distance is less than the depth threshold, the first coordinate can accurately determine where obstacles may exist. Therefore, when the distance between the sample coordinate and the origin of the vehicle coordinate system is less than the depth threshold, the determined reference coordinate, combined with the first coordinate, can also accurately determine where obstacles exist in the vehicle surrounding image, and then perform obstacle detection on the area where obstacles exist to obtain obstacle information.
[0065] In some embodiments, feature extraction is performed on the image around the vehicle to determine a feature map corresponding to the image around the vehicle, a first area corresponding to the reference coordinate in the feature map, and a second area corresponding to the first coordinate in the feature map are determined; based on the first area and the second area in the feature map, target obstacle information is determined.
[0066] The first area may be an area within a preset distance around the first coordinate. For example, the preset distance may be in pixels, such as 100 pixels or 50 pixels, or may be in distance, such as 0.5 cm, 1 cm, 5 cm, etc. This disclosure does not limit this.
[0067] Similarly, the second area may be an area within a preset distance around the reference coordinates.
[0068] In some embodiments, after determining the first area corresponding to the reference coordinate in the feature map and the second area corresponding to the first coordinate in the feature map, features in the first area and the second area in the feature map can be analyzed to obtain target obstacle information.
[0069] In some embodiments, the target obstacle information may include the position, direction, speed, shape, size, type, etc. of the target obstacle in the vehicle coordinate system. The present disclosure is not limited to this.
[0070] In some embodiments, feature extraction can also be performed on the first coordinate and the reference coordinate to obtain a first vector corresponding to the first coordinate and a reference vector corresponding to the reference coordinate. Then, the feature map, the first vector and the reference vector corresponding to the sample image are output to a pre-trained sparse detection model to obtain target obstacle information.
[0071] In some embodiments, based on the sample vehicle surrounding image, the first sample coordinates and reference coordinates corresponding to the sample vehicle surrounding image, and the obstacle information label corresponding to the sample vehicle surrounding image, the feature map corresponding to the sample vehicle surrounding image, the vector corresponding to the first sample coordinate and the vector corresponding to the reference coordinate can be included in the initial sparse detection model to obtain predicted obstacle information. According to the difference between the predicted obstacle information and the obstacle information label, the initial sparse detection model is corrected to obtain a trained sparse detection model.
[0072] The method for determining the first sample coordinates corresponding to the sample vehicle surrounding image is the same as the method for determining the first coordinates corresponding to the vehicle image.
[0073] In some embodiments, when there are multiple images around the vehicle, based on the coverage area corresponding to each image around the vehicle, determine the first image around the vehicle corresponding to the reference coordinate and the second image around the vehicle corresponding to the first coordinate; determine the first area in the feature map corresponding to the first image around the vehicle; determine the second area in the feature map corresponding to the second image around the vehicle with the first coordinate. Then, the first area in the feature map corresponding to the first image around the vehicle and the second area in the feature map corresponding to the second image around the vehicle can be analyzed to obtain target obstacle information.
[0074] In the disclosed embodiment, obstacle detection is first performed on the current vehicle surrounding image of the target vehicle to determine the first position information and depth corresponding to the predicted obstacle, and then the predicted obstacle with a depth less than the depth threshold is determined as a candidate obstacle, and the first coordinate corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle is determined according to the first position information and depth corresponding to the candidate obstacle, and finally the target obstacle information around the target vehicle is determined according to the reference coordinate, the first coordinate and the vehicle surrounding image. Thus, for the area close to the vehicle, the target obstacle information is determined from the vehicle surrounding image in combination with the first coordinate and the reference coordinate within the depth threshold range, and for the area far from the vehicle, the target obstacle information is determined from the vehicle surrounding image only according to the reference coordinate outside the depth threshold range, thereby avoiding the problem of inaccurate depth information of the predicted obstacle at a long distance, which causes the final predicted long-distance obstacle to jump, and improves the position accuracy of the determined short-distance obstacle and the position stability of the long-distance obstacle.
[0075] Figure 2 A flowchart of an obstacle detection method provided by an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the obstacle detection method may include the following steps:
[0076] Step 201 : performing obstacle detection on the current image around the target vehicle to determine first position information and depth corresponding to the predicted obstacle.
[0077] Step 202: Determine the predicted obstacles whose depth is less than the depth threshold as candidate obstacles.
[0078] Step 203 : determining the first coordinates of the candidate obstacle in the vehicle coordinate system of the target vehicle according to the first position information and the depth of the candidate obstacle.
[0079] The specific implementation forms of steps 201 to 203 may refer to the detailed descriptions in other embodiments of the present disclosure and will not be repeated here.
[0080] Step 204, determining the position offset information of the target vehicle between the current moment and the historical moment, wherein the historical moment is the sample image acquisition moment adjacent to the current moment.
[0081] The position offset information may include a change in the vehicle's driving direction and an offset in the vehicle's center position.
[0082] Step 205 , determining the second coordinates of the obstacle at the historical moment in the vehicle coordinate system at the current moment according to the historical position of the obstacle at the historical moment in the vehicle coordinate system at the historical moment and the position offset information.
[0083] It should be noted that since the target vehicle moves between the current moment and the historical moment, there may be an offset between the vehicle coordinate system of the target vehicle at the current moment and the vehicle coordinate system at the historical moment. Therefore, based on the position offset information, the historical position corresponding to the obstacle at the historical moment in the vehicle coordinate system at the historical moment can be offset to obtain the second coordinate corresponding to the obstacle at the historical moment in the vehicle coordinate system at the current moment.
[0084] In some embodiments, the second coordinate may be the coordinate corresponding to the center point of the obstacle at a historical moment in the vehicle coordinate system at the current moment.
[0085] It should be noted that if the obstacle at the historical moment has not moved between the historical moment and the current moment, there is still an obstacle at the position corresponding to the second coordinate in the current vehicle surrounding image. If the obstacle at the historical moment has moved between the historical moment and the current moment, there may be an obstacle around the position corresponding to the second coordinate in the current vehicle surrounding image.
[0086] Step 206, determining target obstacle information around the target vehicle according to the reference coordinates, the first coordinates, the second coordinates and the vehicle surrounding image.
[0087] In some embodiments, feature extraction is performed on the image around the vehicle to determine a feature map corresponding to the image around the vehicle, a first area corresponding to the reference coordinate in the feature map, a second area corresponding to the first coordinate in the feature map, and a third area corresponding to the second coordinate in the feature map are determined; based on the first area, the second area, and the third area, target obstacle information is determined.
[0088] The third area may be an area within a preset distance around the second coordinate. For example, the preset distance may be in pixels. For example, 100 pixels, 50 pixels, or in distances, such as 0.5 cm, 1 cm, 5 cm, etc. This disclosure does not limit this.
[0089] In some embodiments, after determining the first area corresponding to the reference coordinate in the feature map, the second area corresponding to the first coordinate in the feature map, and the third area corresponding to the second coordinate in the feature map, the features of the first area, the second area, and the third area in the feature map can be analyzed to obtain target obstacle information.
[0090] In some embodiments, feature extraction can also be performed on the first coordinate, the reference coordinate, and the second coordinate to obtain a first vector corresponding to the first coordinate, a reference vector corresponding to the reference coordinate, and a second vector corresponding to the second coordinate. Then, the feature map, the first vector, the reference vector, and the second vector corresponding to the sample image are output to a pre-trained sparse detection model to obtain target obstacle information.
[0091] In some embodiments, when there are multiple images around the vehicle, based on the coverage area corresponding to each image around the vehicle, determine the first image around the vehicle corresponding to the reference coordinate, the second image around the vehicle corresponding to the first coordinate, and the third image around the vehicle corresponding to the second coordinate; then determine the reference coordinate, the first area in the feature map corresponding to the first image around the vehicle, determine the first coordinate, the second area in the feature map corresponding to the second image around the vehicle, and determine the second coordinate, the third area in the feature map corresponding to the third image around the vehicle. Then, analyze the first area in the feature map corresponding to the first image around the vehicle, the second area in the feature map corresponding to the second image around the vehicle, and the third area in the feature map corresponding to the third image around the vehicle to obtain target obstacle information.
[0092] In the disclosed embodiment, after determining the first coordinate corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle, the position offset information of the target vehicle between the current moment and the historical moment can also be determined, wherein the historical moment is the sample image acquisition moment adjacent to the current moment, and the second coordinate corresponding to the obstacle in the vehicle coordinate system at the historical moment and the position offset information are determined, and finally, the target obstacle information around the target vehicle is determined according to the reference coordinate, the first coordinate, the second coordinate and the vehicle surrounding image. Thus, for the area close to the vehicle, the target obstacle information is determined from the vehicle surrounding image in combination with the first coordinate, the reference coordinate within the depth threshold range and the second coordinate, and for the area far from the vehicle, the target obstacle information is determined from the vehicle surrounding image in combination with the reference coordinate outside the depth threshold range and the second coordinate, and the target obstacle information can be further determined in combination with the obstacle information at the historical moment to determine the target obstacle information at the current moment, thereby further improving the position accuracy of the determined short-distance obstacles and the position stability and accuracy of the long-distance obstacles.
[0093] In order to implement the above embodiments, the present disclosure also proposes an obstacle detection device.
[0094] Figure 3 A schematic diagram of the structure of an obstacle detection device provided in an embodiment of the present disclosure.
[0095] like Figure 3 As shown, the obstacle detection device 300 may include:
[0096] The first determination module 301 is used to perform obstacle detection on the current vehicle surrounding image of the target vehicle, and determine the first position information and depth corresponding to the predicted obstacle;
[0097] A second determination module 302 is used to determine predicted obstacles with a depth less than a depth threshold as candidate obstacles;
[0098] The third determination module 303 is used to determine the first coordinates corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle according to the first position information and depth corresponding to the candidate obstacle;
[0099] The fourth determination module 304 is used to determine target obstacle information around the target vehicle according to the reference coordinates, the first coordinates and the vehicle surrounding image.
[0100] In some embodiments, a fifth determining module is further included, configured to:
[0101] Determine the position offset information of the target vehicle between the current moment and the historical moment, wherein the historical moment is the sample image acquisition moment adjacent to the current moment;
[0102] Determine the second coordinate of the obstacle at the historical moment in the vehicle coordinate system at the current moment according to the historical position corresponding to the obstacle at the historical moment in the vehicle coordinate system at the historical moment and the position offset information;
[0103] According to the reference coordinates, the first coordinates, the second coordinates and the vehicle surrounding image, target obstacle information around the target vehicle is determined.
[0104] In some embodiments, the fifth determining module is configured to:
[0105] Extract features from the image around the vehicle and determine a feature map corresponding to the image around the vehicle;
[0106] Determine a first region corresponding to the reference coordinate in the feature map, a second region corresponding to the first coordinate in the feature map, and a third region corresponding to the second coordinate in the feature map;
[0107] Target obstacle information is determined based on the first area, the second area, and the third area.
[0108] In some embodiments, the fifth determining module is configured to:
[0109] In the case where there are multiple vehicle surrounding images, based on the coverage area corresponding to each vehicle surrounding image, determine a first vehicle surrounding image corresponding to the reference coordinate, a second vehicle surrounding image corresponding to the first coordinate, and a third vehicle surrounding image corresponding to the second coordinate;
[0110] Determine a reference coordinate, a first region in a feature map corresponding to a first vehicle circumference image;
[0111] Determine the first coordinate and the second region in the feature map corresponding to the second vehicle circumference image;
[0112] Determine a second coordinate, a third region in the feature map corresponding to the third vehicle circumference image.
[0113] In some embodiments, a sixth determining module is further included, configured to:
[0114] Determine the probability of an obstacle appearing at each sample coordinate according to the sample obstacle information corresponding to the sample vehicle at at least one sample time; wherein the sample coordinate is the coordinate point in the vehicle coordinate system corresponding to the sample vehicle;
[0115] The sample coordinates whose probability is greater than the probability threshold are determined as reference coordinates.
[0116] In some embodiments, the sixth determining module is configured to:
[0117] Determine a target sample time at which an obstacle exists at each sample coordinate according to the second position information corresponding to the obstacle at each sample time in the vehicle coordinate system of the sample vehicle;
[0118] The ratio between the number of target sample moments corresponding to each sample coordinate and the total number of sample moments is determined as the probability of an obstacle appearing at each sample coordinate.
[0119] In some embodiments, the distance between the sample coordinates and the origin of the vehicle coordinate system is less than a depth threshold.
[0120] In some embodiments, the fourth determining module 304 is configured to:
[0121] Extract features from the image around the vehicle and determine a feature map corresponding to the image around the vehicle;
[0122] Determine a first region corresponding to the reference coordinate in the feature map, and a second region corresponding to the first coordinate in the feature map;
[0123] Based on the first area and the second area in the feature map, target obstacle information is determined.
[0124] The functions and specific implementation principles of the above modules in the embodiments of the present disclosure can be referred to the above method embodiments, and will not be repeated here.
[0125] The obstacle detection device of the embodiment of the present disclosure first performs obstacle detection on the current vehicle surrounding image of the target vehicle, determines the first position information and depth corresponding to the predicted obstacle, then determines the predicted obstacle with a depth less than the depth threshold as a candidate obstacle, and determines the first coordinate corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle according to the first position information and depth corresponding to the candidate obstacle, and finally determines the target obstacle information around the target vehicle according to the reference coordinate, the first coordinate and the vehicle surrounding image. Thus, for the area close to the vehicle, the target obstacle information is determined from the vehicle surrounding image in combination with the first coordinate and the reference coordinate within the depth threshold range, and for the area far from the vehicle, the target obstacle information is determined from the vehicle surrounding image only according to the reference coordinate outside the depth threshold range, thereby avoiding the problem that the depth information of the predicted obstacle at a long distance is inaccurate, resulting in the final predicted long-distance obstacle jumping, and improving the position accuracy of the determined short-distance obstacle and the position stability of the long-distance obstacle.
[0126] In order to implement the above embodiments, the present disclosure further proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the obstacle detection method proposed in the above embodiments of the present disclosure is implemented.
[0127] Figure 4 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 4The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0128] like Figure 4 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).
[0129] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.
[0130] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.
[0131] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.
[0132] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.
[0133] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0134] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.
[0135] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the obstacle detection method proposed in the above embodiments of the present disclosure is implemented.
[0136] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program, which implements the obstacle detection method proposed in the above embodiments of the present disclosure when the computer program is executed by a processor.
[0137] Figure 5 is a schematic diagram of the structure of the chip proposed in the embodiment of the present disclosure. Figure 5 The structure of the chip 500 is shown, but is not limited to this.
[0138] The chip 500 includes a processing circuit 501 , and the processing circuit 501 is configured to execute any of the above methods.
[0139] In some embodiments, the chip 500 further includes one or more interface circuits 502. Optionally, the interface circuit 502 is connected to the memory 503, and the interface circuit 502 can be used to receive signals from the memory 503 or other devices, and the interface circuit 502 can be used to send signals to the memory 503 or other devices. For example, the interface circuit 502 can read instructions stored in the memory 503 and send the instructions to the processing circuit 501.
[0140] In some embodiments, the interface circuit 502 performs at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 501 performs other steps.
[0141] In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.
[0142] In some embodiments, the chip 500 further includes one or more memories 503 for storing instructions. Optionally, all or part of the memory 503 may be outside the chip 500.
[0143] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.
[0144] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0145] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0146] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.
[0147] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.
[0148] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0149] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0150] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0151] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.
Claims
1. An obstacle detection method, characterized in that: The method comprises: Perform obstacle detection on the current image around the target vehicle to determine the first position information and depth corresponding to the predicted obstacle; Determine the predicted obstacles whose depth is less than the depth threshold as candidate obstacles; Determining, according to the first position information and the depth corresponding to the candidate obstacle, a first coordinate corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle; Target obstacle information around the target vehicle is determined according to the reference coordinates, the first coordinates and the vehicle surrounding image.
2. The method according to claim 1, characterized in that After determining the first coordinates corresponding to the candidate obstacle in the vehicle coordinate system of the target vehicle according to the first position information and the depth corresponding to the candidate obstacle, the method further includes: Determine the position offset information of the target vehicle between the current moment and the historical moment, wherein the historical moment is the sample image acquisition moment adjacent to the current moment; Determine, according to the historical position of the obstacle at the historical moment, corresponding to the vehicle coordinate system at the historical moment, and the position offset information, a second coordinate corresponding to the obstacle at the historical moment in the vehicle coordinate system at the current moment; Target obstacle information around the target vehicle is determined according to the reference coordinates, the first coordinates, the second coordinates and the vehicle surrounding image.
3. The method according to claim 2, characterized in that The determining the target obstacle information around the target vehicle according to the reference coordinates, the first coordinates, the second coordinates and the vehicle surrounding image includes: Extracting features from the image around the vehicle to determine a feature map corresponding to the image around the vehicle; Determine a first region corresponding to the reference coordinate in the feature map, a second region corresponding to the first coordinate in the feature map, and a third region corresponding to the second coordinate in the feature map; The target obstacle information is determined based on the first area, the second area, and the third area.
4. The method according to claim 3, characterized in that The determining the first area corresponding to the reference coordinate in the feature map, the second area corresponding to the first coordinate in the feature map, and the third area corresponding to the second coordinate in the feature map includes: In the case where there are multiple vehicle surrounding images, determining a first vehicle surrounding image corresponding to the reference coordinate, a second vehicle surrounding image corresponding to the first coordinate, and a third vehicle surrounding image corresponding to the second coordinate based on the coverage area corresponding to each of the vehicle surrounding images; Determine the reference coordinates, a first region in a feature map corresponding to the first vehicle surrounding image; Determine a second region in the feature map corresponding to the second vehicle surrounding image, wherein the first coordinate is Determine the second coordinate in a third area in the feature map corresponding to the third vehicle circumference image.
5. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine the probability of an obstacle appearing at each sample coordinate according to sample obstacle information corresponding to the sample vehicle at at least one sample time; wherein the sample coordinate is a coordinate point in a vehicle coordinate system corresponding to the sample vehicle; The sample coordinates whose probabilities are greater than the probability threshold are determined as the reference coordinates.
6. The method according to claim 5, characterized in that The sample obstacle information includes a sample obstacle at a sample time and second position information corresponding to the sample vehicle in a vehicle coordinate system. The determining, based on the sample obstacle information corresponding to the sample vehicle at at least one sample time, a probability of an obstacle appearing at each sample coordinate includes: Determine a target sample time at which an obstacle exists at each sample coordinate according to second position information corresponding to the obstacle at each sample time in the vehicle coordinate system of the sample vehicle; The ratio between the number of target sample moments corresponding to each sample coordinate and the total number of sample moments is determined as the probability of an obstacle appearing at each sample coordinate.
7. The method according to claim 5, characterized in that A distance between the sample coordinates and the origin of the vehicle coordinate system is less than the depth threshold.
8. The method according to claim 1, characterized in that The step of determining target obstacle information around the target vehicle according to the reference coordinates, the first coordinates, and the vehicle surrounding image includes: Extracting features from the image around the vehicle to determine a feature map corresponding to the image around the vehicle; Determine a first region in the feature map corresponding to the reference coordinates, and a second region in the feature map corresponding to the first coordinates; The target obstacle information is determined based on the first area and the second area in the feature map.
9. An obstacle detection device, characterized in that: The device comprises: A first determination module is used to perform obstacle detection on the current vehicle surrounding image of the target vehicle, and determine the first position information and depth corresponding to the predicted obstacle; A second determination module is used to determine predicted obstacles with a depth less than a depth threshold as candidate obstacles; A third determination module, configured to determine a first coordinate corresponding to the candidate obstacle in a vehicle coordinate system of the target vehicle according to the first position information and the depth corresponding to the candidate obstacle; The fourth determination module is used to determine target obstacle information around the target vehicle according to the reference coordinates, the first coordinates and the vehicle surrounding image.
10. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the obstacle detection method according to any one of claims 1 to 8 is implemented.
11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the obstacle detection method as described in any one of claims 1 to 8 is implemented.
12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the obstacle detection method as claimed in any one of claims 1 to 8.
13. A chip, characterized in that: The chip includes a processing unit and an interface circuit, the processing unit obtains program instructions through the interface circuit, the program instructions are executed by the processing unit, and the processing unit is used to execute the obstacle detection method according to any one of claims 1-8.