Image obstacle target detection method, device, electronic device and storage medium

Abstract

The present invention relates to the field of image recognition technology and provides a method, device, electronic device, and storage medium for detecting obstacles in images. The method comprises: acquiring 2D image data and radar point cloud data in the current vehicle's travel direction; determining a detection frame for the obstacle in the image based on the 2D image data; determining radar point cloud data belonging to the obstacle based on the radar point cloud data and the obstacle's detection frame; and constructing a 3D frame for the obstacle based on the radar point cloud data of the obstacle. By associating the obstacle's detection frame in the 2D image with the radar point cloud data, the radar point cloud data belonging to the obstacle is separated and a 3D frame for the obstacle is constructed. This allows for rapid determination of required point cloud data from the radar point cloud data, accelerating the detection and location of the obstacle.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, system, electronic device and storage medium for detecting obstacle targets in images. Background Art

[0002] Obstacles primarily refer to objects encountered on the road during autonomous driving scenarios, such as vehicles, pedestrians, rocks, fallen trees, used tires, and any other unusual protrusions on the ground. Existing methods for detecting common obstacles in autonomous driving scenarios include combining ultrasonic, millimeter-wave, and lidar detection, using drivable area segmentation in images, and traditional CV methods to extract common obstacles.

[0003] For radar detection, lidar detection mainly has a high reflection response to metal objects, and it is not easy to accurately determine the point cloud data belonging to obstacles, that is, the point cloud data division is inaccurate. Summary of the Invention

[0004] In view of the problems existing in the prior art, the present invention provides a method, system, electronic device and storage medium for detecting obstacle targets in images.

[0005] In a first aspect, the present invention provides a method for detecting an obstacle target in an image, comprising:

[0006] Obtain 2D image data and radar point cloud data in the current vehicle travel direction;

[0007] Determine a detection frame of an obstacle target in the image based on the 2D image data;

[0008] Determining the radar point cloud data belonging to the obstacle target according to the radar point cloud data and the detection frame of the obstacle target;

[0009] A 3D frame of the obstacle target is constructed according to the radar point cloud data of the obstacle target.

[0010] In one embodiment, determining the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target includes:

[0011] Establishing a 2D coordinate system based on the 2D image data, and determining a position of the detection frame of the obstacle target on the 2D coordinate system;

[0012] Obtaining a projection point of the radar point cloud data on the 2D image data, determining a position of the projection point, and establishing a corresponding relationship between the radar point cloud data and the projection point;

[0013] Determining a projection point within the detection frame of the obstacle target based on a position of the detection frame of the obstacle target in the 2D coordinate system and a position of the projection point;

[0014] Determine the center area of the detection frame of the obstacle target based on a preset ratio, use the projection point of the center area as a seed point, and use a region growing algorithm to determine the expansion area;

[0015] The radar point cloud data belonging to the obstacle target is determined based on the projection points in the central area and the extended area and the corresponding relationship between the radar point cloud data and the projection points.

[0016] In one embodiment, constructing a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target includes:

[0017] Determine a center point based on the radar point cloud data of the obstacle target, and establish a 3D coordinate system based on the center point;

[0018] Obtaining a radar point that is farthest from the center position point within each quadrant of the 3D coordinate system;

[0019] A first stereo frame is established according to each acquired radar point, and mirror complementation is performed based on the first stereo frame to determine a 3D frame of the obstacle target.

[0020] In one embodiment, determining a detection frame of an obstacle target in an image based on the 2D image data includes:

[0021] Determining feature points in the 2D image data; the feature points are pixels in each frame of the image that intersect with the ground area;

[0022] determining a first detection contour in the 2D image data based on the feature points; the first detection contour is a contour representing an obstacle;

[0023] A second detection contour is determined based on the first detection contour, and the second detection contour is displayed on the 2D image data; the second detection contour is a contour obtained by screening and trimming the first detection contour.

[0024] In one embodiment, determining a first detection contour in the 2D image data according to the feature points includes:

[0025] All the determined feature points are smoothly connected to determine a first detection contour in the 2D image data; wherein two feature points whose distance between adjacent feature points is greater than a preset distance are not smoothly connected.

[0026] In one embodiment, determining the second detection profile according to the first detection profile includes:

[0027] Comparing the first detected contour having two endpoints with each standard contour in the pre-stored standard contour set for similarity, and determining the standard contour with the greatest similarity;

[0028] determining a scaling ratio between a first detection contour having two endpoints and a standard contour having the greatest similarity, and determining a defective contour corresponding to the first detection contour having the two endpoints according to the scaling ratio;

[0029] The incomplete contour and the first detection contour having two endpoints are integrated into a second detection contour.

[0030] In a second aspect, the present invention provides an image obstacle detection device, comprising:

[0031] The acquisition module is used to obtain 2D image data and radar point cloud data in the current vehicle driving direction;

[0032] A recognition module, configured to determine a detection frame of an obstacle target in the image based on the 2D image data;

[0033] a partitioning module, configured to determine the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target;

[0034] A construction module is used to construct a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target.

[0035] In a third aspect, the present invention provides an electronic device comprising a memory and a memory storing a computer program, wherein when the processor executes the program, the steps of the method for detecting obstacle targets in an image according to the first aspect are implemented.

[0036] In a fourth aspect, the present invention provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the obstacle target detection method for an image described in the first aspect.

[0037] The present invention provides an obstacle target detection method, system, electronic device, and storage medium for detecting obstacles in images. By associating the detection frame of the obstacle target in the 2D image with the radar point cloud data, the radar point cloud data belonging to the obstacle target is divided and a 3D frame of the obstacle target is constructed. This allows the required point cloud data to be quickly determined from the radar point cloud data, thereby accelerating the detection and positioning of the obstacle target. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 1 is a flow chart of the method for detecting obstacle targets in images provided by the present invention;

[0040] Figure 2 1 is a schematic structural diagram of an obstacle target detection device for a vehicle image according to the present invention;

[0041] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention; DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0043] The following combination Figure 1-Figure 3 The present invention describes an image obstacle target detection method, system, electronic device and storage medium.

[0044] Figure 1 A schematic diagram showing a flow chart of an obstacle target detection method for an image of the present invention is shown. Figure 1 , the method comprising:

[0045] 11. Obtain 2D image data and radar point cloud data in the current vehicle travel direction;

[0046] 12. Determine the detection frame of the obstacle target in the image based on the 2D image data;

[0047] 13. Determine the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target;

[0048] 14. Construct the 3D box of the obstacle target based on the radar point cloud data of the obstacle target.

[0049] Regarding steps 11-13, it should be noted that, in the present invention, the vehicle is an intelligent vehicle with autonomous driving capabilities, equipped with a forward-facing camera device for capturing 2D image data in the direction of travel. This 2D image data is dynamic video, consisting of multiple frames. Therefore, during the process of detecting obstacles in the 2D image data, steps 11-13 are actually performed on each frame.

[0050] The current vehicle is also equipped with a forward-facing radar transmitter for acquiring radar point cloud data in the vehicle's direction of travel. This radar point cloud data is synchronized with the camera device to achieve synchronous processing of image data and radar point cloud data.

[0051] In this invention, obstacles (such as vehicles ahead, pedestrians, trees blocking the road, fallen boulders, etc.) in the image can be identified as detected obstacle targets based on their image features such as shape, color, and size. The obstacle targets are then marked with a frame on the image, i.e., the obstacle target detection frame.

[0052] In the present invention, the acquired radar point cloud data is data reflected from all objects in front of the current vehicle. Therefore, some radar point cloud data may not have a substantial impact on the detection and location of obstacles in front of the vehicle. In this case, the acquired radar point cloud data needs to be segmented to separate point cloud data suitable for detecting and locating obstacles in front of the vehicle. Here, since the acquired detection frame is the detection frame of the obstacle target, the detection frame of the obstacle target is associated with the radar point cloud data to determine the radar point cloud data belonging to the obstacle target in a combined manner. The detection frame of the obstacle target and the radar point cloud data can be associated by projecting the point cloud data onto the detection frame, or by vertically moving the detection frame to enclose the point cloud data.

[0053] After acquiring the radar point cloud data belonging to the obstacle, a 3D box corresponding to the obstacle can be constructed based on the radar point cloud data of the obstacle. This 3D box is used to simplify the annotation and positioning of the obstacle in the image.

[0054] The obstacle target detection method provided by the present invention associates the detection frame of the obstacle target in the 2D image with the radar point cloud data, divides the radar point cloud data belonging to the obstacle target and constructs a 3D frame of the obstacle target, thereby quickly determining the required point cloud data from the radar point cloud data and accelerating the detection and positioning of the obstacle target.

[0055] In the further description of the above method, the processing process of determining the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection box of the obstacle target is mainly explained as follows:

[0056] Establishing a 2D coordinate system based on the 2D image data and determining the position of the detection frame of the obstacle target on the 2D coordinate system;

[0057] Obtain the projection point of the radar point cloud data on the 2D image data, determine the position of the projection point, and establish the corresponding relationship between the radar point cloud data and the projection point;

[0058] Determine the projection point within the obstacle target's detection frame based on the position of the obstacle target's detection frame in the 2D coordinate system and the position of the projection point.

[0059] The center area of the obstacle detection frame is determined based on a preset ratio, and the projection point of the center area is used as the seed point to determine the expansion area using the region growing algorithm.

[0060] The radar point cloud data belonging to the obstacle target is determined based on the corresponding relationship between the projection points in the central area and the extended area and the radar point cloud data and the projection points.

[0061] In this regard, it should be noted that, in the present invention, a 2D coordinate system is established on the 2D image to determine the position of the detection frame of each obstacle target on the 2D coordinate system, that is, the coordinate area corresponding to each detection frame is divided on the 2D coordinate system.

[0062] Project the radar point cloud data onto the 2D image to obtain the corresponding projected points. For each projected point, determine its corresponding position in the 2D coordinate system. Because multiple radar points may be projected onto the same point, a correspondence between the radar point cloud data and the projected points must be established. This facilitates subsequent determination of the corresponding radar point after the projected point falls within the detection frame.

[0063] By obtaining the coordinate areas corresponding to each detection frame divided on the 2D coordinate system and the position of each projection point, the projection point located within the detection frame of the obstacle target can be determined.

[0064] Due to the uncertainty of the obstacle's shape, the detection frame must cover an area larger than the obstacle's specific outline in the image. To this end, the detection frame includes not only the image area of the obstacle but also the background area. For example, if an image contains a vehicle ahead, the detection frame will include not only the image area of the vehicle ahead but also the background area of the ground or roadside guardrail.

[0065] Therefore, the point cloud data corresponding to the projection point located within the detection frame of the obstacle target not only includes the point cloud data of the obstacle target, but also the point cloud data of the background objects in the image. Therefore, the point cloud data of the background objects must be filtered out.

[0066] In the present invention, the central area of the detection frame of the obstacle target is determined based on a preset ratio (for example, a projection point is determined and a 5*5 ratio range with the projection point as the center point).

[0067] The projection point of the central area is used as the seed point, and the pixel features of the seed point on the image are obtained. Based on these pixel features, the region growing algorithm is used to determine the expansion area. The expansion area is the area outside the central area.

[0068] Then, the projection points in the central area and the extended area are obtained, and the radar point cloud data belonging to the obstacle target is determined based on the corresponding relationship between the projection points in the central area and the extended area and the radar point cloud data and the projection points.

[0069] A further method of the present invention establishes a 2D coordinate system, associates the detection frame of the obstacle target in the 2D image with the projection point of the radar point cloud based on the positions of the detection frame and the projection point on the coordinate system, and divides the radar point cloud data belonging to the obstacle target based on the region growing algorithm, thereby quickly determining the required point cloud data from the radar point cloud data.

[0070] In the further description of the above method, the process of constructing a 3D box of an obstacle target based on the radar point cloud data of the obstacle target is mainly explained as follows:

[0071] Determine the center point based on the radar point cloud data of the obstacle target and establish a 3D coordinate system based on the center point;

[0072] Get the radar point farthest from the center point in each quadrant of the 3D coordinate system;

[0073] A first stereo frame is established based on each acquired radar point, and mirror complementation is performed based on the first stereo frame to determine the 3D frame of the obstacle target.

[0074] It's important to note that, in this invention, since the obstacle is located in front of the vehicle, the radar emission point can only transmit laser light to the rear half of the obstacle surface in the train's direction of travel, acquiring reflected signals and generating point cloud data. Therefore, the radar point cloud data obtained for the obstacle corresponds to the point cloud data for the entire obstacle surface. Therefore, it's not possible to directly construct a 3D bounding box corresponding to the obstacle using only the radar point cloud data for the obstacle.

[0075] To this end, the following processing is required to construct a 3D box corresponding to the obstacle target:

[0076] The center point is determined based on the radar point cloud data of the obstacle target, and a 3D coordinate system is established based on the center point. In this regard, it should be noted that multiple radar points are used to determine a center point, and the center point is used as the center and the shortest distance is used as the radius to cover all radar points.

[0077] After the center point is determined, a 3D coordinate system is established based on the center point. A 3D coordinate system has eight quadrants. In the present invention, the radar point cloud data of the obstacle target falls into multiple quadrants, and then the radar point farthest from the center point in each quadrant is found.

[0078] Then, a stereo frame can be established based on the acquired radar points as the first stereo frame. Then, based on the first stereo frame, mirror complementation is performed to determine the other half of the stereo frame. The two stereo frames are combined to determine the 3D frame of the obstacle target.

[0079] A further method of the present invention determines a stereo frame based on the acquired radar point cloud data, and then determines the 3D frame of the obstacle target based on the stereo frame using a mirror complementary method, thereby quickly determining the complete 3D frame of the obstacle target when the radar point cloud data is incomplete.

[0080] In the further description of the above method, the process of determining the detection frame of the obstacle target in the image based on the 2D image data is mainly explained, including:

[0081] Determining feature points in the 2D image data; the feature points are pixels in each frame of the image that intersect with the ground area;

[0082] Determine a first detection contour in the 2D image data based on the feature points; the first detection contour is a contour representing the obstacle target;

[0083] A second detection contour is determined based on the first detection contour, and the second detection contour is displayed on the 2D image data; the second detection contour is a contour obtained by screening and trimming the first detection contour.

[0084] In this regard, it should be noted that in the present invention, the self-driving intelligent vehicle mainly travels on the specified main roads, and the obstacles such as vehicles, stones, trees, pedestrians, bicycles, etc. that appear on the road will have edges that intersect with the ground in the picture. For this reason, it is necessary to identify the pixel points where the obstacle and the ground area intersect in the image, and these pixel points serve as the feature points required in the detection process of the present invention. Since each obstacle and the ground are a different intersection scene, there is a regularity in the pixel changes of the intersection edge of the obstacle and the ground for different intersection scenes. Therefore, the intersection edge of the obstacle and the ground can be identified based on this pixel change, and then the pixel points on the intersection edge are selected as feature points.

[0085] In this invention, after identifying feature points in an image, these feature points are concatenated to form contour lines, which serve as preliminary detection contours that may represent obstacles. To facilitate the subsequent description of the solution, these detection contours are referred to as first detection contours.

[0086] Regarding the detection contours that may represent obstacle targets, it should be noted that some obstacle targets in the image do not intersect with the ground area as a whole.

[0087] For example, the lower half of a pedestrian on the road has an intersecting edge with the ground area in the image, while the upper half of the pedestrian has an intersecting edge with the building or the sky in the image.

[0088] For example, a tree blown down by the wind lies across the road. Part of its leafy branches intersects with the ground area in the image, while another part intersects with the building or the sky in the image.

[0089] For example, the tires and the lower body of the vehicle in front have an intersecting edge with the ground area in the image, while the upper body has an intersecting edge with the sky in the image.

[0090] Therefore, the first detection contour obtained based on the feature points may include both complete closed-loop contours and incomplete open-loop contours.

[0091] In the present invention, the first detection profile is then screened and trimmed to obtain a more complete detection profile. To distinguish these from the above detection profiles, these complete detection profiles are referred to as second detection profiles.

[0092] For example, an incomplete open-loop contour is repaired to obtain a complete open-loop contour. A complete closed-loop contour is processed to obtain a more standardized complete open-loop contour. These processed contours are classified as the second detection contours mentioned above.

[0093] A further method of the present invention determines the pixel points that intersect with the ground area in each frame of the image data, and based on the pixel points, realizes the outline display of the obstacle target in the image, so as to achieve the purpose of accurately locating the obstacle target in front of the moving vehicle.

[0094] In the further description of the above method, the processing of determining the first detection contour in the 2D image data based on the feature points is mainly explained as follows:

[0095] All the determined feature points are smoothly connected to determine a first detection contour in the 2D image data; wherein two feature points whose distance between adjacent feature points is greater than a preset distance are not smoothly connected.

[0096] It's important to note that there may be multiple obstacles that appear in a single image separated by a distance. For example, there may be gaps between different vehicles on the road. In this case, each obstacle exists independently at its intersection with the ground in the image.

[0097] There are also cases where multiple obstacles appearing in a single image overlap. For example, if a vehicle changes lanes and cuts in, multiple nearby vehicles may be connected to the intersecting edges of the ground area in the image.

[0098] In this invention, obstacle targets are marked using contour lines, so all identified feature points are smoothly connected to obtain individual detection contours. However, since many of the obstacles mentioned above exist independently at the intersection edges with the ground area in the image, a preset distance is required. When the distance between adjacent feature points is greater than the preset distance, it is equivalent to the two feature points belonging to two obstacles with a gap. In this case, the two feature points do not need to be smoothly connected. Using the distance restriction mentioned above, possible obstacles in the image can be distinguished.

[0099] A further method of the present invention can divide a plurality of detection contours in an image by smoothly connecting all feature points and limiting the distance between feature points, thereby achieving simple distinction of areas in the image that may be obstacles.

[0100] In the further description of the above method, the process of determining the second detection contour based on the first detection contour is mainly explained, as follows:

[0101] Comparing the first detected contour having two endpoints with each standard contour in the pre-stored standard contour set for similarity, and determining the standard contour with the greatest similarity;

[0102] determining a scaling ratio between a first detection contour having two endpoints and a standard contour having the greatest similarity, and determining a defective contour corresponding to the first detection contour having the two endpoints according to the scaling ratio;

[0103] The incomplete contour and the first detected contour having two endpoints are integrated into a second detected contour.

[0104] In this regard, it should be noted that, in the present invention, the first detection contour with two endpoints is an incomplete open-loop contour, that is, the contour has an opening. The reason for the opening is that the obstacle target to which the contour corresponds does not have an intersecting edge with the ground area (such as an intersecting edge with the sky).

[0105] In the present invention, a standard contour set is provided, in which a large number of standard contours are stored. These standard contours are overall contours collected at different angles and sizes for different obstacles and the ground in an intersecting state.

[0106] For example, a change in the distance between the vehicle in front of the current vehicle and the current vehicle will result in a different intersection contour between the vehicle in front and the ground area captured by the current vehicle. To this end, the standard contour set stores a reasonable amount of contour data suitable for the intersection between the vehicle in front and the ground.

[0107] For example, as the distance between a pedestrian and the vehicle changes, as does their walking posture, the intersection contours between the pedestrian and the ground area captured by the current vehicle may also vary. To this end, the standard contour set stores a reasonable amount of contour data suitable for the intersection of pedestrians and the ground.

[0108] For other types of obstacles, the principle is the same as the above examples, and contour data of reasonable data will also be stored in the standard contour set.

[0109] In this invention, incomplete open-loop contours need to be repaired, that is, the open-loop contours are supplemented into closed-loop and complete contours. Therefore, the similarity between the detected contour with two endpoints and each standard contour in a pre-stored standard contour set is compared. The standard contour with the greatest similarity is determined as the contour belonging to the same obstacle as the detected contour with two endpoints.

[0110] In the present invention, since the standard contour may be different in size from the detection contour with two endpoints, the partial contour size of the detection contour with two endpoints is converted to the corresponding partial contour size of the standard contour with the greatest similarity to obtain the scaling ratio between the two, and then the incomplete contour corresponding to the first detection contour with two endpoints is determined according to the scaling ratio, that is, the contour of the other part of the detection contour with two endpoints is restored according to the scaling ratio.

[0111] Finally, the incomplete contour and the first detected contour with two endpoints are integrated into a complete contour, the second detected contour. In order to distinguish them from the detected contours before integration, these detected contours after integration are called second detected contours.

[0112] In addition, the first detection contours that do not have two endpoints are actually complete closed-loop contours. In this case, these contours are directly determined as second detection contours.

[0113] A further method of the present invention achieves complete trimming of an incomplete detected contour by comparing the detected contour with a standard contour, thereby obtaining a more complete contour marking of the obstacle target.

[0114] The following describes the device for detecting an obstacle target in an image provided by the present invention. The device for detecting an obstacle target in an image described below and the method for detecting an obstacle target in an image described above can be referred to in correspondence with each other.

[0115] Figure 2 A schematic diagram showing the structure of an obstacle target detection device for an image provided by the present invention is shown. Figure 2 The device includes an acquisition module 21, an identification module 22, a division module 23 and a construction module 24, wherein:

[0116] An acquisition module 21 is used to acquire 2D image data and radar point cloud data in the current vehicle travel direction;

[0117] A recognition module 22 is configured to determine a detection frame of an obstacle target in the image based on the 2D image data;

[0118] a dividing module 23, configured to determine the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target;

[0119] The construction module 24 is configured to construct a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target.

[0120] In further description of the above device, the division module is specifically used to:

[0121] Establishing a 2D coordinate system based on the 2D image data, and determining a position of the detection frame of the obstacle target on the 2D coordinate system;

[0122] Establishing a 2D coordinate system based on the 2D image data and determining the position of the detection frame of the obstacle target on the 2D coordinate system;

[0123] Obtain the projection point of the radar point cloud data on the 2D image data, determine the position of the projection point, and establish the corresponding relationship between the radar point cloud data and the projection point;

[0124] Determine the projection point within the obstacle target's detection frame based on the position of the obstacle target's detection frame in the 2D coordinate system and the position of the projection point.

[0125] The center area of the obstacle detection frame is determined based on a preset ratio, and the projection point of the center area is used as the seed point to determine the expansion area using the region growing algorithm.

[0126] The radar point cloud data belonging to the obstacle target is determined based on the corresponding relationship between the projection points in the central area and the extended area and the radar point cloud data and the projection points.

[0127] In further description of the above device, the building block is specifically used to:

[0128] Determine a center point based on the radar point cloud data of the obstacle target, and establish a 3D coordinate system based on the center point;

[0129] Obtaining a radar point that is farthest from the center position point within each quadrant of the 3D coordinate system;

[0130] A first stereo frame is established according to each acquired radar point, and mirror complementation is performed based on the first stereo frame to determine a 3D frame of the obstacle target.

[0131] In further description of the above device, the identification module is specifically used to:

[0132] Determining feature points in the 2D image data; the feature points are pixels in each frame of the image that intersect with the ground area;

[0133] Determine a first detection contour in the 2D image data based on the feature points; the first detection contour is a contour representing an obstacle target;

[0134] A second detection contour is determined based on the first detection contour, and the second detection contour is displayed on the 2D image data; the second detection contour is a contour obtained by screening and trimming the first detection contour.

[0135] In a further description of the above apparatus, the recognition module, in the process of determining the first detection contour in the 2D image data according to the feature points, is specifically configured to:

[0136] All the determined feature points are smoothly connected to determine a first detection contour in the 2D image data; wherein two feature points whose distance between adjacent feature points is greater than a preset distance are not smoothly connected.

[0137] In a further description of the above device, the recognition module, in the process of determining the second detection profile according to the first detection profile, is specifically configured to:

[0138] Comparing the first detected contour having two endpoints with each standard contour in the pre-stored standard contour set for similarity, and determining the standard contour with the greatest similarity;

[0139] determining a scaling ratio between a first detection contour having two endpoints and a standard contour having the greatest similarity, and determining a defective contour corresponding to the first detection contour having the two endpoints according to the scaling ratio;

[0140] The incomplete contour and the first detection contour having two endpoints are integrated into a second detection contour.

[0141] Since the principles of the apparatus described in the embodiment of the present invention are the same as those of the method described in the above embodiment, more detailed explanations are omitted here.

[0142] It should be noted that, in the embodiment of the present invention, relevant functional modules can be implemented by a hardware processor.

[0143] The image obstacle target detection method provided by the present invention associates the detection box of the obstacle target in the 2D image with the radar point cloud data, divides the radar point cloud data belonging to the obstacle target and constructs a 3D box of the obstacle target, thereby quickly determining the required point cloud data from the radar point cloud data and accelerating the detection and positioning of the obstacle target.

[0144] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 31, a communication interface 32, a memory 33, and a communication bus 34. The processor 31, the communication interface 32, and the memory 33 communicate with each other via the communication bus 34. The processor 31 may call a computer program in the memory 33 to execute the steps of a method for detecting an obstacle target in an image, which may include, for example: acquiring 2D image data and radar point cloud data in the current vehicle travel direction; determining a detection frame of the obstacle target in the image based on the 2D image data; determining the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the obstacle target's detection frame; and constructing a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target.

[0145] In addition, the logic instructions in the above-mentioned memory 33 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0146] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the obstacle target detection method in the image provided by the above methods, and the method includes: obtaining 2D image data and radar point cloud data in the current vehicle driving direction; determining a detection frame of the obstacle target in the image based on the 2D image data; determining the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target; and constructing a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target.

[0147] On the other hand, an embodiment of the present application also provides a processor-readable storage medium, which stores a computer program. The computer program is used to enable the processor to execute the obstacle target detection method in the image provided by the above-mentioned embodiments, for example, including: obtaining 2D image data and radar point cloud data in the current vehicle driving direction; determining the detection frame of the obstacle target in the image based on the 2D image data; determining the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target; and constructing a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target.

[0148] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), solid-state drives (SSDs)), etc.

[0149] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0150] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting obstacles in an image, characterized in that: include: Obtain 2D image data and radar point cloud data in the current vehicle travel direction; Determine a detection frame of an obstacle target in the image based on the 2D image data; Determining the radar point cloud data belonging to the obstacle target according to the radar point cloud data and the detection frame of the obstacle target; Constructing a 3D frame of the obstacle target according to the radar point cloud data of the obstacle target; The step of constructing a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target includes: Determine a center point based on the radar point cloud data of the obstacle target, and establish a 3D coordinate system based on the center point; Obtaining a radar point that is farthest from the center position point within each quadrant of the 3D coordinate system; A first stereo frame is established according to each acquired radar point, and mirror complementation is performed based on the first stereo frame to determine a 3D frame of the obstacle target.

2. The method for detecting obstacle targets in an image according to claim 1, wherein: The determining the radar point cloud data belonging to the obstacle target according to the radar point cloud data and the detection frame of the obstacle target includes: Establishing a 2D coordinate system based on the 2D image data, and determining a position of the detection frame of the obstacle target on the 2D coordinate system; Obtaining a projection point of the radar point cloud data on the 2D image data, determining a position of the projection point, and establishing a corresponding relationship between the radar point cloud data and the projection point; Determining a projection point within the detection frame of the obstacle target based on a position of the detection frame of the obstacle target in the 2D coordinate system and a position of the projection point; Determine the center area of the detection frame of the obstacle target based on a preset ratio, use the projection point of the center area as a seed point, and use a region growing algorithm to determine the expansion area; The radar point cloud data belonging to the obstacle target is determined based on the projection points in the central area and the extended area and the corresponding relationship between the radar point cloud data and the projection points.

3. The method for detecting obstacle targets in an image according to claim 1, wherein: The determining a detection frame of an obstacle target in an image according to the 2D image data includes: Determine feature points in the 2D image data; the feature points are pixels in each frame of the image that intersect with the ground area; Determine a first detection contour in the 2D image data based on the feature points; the first detection contour is a contour representing an obstacle target; A second detection contour is determined based on the first detection contour, and the second detection contour is displayed on the 2D image data; the second detection contour is a contour obtained by screening and trimming the first detection contour.

4. The method for detecting obstacle targets in an image according to claim 3, wherein: The determining a first detection contour in the 2D image data according to the feature points includes: All the determined feature points are smoothly connected to determine a first detection contour in the 2D image data; wherein two feature points whose distance between adjacent feature points is greater than a preset distance are not smoothly connected.

5. The method for detecting obstacle targets in an image according to claim 4, wherein: The determining the second detection profile according to the first detection profile includes: Comparing the first detected contour having two endpoints with each standard contour in the pre-stored standard contour set for similarity, and determining the standard contour with the greatest similarity; determining a scaling ratio between a first detection contour having two endpoints and a standard contour having the greatest similarity, and determining a defective contour corresponding to the first detection contour having the two endpoints according to the scaling ratio; The incomplete contour and the first detection contour having two endpoints are integrated into a second detection contour.

6. An image obstacle detection device, characterized in that: include: The acquisition module is used to obtain 2D image data and radar point cloud data in the current vehicle driving direction; A recognition module, configured to determine a detection frame of an obstacle target in the image based on the 2D image data; a partitioning module, configured to determine the radar point cloud data belonging to the obstacle target based on the radar point cloud data and the detection frame of the obstacle target; A construction module, configured to construct a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target; The step of constructing a 3D frame of the obstacle target based on the radar point cloud data of the obstacle target includes: Determine a center point based on the radar point cloud data of the obstacle target, and establish a 3D coordinate system based on the center point; Obtaining a radar point that is farthest from the center position point within each quadrant of the 3D coordinate system; A first stereo frame is established according to each acquired radar point, and mirror complementation is performed based on the first stereo frame to determine a 3D frame of the obstacle target.

7. The image obstacle detection device according to claim 6, characterized in that: The partitioning module is specifically used for: Establishing a 2D coordinate system based on the 2D image data, and determining a position of the detection frame of the obstacle target on the 2D coordinate system; Obtaining a projection point of the radar point cloud data on the 2D image data, determining a position of the projection point, and establishing a corresponding relationship between the radar point cloud data and the projection point; Determining a projection point within the detection frame of the obstacle target based on a position of the detection frame of the obstacle target in the 2D coordinate system and a position of the projection point; Determine the center area of the detection frame of the obstacle target based on a preset ratio, use the projection point of the center area as a seed point, and use a region growing algorithm to determine the expansion area; The radar point cloud data belonging to the obstacle target is determined based on the projection points in the central area and the extended area and the corresponding relationship between the radar point cloud data and the projection points.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of the method for detecting obstacle targets in an image according to any one of claims 1 to 5 are implemented.

9. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the method for detecting obstacle targets in an image according to any one of claims 1 to 5.

Images