A method and apparatus for object detection

By using inverse perspective transformation and contour line segment processing, the problems of missed detection and false detection in obstacle detection in intelligent driving are solved, achieving efficient and accurate object detection, which is suitable for complex road conditions.

CN115147803BActive Publication Date: 2026-11-17YINWANG INTELLIGENT TECHNOLOGIES CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110333573.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-29
Publication Date
2026-11-17
Estimated Expiration
2041-03-29

AI Technical Summary

Technical Problem

Existing obstacle detection methods suffer from problems such as missed detections and false detections in the field of intelligent driving, have poor versatility, and are difficult to achieve high-precision detection in complex road conditions.

Method used

By performing inverse perspective transformation on the image, the contour lines in the image are extracted, and the position of the object is determined according to the division and projection of the line segment set. Error parameters are calculated using a set data model to improve the detection accuracy and versatility.

Benefits of technology

It enables accurate detection of a variety of different objects, has a wide range of applications, reduces detection costs, and improves detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115147803B_ABST
    Figure CN115147803B_ABST
Patent Text Reader

Abstract

This application provides an object detection method and apparatus. The method includes: performing inverse perspective transformation on a first target image to obtain a second target image, wherein the second target image is an image obtained by mapping the first target image onto a target plane; performing contour line extraction on the second target image to obtain multiple contour line segments; selecting multiple line segments from the multiple contour line segments whose lines pass through a set target point; dividing the multiple line segments into at least one line segment set according to the position of the multiple line segments in the second target image; and performing the following steps for each line segment set: projecting at least two line segments contained in the target line segment set onto the first target image to obtain at least two projected line segments; and determining the position of a target object in the first target image based on the at least two projected line segments, wherein the target line segment set is each line segment set. The solution provided by this application can improve the accuracy and versatility of detecting objects in images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an object detection method and apparatus. Background Technology

[0002] In the field of intelligent driving, such as assisted driving and autonomous driving, vehicles typically need to use autonomous driving systems or advanced driver-assistance systems (ADAS) to detect and judge the current road conditions during driving, and to detect general obstacles, thereby effectively avoiding obstacles and ensuring the safety of vehicle driving.

[0003] Currently, obstacle detection can be achieved using either traditional image processing methods or network processing methods. Traditional image processing methods, based on image processing algorithms and manual feature construction and extraction, can selectively filter obstacles. However, these methods typically require limiting the driving scenario and the types of obstacles to be detected to maintain feature validity, resulting in poor versatility. Network processing methods, primarily data-driven, use machine learning for feature construction and extraction, exhibiting some generalization ability. However, they suffer from missed detections for obstacles whose features differ significantly from those in the training data. Therefore, current obstacle detection methods suffer from issues such as missed detections and false detections, resulting in poor versatility. Summary of the Invention

[0004] This application provides an object detection method and apparatus to improve the accuracy and versatility of object detection in images.

[0005] In a first aspect, this application provides an object detection method, the method comprising:

[0006] A first target image is subjected to inverse perspective transformation to obtain a second target image, wherein the first target image is an image of a scene containing at least one target object, and the second target image is an image obtained by mapping the first target image onto a target plane; contour line extraction is performed on the second target image to obtain multiple contour line segments; among the multiple contour line segments, multiple line segments whose lines pass through a set target point are selected; based on the positions of the multiple line segments in the second target image, the multiple line segments are divided into at least one line segment set, wherein different line segment sets contain different line segments, and any line segment set contains at least two line segments from the multiple line segments; for each line segment set, the following steps are performed: projecting at least two line segments contained in the target line segment set onto the first target image to obtain at least two projected line segments; determining the position of a target object in the first target image based on the at least two projected line segments, wherein the target line segment set is each line segment set.

[0007] In this method, object detection is performed based on the second target image obtained by projecting the first target image onto the target plane. This fully utilizes the affine geometric features of the object's contour, especially its height contour, in the image after inverse perspective transformation of the first target image. This allows for accurate detection of object positions within the image, improving detection accuracy. Furthermore, this method can detect a variety of different objects, making it applicable to a wide range of scenarios and demonstrating good versatility. In addition, this method can accurately detect objects in an image using only a single frame, thus reducing the cost of detection.

[0008] In one possible design, the multiple line segments are divided into at least one set of line segments based on their positions in the second target image. This includes: sorting the multiple line segments according to their starting endpoints in the second target image, wherein the starting endpoint of any line segment is the endpoint of the line segment that is closest to the target point; selecting a line segment not yet assigned to a set of line segments as a reference line segment according to the sorting of the multiple line segments, and performing the following division process on the reference line segment: dividing the multiple line segments excluding the reference line segment and the already assigned line segments... Line segments other than those assigned to the line segment set are designated as line segments to be divided. Among these line segments, target line segments are identified whose included angle with the reference line segment is less than a set threshold. Each target line segment contains at least one line segment. An error parameter is calculated to classify the target line segment and the reference line segment as belonging to the same object contour. When the error parameter is less than or equal to the set threshold, the target line segment and the reference line segment are assigned to the same line segment set. When the error parameter is greater than the set threshold, the target line segment and the reference line segment are not assigned to the same line segment set, and the current division process ends.

[0009] In this method, when dividing the line segment set, a line segment is selected in the order of multiple line segments, and a line segment and a line segment that is more likely to belong to the same object contour line are selected to form a line segment set. Finally, the line segment set is divided, and line segments belonging to different object contour lines are divided into different sets. On the one hand, processing the line segments according to the line segment order can simplify the line segment set division process and improve the efficiency of line segment set division. On the other hand, the distribution of line segments in the contour line of the same object is similar, and dividing the line segment set according to the line segment position can improve the accuracy of line segment set division.

[0010] In one possible design, the multiple line segments are sorted according to the positions of their starting endpoints in the second target image, including: sorting the multiple line segments according to their clockwise or counterclockwise positional distribution order of the starting endpoints in the second target image.

[0011] In this method, all line segments are line segments whose corresponding lines pass through the same target point. Therefore, after sorting the line segments according to the positional distribution of their starting endpoints, when selecting the target line segment corresponding to the reference line segment during the process of dividing the line segment set, it can be ensured that the line segment that is close to the reference line segment and has a high probability of belonging to the same object is selected as accurately as possible.

[0012] In one possible design, calculating the error parameter for classifying the target line segment and the reference line segment into the same object contour includes: calculating the error parameter for classifying the target line segment and the reference line segment into the same object contour according to a set data model, wherein the set data model is used to calculate the error that exists when classifying multiple line segments into the same object contour.

[0013] In this method, the probability value that the target line segment and the reference line segment belong to the same object contour can be calculated quickly and accurately using the established data model.

[0014] In one possible design, calculating the error parameter for classifying the target line segment and the reference line segment into the same object contour includes: performing multiple calculations on the target line segment and the reference line segment to obtain multiple candidate error parameters; wherein each calculation process includes the following steps: randomly selecting at least one line segment from the target line segments; using the reference line segment and the at least one line segment randomly selected from the target line segments as sample line segments; calculating the target parameter corresponding to the sample line segment, and determining the candidate error parameter based on the target parameter; wherein the target parameter includes at least one parameter for characterizing the geometric distribution characteristics of the sample line segment; determining the minimum value among the multiple candidate error parameters; and using the minimum value as the error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0015] In this method, the probability values ​​of the target line segment and the reference line segment belonging to the same object contour are repeatedly calculated, and the minimum value among the multiple probability values ​​is selected as the final judgment basis. This can ensure the accuracy and reliability of the judgment. By repeatedly calculating, the error or misjudgment caused by the uncertainty of a single calculation is reduced.

[0016] In one possible design, determining the candidate error parameter based on the target parameter includes: if the target parameter contains only one parameter for characterizing the geometric distribution characteristics of the sample line segment, then that parameter is used as the candidate error parameter; if the target parameter contains multiple parameters for characterizing the geometric distribution characteristics of the sample line segment, then the multiple parameters are weighted and summed according to their corresponding weights to obtain the candidate error parameter.

[0017] In this method, the weights corresponding to different parameters in the target parameters are applied to the determination of probability values. This allows the emphasis of reference information to be adjusted according to the actual needs of the scenario during implementation, thereby flexibly dividing the line segment set and improving scenario adaptability.

[0018] In one possible design, the sample line segment includes multiple first line segments; the target parameter includes at least one of the following: a parameter with a value of 1 / (1+exp(-x)), where exp() is an exponential function with the natural constant as the base, x is the variance of the proportional parameters of the multiple first line segments, the proportional parameter of any first line segment is the ratio between the length of the first line segment and the distance from the terminating endpoint of the first line segment to the target point, and the terminating endpoint of any first line segment is the endpoint of the first line segment that is farthest from the target point; the variance of the texture direction values ​​of all points contained in at least one second line segment, the at least one second line segment being a line segment obtained by sequentially connecting the starting endpoints of the multiple first line segments according to their order in the multiple line segments; the distance between the target point and the line segment of the at least one second line segment that is closest to the target point; and the angle between the two line segments obtained by connecting the two endpoints of the reference line segment to the target point.

[0019] In this method, the target parameters include geometric feature information or geometric distribution feature information such as the length, texture, and angle of the line segments in the line segment set. Therefore, based on the feature information of each dimension of the line segments in the line segment set, it can be determined whether the line segments in the line segment set belong to the outline of the same object, thereby improving the accuracy of the judgment and thus improving the accuracy of subsequent target detection.

[0020] In one possible design, contour line extraction processing is performed on the second target image to obtain multiple contour line segments in the second target image, including: edge contour detection of the second target image to obtain multiple contour points in the second target image; and obtaining the multiple contour line segments based on the multiple contour points.

[0021] In this method, multiple contour segments are obtained by extracting contours from the image, and then the line segments corresponding to different objects can be determined based on the multiple contour segments for further object detection.

[0022] In one possible design, obtaining the multiple contour segments based on the multiple contour points includes: filtering the multiple contour points to obtain multiple target contour points; performing line detection on the multiple target contour points to obtain the multiple contour segments; or performing line detection on the multiple contour points to obtain the multiple contour segments.

[0023] In this method, by filtering the contour points in the image, the contour points of some objects other than the target object to be detected can be filtered out, reducing the interference of these contour points on the target detection process, thereby improving the detection accuracy and efficiency.

[0024] In one possible design, filtering the plurality of contour points includes: dividing the image region of the second target image into a plurality of grids according to a set grid size; classifying the objects displayed in the plurality of grids using a set classifier; and deleting the contour points located in the target grid if there is an object in the target grid whose category is a set category.

[0025] In this method, a classifier can quickly classify pixels in an image. By deleting the contour points of the region where the object of a set category is located, the number of contour points processed can be reduced without affecting the detection of the target object, thereby improving processing speed and efficiency.

[0026] Secondly, this application provides an object detection device, the device comprising:

[0027] A perspective transformation unit is used to perform inverse perspective transformation processing on a first target image to obtain a second target image, wherein the first target image is an image of a scene containing at least one target object, and the second target image is an image obtained by mapping the first target image onto a target plane; an image processing unit is used to perform contour line extraction processing on the second target image to obtain multiple contour line segments in the second target image; a selection unit is used to select multiple line segments whose lines pass through a set target point from the multiple contour line segments; a segmentation unit is used to divide the multiple line segments into at least one line segment set according to the position of the multiple line segments in the second target image, wherein different line segment sets contain different line segments, and any line segment set contains at least two line segments from the multiple line segments; a positioning unit is used to perform the following steps for each line segment set: projecting at least two line segments contained in the target line segment set onto the first target image to obtain at least two projected line segments; determining the position of a target object in the first target image according to the at least two projected line segments, wherein the target line segment set is each line segment set.

[0028] In one possible design, the object detection device further includes an acquisition unit for acquiring the first target image.

[0029] In one possible design, when the partitioning unit divides the multiple line segments into at least one set of line segments based on their positions in the second target image, it specifically performs the following steps: sorting the multiple line segments according to the positions of their starting endpoints in the second target image, wherein the starting endpoint of any line segment is the endpoint closest to the target point among the two endpoints of the line segment; sequentially selecting a line segment not yet assigned to a set of line segments as a reference line segment according to the sorting of the multiple line segments, and performing the following partitioning process on the reference line segment: dividing the multiple line segments excluding the reference line segment... Line segments other than those already assigned to a line segment set are designated as line segments to be divided. Among these, target line segments with an angle less than a set threshold with respect to the reference line segment are identified, wherein each target line segment contains at least one line segment. An error parameter is calculated to classify the target line segment and the reference line segment as belonging to the same object contour. When the error parameter is less than or equal to the set threshold, the target line segment and the reference line segment are assigned to the same line segment set. When the error parameter is greater than the set threshold, the target line segment and the reference line segment are not assigned to the same line segment set, and the current division process ends.

[0030] In one possible design, when the segmentation unit sorts the multiple line segments according to the positions of their starting endpoints in the second target image, it is specifically used to sort the multiple line segments according to the clockwise or counterclockwise positional distribution order of their starting endpoints in the second target image.

[0031] In one possible design, when the partitioning unit calculates the error parameter for classifying the target line segment and the reference line segment into the same object contour, it is specifically used to: calculate the error parameter for classifying the target line segment and the reference line segment into the same object contour according to a set data model, wherein the set data model is used to calculate the error that exists when classifying multiple line segments into the same object contour.

[0032] In one possible design, when the partitioning unit calculates the error parameter for classifying the target line segment and the reference line segment into the same object contour, it specifically performs the following steps: performing multiple calculations on the target line segment and the reference line segment to obtain multiple candidate error parameters; wherein each calculation process includes the following steps: randomly selecting at least one line segment from the target line segments; using the reference line segment and the at least one line segment randomly selected from the target line segments as sample line segments; calculating the target parameter corresponding to the sample line segment, and determining the candidate error parameter based on the target parameter; wherein the target parameter includes at least one parameter for characterizing the geometric distribution characteristics of the sample line segment; determining the minimum value among the multiple candidate error parameters; and using the minimum value as the error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0033] In one possible design, the sample line segment includes multiple first line segments; the target parameter includes at least one of the following: a parameter with a value of 1 / (1+exp(-x)), where exp() is an exponential function with the natural constant as the base, x is the variance of the proportional parameters of the multiple first line segments, the proportional parameter of any first line segment is the ratio between the length of the first line segment and the distance from the terminating endpoint of the first line segment to the target point, and the terminating endpoint of any first line segment is the endpoint of the first line segment that is farthest from the target point; the variance of the texture direction values ​​of all points contained in at least one second line segment, the at least one second line segment being a line segment obtained by sequentially connecting the starting endpoints of the multiple first line segments according to their order in the multiple line segments; the distance between the target point and the line segment of the at least one second line segment that is closest to the target point; and the angle between the two line segments obtained by connecting the two endpoints of the reference line segment to the target point.

[0034] In one possible design, when the image processing unit performs contour line extraction processing on the second target image and obtains multiple contour line segments in the second target image, it is specifically used to: perform edge contour detection on the second target image to obtain multiple contour points in the second target image; and obtain the multiple contour line segments based on the multiple contour points.

[0035] In one possible design, when the image processing unit obtains the multiple contour segments based on the multiple contour points, it is specifically used to: filter the multiple contour points to obtain multiple target contour points; perform line detection on the multiple target contour points to obtain the multiple contour segments; or perform line detection on the multiple contour points to obtain the multiple contour segments.

[0036] In one possible design, when the image processing unit filters the plurality of contour points, it specifically performs the following steps: dividing the image region of the second target image into a plurality of grids according to a set grid size; classifying the objects displayed in the plurality of grids using a set classifier; and deleting the contour points located in the target grid if there is an object in the target grid whose category is a set category.

[0037] Thirdly, this application provides an object detection device, including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the calculation program stored in the memory to implement the method described in the first aspect or any possible design of the first aspect.

[0038] Fourthly, this application provides an object detection device, including at least one processor and an interface; the interface is used to provide program instructions or data to the at least one processor; the at least one processor is used to execute the program instructions to implement the method described in the first aspect or any possible design of the first aspect.

[0039] Fifthly, this application provides a computer-readable storage medium storing a computer program that, when run on an object detection device, causes the object detection device to perform the method described in the first aspect or any possible design of the first aspect.

[0040] In a sixth aspect, this application provides a computer program product comprising a computer program or instructions that, when executed by an object detection device, implement the method described in the first aspect or any possible design of the first aspect.

[0041] In a seventh aspect, this application provides a chip system including at least one processor and an interface, the interface being used to provide program instructions or data to the at least one processor, the at least one processor being used to execute the program instructions to implement the method described in the first aspect or any possible design of the first aspect.

[0042] In one possible design, the chip system further includes a memory for storing program instructions and data.

[0043] In one possible design, the chip system consists of chips, or includes chips and other discrete devices.

[0044] For the beneficial effects described in aspects two through seven above, please refer to the description of the beneficial effects in aspect one above, which will not be repeated here. Attached Figure Description

[0045] Figure 1 A schematic diagram of an object detection method provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram illustrating contour line extraction processing of a second target image provided in an embodiment of this application;

[0047] Figure 3 A schematic diagram of an inverse perspective transformation provided in an embodiment of this application;

[0048] Figure 4 A schematic diagram illustrating the selection of line segments in a contour line segment in a second target image, provided as an embodiment of this application;

[0049] Figure 5 A schematic diagram illustrating the selection of a target line segment based on a reference line segment, provided for an embodiment of this application;

[0050] Figure 6 A schematic diagram of a target line segment provided in an embodiment of this application;

[0051] Figure 7 A schematic diagram illustrating the determination of the position of an object in a first target image, provided as an embodiment of this application;

[0052] Figure 8 A schematic diagram of an object detection method provided in an embodiment of this application;

[0053] Figure 9 A flowchart illustrating an object detection method provided in an embodiment of this application;

[0054] Figure 10 A flowchart illustrating an object detection method provided in an embodiment of this application;

[0055] Figure 11 A flowchart illustrating an object detection method provided in an embodiment of this application;

[0056] Figure 12 This is a schematic diagram of an object detection device provided in an embodiment of this application;

[0057] Figure 13 This is a schematic diagram of the structure of an object detection device provided in an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0059] For ease of understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0060] 1) Object Detection: Object detection refers to the process of locating multiple target objects in an image. In some implementation scenarios, object detection can not only determine the location of a target object in an image but also identify information such as the object's category. Generally, after identifying the location of a target object in the image, a rectangular box can be used to mark the object to inform the user; this rectangular box is also called a bounding box.

[0061] 2) Inverse Perspective Mapping (IPM): In images captured by a forward-facing camera, due to perspective effects, the outlines of objects that are normally parallel may appear to intersect. IPM transformation is a method used to eliminate this perspective effect. Currently, the basic principle can be abstracted and simplified by utilizing the transformation relationships between various coordinate systems during camera imaging, obtaining the correspondence between the world coordinate system and the image coordinate system, and then formulaically describing the coordinate relationships of inverse perspective transformation.

[0062] 3) Region of Interest (ROI): In image processing, the region in the image to be processed, defined by means of a box, circle, ellipse, irregular polygon, etc., is called the region of interest. In other words, the region of interest is a region of the image selected from the image.

[0063] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0064] The object detection method provided in this application can be applied to various target detection fields, such as security, image processing, video processing, land transportation, maritime transportation, drone monitoring, navigation, smart homes, and autonomous driving. The following explanation uses an autonomous driving scenario as an example.

[0065] In the field of autonomous driving, the driving safety of autonomous vehicles is paramount. Autonomous vehicles typically rely on autonomous driving systems or Advanced Driver Assistance Systems (ADAS) to promptly detect and assess current road conditions, effectively avoiding obstacles and ensuring driving safety. However, due to the complexity of road conditions and the diversity of road users, it is currently difficult to achieve high-precision detection of common obstacles using a single solution. Unreliable detection results can pose significant safety hazards to autonomous driving systems. For example, false detections can affect the decisions of ADAS systems, leading to frequent triggering of the Forward Collision Warning System (FCWS), resulting in reduced driving efficiency. Missed detections, on the other hand, can easily cause serious vehicle collisions.

[0066] Currently, existing obstacle detection methods in the field of autonomous driving mainly include detection methods based on monocular cameras, binocular cameras, radar, and lidar, and mainly include the following:

[0067] Method 1: Acquire the current frame image and historical frame images through the vehicle-mounted front-view camera, and reconstruct a dense 3D point cloud of the road surface using a triangulation algorithm to obtain the relative depth of the pixels in the image. This depth information is then used to further detect obstacles.

[0068] However, this method requires processing dozens of historical frame images to obtain a dense 3D point cloud of the road surface, which results in a large computational load. In addition, the triangulation algorithm has high requirements for feature matching accuracy, which means that the algorithm is quite sensitive to the accuracy of feature matching and has poor versatility.

[0069] Method 2: Collect images of the current surrounding environment using the vehicle's front-view camera, subtract the image from the previous frame in the RGB (red, green, blue) color gamut, and then use inverse perspective transformation to map the difference image obtained after subtraction onto the road surface. Finally, calculate the angle histogram of the mapped image to obtain the candidate locations of obstacles.

[0070] However, this method mainly relies on the pixel difference between two frames of images to obtain the candidate position of the obstacle. Therefore, it has the problem of false detection and false negative detection for static obstacles.

[0071] Method 3: Obtain a point cloud dataset by emitting lasers from the vehicle-mounted LiDAR to scan obstacles ahead, then fuse the image information collected by the vehicle-mounted camera, filter the point cloud dataset, and use the filtered point cloud data to detect obstacles with a certain height.

[0072] This method is based on the information fusion technology of LiDAR and camera. The implementation cost of the solution is relatively high. In addition, when the 3D point cloud is too dense, it is also easy to cause a certain degree of false detection.

[0073] In summary, most existing obstacle detection methods suffer from problems such as missed detections and false detections, resulting in low detection efficiency and poor versatility.

[0074] To address this problem, embodiments of this application provide an object detection method and apparatus for efficient and accurate object detection. This method can be applied to obstacle detection in intelligent driving fields such as autonomous driving and assisted driving, achieving relatively accurate obstacle detection under various road conditions.

[0075] For ease of explanation, the following description will use the object detection method provided in the embodiments of this application, executed by an object detection device, as an example.

[0076] The object detection method provided in this application can perform object detection by combining images acquired by an image acquisition device. This method can be applied to object detection devices with data processing capabilities. The image acquisition device can be integrated into the object detection device, or the image acquisition device can transmit images to the object detection device in real time. Furthermore, the image acquisition device can also detect images in saved multimedia files.

[0077] By way of example and not limitation, the object detection device may be a vehicle with data processing capabilities, or an in-vehicle device with data processing capabilities within a vehicle, or a sensor with image acquisition and processing capabilities. In-vehicle devices may include, but are not limited to, in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, in-vehicle units, electronic control units (ECUs), domain controllers (DCs), etc. The object detection device may also be other electronic devices with data processing capabilities, including but not limited to smart home devices (e.g., televisions), smart robots, mobile terminals (e.g., mobile phones, tablets), wearable devices (e.g., smartwatches), etc. The object detection device may also be a controller, chip, or other components within a smart device.

[0078] The image acquisition device described in this application embodiment may be a camera, monocular camera, binocular camera, near-infrared camera, video camera, cockpit camera, dashcam (i.e., recording terminal), reversing camera, etc., and no specific limitations are imposed in this application embodiment.

[0079] For example, in the field of intelligent driving such as autonomous driving and assisted driving, the shooting area of ​​the image acquisition device can be the external environment of the vehicle. For instance, when the vehicle is moving forward, the shooting area can be the area in front of the vehicle; when the vehicle is reversing, the shooting area can be the area behind the vehicle; when the image acquisition device is a 360-degree multi-angle camera, the shooting area can be the area within a 360-degree range around the vehicle, etc.

[0080] The object detection method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described below are only some embodiments of this application, and not all embodiments.

[0081] Figure 1 This is a schematic diagram of an object detection method provided in an embodiment of this application. See below. Figure 1 The object detection method provided in the embodiments of this application will be described in detail.

[0082] S101: The object detection device performs inverse perspective transformation on the first target image to obtain a second target image, wherein the first target image is an image of a scene containing at least one target object, and the second target image is an image obtained by mapping the first target image onto a target plane.

[0083] In this embodiment of the application, the object detection device can detect target objects contained in an image captured for a certain scene, determine the position of the target object in the image, and then determine the position of the target object in the scene based on the position of the target object in the image.

[0084] When performing object detection, the object detection device first acquires a first target image to be detected. The first target image is an image captured of a scene containing at least one target object. The object detection device can acquire the first target image by receiving an image sent by an image acquisition device or by receiving an image input by a user.

[0085] After the object detection device acquires the first target image, it performs an inverse perspective transformation on the first target image to obtain a second target image, wherein the second target image is the image obtained by mapping the first target image onto the target plane.

[0086] In some embodiments of this application, the first target image may be a complete image taken for a certain scene, or the first target image may be a partial image corresponding to a region of interest in a certain image. The target plane may be the same plane in which most target objects in the scene are located.

[0087] For example, in an autonomous driving scenario, the target plane can be an ideal ground plane. As another example, in a maritime traffic scenario, the target plane can be an ideal sea level.

[0088] Given that images acquired by image acquisition devices in most scenarios will suffer from a certain degree of geometric distortion due to perspective effects, object detection devices can first project the acquired first target image onto an ideal ground plane to obtain a second target image, thereby restoring the true geometric characteristics of the object in the first target image.

[0089] The object detection device can map the first target image onto an ideal ground plane using a homography matrix to obtain the second target image. The homography matrix can be used to describe the positional mapping relationship between the object in the world coordinate system and the pixel coordinate system.

[0090] It should be noted that after mapping the first target image taken for a certain scene onto an ideal ground plane, the resulting second target image can also be understood as an image of the scene from a top-down angle. Therefore, in the embodiments of this application, the second target image can also be understood as a bird's-eye view.

[0091] S102: The object detection device performs contour line extraction processing on the second target image to obtain multiple contour line segments in the second target image.

[0092] After obtaining the second target image corresponding to the first target image, the object detection device can first perform edge contour detection on the second target image to obtain multiple contour points in the second target image, and then obtain the multiple contour line segments based on the multiple contour points.

[0093] The object detection device can use an edge contour detection algorithm based on the Canny operator to extract contour points and obtain the multiple contour points; it can also use the Hough transform algorithm to perform line detection on the multiple contour points and obtain the multiple contour line segments.

[0094] As an optional implementation, when the object detection device determines the plurality of contour lines, it can directly perform straight line detection on the plurality of contour points to obtain the plurality of contour lines.

[0095] As another optional implementation, when the object detection device determines the plurality of contour segments, it can first filter the plurality of contour points obtained by edge contour detection to obtain a plurality of target contour points, and then perform line detection on the plurality of target contour points to obtain the plurality of contour segments.

[0096] The object detection device can divide the image area of ​​the second target image into multiple grids according to a set grid size, and then use a set classifier to classify the objects displayed in the multiple grids. If, after classification, there is an object in the target grid whose category is a set category, the contour points located in the target grid are deleted, thereby filtering the multiple contour points.

[0097] In some embodiments of this application, the set classifier may be a lightweight classifier, such as a classifier based on support vector machine (SVM) or a classifier based on extreme gradient boosting (XGBoost).

[0098] The object detection device can use the classifier described above to filter the contour points in the grid region where the object of the set category is located in the second target image, and delete the contour points in the grid region where the object of the set category is more likely to be located, thereby avoiding the interference of pixels in these regions on the object detection process.

[0099] For example, in the field of autonomous driving, when the target plane is set as an ideal ground plane, the set category can be a road surface. The object detection device can use a lightweight road surface classifier to classify the pixels in the second target image, determine the road surface pixels, and delete the contour points in the grid area corresponding to the road surface pixels, thereby filtering out the contour points in the grid that are more likely to be road surface areas and avoiding interference from the road surface areas in the object detection process.

[0100] For example, in a maritime traffic scenario, when the target plane is set to an ideal sea level, the set category can be sea level, and the corresponding contour point filtering method can refer to the above-mentioned ground contour point filtering method, which will not be repeated here.

[0101] For example, Figure 2 This is a schematic diagram illustrating contour line extraction processing of a second target image, provided as an embodiment of this application. Figure 2 The schematic diagram (a) shows the second target image obtained after performing inverse perspective transformation on the image captured by the camera. After performing the edge contour detection described above on the second target image, the result is as follows: Figure 2 The contour map shown in diagram (b) represents the contours of multiple objects in the second target image shown in diagram (a). By filtering the contour points in this contour map and performing line detection on the filtered target contour points, the following is obtained: Figure 2 The multiple line segments shown in diagram (c) are where the contour points of the road surface have been removed to avoid the influence of road surface elements on object detection. The multiple line segments shown in diagram (c) represent the contour line segments of the object extracted from the second target image shown in diagram (a), which are used to determine the position of the object in the second target image.

[0102] S103: The object detection device selects multiple line segments whose straight lines pass through a set target point from among the multiple contour line segments.

[0103] After performing an inverse perspective transformation on the image, the height contour lines of objects with a certain height in the image will be significantly elongated, and the contour lines of the objects at different heights will all be close to the same point after the transformation. Therefore, after the object detection device performs an inverse perspective transformation on the first target image, the contour line segments corresponding to the height of the objects in the second target image will all be close to the same feature point in the second target image.

[0104] For example, Figure 3 This is a schematic diagram illustrating an inverse perspective transformation provided in an embodiment of this application. Wherein, Figure 3 The schematic diagram (a) in the figure is a schematic diagram of a first target image. Figure 3In diagram (a), quadrilateral ABCD represents the outline of an object in the road in the first target image, where line segments AC and BD represent the outline of the object in the height direction. Figure 3 The second target image obtained after performing inverse perspective transformation on the first target image shown in schematic diagram (a) is as follows: Figure 3 As shown in diagram (b), quadrilateral EFGH is the result of inverse perspective transformation of quadrilateral ABCD in diagram (a), with points E, F, G, and H corresponding to points A, B, C, and D respectively. (Comparison) Figure 3 As shown in diagrams (a) and (b), the outline of an object in the height direction is significantly elongated after inverse perspective transformation, while the outline of the object in the width direction shows little change in length after inverse perspective transformation.

[0105] Based on the above characteristics, after the object detection device performs contour line extraction processing on the second target image to obtain the multiple contour lines, it can first select the line segment corresponding to the height contour of the object from the multiple contour lines, and then perform object detection based on the selected line segment.

[0106] In specific implementation, the object detection device can select multiple line segments whose straight lines pass through a set target point, or select multiple line segments whose straight lines pass through a set area centered on the target point, and then analyze and process the multiple line segments.

[0107] Optionally, the target point is a feature point in the second target image corresponding to the optical center of the image acquisition device. For example, the target point can be a feature point at the center of the bottom edge of the second target image.

[0108] For example, Figure 4 This is a schematic diagram illustrating the selection of line segments from contour line segments in a second target image, provided as an embodiment of this application. For example... Figure 4 As shown, line segments L1 to L8 are the eight contour line segments extracted from the second target image, and point P is the target point set in the second target image. Among the eight contour line segments, the lines containing line segments L1, L2, L6, and L7 all pass through the target point, i.e., point P. Therefore, line segments L1, L2, L6, and L7 can be selected from the eight contour line segments as multiple selected line segments, and object detection can be performed based on line segments L1, L2, L6, and L7.

[0109] S104: The object detection device divides the multiple line segments into at least one set of line segments according to their positions in the second target image. Different sets of line segments contain different line segments, and any set of line segments contains at least two line segments from the multiple line segments.

[0110] After selecting multiple line segments from multiple contour line segments, the object detection device divides the multiple line segments into a line segment set, and then performs object detection based on the divided line segment set.

[0111] Specifically, the object detection device can execute S104 through the following steps A1 to A2:

[0112] A1: The object detection device sorts the multiple line segments according to the positions of their starting endpoints in the second target image, wherein the starting endpoint of any line segment is the endpoint of the line segment that is closest to the target point.

[0113] A2: The object detection device, according to the sorting of the multiple line segments, sequentially selects a line segment that has not been assigned to the line segment set as a reference line segment, and performs the following division process B1 to B4 on the reference line segment:

[0114] B1: Take the line segments other than the reference line segment and the line segments that have been assigned to the line segment set as the line segments to be divided.

[0115] B2: Determine a target line segment from the line segments to be divided that has an angle less than a set threshold with respect to the reference line segment, wherein the target line segment contains at least one line segment.

[0116] B3: Calculate the error parameter that classifies the target line segment and the reference line segment into the same object contour.

[0117] B4: When the error parameter is less than or equal to the set threshold, the target line segment and the reference line segment are assigned to the same line segment set; when the error parameter is greater than the set threshold, the target line segment and the reference line segment are not assigned to the same line segment set, and the current assignment process ends.

[0118] The smaller the error parameter calculated above, the greater the likelihood that the reference line segment and the target line segment belong to the same object contour.

[0119] In some embodiments of this application, the object detection device can apply the above-mentioned line segment set partitioning method to the RANSAC algorithm model that uses the random sample consensus (RANSAC) algorithm, and use the model to complete the partitioning of the line segment set.

[0120] In some embodiments of this application, when the object detection device sorts the plurality of line segments, it can sort the plurality of line segments according to the clockwise or counterclockwise positional distribution order of the starting endpoints of the line segments in the second target image.

[0121] In some embodiments of this application, the angle between any target line segment and a reference line segment is the angle formed at a point where the target line segment and the reference line segment intersect. When the object detection device determines a target line segment among the line segments to be segmented whose angle with the reference line segment is less than a set threshold, it can connect the starting endpoint of the reference line segment to the target point to obtain a first auxiliary line segment. When the angle between the line segment obtained by connecting the starting endpoint of a line segment to the target point and the first auxiliary line segment is less than a set angle, it is considered that the angle between the line segment and the reference line segment is less than the set threshold.

[0122] For example, Figure 5 This is a schematic diagram illustrating the selection of a target line segment based on a reference line segment, as provided in an embodiment of this application. Figure 5 As shown, based on Figure 4 In the second target image shown, among the selected line segments, if the currently selected line segment L1 is used as the reference line segment, the angle setting threshold is... Then, connecting the starting endpoint A0 of line segment L1 with the target point yields the auxiliary line segment L. 01 , and the auxiliary line segment L 01 The included angle is within the range of the set threshold. Figure 5 The angle range corresponding to the sector shown. When selecting the target line segment corresponding to line segment L1, for Figure 5 The auxiliary line segment L is obtained by connecting the starting endpoint B0 of the line segment L2 shown to the target point. 02 Due to auxiliary line segment L 02 With auxiliary line segment L 01 The included angle δ between them is less than the set angle. Then line segment L2 can be used as a target line segment corresponding to line segment L1, and so on, all reference line segments corresponding to line segment L1 can be determined.

[0123] In this application, when the object detection device calculates the error parameter that classifies the target line segment and the reference line segment into the same object contour, it can do so in, but is not limited to, the following ways:

[0124] In a first embodiment, the object detection device can perform multiple calculations on the target line segment and the reference line segment to obtain multiple alternative error parameters. Each calculation process includes the following steps:

[0125] C1: Randomly select at least one line segment from the target line segments.

[0126] C2: The reference line segment and at least one line segment randomly selected from the target line segment are used as sample line segments.

[0127] C3: Calculate the target parameters corresponding to the sample line segments, and determine the alternative error parameters based on the target parameters; wherein, the target parameters include at least one parameter used to characterize the geometric distribution features of the sample line segments.

[0128] C4: Determine the minimum value among the plurality of alternative error parameters.

[0129] C5: The minimum value is used as an error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0130] In the second embodiment, the algorithm for calculating the error parameter that classifies the target line segment and the reference line segment into the same object contour can be implemented by a set data model. This set data model is used to calculate the probability value of multiple line segments belonging to the same object. The method described above for calculating probability values ​​can be used to calculate the error parameter that classifies the target line segment and the reference line segment into the same object contour.

[0131] Optionally, in the first embodiment described above, the sample line segment includes multiple first line segments, and the target parameters calculated by the object detection device include at least one of the following four parameters:

[0132] 1) A parameter with a value of 1 / (1+exp(-x)), which can be called a proportional constraint parameter in this embodiment of the application, where exp() is an exponential function with the natural constant as the base, x is the variance of the proportional parameters of the plurality of first line segments, the proportional parameter of any first line segment is the ratio between the length of the first line segment and the distance from the termination endpoint of the first line segment to the target point, and the termination endpoint of any first line segment is the endpoint of the first line segment that is farthest from the target point.

[0133] For example, Figure 6 This is a schematic diagram of a target line segment provided in an embodiment of this application. Figure 6 As shown in the figure, assuming that the sample line segments contain 3 first line segments, which are line segment L9, line segment L, etc., in order of line segment order, are line segment L9, line segment L9, etc. 10 and line segment L 11 In this context, line segment L9 also serves as a reference line segment, with point P as the target point. Taking line segment L9 as an example, if the starting endpoint of line segment L9 is A1 and the ending endpoint is A2, then the proportional parameter corresponding to line segment L9 is the ratio of the length of auxiliary line segment A1P to the length of auxiliary line segment A2P. 10 Line segment L11 Similarly, line segment L9 and line segment L are calculated using this method. 10 Line segment L 11 By calculating the variance of the three proportional parameters, we can obtain the proportional constraint parameters corresponding to the sample line segments.

[0134] 2) The variance of the texture direction values ​​of all points contained in at least one second line segment, wherein the at least one second line segment is a line segment obtained by sequentially connecting the starting endpoints of the plurality of first line segments according to the sorting of the plurality of first line segments in the plurality of line segments. This parameter can be referred to as a texture constraint parameter in this embodiment of the application.

[0135] The object detection device can use the wavelet (Gabor) operator to calculate the texture direction (texture orientation) value of each point on the second line segment.

[0136] Among them, the texture direction of any point on any second line segment is the texture direction of the corresponding pixel in the first target image or the second target image.

[0137] For example, such as Figure 6 As shown in the figure, assuming that the three first line segments contained in the sample line segments are ordered as line segment L9, line segment L9, line segment L9, and line segment L9 among multiple line segments selected from multiple contour line segments, respectively. 10 Line segment L 11 In this case, the starting endpoint of line segment L9 is A1, and the ending endpoint is A2. 10 The starting endpoint is B1, the ending endpoint is B2, and the line segment L... 11 The starting endpoint is C1, and the ending endpoint is C2. The object detection device can then connect points A1 and B1 to obtain auxiliary line segment L. 12 Connect points B1 and C1 to obtain auxiliary line segment L. 13 Line segment L 12 Line segment L 13 As the second line segment, then calculate line segment L separately. 12 Line segment L 13 The texture direction values ​​corresponding to each point are then used to obtain the line segment L. 12 and line segment L 13 The variance of the texture direction values ​​corresponding to all points is calculated to obtain the texture constraint parameters corresponding to the sample line segments.

[0138] 3) The distance between the target point and the target point of at least one second line segment. This parameter can be called a distance constraint parameter in this embodiment of the application. The at least one second line segment is a line segment obtained by connecting the starting endpoints of the plurality of first line segments in sequence according to the order of the plurality of first line segments in the plurality of line segments.

[0139] For example, such as Figure 6 As shown in the diagram, the object detection device connects points A1 and B1 to obtain auxiliary line segment L. 12 Connect points B1 and C1 to obtain auxiliary line segment L. 13 Then, determine line segment L. 12 Line segment L 13 The line segment closest to point P is line segment L. 12 Then line segment L 12 The distance to point P is determined as the distance constraint parameter for the sample line segment. Wherein, line segment L... 12 The distance to point P is the distance from point P to line segment L. 12 perpendicular segment L 03 The length.

[0140] 4) The angle between the two line segments obtained by connecting the two endpoints of the reference line segment to the target point is referred to as the angle constraint parameter in this embodiment of the application.

[0141] In some embodiments of this application, after the object detection device determines the above-mentioned target parameters, the following method can be used to determine the candidate error parameters based on the target parameters: if the object detection device determines that the target parameters contain only one parameter, then the parameter is used as the candidate error parameter; if the object detection device determines that the target parameters contain multiple parameters, then the multiple parameters are weighted and summed according to the weights corresponding to the multiple parameters in the target parameters to obtain the candidate error parameters.

[0142] The weights of the parameters included in the target parameters can be user-inputted, and the weight of the same parameter can be different in different scenarios.

[0143] S105: The object detection device performs the following steps for each set of line segments: projecting at least two line segments contained in the target set of line segments onto the first target image to obtain at least two projected line segments; determining the position of a target object in the first target image based on the at least two projected line segments, wherein the target set of line segments is each set of line segments.

[0144] In some embodiments of this application, after the object detection device divides the line segment set into the second target image in the above manner, it treats each line segment set as a target line segment set and performs the following processing: at least two line segments contained in the target line segment set are projected onto the first target image to obtain at least two projected line segments. If the at least two projected line segments cannot form a closed bounding box, the at least two projected line segments can form a closed bounding box by connecting the start endpoint / end endpoint of every two adjacent projected line segments, or by connecting the start endpoint of one projected line segment and the end endpoint of the other projected line segment in every two adjacent projected line segments. The bounding box is then used as the bounding box of a target object in the first target image, and the position enclosed by the bounding box is used as the position of the target object in the first target image.

[0145] In some embodiments of this application, when the object detection device processes a set of target line segments, if at least two line segments in the set of target line segments cannot form a closed bounding box, the object detection device may first connect the start endpoints / end endpoints of every two adjacent line segments, or connect the start endpoint of one line segment to the end endpoint of the other line segment, so that the at least two line segments can form a closed bounding box, and then project all the line segments that form the bounding box onto the first target image to obtain a bounding box corresponding to a target object.

[0146] For example, Figure 7 This is a schematic diagram illustrating how to determine the position of an object in a first target image, as provided in an embodiment of this application. Based on Figure 4 A schematic diagram of the second target image shown, in Figure 7 In the diagram, (a) shows the process from... Figure 4 The diagram illustrates the second target image after segmenting selected line segments L1, L2, L7, and L8 from the outline line segments. This second target image contains two line segment sets: a first set containing line segments L1 and L2, and a second set containing line segments L7 and L8. Since the geometrically contained line segments L1 and L2 in the first set cannot form a closed bounding box, the object detection device can connect the starting endpoints of line segments L1 and L2 to obtain line segment L... 14 And the terminating endpoints of line segment L1 and line segment L2 are used to obtain line segment L. 15 This makes line segment L1, line segment L2, and line segment L... 14 Line segment L 15 A closed bounding box is formed, and the second set of line segments is similar. Then, the object detection device projects the line segments of each bounding box in the second target image onto the first target image, obtaining... Figure 7The first target image shown in (b) is a schematic diagram, wherein each bounding box projected into the first target image can serve as a bounding box of a target object, used to mark the location of the target object.

[0147] It should be noted that the step numbers in the various embodiments described in this application are only an example of the execution flow and do not constitute a restriction on the order of execution of the steps. There is no strict execution order between steps that do not have a temporal dependency relationship with each other in this application.

[0148] In the above embodiments, the object detection device performs object detection based on the second target image obtained after projecting the first target image onto the target plane. This method can make full use of the affine geometric features of the object contour after the first target image undergoes inverse perspective transformation to perform object detection, thereby improving the accuracy of object detection. Furthermore, this solution can detect a variety of different objects, has a wide range of applicable scenarios, and is highly versatile.

[0149] The object detection method provided in the above embodiments of this application can be applied to the field of autonomous driving to detect obstacles on the road surface. For example, the on-board equipment of an autonomous vehicle can use this object detection method to detect objects in images captured by a front-mounted camera. The following description is based on specific embodiments. Hereinafter, an object detection device will still be used as an example to illustrate the implementation of the object detection method.

[0150] Figure 8 This is a schematic diagram of an object detection method provided in an embodiment of this application. Figure 8 As shown, after the object detection device acquires the current frame image to be detected, it can directly use the image as the first target image to perform inverse perspective transformation to obtain the second target image, i.e., the bird's-eye view. Alternatively, it can use the image of the region of interest in the image as the first target image to perform inverse perspective transformation to obtain the second target image. The device can then perform contour line extraction processing on the second target image to obtain the processed contour map, which contains multiple contour line segments.

[0151] In this context, on an ideal ground plane, obstacles have an outer contour line that closely follows the road surface. Therefore, based on the assumption of an ideal ground plane, the object detection device can map the first target image onto the ground plane according to the calibration parameters of the camera that captured the image, obtaining a second target image, and then extracting the outer contour line from the second target image. Optionally, after obtaining the contour image, the object detection device can perform noise processing on the contour image to remove road surface noise points.

[0152] Based on the IPM transform, contour segments with a certain height will be significantly elongated. For objects with a certain height on the road surface, the outer contour of their height will be close to the optical center. Therefore, in the following embodiments, a local RANSAC algorithm model based on optical center constraints can be used to iteratively sample contour segments and divide the segment set.

[0153] In the RANSAC algorithm model, the coordinates of a pixel in the image coordinate system can be represented as a point. n = (x, y), where point n Let x represent the nth endpoint, and y represent the x-coordinate and y-coordinate in the image coordinate system. A contour line segment in an image can be represented as a line in the image coordinate system. n =(point n1 ,point n2 ), where line n Point represents the nth line segment. n1 Point represents the starting endpoint of the nth line segment. n2 This represents the endpoint of the nth line segment. n is a positive integer. By sorting the line segments in the image clockwise according to their starting endpoints, we can obtain a sequence containing (line1, line2, ..., line...). n The model.

[0154] After the object detection device obtains multiple contour line segments in the second target image by performing line detection, when using the RANSAC algorithm model to perform line detection / clustering (i.e., dividing the line segment set) on the contour line segments in the second target image, based on the fact that the same height will be stretched to the same length after IPM transformation, contour ratio constraints can be added to the RANSAC algorithm model. That is, if it is determined that the length of some line segments is close to the ratio of the distance from their termination endpoint to the optical center, then these line segments are tended to belong to the same object.

[0155] Furthermore, the contour lines of an object's tail facing the camera will form textures with similar contour directions in the image after IPM transformation. Based on this, the RANSAC algorithm model can also use wavelet operators to detect the texture direction of line segments connecting object vertices in a bird's-eye view, i.e., the line connecting the starting points of two adjacent line segments. The determined texture direction parameters are used as reference information to determine whether different line segments belong to the same object.

[0156] After dividing the contour line segments into line segment sets using the RANSAC algorithm model, the object detection device backprojects the line segments of different line segment sets obtained into the first target image to obtain the final bounding boxes of different target objects.

[0157] This method enhances the outer contour features of objects on the road surface by performing IPM transformation on the captured image. Then, the RANSAC algorithm is used to extract contour segments with object features. The candidate model contours obtained after extracting and dividing the contour lines into sets (i.e., the line segments of each set) are back-projected onto the original image plane to obtain the final bounding box of the object. This method fully utilizes the geometric features of the object contours after IPM transformation to filter out all candidate obstacle contours on the road surface that possess these features, while simultaneously addressing the issues of obstacle diversity and distance, thus simplifying the algorithm process. This method is applicable to object detection on both structured and unstructured roads.

[0158] The execution flow of the above object detection method will be described below with reference to specific embodiments.

[0159] Example 1

[0160] Figure 9 This is a flowchart illustrating an object detection method provided in an embodiment of this application. The method is illustrated using the RANSAC algorithm model for segment partitioning as an example, involving the overall iterative process of the algorithm and the process of the RANSAC algorithm model filtering candidate segment sets. Specifically, the RANSAC algorithm model randomly samples local segment data, and based on the unique geometric features of the object contour, scores each sampled segment set using proportional constraints, texture constraints, distance constraints, and angle constraints to obtain the partitioned segment set.

[0161] like Figure 9 As shown, the process of this method includes:

[0162] S900: The image acquisition device acquires the current image frame.

[0163] The image acquisition device captures an image, obtains the current image frame, uses the current image frame as the first target image, and sends it to the object detection device. The image acquisition device can be a camera.

[0164] S901: The object detection device performs IPM transformation on the current image frame.

[0165] The object detection device performs IPM transformation on the current image frame, i.e., the first target image, using a homography matrix to convert the image plane of the first target image into an ideal ground plane, thereby obtaining the second target image, which is the bird's-eye view.

[0166] S902: The object detection device obtains a contour map.

[0167] The object detection device is based on the Canny operator and uses an edge contour detection algorithm to extract the initial contour points in the second target image, thus obtaining a contour map containing multiple contour points.

[0168] S903: The object detection device acquires an initial candidate set of line segments.

[0169] The object detection device performs line detection on the multiple contour points based on the Hough transform to obtain multiple contour line segments, and selects multiple line segments whose lines pass through a set target point from the multiple contour line segments, and uses the multiple line segments as an initial line segment candidate set.

[0170] S904: The object detection device traverses the line segments in the initial line segment candidate set in a counterclockwise direction using the RANSAC algorithm model.

[0171] The object detection device sorts the multiple line segments counterclockwise according to their starting endpoints. Here, the starting point is defined as the segment that is closest to the optical center, meaning the detection algorithm traverses the line segments counterclockwise.

[0172] The detection algorithm can be implemented using the RANSAC algorithm model.

[0173] S905: The object detection device filters the RANSAC algorithm model to divide the line segment set of the initial line segment candidate set.

[0174] In this step, the object detection device traverses each line segment counterclockwise using the RANSAC algorithm model, which includes the following steps:

[0175] Step 1: Take the current line segment being processed and its adjacent line segments within the set angle range as the local sampling set of the RANSAC algorithm model.

[0176] The current line segment is the reference line segment. The range of the left and right adjacent points of the current line segment, not exceeding a set angle, refers to the set angle range around the auxiliary line segment obtained by connecting the starting endpoint of the current line segment and the target point. For details, please refer to the description in the above embodiments. The line segments included in the sampling set correspond to the reference line segment and the target line segment in the above embodiments.

[0177] Step 2: The object detection device uses the local sample set as input to the RANSAC algorithm model and sets the number of iterations for the RANSAC algorithm model.

[0178] The number of iterations can be input by the user. The object detection device iteratively executes steps 3 to 4 according to the stated number of iterations.

[0179] Step 3: The object detection device randomly samples some line segments from the local sampling set to obtain multiple line segments containing the current line segment, and calculates the score of the RANSAC algorithm model. This final score is the error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0180] Among them, multiple line segments including the current line segment are sample line segments, and the score of the RANSAC algorithm model is the target parameter corresponding to the sample line segment.

[0181] When determining the score of the RANSAC algorithm model, the object detection device needs to calculate the following four scores:

[0182] 1) Proportional constraint score (proportional constraint parameters).

[0183] The object detection device first calculates the ratio of the length of each line segment to the maximum length from the endpoint of the line segment to the optical center. Since the input of the RANSAC algorithm model contains at least two line segments, there must exist at least two ratio values. Then, the corresponding ratio constraint score is calculated using the formula 1 / (1+exp(-var(disti / distio))), where exp() is an exponential function with the natural constant as the base, var() is the variance function, disti represents the length of the line segment itself, and distio represents the maximum distance from the endpoint of the line segment to the optical center, i.e., the distance from the endpoint of the line segment to the optical center.

[0184] 2) Texture constraint score (texture constraint parameters).

[0185] The object detection device connects the starting endpoints of each line segment in sequence according to the sorting of multiple line segments in the input RANSAC algorithm model to obtain at least one line segment. It then uses wavelet operators to find the texture orientation of all points contained in the at least one line segment and calculates the texture orientation variance of these points.

[0186] 3) Distance constraint score (distance constraint parameters).

[0187] The object detection device determines the shortest distance to the optical center among the line segments obtained by sequentially connecting the starting endpoints of each line segment according to the sorting of multiple line segments input to the RANSAC algorithm model.

[0188] 4) Angle constraint score (angle constraint parameters).

[0189] The object detection device calculates the angle formed by the starting and ending points of the current line segment to the optical center.

[0190] Step 4: Based on the algorithm's running scenario, determine the weights corresponding to different scores, apply the weights to each constraint score, and obtain the final score.

[0191] Step 5: After the iterations of Steps 3 and 4 are completed, the RANSAC algorithm model with the lowest score in the multiple iterations is taken as the final RANSAC algorithm model of the current line segment. If the score of this model is less than the set threshold, it means that a model exists; otherwise, it means that a model exists.

[0192] Step 6: If a model exists, remove the set of line segments in the current final RANSAC algorithm model from the initial set of line segments, that is, mark the line segments in the set as deleted.

[0193] Step 7: Repeat S904 to process the next line segment using the RANSAC algorithm model.

[0194] S906: The object detection device performs object detection based on the divided set of line segments.

[0195] The object detection device backprojects the set of line segments within all RANSAC algorithm models that meet the conditions onto the image plane of the first target image, and determines the position of at least one target object in the first target image based on the obtained projected line segments.

[0196] For specific implementation details of the above steps, please refer to [link / reference]. Figure 1 The implementation methods of the provided object detection method will not be repeated here.

[0197] In the above embodiments, by introducing camera calibration information, a homography matrix required for IPM transformation from the image plane to the ideal ground plane is constructed. This restores and enhances the geometric attributes of targets within the original image frame. The restoration mainly targets landmarks located on the ground plane, such as lane lines and sidewalks. For objects with height, their outlines will be significantly elongated. Therefore, under the new data representation, it is more conducive to using the RANSAC algorithm for model selection. This embodiment fully utilizes the significant affine geometric features formed by objects in the image after IPM transformation. Especially for tall and distant targets, the outlines formed by the objects in the bird's-eye view will approach or pass through the optical center, and the outlines corresponding to their height will be significantly elongated. Based on the above geometric constraints, a semi-random RANSAC algorithm is used, along with a custom outline model and scoring items, to iteratively select the outlines of target objects. At the same time, this method can perform ranging based on camera calibration information, integrating detection and ranging algorithms into the same iterative model. Therefore, it can simultaneously return distance and detection information, thereby simplifying the overall algorithm process.

[0198] Example 2

[0199] Figure 10This is a flowchart illustrating an object detection method provided in an embodiment of this application. The method is illustrated using the RANSAC algorithm model for segment partitioning as an example, involving the overall iterative process of the algorithm and the process of the RANSAC algorithm model filtering candidate segment sets. Specifically, the RANSAC algorithm model randomly samples local segment data, and based on the unique geometric features of the object contour, scores each sampled segment set using proportional constraints, texture constraints, distance constraints, and angle constraints to obtain the partitioned segment set.

[0200] like Figure 10 As shown, the process of this method includes:

[0201] S1000: The image acquisition device acquires the current image frame.

[0202] The image acquisition device captures an image, obtains the current image frame, uses the current image frame as the first target image, and sends it to the object detection device. The image acquisition device can be a camera.

[0203] S1001: The object detection device performs IPM transformation on the current image frame.

[0204] The object detection device performs IPM transformation on the current image frame, i.e., the first target image, using a homography matrix to convert the image plane of the first target image into an ideal ground plane, thereby obtaining the second target image, which is the bird's-eye view.

[0205] S1002: The object detection device obtains a contour map.

[0206] The object detection device is based on the Canny operator and uses an edge contour detection algorithm to extract the initial contour points in the second target image, thus obtaining a contour map containing multiple contour points.

[0207] S1003: The object detection device divides the contour map into a grid.

[0208] The object detection device divides the contour map, i.e. the second target image, into a grid of a set size.

[0209] S1004: The object detection device uses a classifier to delete the outline points of the road surface area.

[0210] The object detection device uses a lightweight road surface classifier to perform preliminary filtering on road surface pixels, deleting contour points in areas classified as road surfaces.

[0211] The image processed in the following steps is an image of the road surface region where contour points have been deleted, determined based on the classification results of the road surface classifier and the contour map.

[0212] S1005: The object detection device acquires an initial candidate set of line segments.

[0213] The object detection device performs line detection on the multiple contour points based on the Hough transform to obtain multiple contour line segments, and selects multiple line segments whose lines pass through a set target point from the multiple contour line segments, and uses the multiple line segments as an initial line segment candidate set.

[0214] S1006: The object detection device traverses the line segments in the initial line segment candidate set in a counterclockwise direction using the RANSAC algorithm model.

[0215] The object detection device sorts the multiple line segments counterclockwise according to their starting endpoints. Here, the starting point is defined as the segment that is closest to the optical center, meaning the detection algorithm traverses the line segments counterclockwise.

[0216] The detection algorithm can be implemented using the RANSAC algorithm model.

[0217] S1007: The object detection device filters the RANSAC algorithm model to divide the line segment set of the initial line segment candidate set.

[0218] In this step, the object detection device traverses each line segment counterclockwise using the RANSAC algorithm model, which includes the following steps:

[0219] Step 1: Take the current line segment being processed and its adjacent line segments within the set angle range as the local sampling set of the RANSAC algorithm model.

[0220] The current line segment is the reference line segment. The range of the left and right adjacent points of the current line segment, not exceeding a set angle, refers to the set angle range around the auxiliary line segment obtained by connecting the starting endpoint of the current line segment and the target point. For details, please refer to the description in the above embodiments. The line segments included in the sampling set correspond to the reference line segment and the target line segment in the above embodiments.

[0221] Step 2: The object detection device uses the local sample set as input to the RANSAC algorithm model and sets the number of iterations for the RANSAC algorithm model.

[0222] The number of iterations can be input by the user. The object detection device iteratively executes steps 3 to 4 according to the stated number of iterations.

[0223] Step 3: The object detection device randomly samples some line segments from the local sampling set to obtain multiple line segments containing the current line segment, and calculates the score of the RANSAC algorithm model. This final score is the error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0224] Among them, multiple line segments including the current line segment are sample line segments, and the score of the RANSAC algorithm model is the target parameter corresponding to the sample line segment.

[0225] When determining the score of the RANSAC algorithm model, the object detection device needs to calculate the following three scores first:

[0226] 1) Proportional constraint score (proportional constraint parameters).

[0227] The object detection device first calculates the ratio of the length of each line segment to the maximum length from the endpoint of the line segment to the optical center. Since the input of the RANSAC algorithm model contains at least two line segments, there must exist at least two ratio values. Then, the corresponding ratio constraint score is calculated using the formula 1 / (1+exp(-var(disti / distio))), where exp() is an exponential function with the natural constant as the base, var() is the variance function, disti represents the length of the line segment itself, and distio represents the maximum distance from the endpoint of the line segment to the optical center, i.e., the distance from the endpoint of the line segment to the optical center.

[0228] 2) Distance constraint score (distance constraint parameters).

[0229] The object detection device determines the shortest distance to the optical center among the line segments obtained by sequentially connecting the starting endpoints of each line segment according to the sorting of multiple line segments input to the RANSAC algorithm model.

[0230] 3) Angle constraint score (angle constraint parameters).

[0231] The object detection device calculates the angle formed by the starting and ending points of the current line segment to the optical center.

[0232] Step 4: Based on the algorithm's running scenario, determine the weights corresponding to different scores, apply the weights to each constraint score, and obtain the final score.

[0233] Step 5: After the iterations of Steps 3 and 4 are completed, the RANSAC algorithm model with the lowest score in the multiple iterations is taken as the final RANSAC algorithm model of the current line segment. If the score of the model is less than the set threshold, it means that a model exists; otherwise, it means that a model exists.

[0234] Step 6: If a model exists, remove the set of line segments in the current final RANSAC algorithm model from the initial set of line segments, and mark the line segments in that set as deleted.

[0235] Step 7: Repeat S1004 to process the next line segment using the RANSAC algorithm model.

[0236] S1008: The object detection device performs object detection based on the divided set of line segments.

[0237] The object detection device backprojects the set of line segments within all RANSAC algorithm models that meet the conditions onto the image plane of the first target image, and determines the position of at least one target object in the first target image based on the obtained projected line segments.

[0238] For specific implementation details of the above steps, please refer to [link / reference]. Figure 1 The implementation methods of the provided object detection method will not be repeated here.

[0239] In the above embodiments, by introducing camera calibration information, a homography matrix required for IPM transformation from the image plane to the ideal ground plane is constructed. This restores and enhances the geometric attributes of targets within the original image frame. The restoration mainly targets landmarks located on the ground plane, such as lane lines and sidewalks. For objects with height, their outlines will be significantly elongated. Therefore, under the new data representation, it is more conducive to using the RANSAC algorithm for model selection. This embodiment fully utilizes the significant affine geometric features formed by objects in the image after IPM transformation. Especially for tall and distant targets, the outlines formed by the objects in the bird's-eye view will approach or pass through the optical center, and the outlines corresponding to their height will be significantly elongated. Based on the above geometric constraints, a semi-random RANSAC algorithm is used, along with a custom outline model and scoring items, to iteratively select the outlines of target objects. At the same time, this method can perform ranging based on camera calibration information, integrating detection and ranging algorithms into the same iterative model. Therefore, it can simultaneously return distance and detection information, thereby simplifying the overall algorithm process. In addition, the algorithm uses a lightweight classifier to remove contour points in the road surface area, which improves the algorithm's efficiency.

[0240] Example 3

[0241] Figure 11 This is a flowchart illustrating an object detection method provided in an embodiment of this application. The method is illustrated using the RANSAC algorithm model for segment partitioning as an example, involving the overall iterative process of the algorithm and the process of the RANSAC algorithm model filtering candidate segment sets. Specifically, the RANSAC algorithm model randomly samples local segment data, and based on the unique geometric features of the object contour, scores each sampled segment set using proportional constraints, texture constraints, distance constraints, and angle constraints to obtain the partitioned segment set.

[0242] like Figure 11 As shown, the process of this method includes:

[0243] S1100: The image acquisition device acquires the current image frame.

[0244] The image acquisition device captures an image, obtains the current image frame, uses the current image frame as the first target image, and sends it to the object detection device. The image acquisition device can be a camera.

[0245] S1101: The object detection device performs IPM transformation on the current image frame.

[0246] The object detection device performs IPM transformation on the current image frame, i.e., the first target image, using a homography matrix to convert the image plane of the first target image into an ideal ground plane, thereby obtaining the second target image, which is the bird's-eye view.

[0247] S1102: The object detection device obtains a contour map.

[0248] The object detection device is based on the Canny operator and uses an edge contour detection algorithm to extract the initial contour points in the second target image, thus obtaining a contour map containing multiple contour points.

[0249] S1103: The object detection device divides the contour map into a grid.

[0250] The object detection device divides the contour map, i.e. the second target image, into a grid of a set size.

[0251] S1104: The object detection device uses a classifier to delete the outline points of the road surface area.

[0252] The object detection device uses a lightweight road surface classifier to perform preliminary filtering on road surface pixels, deleting contour points in areas classified as road surfaces.

[0253] The image processed in the following steps is an image of the road surface region where contour points have been deleted, determined based on the classification results of the road surface classifier and the contour map.

[0254] S1105: The object detection device acquires an initial candidate set of line segments.

[0255] The object detection device performs line detection on the multiple contour points based on the Hough transform to obtain multiple contour line segments, and selects multiple line segments whose lines pass through a set target point from the multiple contour line segments, and uses the multiple line segments as an initial line segment candidate set.

[0256] S1106: The object detection device traverses the line segments in the initial line segment candidate set in a counterclockwise direction using the RANSAC algorithm model.

[0257] The object detection device sorts the multiple line segments counterclockwise according to their starting endpoints. Here, the starting point is defined as the segment that is closest to the optical center, meaning the detection algorithm traverses the line segments counterclockwise.

[0258] The detection algorithm can be implemented using the RANSAC algorithm model.

[0259] S1107: The object detection device filters the RANSAC algorithm model to divide the line segment set of the initial line segment candidate set.

[0260] In this step, the object detection device traverses each line segment counterclockwise using the RANSAC algorithm model, which includes the following steps:

[0261] Step 1: Take the current line segment being processed and its adjacent line segments within the set angle range as the local sampling set of the RANSAC algorithm model.

[0262] The current line segment is the reference line segment. The range of the left and right adjacent points of the current line segment, not exceeding a set angle, refers to the set angle range around the auxiliary line segment obtained by connecting the starting endpoint of the current line segment and the target point. For details, please refer to the description in the above embodiments. The line segments included in the sampling set correspond to the reference line segment and the target line segment in the above embodiments.

[0263] Step 2: The object detection device uses the local sample set as input to the RANSAC algorithm model and sets the number of iterations for the RANSAC algorithm model.

[0264] The number of iterations can be input by the user. The object detection device iteratively executes steps 3 to 4 according to the stated number of iterations.

[0265] Step 3: The object detection device randomly samples some line segments from the local sampling set to obtain multiple line segments containing the current line segment, and calculates the score of the RANSAC algorithm model. This final score is the error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0266] Among them, multiple line segments including the current line segment are sample line segments, and the score of the RANSAC algorithm model is the target parameter corresponding to the sample line segment.

[0267] When determining the score of the RANSAC algorithm model, the object detection device needs to calculate the following four scores:

[0268] 1) Proportional constraint score (proportional constraint parameters).

[0269] The object detection device first calculates the ratio of the length of each line segment to the maximum length from the endpoint of the line segment to the optical center. Since the input of the RANSAC algorithm model contains at least two line segments, there must exist at least two ratio values. Then, the corresponding ratio constraint score is calculated using the formula 1 / (1+exp(-var(disti / distio))), where exp() is an exponential function with the natural constant as the base, var() is the variance function, disti represents the length of the line segment itself, and distio represents the maximum distance from the endpoint of the line segment to the optical center, i.e., the distance from the endpoint of the line segment to the optical center.

[0270] 2) Texture constraint score (texture constraint parameters).

[0271] The object detection device connects the starting endpoints of each line segment in sequence according to the sorting of multiple line segments in the input RANSAC algorithm model to obtain at least one line segment. It then uses wavelet operators to find the texture orientation of all points contained in the at least one line segment and calculates the texture orientation variance of these points.

[0272] When the object detection device determines the texture orientation of any endpoint, it uses the texture orientation of the pixel corresponding to that endpoint in the source image, i.e., the first target image, as the texture orientation of that endpoint.

[0273] 3) Distance constraint score (distance constraint parameters).

[0274] The object detection device determines the shortest distance to the optical center among the line segments obtained by sequentially connecting the starting endpoints of each line segment according to the sorting of multiple line segments input to the RANSAC algorithm model.

[0275] 4) Angle constraint score (angle constraint parameters).

[0276] The object detection device calculates the angle formed by the starting and ending points of the current line segment to the optical center.

[0277] Step 4: Based on the algorithm's running scenario, determine the weights corresponding to different scores, apply the weights to each constraint score, and obtain the final score.

[0278] Step 5: After the iterations of Steps 3 and 4 are completed, the RANSAC algorithm model with the lowest score in the multiple iterations is taken as the final RANSAC algorithm model of the current line segment. If the score of the model is less than the set threshold, it means that a model exists; otherwise, it means that a model exists.

[0279] Step 6: If a model exists, remove the set of line segments in the current final RANSAC algorithm model from the initial set of line segments, and mark the line segments in that set as deleted.

[0280] Step 7: Repeat S1104 to process the next line segment using the RANSAC algorithm model.

[0281] S1108: The object detection device performs object detection based on the divided set of line segments.

[0282] The object detection device backprojects the set of line segments within all RANSAC algorithm models that meet the conditions onto the image plane of the first target image, and determines the position of at least one target object in the first target image based on the obtained projected line segments.

[0283] In the above embodiments, by introducing camera calibration information, a homography matrix required for IPM transformation from the image plane to the ideal ground plane is constructed. This restores and enhances the geometric attributes of targets within the original image frame. The restoration mainly targets landmarks located on the ground plane, such as lane lines and sidewalks. For objects with height, their outlines will be significantly elongated. Therefore, under the new data representation, it is more conducive to using the RANSAC algorithm for model selection. This embodiment fully utilizes the significant affine geometric features formed by objects in the image after IPM transformation. Especially for tall and distant targets, the outlines formed by the objects in the bird's-eye view will approach or pass through the optical center, and the outlines corresponding to their height will be significantly elongated. Based on the above geometric constraints, a semi-random RANSAC algorithm is used, along with a custom outline model and scoring items, to iteratively select the outlines of target objects. At the same time, this method can perform ranging based on camera calibration information, integrating detection and ranging algorithms into the same iterative model. Therefore, it can simultaneously return distance and detection information, thereby simplifying the overall algorithm process. In addition, the algorithm uses a lightweight classifier to remove contour points in the road surface area, which improves the algorithm's efficiency.

[0284] Based on the above embodiments and the same concept, this application also provides an object detection device, such as... Figure 12 As shown, the object detection device 1200 may include: a perspective transformation unit 1201, an image processing unit 1202, a selection unit 1203, a division unit 1204, and a positioning unit 1205.

[0285] The perspective transformation unit 1201 is used to perform inverse perspective transformation processing on the first target image to obtain a second target image, wherein the first target image is an image of a scene containing at least one target object, and the second target image is an image obtained by mapping the first target image onto a target plane.

[0286] The image processing unit 1202 is used to perform contour line extraction processing on the second target image to obtain multiple contour line segments in the second target image.

[0287] The selection unit 1203 is used to select, among the plurality of contour line segments, multiple line segments whose straight lines pass through a set target point.

[0288] The segmentation unit 1204 is used to divide the multiple line segments into at least one line segment set according to the position of the multiple line segments in the second target image, wherein different line segment sets contain different line segments, and any line segment set contains at least two line segments from the multiple line segments; for each line segment set, the following steps are performed: projecting at least two line segments contained in the target line segment set onto the first target image to obtain at least two projected line segments.

[0289] The positioning unit 1205 is used to determine the position of a target object in the first target image based on the at least two projected line segments, wherein the target line segment set is each line segment set.

[0290] In one possible design, the object detection device 1200 further includes an acquisition unit 1206 for acquiring the first target image.

[0291] In one possible design, when the partitioning unit 1204 divides the multiple line segments into at least one line segment set based on their positions in the second target image, it is specifically used to: sort the multiple line segments according to the positions of their starting endpoints in the second target image, wherein the starting endpoint of any line segment is the endpoint of the line segment that is closest to the target point; according to the sorting of the multiple line segments, sequentially select a line segment that has not been divided into a line segment set as a reference line segment, and perform the following partitioning process on the reference line segment: partition the multiple line segments except for the one where the starting endpoint is located in the target image; The reference line segment and other line segments not included in the line segment set are designated as line segments to be divided. Among these line segments, target line segments are identified whose included angle with the reference line segment is less than a set threshold. Each target line segment contains at least one line segment. An error parameter is calculated to classify the target line segment and the reference line segment as belonging to the same object contour. When the error parameter is less than or equal to the set threshold, the target line segment and the reference line segment are assigned to the same line segment set. When the error parameter is greater than the set threshold, the target line segment and the reference line segment are not assigned to the same line segment set, and the current division process ends.

[0292] In one possible design, when the segmentation unit 1204 sorts the multiple line segments according to the positions of their starting endpoints in the second target image, it is specifically used to sort the multiple line segments according to the clockwise or counterclockwise positional distribution order of their starting endpoints in the second target image.

[0293] In one possible design, when the segmentation unit 1204 calculates the error parameter for classifying the target line segment and the reference line segment into the same object contour, it is specifically used to: calculate the error parameter for classifying the target line segment and the reference line segment into the same object contour according to a set data model, wherein the set data model is used to calculate the error that exists when classifying multiple line segments into the same object contour.

[0294] In one possible design, when the segmentation unit 1204 calculates the error parameter for classifying the target line segment and the reference line segment into the same object contour, it is specifically used to: perform multiple calculation processes on the target line segment and the reference line segment to obtain multiple candidate error parameters; wherein each calculation process includes the following steps: randomly selecting at least one line segment from the target line segment; using the reference line segment and the at least one line segment randomly selected from the target line segment as sample line segments; calculating the target parameter corresponding to the sample line segment, and determining the candidate error parameter based on the target parameter; wherein the target parameter includes at least one parameter for characterizing the geometric distribution characteristics of the sample line segment; determining the minimum value among the multiple candidate error parameters; and using the minimum value as the error parameter for classifying the target line segment and the reference line segment into the same object contour.

[0295] In one possible design, the sample line segment includes multiple first line segments; the target parameter includes at least one of the following: a parameter with a value of 1 / (1+exp(-x)), where exp() is an exponential function with the natural constant as the base, x is the variance of the proportional parameters of the multiple first line segments, the proportional parameter of any first line segment is the ratio between the length of the first line segment and the distance from the terminating endpoint of the first line segment to the target point, and the terminating endpoint of any first line segment is the endpoint of the first line segment that is farthest from the target point; the variance of the texture direction values ​​of all points contained in at least one second line segment, the at least one second line segment being a line segment obtained by sequentially connecting the starting endpoints of the multiple first line segments according to their order in the multiple line segments; the distance between the at least one second line segment closest to the target point and the target point, the at least one second line segment being a line segment obtained by sequentially connecting the starting endpoints of the multiple first line segments according to their order in the multiple line segments; and the angle between two line segments obtained by connecting the two endpoints of the reference line segment to the target point.

[0296] In one possible design, when the image processing unit 1202 performs contour line extraction processing on the second target image and obtains multiple contour line segments in the second target image, it is specifically used to: perform edge contour detection on the second target image to obtain multiple contour points in the second target image; and obtain the multiple contour line segments based on the multiple contour points.

[0297] In one possible design, when the image processing unit 1202 obtains the plurality of contour lines based on the plurality of contour points, it is specifically used to: filter the plurality of contour points to obtain a plurality of target contour points; perform line detection on the plurality of target contour points to obtain the plurality of contour lines; or perform line detection on the plurality of contour points to obtain the plurality of contour lines.

[0298] In one possible design, when the image processing unit 1202 filters the plurality of contour points, it is specifically used to: divide the image region of the second target image into a plurality of grids according to a set grid size; classify the objects displayed in the plurality of grids using a set classifier; and if there is an object in the target grid whose category is a set category, then delete the contour points located in the target grid.

[0299] The division of units in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into a single processor or controller, exist as separate physical units, or be integrated into a single unit. The integrated units described above can be implemented in hardware or as software functional units.

[0300] For example, the perspective transformation unit 1201, the image processing unit 1202, the selection unit 1203, the division unit 1204, and the positioning unit 1205 may be different processors or controllers; or the perspective transformation unit 1201, the image processing unit 1202, the selection unit 1203, the division unit 1204, and the positioning unit 1205 may be the same processor or controller; or, some functional units of the perspective transformation unit 1201, the image processing unit 1202, the selection unit 1203, the division unit 1204, and the positioning unit 1205 may be integrated into one processor or controller, and other functional units may be integrated into another processor or controller.

[0301] The processor or controller may be, for example, a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, etc., described in conjunction with the disclosure of this application. The processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0302] The acquisition unit 1206 can be an interface circuit of the object detection device 1200, used to receive data from other devices, such as receiving a first target image sent by an image acquisition device. When the object detection device is implemented as a chip, the acquisition unit 1206 can be an interface circuit of the chip used to receive data from or send data to other chips or devices.

[0303] Figure 12 One or more of the various units within can be implemented using software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

[0304] Based on the above embodiments and the same concept, this application also provides an object detection device for implementing the obstacle detection method provided in this application. Figure 13 As shown, the object detection device 1300 may include: one or more processors 1301, a memory 1302, and one or more computer programs (not shown in the figure).

[0305] In one implementation, the aforementioned devices can be coupled via one or more communication lines 1303. The memory 1302 stores one or more computer programs, each including instructions; the processor 1301 invokes the instructions stored in the memory 1302, causing the object detection device 1300 to execute the object detection method provided in this embodiment.

[0306] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0307] In the embodiments of this application, the memory can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. The memory in the embodiments of this application may also be circuitry or any other means capable of implementing storage functions.

[0308] In one implementation, the object detection device 1300 may further include a communication interface 1304 for communicating with other devices via a transmission medium. For example, the communication interface 1304 may receive a first target image from an image acquisition device. In this embodiment, the communication interface may be a transceiver, a circuit, a bus, a module, or other types of communication interface. In this embodiment, when the communication interface is a transceiver, the transceiver may include an independent receiver, an independent transmitter, or a transceiver with integrated transceiver functions, or an interface circuit.

[0309] In some embodiments of this application, the processor 1301, memory 1302, and communication interface 1304 can be interconnected via a communication line 1303. The communication line 1303 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication line 1303 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0310] The methods provided in this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., digital video disc (DVD), or semiconductor media (e.g., SSD), etc.

[0311] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An object detection method, characterized in that, include: The first target image is subjected to inverse perspective transformation to obtain the second target image, wherein the first target image is an image of a scene containing at least one target object, and the second target image is an image obtained by mapping the first target image onto a target plane; The second target image is subjected to contour line extraction processing to obtain multiple contour line segments in the second target image; Among the plurality of contour line segments, select a plurality of line segments whose straight lines pass through a set target point; wherein, the set target point is the feature point in the second target image corresponding to the optical center of the image acquisition device that acquires the first target image; Based on the positions of the multiple line segments in the second target image, the multiple line segments are divided into at least one set of line segments, wherein different set of line segments contain different line segments, and any set of line segments contains at least two line segments from the multiple line segments. For each set of line segments, the following steps are performed: at least two line segments contained in the target set of line segments are projected onto the first target image to obtain at least two projected line segments; the position of a target object in the first target image is determined based on the at least two projected line segments, wherein the target set of line segments is each set of line segments.

2. The method according to claim 1, characterized in that, Based on the positions of the multiple line segments in the second target image, the multiple line segments are divided into at least one set of line segments, including: Based on the positions of the starting endpoints of the multiple line segments in the second target image, the multiple line segments are sorted, wherein the starting endpoint of any line segment is the endpoint of the line segment that is closest to the target point among the two endpoints of the line segment; According to the order of the multiple line segments, select one line segment that has not been assigned to the line segment set as a reference line segment, and perform the following division process on the reference line segment: The line segments other than the reference line segment and the line segments that have been assigned to the line segment set are taken as the line segments to be divided. In the line segments to be divided, a target line segment is determined whose angle with the reference line segment is less than a set threshold, wherein the target line segment contains at least one line segment; Calculate the error parameter that classifies the target line segment and the reference line segment into the same object contour; When the error parameter is less than or equal to the set threshold, the target line segment and the reference line segment are assigned to the same line segment set. When the error parameter is greater than the set threshold, the target line segment and the reference line segment are not assigned to the same line segment set, and the current division process ends.

3. The method according to claim 2, characterized in that, Based on the positions of the starting endpoints of the multiple line segments in the second target image, the multiple line segments are sorted, including: The multiple line segments are sorted according to their clockwise or counterclockwise positional distribution from the starting endpoints in the second target image.

4. The method according to claim 2 or 3, characterized in that, Calculating the error parameters that classify the target line segment and the reference line segment into the same object contour includes: Based on the set data model, the error parameter for classifying the target line segment and the reference line segment into the same object contour is calculated, wherein the set data model is used to calculate the error that exists when classifying multiple line segments into the same object contour.

5. The method according to claim 2 or 3, characterized in that, Calculating the error parameters that classify the target line segment and the reference line segment into the same object contour includes: Multiple calculation processes are performed on the target line segment and the reference line segment to obtain multiple candidate error parameters. Each calculation process includes the following steps: randomly selecting at least one line segment from the target line segment; using the reference line segment and the at least one randomly selected line segment from the target line segment as sample line segments; calculating the target parameter corresponding to the sample line segment; and determining the candidate error parameters based on the target parameter. The target parameter includes at least one parameter characterizing the geometric distribution features of the sample line segment. Determine the minimum value among the plurality of alternative error parameters; The minimum value is used as an error parameter for classifying the target line segment and the reference line segment into the same object contour.

6. The method according to claim 5, characterized in that, The sample line segment includes multiple first line segments; the target parameter includes at least one of the following: Values The parameters are: exp() is an exponential function with the natural constant as the base, x is the variance of the proportional parameters of the plurality of first line segments, the proportional parameter of any first line segment is the ratio between the length of the first line segment and the distance from the terminating endpoint of the first line segment to the target point, and the terminating endpoint of any first line segment is the endpoint of the first line segment that is farthest from the target point. The variance of the texture direction values ​​of all points contained in at least one second line segment, wherein the at least one second line segment is a line segment obtained by sequentially connecting the starting endpoints of the plurality of first line segments according to the sorting of the plurality of first line segments in the plurality of line segments; The distance between the target point and the at least one second line segment that is closest to the target point; The angle between the two line segments obtained by connecting the two endpoints of the reference line segment to the target point.

7. The method according to any one of claims 1 to 6, characterized in that, The second target image is subjected to contour line extraction processing to obtain multiple contour line segments in the second target image, including: Edge contour detection is performed on the second target image to obtain multiple contour points in the second target image; Based on the multiple contour points, the multiple contour line segments are obtained.

8. The method according to claim 7, characterized in that, Based on the plurality of contour points, the plurality of contour line segments are obtained, including: Filtering the multiple contour points yields multiple target contour points; line detection is performed on the multiple target contour points to obtain multiple contour line segments; or Line detection is performed on the multiple contour points to obtain the multiple contour line segments.

9. The method according to claim 8, characterized in that, Filtering the plurality of contour points includes: The image region of the second target image is divided into multiple grids according to the set grid size; The objects displayed in the multiple grids are classified using a set classifier; If, among the multiple grids, there exists an object displayed in the target grid whose category is a set category, then the contour points located in the target grid will be deleted.

10. An object detection device, characterized in that, include: A perspective transformation unit is used to perform inverse perspective transformation on a first target image to obtain a second target image, wherein the first target image is an image of a scene containing at least one target object, and the second target image is an image obtained by mapping the first target image onto a target plane; An image processing unit is used to perform contour line extraction processing on the second target image to obtain multiple contour line segments in the second target image; The selection unit is used to select, from the plurality of contour line segments, multiple line segments whose straight lines pass through a set target point; wherein, the set target point is the feature point in the second target image corresponding to the optical center of the image acquisition device that acquires the first target image; The segmentation unit is used to divide the multiple line segments into at least one set of line segments according to their positions in the second target image, wherein different set of line segments contain different line segments, and any set of line segments contains at least two line segments from the multiple line segments. A positioning unit is configured to perform the following steps for each set of line segments: projecting at least two line segments contained in the target set of line segments onto the first target image to obtain at least two projected line segments; determining the position of a target object in the first target image based on the at least two projected line segments, wherein the target set of line segments is each set of line segments.

11. The object detection device according to claim 10, characterized in that, When the segmentation unit divides the multiple line segments into at least one set of line segments based on their positions in the second target image, it is specifically used for: Based on the positions of the starting endpoints of the multiple line segments in the second target image, the multiple line segments are sorted, wherein the starting endpoint of any line segment is the endpoint of the line segment that is closest to the target point among the two endpoints of the line segment; According to the order of the multiple line segments, select one line segment that has not been assigned to the line segment set as a reference line segment, and perform the following division process on the reference line segment: The line segments other than the reference line segment and the line segments that have been assigned to the line segment set are taken as the line segments to be divided. In the line segments to be divided, a target line segment is determined whose angle with the reference line segment is less than a set threshold, wherein the target line segment contains at least one line segment; Calculate the error parameter that classifies the target line segment and the reference line segment into the same object contour; When the probability value is less than or equal to a set threshold, the target line segment and the reference line segment are assigned to the same line segment set. When the error parameter is greater than the set threshold, the target line segment and the reference line segment are not assigned to the same line segment set, and the current division process ends.

12. The object detection device according to claim 11, characterized in that, When the segmentation unit sorts the multiple line segments according to the positions of their starting endpoints in the second target image, it is specifically used for: The multiple line segments are sorted according to their clockwise or counterclockwise positional distribution from the starting endpoints in the second target image.

13. The object detection device according to claim 11 or 12, characterized in that, When the segmentation unit calculates the error parameter that classifies the target line segment and the reference line segment into the same object contour, it is specifically used for: Based on the set data model, the error parameter for classifying the target line segment and the reference line segment into the same object contour is calculated, wherein the set data model is used to calculate the error that exists when classifying multiple line segments into the same object contour.

14. The object detection device according to claim 11 or 12, characterized in that, When the segmentation unit calculates the error parameter that classifies the target line segment and the reference line segment into the same object contour, it is specifically used for: Multiple calculation processes are performed on the target line segment and the reference line segment to obtain multiple candidate error parameters. Each calculation process includes the following steps: randomly selecting at least one line segment from the target line segment; using the reference line segment and the at least one randomly selected line segment from the target line segment as sample line segments; calculating the target parameter corresponding to the sample line segment; and determining the candidate error parameters based on the target parameter. The target parameter includes at least one parameter characterizing the geometric distribution features of the sample line segment. Determine the minimum value among the plurality of alternative error parameters; The minimum value is used as an error parameter for classifying the target line segment and the reference line segment into the same object contour.

15. The object detection device according to claim 14, characterized in that, The sample line segment includes multiple first line segments; the target parameter includes at least one of the following: Values The parameters are: exp() is an exponential function with the natural constant as the base, x is the variance of the proportional parameters of the plurality of first line segments, the proportional parameter of any first line segment is the ratio between the length of the first line segment and the distance from the terminating endpoint of the first line segment to the target point, and the terminating endpoint of any first line segment is the endpoint of the first line segment that is farthest from the target point. The variance of the texture direction values ​​of all points contained in at least one second line segment, wherein the at least one second line segment is a line segment obtained by sequentially connecting the starting endpoints of the plurality of first line segments according to the sorting of the plurality of first line segments in the plurality of line segments; The distance between the target point and the at least one second line segment that is closest to the target point; The angle between the two line segments obtained by connecting the two endpoints of the reference line segment to the target point.

16. The object detection device according to any one of claims 10 to 15, characterized in that, The image processing unit performs contour line extraction processing on the second target image. When multiple contour line segments are obtained in the second target image, it is specifically used for: Edge contour detection is performed on the second target image to obtain multiple contour points in the second target image; Based on the multiple contour points, the multiple contour line segments are obtained.

17. The object detection device according to claim 16, characterized in that, When the image processing unit obtains the multiple contour line segments based on the multiple contour points, it is specifically used for: Filtering the multiple contour points yields multiple target contour points; line detection is performed on the multiple target contour points to obtain multiple contour line segments; or Line detection is performed on the multiple contour points to obtain the multiple contour line segments.

18. The object detection device according to claim 17, characterized in that, When the image processing unit filters the plurality of contour points, it is specifically used for: The image region of the second target image is divided into multiple grids according to the set grid size; The objects displayed in the multiple grids are classified using a set classifier; If, among the multiple grids, there exists an object displayed in the target grid whose category is a set category, then the contour points located in the target grid will be deleted.

19. An object detection device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is used to execute the calculation program stored in the memory to implement the method as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on the object detection device, causes the object detection device to perform the method as described in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Artificial mark detection method applied to augmented reality

    CN105740818A

  • Lane line detection method and system

    CN108256445A