Method, device and storage medium for detecting edge of vehicle drivable area

Abstract

The present invention discloses a method, device and storage medium for detecting the regional edge of a vehicle drivable area. The method comprises: when a vehicle is traveling in the driving area, acquiring frame images in the driving area at different time points to obtain a frame image set, wherein the frame image set at least comprises: a first frame image acquired at the current moment and at least one second frame image acquired at a historical moment; detecting edge objects in the first frame image and the second frame image respectively, identifying a first edge object set in the first frame image and a second edge object set in the second frame image; matching the edge objects in the first edge object set and the second edge object set in sequence; and determining whether the edge object detected in the first frame image is a target edge based on the matching result. The present invention solves the problem that in the process of detecting the regional edge in the vehicle driving area, the detection result of the previous frame is inaccurately predicted.

Description

Technical Field

[0001] The present invention relates to the field of intelligent automobile environment perception, and in particular to a method, a device and a storage medium for detecting an edge of a vehicle drivable area. Background Art

[0002] With the rapid development of science and technology, assisted driving and intelligent driving have become the hottest emerging technology industries nowadays. Among them, lane keeping and drivable area detection are important components: lane keeping refers to the use of cameras to identify the marking lines of the driving lanes when the vehicle is driving. If the vehicle approaches the identified marking lines and may leave the driving lane, it will remind the driver to take safety measures; the drivable area detection generally processes the laser radar point cloud data to detect various boundaries of the road (such as curbs, bushes, fences, etc.) to achieve the purpose of obtaining the drivable area of ​​the car. The trend of lane lines and road boundaries is always consistent with the trend of the road, and they are generally smooth and continuous. However, whether it is an active sensor such as radar or a passive sensor such as a camera, it will be affected by the environment, resulting in inaccurate detection. For example, when a vehicle passes by, the lane lines and road boundaries will be blocked, and the detection results will be interrupted; for example, when the lighting conditions are uneven, the lane line detection results are also prone to inaccuracy, intermittent or even false detection.

[0003] At present, in order to solve the problem of inaccurate detection caused by the influence of the environment on automobile sensors, most of the related technologies are based on the vehicle's motion state to establish a model, and then use Kalman filtering, extended Kalman filtering or particle filtering and other methods to predict the detection of the current frame based on the detection results of the previous frame, and update the detection results of the current frame. For example, according to the Kalman filter, curve or target point tracking is first performed according to the vehicle's uniform acceleration ideal motion model, and the boundary curve coefficient or the (x, y, z) coordinates of the selected target point are used as observation values, and then the state equation and observation equation are established, the Kalman gain is calculated, and finally the boundary state is updated. Among them, the process of establishing the state equation needs to find the change relationship of the same observation value in the previous and next frames according to the vehicle motion model, which requires complex model calculation. When calculating the Kalman gain, it is necessary to continuously adjust the process noise Q and observation noise R according to the test results to find the optimal state between the prediction result and the detection result. From the above process, it can be seen that the existing optimization methods are relatively complex whether it is from the early model establishment or the later parameter adjustment.

[0004] As can be seen from the above, the existing technology has the following problems: (1) The vehicle motion models established by the existing technology are mostly ideal uniform acceleration motion models, but in actual scenarios, it is difficult to ensure that the vehicle's motion is uniformly accelerated. (2) The existing technology uses the detection results of the previous frame to predict the detection results of the current frame, which is inaccurate. (3) The existing filtering algorithm requires information other than the detection results to optimize the detection results. For example, the Kalman filter requires information such as the vehicle's driving speed, acceleration and driving direction to establish the state transfer matrix and the drive matrix. (4) Most of the existing algorithms require a certain fixed relationship between the left and right boundaries as a constraint. (5) The parameters that need to be debugged in the existing filtering algorithm are noise terms. This parameter debugging is relatively difficult and does not have much physical or data basis.

[0005] With respect to the problem in the prior art of inaccurately predicting the detection result of a current frame using the detection result of a previous frame during the detection of the edge of a region within a vehicle driving area, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a method, device and storage medium for detecting the edge of a region in which a vehicle can travel, so as to at least solve the technical problem that in the process of detecting the edge of a region in a vehicle traveling area, the detection result of a previous frame is used to predict the detection result of a current frame inaccurately.

[0007] According to one aspect of an embodiment of the present invention, a method for detecting an area edge of a vehicle drivable area is provided, comprising: when a vehicle is traveling in the driving area, acquiring frame images in the driving area at different time points to obtain a frame image set, wherein the frame image set at least includes: a first frame image acquired at a current moment and at least one second frame image acquired at a historical moment; detecting edge objects in the first frame image and the second frame image respectively, identifying a first edge object set in the first frame image and a second edge object set in the second frame image; matching edge objects in the first edge object set and the second edge object set in sequence, wherein the edge objects in the two edge object sets are used to characterize objects that are located in the driving area and have area edge attributes; based on the matching result, determining whether the edge object detected in the first frame image is a target edge, wherein the target edge is used to characterize an area edge that conforms to the current road edge trend and is located in the driving area when the vehicle is traveling at the current moment.

[0008] Optionally, in the process of detecting edge objects in the frame image set, edge objects detected in the first frame image are accumulated and the total number of edge objects detected in the first frame image is obtained by statistics.

[0009] Optionally, when the first edge object set is not empty and the second edge object set is empty, edge objects detected in the first frame image are accumulated and the detected edge objects are saved in the edge object set of the second frame image.

[0010] Optionally, when the second edge object set is not empty, edge objects in the first edge object set and the second edge object set are matched in sequence, including: obtaining a coefficient distance matrix of edge objects in the two edge object sets, wherein the coefficient distance matrix records the distance between the curve coefficients of the i-th edge object in the first edge object set and the j-th edge object in the second edge object set; based on the coefficient distance matrix, determining the successfully matched edge objects in the two edge object sets, wherein the successfully matched edge objects represent the same area edge displayed in different frame images.

[0011] Optionally, if the distance between the curve coefficients of the ith edge object and the jth edge object is less than or equal to a preset threshold, there are successfully matched edge objects in the two edge object sets, wherein the ith edge object and the jth edge object are one-to-one matched edge objects.

[0012] Optionally, based on the matching result, determining whether the edge object detected in the first frame image is a target edge includes: obtaining a curve coefficient difference between the ith edge object and the jth edge object; judging whether the curve coefficient difference is within a preset range; if the curve coefficient difference is within the preset range, determining that the ith edge object detected in the first frame image is a target edge; otherwise, at least re-detecting the edge object of a previous frame image adjacent to the first frame image, and updating the curve coefficient of the previous frame image based on the result of the re-detection.

[0013] Optionally, if there is an edge object that fails to match in the first edge object set, the edge object is a newly detected edge object, and / or if there is an edge object that fails to match in the second edge object set, the edge object is a disappeared edge object.

[0014] Optionally, after determining whether the edge object detected in the first frame image is a target edge based on the matching result, the method further includes: counting the number of matching failures of any edge object in the second edge object set; if the number of matching failures is greater than or equal to a set value, removing the edge object from the second edge object set.

[0015] According to another aspect of an embodiment of the present invention, there is also provided an area edge detection device for a vehicle drivable area, comprising: an acquisition module, used to acquire frame images in the driving area at different time points during the driving of the vehicle in the driving area, and obtain a frame image set, wherein the frame image set at least includes: a first frame image acquired at a current moment and at least one second frame image acquired at a historical moment; a detection module, used to detect edge objects in the first frame image and the second frame image respectively, and identify a first edge object set in the first frame image and a second edge object set in the second frame image; a matching module, used to match edge objects in the first edge object set and the second edge object set in sequence, wherein the edge objects in the two edge object sets are used to characterize objects that are located in the driving area and have area edge attributes; an analysis module, used to determine whether the edge object detected in the first frame image is a target edge based on the matching result, wherein the target edge is used to characterize the area edge that conforms to the current road edge trend and is located in the driving area when the vehicle is driving at the current moment.

[0016] Optionally, the device for detecting an area edge of a vehicle drivable area further includes: a first statistical module, configured to accumulate edge objects detected in a first frame image during the process of detecting edge objects in a frame image set, and obtain a total number of edge objects detected in the first frame image by counting.

[0017] Optionally, the area edge detection device for the vehicle drivable area also includes: an accumulation module, used to accumulate edge objects detected in the first frame image when the first edge object set is not empty and the second edge object set is empty; and a storage module, used to save the detected edge objects in the edge object set of the second frame image.

[0018] Optionally, when the second edge object set is not empty, the matching module includes: a first acquisition unit, used to obtain a coefficient distance matrix of edge objects in the two edge object sets, wherein the coefficient distance matrix records the distance between the curve coefficients of the i-th edge object in the first edge object set and the j-th edge object in the second edge object set; and a processing unit, used to determine the successfully matched edge objects in the two edge object sets based on the coefficient distance matrix, wherein the successfully matched edge objects represent the same area edge displayed in different frame images.

[0019] Optionally, the matching module also includes: an analysis unit, which is used to determine that if the distance between the curve coefficients of the i-th edge object and the j-th edge object is less than or equal to a preset threshold, there is a successfully matched edge object in the two edge object sets, wherein the i-th edge object and the j-th edge object are one-to-one matched edge objects.

[0020] Optionally, the matching module also includes: a second acquisition unit, used to obtain the curve coefficient difference between the i-th edge object and the j-th edge object; a judgment unit, used to judge whether the curve coefficient difference is within a preset range; a determination unit, used to determine that the i-th edge object detected in the first frame image is the target edge if the curve coefficient difference is within the preset range; and a re-detection unit, used to re-detect at least the edge object of the previous frame image adjacent to the first frame image if the curve coefficient difference is not within the preset range, and update the curve coefficient of the previous frame image based on the result of the re-detection.

[0021] Optionally, the area edge detection device for the vehicle drivable area also includes: a first determination module, used to determine that if there is an edge object that fails to match in the first edge object set, the edge object is a newly detected edge object; and a second determination module, used to determine that if there is an edge object that fails to match in the second edge object set, the edge object is a disappeared edge object.

[0022] Optionally, the area edge detection device for the vehicle drivable area also includes: a second statistical module, used to count the number of matching failures of any edge object in the second edge object set; and a removal module, used to remove the edge object from the second edge object set if the number of matching failures is greater than or equal to a set value.

[0023] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the area edge detection method of the vehicle drivable area in the above embodiment.

[0024] According to another aspect of an embodiment of the present invention, a processor is further provided. The processor is used to run a program. When the program is run, the method for detecting an edge of a vehicle drivable area in the above embodiment is executed.

[0025] In an embodiment of the present invention, during the process of a vehicle traveling in a driving area, frame images in the driving area at different time points are obtained to obtain a frame image set, wherein the frame image set at least includes: a first frame image collected at the current moment and at least one second frame image collected at a historical moment; edge objects in the first frame image and the second frame image are detected respectively, and a first edge object set in the first frame image and a second edge object set in the second frame image are identified; edge objects in the first edge object set and the second edge object set are matched in sequence, wherein the edge objects in the two edge object sets are used to characterize objects located in the driving area and having regional edge attributes; based on the matching result, it is determined whether the edge object detected in the first frame image is a target edge, wherein the target edge is used to characterize the regional edge that is in line with the current road edge trend and located in the driving area when the vehicle is traveling at the current moment. It is easy to notice that through the above-mentioned filtering method for optimizing the edge detection of the drivable area, the purpose of filtering and filling the false detection is achieved, thereby achieving the technical effect of correcting the abnormal value of the detection result, thereby solving the technical problem of inaccurate prediction of the current frame detection result using the previous frame detection result in the process of detecting the regional edge in the vehicle driving area. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a flow chart of a method for detecting an edge of a vehicle drivable area according to an embodiment of the present invention;

[0028] Figure 2 is a flow chart of an optional method for detecting an edge of a vehicle drivable area according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of an optional lane line and curb state near a bus stop according to an embodiment of the present invention;

[0030] Figure 4 4 is a schematic diagram of a device for detecting an edge of a vehicle drivable area according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Example 1

[0034] According to an embodiment of the present invention, a method for detecting an area edge of a vehicle drivable area is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] Figure 1 is a method for detecting the edge of a vehicle drivable area according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0036] Step S102, when the vehicle is traveling in the driving area, frame images in the driving area at different time points are obtained to obtain a frame image set, wherein the frame image set includes at least: a first frame image collected at the current moment and at least one second frame image collected at a historical moment.

[0037] The frame image in the above step S102 can be obtained through a video acquisition device, for example, through a front-view camera on a vehicle, but is not limited thereto.

[0038] Step S104 , detecting edge objects in the first frame image and the second frame image respectively, and identifying a first edge object set in the first frame image and a second edge object set in the second frame image.

[0039] The edge object in the above step S104 may be a lane line, a curb, a fence, a green belt, a water barrier, or a temporary construction barrier, but is not limited thereto.

[0040] In the above step S104, the edge objects of the first frame image are detected, which may include the curve coefficient, the number of occurrences and the number of disappearances of the edge objects, but are not limited thereto. The data detected here can be stored to form a first edge object set, that is, the current frame detection target set. The edge objects of the second frame image are detected, which may include the curve coefficient, the number of occurrences and the number of disappearances of the edge objects, but are not limited thereto. The data detected here can be stored to form a second edge object set, that is, the past frame detection target set, that is, the set of all edge objects detected before the current frame. The number of edge objects detected can be one or more.

[0041] It should be noted that in order to smooth the data and effectively remove noise during edge detection, a curve model is generally used to fit the edge. The more commonly used curve model is the polynomial model. Considering the actual application, in Figure 3 In the areas such as the bus station shown, a quartic polynomial curve can better fit the edge trend, so the embodiment of the present invention uses a quartic polynomial as an example.

[0042] It should be noted here that the quartic polynomial model can be:

[0043] y=c0+c1x+c2x 2 +c3x 3 +c4x 4

[0044] Among them, c0, c1, c2, c3, c4 are curve coefficients, x is the horizontal coordinate of the edge object, and y is the vertical coordinate of the edge object.

[0045] Step S106 : sequentially matching edge objects in the first edge object set and the second edge object set, wherein the edge objects in the two edge object sets are used to represent objects that are located in the driving area and have area edge attributes.

[0046] The object with the region edge attribute in the above step S106 represents the region edge, which may be a lane line, a curb, a fence, a green belt, a water barrier, or a temporary construction barrier, but is not limited thereto.

[0047] In the above step S106, the edge objects in the first edge object set and the second edge object set can be matched one by one according to the curve coefficients stored in the two sets, including: calculating the coefficient distance matrix of the edge objects in the two sets, and determining the successfully matched edge objects in the two edge object sets based on the coefficient distance matrix, but not limited to this.

[0048] Step S108, based on the matching result, determining whether the edge object detected in the first frame image is a target edge, wherein the target edge is used to represent the edge of a region that conforms to the current road edge trend and is located within the driving area when the vehicle is driving at the current moment.

[0049] In the above step S108, for the matched edge object, the curve coefficient difference stored in the first edge object set and the second edge object set can be obtained, and based on the comparison result between the coefficient difference and the set threshold, it is determined whether it is an edge object on the current vehicle driving road.

[0050] In an embodiment of the present invention, during the process of a vehicle traveling in a driving area, frame images in the driving area at different time points are obtained to obtain a frame image set, wherein the frame image set at least includes: a first frame image collected at the current moment and at least one second frame image collected at a historical moment; edge objects in the first frame image and the second frame image are detected respectively, and a first edge object set in the first frame image and a second edge object set in the second frame image are identified; edge objects in the first edge object set and the second edge object set are matched in sequence, wherein the edge objects in the two edge object sets are used to characterize objects located in the driving area and having regional edge attributes; based on the matching result, it is determined whether the edge object detected in the first frame image is a target edge, wherein the target edge is used to characterize the regional edge that is in line with the current road edge trend and located in the driving area when the vehicle is traveling at the current moment. It is easy to notice that through the above-mentioned filtering method for optimizing the edge detection of the drivable area, the purpose of filtering and filling the false detection is achieved, thereby achieving the technical effect of correcting the abnormal value of the detection result, thereby solving the technical problem of inaccurate prediction of the current frame detection result using the previous frame detection result in the process of detecting the regional edge in the vehicle driving area.

[0051] Optionally, in the process of detecting edge objects in the frame image set, edge objects detected in the first frame image are accumulated and the total number of edge objects detected in the first frame image is obtained by statistics.

[0052] The total number of edge objects detected in the first frame image in the above steps can be obtained by a counter. Every time an edge object is detected in a frame image, the value of the counter is increased by 1 and saved.

[0053] Optionally, when the first edge object set is not empty and the second edge object set is empty, edge objects detected in the first frame image are accumulated and the detected edge objects are saved in the edge object set of the second frame image.

[0054] In an optional embodiment, if the second edge object set is empty, it means that the currently detected frame image is the first frame, then the number of occurrences of the edge object in the current frame image detection result can be increased by 1, and the edge object in the current frame image detection result can be stored in the second edge object set.

[0055] Optionally, when the second edge object set is not empty, edge objects in the first edge object set and the second edge object set are matched in sequence, including: obtaining a coefficient distance matrix of edge objects in the two edge object sets, wherein the coefficient distance matrix records the distance between the curve coefficients of the i-th edge object in the first edge object set and the j-th edge object in the second edge object set; based on the coefficient distance matrix, determining the successfully matched edge objects in the two edge object sets, wherein the successfully matched edge objects represent the same area edge displayed in different frame images.

[0056] In an optional embodiment, when the second edge object set is not empty, matching edge objects in the first edge object set and the second edge object set one by one according to curve coefficients stored in the two sets includes:

[0057] Calculate the coefficient distance matrix Dis of edge objects in two sets:

[0058]

[0059] Where Dis(i, j) represents the distance between the curve coefficients of the i-th edge object in the second edge object set and the j-th edge object in the first edge object set, c 0i , c 1i , c 2i , c 3i , c 4i is the curve coefficient of the ith edge object in the second edge object set, c 0j , c 1j , c 2j , c 3j , c 4j is the curve coefficient of the jth edge object in the first edge object set.

[0060] Based on the coefficient distance matrix, the successfully matched edge objects in the two edge object sets are determined.

[0061] Optionally, if the distance between the curve coefficients of the ith edge object and the jth edge object is less than or equal to a preset threshold, there are successfully matched edge objects in the two edge object sets, wherein the ith edge object and the jth edge object are one-to-one matched edge objects.

[0062] The preset threshold in the above steps can be set according to user needs. Since the curve difference between adjacent frame images is generally small, the threshold can be set at about 0.5, but is not limited thereto.

[0063] In an optional embodiment, when Dis(i, j) is less than a set threshold, two edge objects match, that is, they are the same boundary in the real scene, such as lane lines, curbs, fences, etc. It should be noted that at this time, the same object may match multiple objects. Users can use the KM algorithm or other equivalent algorithms to solve this problem, which will not be described here.

[0064] Optionally, based on the matching result, determining whether the edge object detected in the first frame image is a target edge includes: obtaining a curve coefficient difference between the ith edge object and the jth edge object; judging whether the curve coefficient difference is within a preset range; if the curve coefficient difference is within the preset range, determining that the ith edge object detected in the first frame image is a target edge; otherwise, at least re-detecting the edge object of a previous frame image adjacent to the first frame image, and updating the curve coefficient of the previous frame image based on the result of the re-detection.

[0065] The preset range value in the above step can be obtained by testing the coefficient difference distribution statistics of adjacent frames of the same curve through multiple frames of data.

[0066] In an optional embodiment, for the matched edge object, obtain its curve coefficient in the current frame detection target set and the past frame detection target set, and find the difference between the two. If the absolute value of the coefficient difference is within the set threshold range, it means that there is no problem with the detection result of the current frame, and it is smooth and continuous with the previous frame. The curve coefficient of the current frame is used to update the curve coefficient of the past frame, and the number of edge object occurrences is increased by 1. If the coefficient difference is too large, it means that there may be a problem with the current frame detection. At this time, it is necessary to compare the current frame detection result with the detection results of the previous two frames retained. If the difference between the curve coefficient of the current frame and the curve coefficient of the previous two frames is within the preset range, it means that the detection result of the current frame is correct, and the previous frame updated the wrong prediction value. The curve coefficient of the current frame is used to update the curve coefficient of the past frame, and the corresponding number of edge object occurrences is updated; otherwise, keep the previous curve coefficient unchanged, do not update, and do not update the number of edge object occurrences.

[0067] Optionally, if there is an edge object that fails to match in the first edge object set, the edge object is a newly detected edge object, and / or if there is an edge object that fails to match in the second edge object set, the edge object is a disappeared edge object.

[0068] In an optional embodiment, for an edge object that is not matched, if the edge object is an edge object detected in the current frame, it means that a new edge is detected, a new edge object is created, and its number of occurrences is set to 1, and then stored in the past frame detection target set; if the object is an edge object detected in the past frame detection target set, but is not detected in the current frame detection target set, then the number of disappearances of the edge object is increased by 1.

[0069] Optionally, after determining whether the edge object detected in the first frame image is a target edge based on the matching result, the method further includes: counting the number of matching failures of any edge object in the second edge object set; if the number of matching failures is greater than or equal to a set value, removing the edge object from the second edge object set.

[0070] In an optional embodiment, the number of times an edge object disappears is read from a counter. If the number of times an edge object disappears is greater than a set threshold (such as 3), the edge object is removed from the past frame detection target set. That is, if an edge object disappears more than three times, the edge object is considered to have disappeared and the edge object is no longer displayed. The remaining edge objects whose number of appearances is greater than the set threshold are filtered edge objects and participate in subsequent processing of the current frame detection target set, such as control planning of smart driving vehicles.

[0071] Combine the following Figure 2 and Figure 3 As shown, the functions achieved by the solution of the present application in an application scenario where vehicles are traveling on a two-lane road near a bus stop are described in detail.

[0072] Step S201, obtaining a current frame detection target set N.

[0073] In the above step S201, edge objects in the current frame image captured by the camera device installed on the car are detected, including the curve coefficient, the number of appearances and the number of disappearances of the edge objects, but not limited thereto, and the detected data are stored to form a first edge object set, that is, the current frame detection target set. Among them, the edge objects are targets in the embodiment of the present invention, such as lane lines, curbs, fences, green belts, water barriers, temporary construction barriers, etc., but not limited thereto.

[0074] For example, Figure 3As shown in the figure, the car driving on the right detects the lane line, curb and bus stop in the photo when detecting the current moment photo. The curve parameters of the curves that the lane line, curb and bus stop shape conform to are identified, and the number of times the lane line, curb and bus stop appear is increased by 1 using a counter, and the number of times it disappears is not updated. Here, the curve coefficients, number of appearances and number of disappearances of the lane line, curb and bus stop are stored as the current frame detection target set N.

[0075] Step S202, detecting whether the past frame detection target set M is empty.

[0076] If it is empty, execute S203; if it is not empty, execute S204.

[0077] Step S203, if the past frame detection target set M is empty, the current frame is the first frame, and the current frame target N is stored in the past frame detection target set M, and the number of occurrences of each target is increased by 1.

[0078] In step S203 , if the car on the right has just started and there is no data in the past frame detection target set M, the data detected in the current frame detection target set N is stored in the past frame detection target set M.

[0079] Step S204: if the target set M detected in the past frame is not empty, and the current frame is not the first frame, the targets in the sets M and N, namely the edge objects, are matched one-to-one.

[0080] In the above step S204, if the car has been driving for a period of time and data is stored in the past frame detection target set M, the coefficient distance matrix of the targets in the two sets can be calculated based on the curve coefficients of the targets stored in the current frame detection target set N and the past frame detection target set M, and the targets in the two sets can be matched one by one by comparing the coefficient distance with the set threshold.

[0081] For example, calculate the coefficient distance between the curve coefficient of the lane line in the current frame detection target set N and the curve coefficient of the lane line in the past frame detection target set M, the curve coefficient of the curb, and the curve coefficient of the bus stop. If the calculated coefficient distances are 0.2, 0.6, and 1.2, respectively, compare them with the set threshold of 0.5, where the coefficient distance 0.2<0.5, then the lane line has been matched, proving that it still exists and is still the same lane line. The judgment method for curbs and bus stops is the same as above.

[0082] Step S205: for the unmatched past frame target mi, increase the target mark disappearance times by 1, and execute S212.

[0083] In the above step S205, when the car leaves the vicinity of the bus stop, if the bus stop is not matched, the disappearance count of the bus stop is increased by 1, and the disappearance count of the bus stop in the past frame detection target set M is updated.

[0084] Step S206, for the unmatched current frame target nj, a new target is created, and the number of occurrences of the above target is set to 1, stored in the past frame detection target set M, and S212 is executed.

[0085] In the above step S206, when the car enters the bus stop, if the bus stop is not matched, a new edge object bus stop is created, and the number of occurrences of the bus stop is set to 1 and stored in the past frame detection target set M.

[0086] Step S207, for the matched targets mi and nj, that is, the targets mi and nj are matched successfully, it is determined whether the target curve coefficient difference satisfies condition 1.

[0087] Condition 1 in the above steps may be that the absolute value of the coefficient difference is within a preset range, wherein the preset range can be obtained by testing the coefficient difference distribution statistics of adjacent frames of the same curve through multiple frames of data.

[0088] In the above step S207, for the matched lane line, the difference between its curve coefficient in the current frame detection result set N and the curve coefficient in the past frame detection target set M is calculated to determine whether the coefficient difference is within a preset range.

[0089] If condition 1 is met, execute S208; if condition 1 is not met, execute S209.

[0090] Step S208, if the target curve coefficient difference is within the preset range, the curve coefficient of the previous frame detection target set mi is updated using the curve coefficient of the current frame detection target set nj, and the number of occurrences of the mi target is increased by 1, and S212 is executed.

[0091] In the above step S208, if the lane line coefficient difference is within the preset range, the curve coefficient of the lane line in the past frame detection target set is updated using the curve parameters of the lane line in the current frame detection target set, the number of occurrences of the lane line is increased by 1, and the number of occurrences of the lane line in the past frame detection target set M is updated.

[0092] Step S209, if the target curve coefficient difference is not within the preset range, determine whether the coefficient difference between the curve coefficient of nj in the current frame detection target set and the curve coefficient retained in mi in the past frame detection target set (the detection coefficient detected in the previous two frames) meets condition 1.

[0093] In the above step S209, if the lane line coefficient difference is not within the preset range, the difference between the curve coefficient of the lane line in the current frame detection result set N and the curve coefficient detected in the first two frames in the past frame detection target set M is calculated to determine whether the coefficient difference is within the preset range.

[0094] If condition 1 is met, execute S210; if condition 1 is not met, execute S211.

[0095] Step S210, if condition 1 is met, the curve coefficient of nj in the current frame detection target set is used to update the curve coefficient of mi in the past frame detection target set, and the number of target appearances is adjusted, and S212 is executed.

[0096] In the above step S210, if the lane line coefficient difference is within the preset range, it means that the detection of the previous frame is wrong. The curve parameters of the lane line in the current frame detection target set N are used to update the curve coefficient of the lane line in the previous frame detection target set M, and the number of occurrences of the lane line is increased by 1.

[0097] Step S211, if condition 1 is not met, the previous curve coefficient is kept unchanged, no update is made, and the target occurrence number is not updated.

[0098] In the above step S210, if the lane line coefficient difference is not within the preset range, it means that the current frame detection fails, and the curve coefficient of the lane line of the previous frame in the past frame detection target set M remains unchanged, and the number of occurrences of the lane line is not updated.

[0099] Step S212, updating the target mi of the target set detected in the past frame.

[0100] Step S213, determine whether the past frame detection target set M meets condition 2, and the target that meets the condition is the current final detection target.

[0101] Condition 2 of S213 in the above step can be that the number of target disappearances is greater than the set threshold, and the number of target appearances is greater than the set threshold. The number of target disappearances is read from the counter. If the number of target disappearances is greater than the set threshold (such as 3), the target is removed from the past frame detection target set, that is, if the target disappears more than three times, it is considered that the target disappears and the target is no longer displayed. The remaining targets whose number of target appearances is greater than the set threshold (such as 3) are filtered targets and participate in the subsequent processing of the current frame detection target set, such as the control planning of smart driving cars.

[0102] In the above step S213, if the number of times the bus stop disappears is 5, which is greater than the set threshold value 3, the bus stop is removed from the past frame detection target set N, and it is considered that the bus stop disappears. If the number of times the remaining edge objects such as lane lines and curbs appear is 6 and 7 respectively, which is greater than the set threshold value 3, that is, the lane lines and curbs still exist, they continue to participate in subsequent processing.

[0103] Example 2

[0104] According to an embodiment of the present invention, a device for detecting the edge of a vehicle-drivable area is also provided. The device may be a device for detecting the edge of a vehicle-drivable area in the above embodiment. The specific implementation scheme and application scenario are the same as those in the above embodiment and will not be described in detail herein.

[0105] Figure 4 is a schematic diagram of a device for detecting an edge of a vehicle drivable area according to an embodiment of the present invention. Figure 4 As shown, the device comprises:

[0106] The acquisition module 42 is used to acquire frame images within the driving area at different time points when the vehicle is driving within the driving area, and obtain a frame image set, wherein the frame image set at least includes: a first frame image acquired at the current moment and at least one second frame image acquired at a historical moment.

[0107] The detection module 44 is used to detect edge objects in the first frame image and the second frame image respectively, and identify a first edge object set in the first frame image and a second edge object set in the second frame image.

[0108] The matching module 46 is used to sequentially match the edge objects in the first edge object set and the second edge object set, wherein the edge objects in the two edge object sets are used to represent objects located in the driving area and having area edge attributes.

[0109] Optionally, when the second edge object set is not empty, the matching module includes: a first acquisition unit, used to obtain a coefficient distance matrix of edge objects in the two edge object sets, wherein the coefficient distance matrix records the distance between the curve coefficients of the i-th edge object in the first edge object set and the j-th edge object in the second edge object set; and a processing unit, used to determine the successfully matched edge objects in the two edge object sets based on the coefficient distance matrix, wherein the successfully matched edge objects represent the same area edge displayed in different frame images.

[0110] Optionally, the matching module also includes: an analysis unit, which is used to determine that if the distance between the curve coefficients of the i-th edge object and the j-th edge object is less than or equal to a preset threshold, there is a successfully matched edge object in the two edge object sets, wherein the i-th edge object and the j-th edge object are one-to-one matched edge objects.

[0111] Optionally, the matching module also includes: a second acquisition unit, used to obtain the curve coefficient difference between the i-th edge object and the j-th edge object; a judgment unit, used to judge whether the curve coefficient difference is within a preset range; a determination unit, used to determine that the i-th edge object detected in the first frame image is the target edge if the curve coefficient difference is within the preset range; and a re-detection unit, used to re-detect at least the edge object of the previous frame image adjacent to the first frame image if the curve coefficient difference is not within the preset range, and update the curve coefficient of the previous frame image based on the result of the re-detection.

[0112] The analysis module 48 is used to determine whether the edge object detected in the first frame image is a target edge based on the matching result, wherein the target edge is used to represent the edge of the area that conforms to the current road edge trend and is located in the driving area when the vehicle is driving at the current moment.

[0113] Optionally, the device for detecting an area edge of a vehicle drivable area further includes: a first statistical module, configured to accumulate edge objects detected in a first frame image during the process of detecting edge objects in a frame image set, and obtain a total number of edge objects detected in the first frame image by counting.

[0114] Optionally, the area edge detection device for the vehicle drivable area also includes: an accumulation module, used to accumulate edge objects detected in the first frame image when the first edge object set is not empty and the second edge object set is empty; and a storage module, used to save the detected edge objects in the edge object set of the second frame image.

[0115] Optionally, the area edge detection device for the vehicle drivable area also includes: a first determination module, used to determine that if there is an edge object that fails to match in the first edge object set, the edge object is a newly detected edge object; and a second determination module, used to determine that if there is an edge object that fails to match in the second edge object set, the edge object is a disappeared edge object.

[0116] Optionally, the area edge detection device for the vehicle drivable area also includes: a second statistical module, used to count the number of matching failures of any edge object in the second edge object set; and a removal module, used to remove the edge object from the second edge set if the number of matching failures is greater than or equal to a set value.

[0117] Example 3

[0118] According to an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the area edge detection method of the vehicle drivable area in the above embodiment.

[0119] Example 4

[0120] According to an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein when the program is run, the method for detecting an edge of a vehicle drivable area in the above embodiment is executed.

[0121] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0122] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0123] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0125] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0127] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting the edge of a vehicle drivable area, characterized in that: include: When the vehicle is traveling in the driving area, frame images in the driving area at different time points are acquired to obtain a frame image set, wherein the frame image set at least includes: a first frame image acquired at a current moment and at least one second frame image acquired at a historical moment; Respectively detecting edge objects in the first frame image and the second frame image, and identifying a first edge object set in the first frame image and a second edge object set in the second frame image; Matching edge objects in the first edge object set and the second edge object set in sequence according to curve coefficients of edge objects in the first edge object set and curve coefficients of edge objects in the second edge object set, wherein edge objects in both edge object sets are used to represent objects located in the driving area and having area edge attributes; Based on the matching result, determining whether the edge object detected in the first frame image is a target edge, wherein the target edge is used to represent the edge of a region that conforms to the current road edge trend and is located in the driving area when the vehicle is driving at the current moment; The step of determining whether the edge object detected in the first frame of image is a target edge based on the matching result includes: In response to the existence of a successfully matched edge object, it is determined whether the edge object detected in the first frame image is a target edge according to curve coefficients of the successfully matched edge object in the first edge object set and the second edge object set.

2. The method according to claim 1, characterized in that The method further includes: in the process of detecting edge objects in the frame image set, accumulating edge objects detected in the first frame image, and obtaining the total number of edge objects detected in the first frame image by counting.

3. The method according to claim 2, characterized in that The method further includes: when the first edge object set is not empty and the second edge object set is empty, accumulating edge objects detected in the first frame image, and saving the detected edge objects into the edge object set of the second frame image.

4. The method according to claim 2, characterized in that: When the second edge object set is not empty, sequentially matching edge objects in the first edge object set and the second edge object set includes: Obtaining a coefficient distance matrix of edge objects in two edge object sets, wherein the coefficient distance matrix records the distance between the curve coefficients of the i-th edge object in the first edge object set and the j-th edge object in the second edge object set; Based on the coefficient distance matrix, successfully matched edge objects in two edge object sets are determined, wherein the successfully matched edge objects represent the same region edge displayed in different frame images.

5. The method according to claim 4, characterized in that If the distance between the curve coefficients of the i-th edge object and the j-th edge object is less than or equal to a preset threshold, there is a successfully matched edge object in the two edge object sets, wherein the i-th edge object and the j-th edge object are one-to-one matched edge objects.

6. The method according to claim 5, characterized in that Determining whether the edge object detected in the first frame of image is a target edge based on the matching result includes: Obtaining a curve coefficient difference between the ith edge object and the jth edge object; Determine whether the curve coefficient difference is within a preset range; If the curve coefficient difference is within the preset range, determining that the i-th edge object detected in the first frame of image is the target edge; Otherwise, at least the edge object of the previous frame image adjacent to the first frame image is re-detected, and the curve coefficient of the previous frame image is updated based on the result of the re-detection.

7. The method according to claim 4 or 5, characterized in that: If there is an edge object that fails to match in the first edge object set, the edge object is a newly detected edge object, and / or if there is an edge object that fails to match in the second edge object set, the edge object is a disappeared edge object.

8. The method according to claim 7, characterized in that After determining whether the edge object detected in the first frame image is a target edge based on the matching result, the method further includes: Counting the number of times that any edge object in the second edge object set fails to match; If the number of matching failures is greater than or equal to a set value, the edge object is removed from the second edge object set.

9. A device for detecting the edge of a vehicle drivable area, characterized in that: include: An acquisition module is used to acquire frame images in the driving area at different time points during the driving of the vehicle in the driving area to obtain a frame image set, wherein the frame image set at least includes: a first frame image acquired at a current moment and at least one second frame image acquired at a historical moment; A detection module, configured to detect edge objects in the first frame image and the second frame image respectively, and identify a first set of edge objects in the first frame image and a second set of edge objects in the second frame image; a matching module, configured to sequentially match edge objects in the first edge object set and the second edge object set according to curve coefficients of edge objects in the first edge object set and curve coefficients of edge objects in the second edge object set, wherein the edge objects in the two edge object sets are used to represent objects located in the driving area and having area edge attributes; an analysis module, configured to determine whether the edge object detected in the first frame of image is a target edge based on the matching result, wherein the target edge is used to represent an edge of a region that conforms to the current road edge trend and is located in the driving area when the vehicle is driving at the current moment; The analysis module is further configured to determine, in response to the presence of a successfully matched edge object, whether the edge object detected in the first frame image is a target edge according to curve coefficients of the successfully matched edge object in the first edge object set and the second edge object set respectively.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the area edge detection method for the vehicle drivable area according to any one of claims 1 to 8.

Images