A collision warning method, device, electronic device and storage medium

Abstract

The present disclosure provides a method, device, electronic device and storage medium for collision warning, wherein the method includes: obtaining a target image captured by a camera device currently set on a vehicle; performing target detection on the target image; performing curve fitting on the lane lines in the target image based on the position information of each pixel point detected belonging to the lane line, and obtaining a fitting curve representing the position of each lane line in the target image; and issuing a collision warning based on the position information of the detected target object in the target image and the fitting curve of each lane line. The embodiment of the present disclosure can determine the fitting curve of the lane line by using curve fitting, so that the relationship between the target object and each lane line can be determined based on the position information of the target object in the target image and each fitting curve, thereby realizing a vehicle collision warning at the same lane level with high accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a collision warning method, device, electronic device and storage medium. Background Art

[0002] One of the important warning functions in the current advanced driver assistance systems (ADAS) is forward collision warning. The forward collision warning function avoids potential collision risks by sensing the vehicle in front and calculating the collision time. Forward collision warning can improve driving safety, especially when the driver is distracted, fatigued, or sleepy.

[0003] Related vehicle forward collision warning methods mainly calculate the relative collision time based on the relative position and speed relationship of the vehicles to determine whether to issue an alarm. However, for vehicles ahead on complex or special lanes, missed alarms and false alarms often occur. For example, for narrow lanes in the countryside, it is easy to judge the vehicle in the opposite lane next to it as the vehicle in the lane, resulting in false alarms. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a collision warning method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a collision warning method, the method comprising:

[0006] Acquire a target image captured by a camera device currently installed on the vehicle;

[0007] Performing target detection on the target image;

[0008] Based on the position information of each pixel point on the lane line detected, curve fitting is performed on the lane line in the target image to obtain a fitting curve representing the position of each lane line in the target image;

[0009] Based on the position information of the detected target object in the target image and the fitting curves of each lane line, a collision warning is issued.

[0010] By using the above-mentioned collision warning method, the acquired target image can be firstly subjected to target detection. In this way, the lane line can be curve-fitted using the detected position information of each pixel point belonging to the lane line to obtain a fitting curve representing the lane line, and then a collision warning can be issued using the detected position information of the target object in the target image and the fitting curves of each lane line. The disclosed embodiment can determine the fitting curve of the lane line using curve fitting, so that the relationship between the target object and each lane line can be determined based on the position information of the target object in the target image and each fitting curve, thereby achieving a vehicle collision warning at the same lane level with high accuracy.

[0011] In a possible implementation, issuing a collision warning based on the detected position information of the target object in the target image and the fitting curve of each lane line includes:

[0012] Determining whether the target object is in the lane where the current vehicle is located based on the position information of the detected target object in the target image and the fitting curve of each lane line;

[0013] When it is determined that the target object is in the lane where the current vehicle is located, a collision warning is issued based on the fitting curve of the lane line of the lane and the position information of the target object in the target image.

[0014] Here, we can first determine whether the target object is in the lane where the current vehicle is located, and then perform a collision warning based on the target object in the same lane to avoid false detection in different lanes and improve the accuracy of the collision warning.

[0015] In a possible implementation manner, determining whether the target object is in the lane where the current vehicle is located based on the position information of the detected target object in the target image and the fitting curve of each lane line includes:

[0016] Based on the longitudinal position of the target object in the target image and the fitting curves of the two lane lines of the lane where the current vehicle is located, determine the lateral position on the fitting curves of the two lane lines of the lane where the current vehicle is located that corresponds to the longitudinal position;

[0017] If the lateral position of the target object in the target image is between two lateral positions corresponding to the two lane lines respectively, it is determined that the target object is in the lane where the current vehicle is located.

[0018] Here, considering the relative position relationship between the target object and the lane line where the current vehicle is located in the target image, the longitudinal position of the target object in the target image can be projected onto the fitting curve to determine the two lateral positions corresponding to the lane line. Based on the lateral position of the target object in the target image and the relative position relationship between the two lateral positions corresponding to the lane line, the relative position relationship between the target object and the two lane lines can be determined, thereby realizing collision warning under curves.

[0019] In a possible implementation, issuing a collision warning based on the fitting curve of the lane line of the lane and the position information of the target object in the target image includes:

[0020] Based on the position information of the target object in the target image, determining a target curve segment located between the current vehicle and the target object in the fitting curve of two lane lines of the lane;

[0021] Calculating the actual travel distance between the target object and the current vehicle based on the target curve segments respectively corresponding to the two lane lines;

[0022] Determining an estimated collision time between the target object and the current vehicle based on the actual travel distance;

[0023] When the estimated collision time is within the target time, a collision warning is issued.

[0024] Taking into account the possibility of curves in the actual lane, the actual driving distance between the two vehicles can be determined based on the fitting curve, and then the estimated collision time can be determined, which is more in line with actual application scenarios.

[0025] In a possible implementation, each pixel point on the lane line is determined according to the following steps:

[0026] Performing semantic segmentation on the target image based on the trained first semantic segmentation model to determine a plurality of lane line pixel points corresponding to the same lane line semantic label; wherein the lane line semantic labels of different lane lines are different;

[0027] Multiple lane line pixel points corresponding to the same lane line semantic label are determined as pixel points belonging to the same lane line.

[0028] Here, different lane line semantic labels can be set for different lane lines, for example, a left lane line semantic label and a right lane line semantic label, so as to realize the detection of different lane lines at the semantic level.

[0029] In a possible implementation, each pixel point on the lane line is determined according to the following steps:

[0030] Performing semantic segmentation on the target image based on the trained second semantic segmentation model to determine a plurality of lane line pixel points corresponding to the lane line semantic label; wherein each lane line has the same lane line semantic label;

[0031] Randomly selecting a preset number of lane line pixel points from the multiple lane line pixel points as initial clustering centers of the lane lines;

[0032] Determine the distances between the plurality of lane line pixels and each initial cluster center, and divide the plurality of lane line pixels into lane lines where the cluster center with the smallest distance is located;

[0033] Determine a new cluster center corresponding to each lane line, and based on the new cluster center, return to execute the step of dividing the multiple lane line pixel points into lane lines where the cluster center with the smallest distance is located, until the division convergence condition is met, and obtain each pixel point belonging to each lane line.

[0034] Here, we can first perform indiscriminate detection of lane lines, and then cluster the lane lines based on the distances between multiple lane line pixels and the clustering centers of the lane lines, thereby achieving detection of different lane lines at the semantic level.

[0035] In a possible implementation, the performing curve fitting on the lane lines in the target image based on the detected position information of each pixel point on the lane line to obtain a fitting curve representing the position of each lane line in the target image includes:

[0036] For the pixel points included in a lane line, construct a fitting curve equation with the longitudinal position variable of the pixel points included in the lane line in the target image as an independent variable and the lateral position variable of the pixel points included in the lane line in the target image as a dependent variable;

[0037] At least some of the pixel points included in the lane line are selected, and based on the longitudinal position and lateral position of the selected pixel points in the target image, equation parameter values ​​in the constructed fitting curve equation are determined, and the fitting curve equation containing the equation parameter values ​​is used to characterize the fitting curve of the position of the lane line in the target image.

[0038] Here, the fitting curve equation can be constructed in advance, and then the equation parameter values ​​in the fitting curve equation can be solved based on the position information (including longitudinal position and lateral position) of the pixel point of the current lane line in the target image, so that the fitting curve can be determined, and the operation is simple.

[0039] In a possible implementation, determining the equation parameter values ​​in the constructed fitting curve equation based on the longitudinal position and the lateral position of the selected pixel point in the target image includes:

[0040] Determining an output result of the fitting curve equation based on the constructed fitting curve equation and the longitudinal position of the selected pixel point in the target image;

[0041] Determining an objective function including equation parameters of the fitting curve equation based on an output result of the fitting curve equation and a lateral position of the selected pixel point in the target image;

[0042] Determine the values ​​of the equation parameters of the fitting curve equation when the value of the objective function is minimized.

[0043] In a possible implementation, selecting at least some of the pixels from the pixels included in the lane line includes:

[0044] For each pixel point of the lane line, obtaining a semantic score of the pixel point belonging to the lane line semantic label;

[0045] The pixels are ranked in descending order according to the semantic scores to obtain a ranking result, and some pixels are selected according to the ranking result.

[0046] In order to make the fitting curve of the fitted lane line closer to the real lane line, here, before performing curve fitting, each pixel point of the lane line can be screened, and the screened pixel points with relatively high semantic scores can more accurately represent the lane line.

[0047] In a possible implementation, determining the equation parameter values ​​in the constructed fitting curve equation based on the longitudinal position and the lateral position of the selected pixel point in the target image includes:

[0048] Based on a first conversion relationship between the image coordinate system where the target image is located and the world coordinate system, and a second conversion relationship between the image coordinate system where the bird's-eye view is located and the world coordinate system, the selected pixel point is projected to the image coordinate system where the bird's-eye view is located, to obtain a longitudinal position and a lateral position of the pixel point in the bird's-eye view;

[0049] Based on the longitudinal position and the lateral position of the selected pixel point in the bird's-eye view, the equation parameter values ​​of the constructed fitting curve equation are determined.

[0050] In order to facilitate curve fitting, the vertical and horizontal positions of the pixels in the bird's-eye view can be obtained based on the conversion relationship between the image coordinate system of the target image and the image coordinate system of the bird's-eye view, and then the equation parameter values ​​of the fitting curve equation can be determined. Since the influence of the viewing angle is eliminated in the bird's-eye view, curve fitting is easier to operate.

[0051] In a second aspect, an embodiment of the present disclosure further provides a collision warning device, the device comprising:

[0052] An acquisition module is used to acquire a target image captured by a camera device currently installed on the vehicle;

[0053] A detection module, used for performing target detection on the target image;

[0054] A fitting module, for performing curve fitting on the lane lines in the target image based on the position information of each pixel point on the lane lines detected, so as to obtain a fitting curve representing the position of each lane line in the target image;

[0055] The warning module is used to issue a collision warning based on the position information of the detected target object in the target image and the fitting curves of each lane line.

[0056] In the third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the collision warning method as described in the first aspect and any of its various embodiments are performed.

[0057] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the collision warning method as described in the first aspect and any of its various embodiments are executed.

[0058] For a description of the effects of the above-mentioned collision warning device, electronic device, and computer-readable storage medium, please refer to the description of the above-mentioned collision warning method, which will not be repeated here.

[0059] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0061] Figure 1 A flow chart of a collision warning method provided by an embodiment of the present disclosure is shown;

[0062] Figure 2 A schematic diagram showing an application of a collision warning method provided by an embodiment of the present disclosure is shown;

[0063] Figure 3 A schematic diagram of a collision warning device provided by an embodiment of the present disclosure is shown;

[0064] Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0066] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0067] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0068] Research has found that the relevant vehicle forward collision warning methods mainly calculate the relative collision time based on the relative position and speed relationship of the vehicles to determine whether to issue an alarm. However, for vehicles ahead on complex or special lanes, there are often omissions and false alarms. For example, for narrow lanes in the countryside, it is easy to judge the vehicle in the next opposite lane as the vehicle in the lane, resulting in false alarms.

[0069] Based on the above research, the present disclosure provides a collision warning method, device, electronic device and storage medium with high detection accuracy.

[0070] To facilitate understanding of the present embodiment, a collision warning method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the collision warning method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the collision warning method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0071] See also Figure 1 FIG. 1 is a flowchart of a collision warning method provided by an embodiment of the present disclosure, the method comprising steps S101 to S104, wherein:

[0072] S101: Acquire a target image captured by a camera device currently installed on the vehicle;

[0073] S102: Performing target detection on the target image;

[0074] S103: performing curve fitting on the lane lines in the target image based on the position information of each pixel point on the lane lines detected, to obtain a fitting curve representing the position of each lane line in the target image;

[0075] S104: Based on the position information of the detected target object in the target image and the fitting curves of each lane line, a collision warning is issued.

[0076] In order to facilitate understanding of the collision warning method provided by the embodiment of the present disclosure, the application scenario of the method is briefly described below. The above-mentioned collision warning method can be mainly applied to Advanced Driver Assistance Systems (ADAS), which can avoid potential collision risks through forward collision warning, especially when the driver is distracted, fatigued and sleepy, and can play a great role.

[0077] In the collision warning scheme provided in the related art that calculates the relative collision time based on the relative position and speed relationship of the vehicles, missed reports and false reports often occur for the front vehicles on complex or special lanes. For example, for large curves or other non-straight lane line scenarios, the front vehicle judged based on the relative position of the vehicles is not the real front vehicle on the driving path of the vehicle, and missed reports and false reports may occur. For another example, for narrow lanes in the countryside, when the road is too narrow, it is easy to judge the vehicle in the opposite lane next door as the vehicle in the lane, resulting in false reports.

[0078] It is precisely to solve the above problems that the embodiments of the present disclosure provide a curve fitting solution based on lane lines to achieve collision warning with high accuracy.

[0079] The target image in the embodiment of the present disclosure may be an image captured by a camera installed on the current vehicle. In order to better realize the forward collision warning, the camera may be arranged to face forward. In this way, the image information of the vehicle ahead can be captured at any time during the vehicle's movement, thereby realizing the forward collision warning.

[0080] For the acquired target image, the disclosed embodiment can perform lane line detection on the one hand, and can perform target detection on the lane line on the other hand. For lane line detection, this can be implemented based on semantic segmentation. For example, each pixel point belonging to the lane line can be determined by a trained semantic segmentation model, and then the detected lane line can be obtained by combining the pixels. For the detection of targets on the lane line, this can be the detection of pedestrian target objects, and it can also be the detection of vehicle target objects. For example, the attribute information of the vehicle is determined by image detection, and then the position of the vehicle in the target image is determined. For another example, the target vehicle is directly identified from the target image through a trained vehicle detection model. The following examples will be explained with the target vehicle as the target object.

[0081] Here, before performing a vehicle collision warning, a curve fitting may be performed on the lane line to obtain a fitting curve representing the lane line.

[0082] In the disclosed embodiment, curve fitting can be implemented based on the position information of each pixel point on the lane line. The process of implementing curve fitting here can be based on the process of solving the equation parameters of the constructed fitting curve equation. When the equation parameter values ​​of the fitting curve equation are solved, the fitting curve represented by the fitting curve equation is determined.

[0083] By using the fitting curve and the position information of the target object in the target image, the relationship between the target object and each lane line can be determined, and then the forward vehicle collision warning within the same lane line can be realized.

[0084] Considering the key role of lane line detection in the subsequent implementation of lane line curve fitting, the process of detecting lane lines can be explained in the following two aspects.

[0085] First aspect: The embodiments of the present disclosure can realize the detection of different lane lines, which can be realized by the following steps:

[0086] Step 1: semantically segment the target image based on the trained first semantic segmentation model to determine a plurality of lane line pixels corresponding to the same lane line semantic label; wherein the lane line semantic labels of different lane lines are different;

[0087] Step 2: multiple lane line pixel points corresponding to the same lane line semantic label are determined as pixel points belonging to the same lane line.

[0088] During the training of the first semantic segmentation model, different lane line semantic labels may be set for different lane lines. For example, the left lane line may be set to label 1, and the right lane line may be set to label 2, thereby enabling detection of different lane lines.

[0089] Among them, the above-mentioned first semantic segmentation model can be based on pixel-level semantic annotation. In this way, when the target image is semantically segmented using the first semantic segmentation model, multiple lane line pixel points corresponding to the same lane line semantic label can be determined, and the multiple lane line pixel points corresponding to the same lane line semantic label can be combined to obtain a detected lane line. For example, by combining multiple lane line pixel points corresponding to label 1, the left lane line can be detected.

[0090] Second aspect: The embodiment of the present disclosure can first perform unified lane line detection, and then implement differentiated detection of different lane lines based on clustering. This can be achieved through the following steps:

[0091] Step 1: semantically segment the target image based on the trained second semantic segmentation model to determine a plurality of lane line pixels corresponding to the lane line semantic labels; wherein each lane line has the same lane line semantic label;

[0092] Step 2: Randomly select a preset number of lane line pixel points from multiple lane line pixel points as initial clustering centers of the lane lines;

[0093] Step 3: Determine the distances between multiple lane line pixels and each initial cluster center, and divide the multiple lane line pixels into lane lines where the cluster center with the smallest distance is located;

[0094] Step 4: Determine the new cluster center corresponding to each lane line, and based on the new cluster center, return to execute the step of dividing multiple lane line pixels into lane lines where the cluster center with the smallest distance is located, until the division convergence condition is met, and obtain each pixel point belonging to each lane line.

[0095] Here, in the process of training the second semantic segmentation model, the same lane line semantic label can be set for different lane lines. For example, the left lane line and the right lane line can both be set to label 1.

[0096] Among them, the above-mentioned second semantic segmentation model can also be based on pixel-level semantic annotation. In this way, when the target image is semantically segmented using the second semantic segmentation model, multiple lane line pixel points corresponding to the lane line semantic labels can be determined, that is, the pixel points pointing to the lane line can be found from the target image.

[0097] Here, lane line division can be achieved based on pixel clustering. First, the initial cluster center of the lane line can be selected, and then the distance between multiple lane line pixels and each initial cluster center can be determined, and the lane line can be aggregated based on the minimum distance. A new cluster center can be determined based on the lane line pixels of the current aggregation, and then the next aggregation is performed based on the minimum distance, and so on, until the divided lane lines are obtained.

[0098] In specific applications, clustering algorithms such as mean-shift can be used to implement the above clustering process. The division convergence condition here can be that the number of clustering times reaches a preset number, such as 15 times, or that the cluster center no longer changes or changes very little, or other conditions, which are not specifically limited here.

[0099] The collision warning method provided by the embodiment of the present disclosure can perform curve fitting on the lane line based on the position information of each pixel point on the lane line to obtain a fitting curve representing the lane line, which specifically includes the following steps:

[0100] Step 1: for the pixel points included in a lane line, construct a fitting curve equation with the longitudinal position variable of the pixel points included in the lane line in the target image as the independent variable and the lateral position variable of the pixel points included in the lane line in the target image as the dependent variable;

[0101] Step 2: Select at least some of the pixel points included in the lane line, and determine the equation parameter values ​​in the constructed fitting curve equation based on the longitudinal position and lateral position of the selected pixel points in the target image, and use the fitting curve equation containing the equation parameter values ​​to represent the fitting curve of the position of the lane line in the target image.

[0102] Considering the corresponding relationship between the fitting curve and the fitting curve equation, the fitting curve equation can be constructed in advance, and then the equation parameter values ​​of the fitting curve equation can be solved by known data. The corresponding fitting curve equation can be used to characterize the fitting curve through the solved equation parameter values.

[0103] The fitting curve equation here may represent the correspondence between the longitudinal position variable of the pixel points included in the lane line in the target image and the lateral position variable of the pixel points included in the lane line in the target image. For example, it may be constructed according to the following equation:

[0104] x=ay 3 +by 2 +cy+d

[0105] Among them, {a, b, c, d} can represent the equation parameters of the fitting curve equation, y can represent the independent variable of the fitting curve equation, and x can represent the dependent variable of the fitting curve equation.

[0106] Here, the longitudinal and lateral positions of some pixels selected from each pixel point of the lane line in the target image are substituted into the above equation as the observation data of the fitting curve equation, and the values ​​of {a, b, c, d} can be solved, that is, the equation parameter values ​​of the fitting curve equation can be determined, and then the fitting curve equation with the equation parameter values ​​can be obtained, and the fitting curve equation can represent the corresponding fitting curve.

[0107] In the embodiment of the present disclosure, the process of solving the equation parameter value may be a process of solving the minimum value of the objective function of the constructed equation parameter including the fitting curve equation, which is specifically implemented by the following steps:

[0108] Step 1: Based on the constructed fitting curve equation and the longitudinal position of the selected pixel point in the target image, determine the output result of the fitting curve equation;

[0109] Step 2: Based on the output result of the fitting curve equation and the lateral position of the selected pixel point in the target image, determine the objective function including the equation parameters of the fitting curve equation;

[0110] Step 3: Determine the parameter values ​​of the fitting curve equation when the value of the objective function is minimized.

[0111] Here, the longitudinal position of the selected pixel point in the target image can be substituted into the constructed fitting curve equation, and the output result obtained points to the output lateral position of the pixel point in the target image. Here, combined with the lateral position of the selected pixel point in the target image (ie, the true lateral position), the difference between the two lateral positions is determined. The smaller the difference, the better the fitting effect.

[0112] In the disclosed embodiment, the equation parameter value of the fitting curve equation is obtained by finding the minimum value of the objective function, and the equation parameter value may be a parameter value pointing to the lowest lateral position difference.

[0113] Considering the huge number of pixel points corresponding to lane lines, directly participating in the fitting operation process will consume a lot of computing resources. Therefore, the embodiment of the present disclosure provides a solution for filtering pixel points of lane lines, which can be implemented by the following steps:

[0114] Step 1: for each pixel of the lane line, obtain a semantic score of the pixel belonging to the lane line semantic label;

[0115] Step 2: Rank each pixel in descending order according to the semantic score to obtain the ranking result, and select some pixels according to the ranking result.

[0116] Here, pixels with relatively high semantic scores often refer to high-quality pixels such as those with centered lane lines and high pixel clarity. Therefore, pixel bands can be screened based on the ranking results of semantic scores, thereby ensuring that the fitted curve is complete enough and reducing the computational cost of fitting.

[0117] Here, in order to eliminate the influence of the viewing angle, before performing curve fitting, the vertical and horizontal positions of the pixels in the target image can be converted to the bird's-eye view. The specific steps are as follows:

[0118] Step 1: Based on the first transformation relationship between the image coordinate system where the target image is located and the world coordinate system, and the second transformation relationship between the image coordinate system where the bird's-eye view is located and the world coordinate system, the selected pixel point is projected to the image coordinate system where the bird's-eye view is located, and the vertical position and horizontal position of the pixel point in the bird's-eye view are obtained;

[0119] Step 2: Based on the longitudinal position and lateral position of the selected pixel point in the bird's-eye view, determine the equation parameter values ​​of the constructed fitting curve equation.

[0120] After projection, the straight lane lines that would have converged on the horizon of the two-dimensional target image will become vertical and parallel, and although the curves will not be vertical, they will also become easy to fit in the bird's-eye view.

[0121] The collision warning method provided by the embodiment of the present disclosure can first determine whether the target object is in the lane where the current vehicle is located based on the position information of the detected target object in the target image and the fitting curve of each lane line, when fitting the fitting curve of each lane line according to the above method. Once it is determined that the target object and the current vehicle are in the same lane, a collision warning can be issued based on the fitting curve of the lane line of the lane and the position information of the target object in the target image.

[0122] The disclosed embodiment can determine whether the target object is in the lane where the current vehicle is located according to the following steps:

[0123] Step 1: Based on the longitudinal position of the target object in the target image and the fitting curves of the two lane lines of the lane where the current vehicle is located, determine the lateral position corresponding to the longitudinal position on the fitting curves of the two lane lines of the lane where the current vehicle is located;

[0124] Step 2: If the lateral position of the target object in the target image is between two lateral positions corresponding to the two lane lines, it is determined that the target object is in the lane where the current vehicle is located.

[0125] Here, first, based on the longitudinal position of the target object in the target image and the fitting curve of the two lane lines of the lane where the current vehicle is located, the lateral position on the fitting curve of the two lane lines corresponding to the longitudinal position can be determined. In the embodiment of the present disclosure, the longitudinal position can be substituted into the fitting curve equation of the two lane lines to obtain the lateral position of the two lane lines at y, x l and x r If the lateral position x of the target object in the target image is between the two lateral positions corresponding to the two lane lines, that is, x l <x<x r , the target object is located in the lane where the current vehicle is located, otherwise the target object is located in other lanes.

[0126] In the embodiment of the present disclosure, for a target object located in the lane where the current vehicle is located, a collision warning for the target object can be implemented. This can be implemented specifically by the following steps:

[0127] Step 1: Based on the position information of the target object in the target image, determine a target curve segment located between the current vehicle and the target object in the fitting curve of the two lane lines of the lane;

[0128] Step 2: Calculate the actual driving distance between the target object and the current vehicle based on the target curve segments corresponding to the two lane lines;

[0129] Step 3: Determine the estimated collision time between the target object and the current vehicle based on the actual driving distance;

[0130] When the estimated collision time is within the target time, a collision warning is issued.

[0131] Here, firstly, the target curve segment between the current vehicle and the target vehicle can be determined from the fitting curves of the two lane lines of the lane where the current vehicle is located, such as Figure 2 An example of a target curve segment is shown. Based on this target curve segment, the actual driving distance between the two vehicles can be determined, and then the estimated collision time can be determined in combination with the current driving speed of the vehicle.

[0132] It should be noted that the target curve segment here can be selected as the shorter target curve segment determined by the two fitting curves, so as to further ensure the timeliness of the collision warning.

[0133] Here, when it is determined that the estimated collision time is short, for example, less than 2 minutes, a collision warning message may be issued. The collision warning message here may be implemented in the form of flashing indicator lights or in a semantic manner, and the embodiments of the present disclosure do not impose specific restrictions on this.

[0134] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0135] Based on the same inventive concept, an embodiment of the present disclosure also provides a collision warning device corresponding to the collision warning method. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned collision warning method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0136] Reference Figure 3 , which is a schematic diagram of a collision warning device provided by an embodiment of the present disclosure, the device includes: an acquisition module 301, a detection module 302, a fitting module 303 and a warning module 304; wherein,

[0137] An acquisition module 301 is used to acquire a target image captured by a camera device currently installed on the vehicle;

[0138] A detection module 302, used to perform target detection on a target image;

[0139] A fitting module 303 is used to perform curve fitting on the lane lines in the target image based on the position information of each pixel point on the lane line detected, so as to obtain a fitting curve representing the position of each lane line in the target image;

[0140] The warning module 304 is used to issue a collision warning based on the position information of the detected target object in the target image and the fitting curves of each lane line.

[0141] The device for collision warning described above can first perform target detection on the acquired target image. In this way, the detected position information of each pixel point belonging to the lane line can be used to perform curve fitting on the lane line to obtain a fitting curve representing the lane line, and then the detected position information of the target object in the target image and the fitting curve of each lane line can be used to issue a collision warning. The disclosed embodiment can determine the fitting curve of the lane line by using curve fitting, and can determine the relationship between the target object and each lane line based on the position information of the target object in the target image and each fitting curve, thereby achieving a vehicle collision warning at the same lane level with high accuracy.

[0142] In a possible implementation, the warning module 304 is configured to issue a collision warning based on the detected position information of the target object in the target image and the fitting curves of each lane line according to the following steps:

[0143] Based on the position information of the detected target object in the target image and the fitting curves of each lane line, determine whether the target object is in the lane where the current vehicle is located;

[0144] When it is determined that the target object is in the lane where the current vehicle is located, a collision warning is issued based on the fitting curve of the lane line of the lane and the position information of the target object in the target image.

[0145] In a possible implementation, the warning module 304 is configured to determine whether the target object is in the lane where the current vehicle is located based on the detected position information of the target object in the target image and the fitting curves of each lane line according to the following steps:

[0146] Based on the longitudinal position of the target object in the target image and the fitting curves of the two lane lines of the lane where the current vehicle is located, determine the lateral position on the fitting curves of the two lane lines of the lane where the current vehicle is located that corresponds to the longitudinal position;

[0147] If the lateral position of the target object in the target image is located between two lateral positions corresponding to the two lane lines, it is determined that the target object is in the lane where the current vehicle is located.

[0148] In a possible implementation, the warning module 304 is configured to issue a collision warning based on the fitting curve of the lane line of the lane and the position information of the target object in the target image according to the following steps:

[0149] Based on the position information of the target object in the target image, determine a target curve segment located between the current vehicle and the target object in the fitting curve of the two lane lines of the lane;

[0150] Based on the target curve segments corresponding to the two lane lines, the actual driving distance between the target object and the current vehicle is calculated;

[0151] Determine the estimated collision time between the target object and the current vehicle based on the actual travel distance;

[0152] When the estimated collision time is within the target time, a collision warning is issued.

[0153] In a possible implementation, the detection module 302 is used to determine each pixel point on the lane line according to the following steps:

[0154] Performing semantic segmentation on the target image based on the trained first semantic segmentation model to determine a plurality of lane line pixel points corresponding to the same lane line semantic label; wherein the lane line semantic labels of different lane lines are different;

[0155] Multiple lane line pixel points corresponding to the same lane line semantic label are determined as pixel points belonging to the same lane line.

[0156] In a possible implementation, the detection module 302 determines each pixel point on the lane line according to the following steps:

[0157] Performing semantic segmentation on the target image based on the trained second semantic segmentation model to determine a plurality of lane line pixel points corresponding to the lane line semantic label; wherein each lane line has the same lane line semantic label;

[0158] Randomly select a preset number of lane line pixel points from multiple lane line pixel points as the initial clustering centers of the lane lines;

[0159] Determine the distances between a plurality of lane line pixels and each initial cluster center, and divide the plurality of lane line pixels into lane lines where the cluster center with the smallest distance is located;

[0160] Determine the new cluster center corresponding to each lane line, and based on the new cluster center, return to execute the step of dividing the multiple lane line pixels into the lane line where the cluster center with the smallest distance is located, until the division convergence condition is met, and obtain the pixel points belonging to each lane line.

[0161] In a possible implementation, the fitting module 303 is used to perform curve fitting on the lane lines in the target image based on the position information of each pixel point detected on the lane lines, so as to obtain a fitting curve representing the position of each lane line in the target image:

[0162] For the pixel points included in a lane line, construct a fitting curve equation with the longitudinal position variable of the pixel points included in the lane line in the target image as an independent variable and the lateral position variable of the pixel points included in the lane line in the target image as a dependent variable;

[0163] At least some of the pixel points included in the lane line are selected, and based on the longitudinal position and lateral position of the selected pixel points in the target image, equation parameter values ​​in the constructed fitting curve equation are determined, and the fitting curve equation containing the equation parameter values ​​is used to represent the fitting curve of the position of the lane line in the target image.

[0164] In a possible implementation, the fitting module 303 is used to determine the equation parameter values ​​in the constructed fitting curve equation based on the longitudinal position and the lateral position of the selected pixel point in the target image according to the following steps:

[0165] Based on the constructed fitting curve equation and the longitudinal position of the selected pixel point in the target image, an output result of the fitting curve equation is determined;

[0166] Determine an objective function including equation parameters of the fitting curve equation based on an output result of the fitting curve equation and a lateral position of the selected pixel point in the target image;

[0167] Determine the values ​​of the equation parameters of the fitted curve equation when the value of the objective function is minimized.

[0168] In a possible implementation, the fitting module 303 is configured to select some pixel points from the pixel points included in the lane line according to the following steps:

[0169] For each pixel of the lane line, obtain a semantic score of the pixel belonging to the lane line semantic label;

[0170] Rank each pixel in descending order according to the semantic score to obtain a ranking result, and select some pixels according to the ranking result.

[0171] In a possible implementation, the fitting module 303 is used to determine the equation parameter values ​​in the constructed fitting curve equation based on the longitudinal position and the lateral position of the selected pixel point in the target image according to the following steps:

[0172] Based on a first transformation relationship between an image coordinate system where the target image is located and a world coordinate system, and a second transformation relationship between an image coordinate system where the bird's-eye view is located and a world coordinate system, the selected pixel point is projected to the image coordinate system where the bird's-eye view is located, and a longitudinal position and a lateral position of the pixel point in the bird's-eye view are obtained;

[0173] Based on the longitudinal position and the lateral position of the selected pixel point in the bird's-eye view, the equation parameter values ​​of the constructed fitting curve equation are determined.

[0174] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0175] The present disclosure also provides an electronic device, such as Figure 4 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, including: a processor 401, a memory 402, and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401 (for example, Figure 3 In the device, the execution instructions corresponding to the acquisition module 301, the detection module 302, the fitting module 303, and the early warning module 304 are obtained, and when the electronic device is running, the processor 401 communicates with the memory 402 through the bus 403, and the machine-readable instructions are executed by the processor 401 to perform the following processing:

[0176] Acquire a target image captured by a camera device currently installed on the vehicle;

[0177] Perform target detection on the target image;

[0178] Based on the position information of each pixel point on the lane line detected, curve fitting is performed on the lane line in the target image to obtain a fitting curve representing the position of each lane line in the target image;

[0179] Based on the position information of the detected target object in the target image and the fitting curves of each lane line, a collision warning is issued.

[0180] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the collision warning method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0181] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the collision warning method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0182] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0183] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0186] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0187] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A collision warning method, characterized in that: The method comprises: Acquire a target image captured by a camera device currently installed on the vehicle; Performing target detection on the target image; Based on the position information of each pixel point on the lane line detected, curve fitting is performed on the lane line in the target image to obtain a fitting curve representing the position of each lane line in the target image; the fitting curve is obtained according to a fitting curve equation represented by an equation parameter value; the equation parameter value is determined by the longitudinal position and the lateral position of the pixel point in the target image; Determining whether the target object is in the lane where the current vehicle is located based on the position information of the detected target object in the target image and the fitting curve of each lane line; If yes, determining a target curve segment located between the current vehicle and the target object in the fitting curve of two lane lines of the lane based on the position information of the target object in the target image; Calculating the actual travel distance between the target object and the current vehicle based on the target curve segments respectively corresponding to the two lane lines; Based on the actual driving distance, an estimated collision time between the target object and the current vehicle is determined; and when the estimated collision time is within a target time, a collision warning is issued.

2. The method according to claim 1, characterized in that The determining whether the target object is in the lane where the current vehicle is located based on the position information of the detected target object in the target image and the fitting curves of each lane line includes: Based on the longitudinal position of the target object in the target image and the fitting curves of the two lane lines of the lane where the current vehicle is located, determine the lateral position on the fitting curves of the two lane lines of the lane where the current vehicle is located that corresponds to the longitudinal position; If the lateral position of the target object in the target image is between two lateral positions corresponding to the two lane lines respectively, it is determined that the target object is in the lane where the current vehicle is located.

3. The method according to claim 1 or 2, characterized in that: Follow the steps below to determine each pixel that belongs to the lane line: Performing semantic segmentation on the target image based on the trained first semantic segmentation model to determine a plurality of lane line pixel points corresponding to the same lane line semantic label; wherein the lane line semantic labels of different lane lines are different; Multiple lane line pixel points corresponding to the same lane line semantic label are determined as pixel points belonging to the same lane line.

4. The method according to claim 1 or 2, characterized in that: Follow the steps below to determine each pixel that belongs to the lane line: Performing semantic segmentation on the target image based on the trained second semantic segmentation model to determine a plurality of lane line pixel points corresponding to the lane line semantic label; wherein each lane line has the same lane line semantic label; Randomly selecting a preset number of lane line pixel points from the multiple lane line pixel points as initial clustering centers of the lane lines; Determine the distances between the plurality of lane line pixels and each initial cluster center, and divide the plurality of lane line pixels into lane lines where the cluster center with the smallest distance is located; Determine a new cluster center corresponding to each lane line, and based on the new cluster center, return to execute the step of dividing the multiple lane line pixel points into lane lines where the cluster center with the smallest distance is located, until the division convergence condition is met, and obtain each pixel point belonging to each lane line.

5. The method according to any one of claims 1 to 4, characterized in that: The method of performing curve fitting on the lane lines in the target image based on the position information of each pixel point on the lane lines detected to obtain a fitting curve representing the position of each lane line in the target image includes: For the pixel points included in a lane line, construct a fitting curve equation with the longitudinal position variable of the pixel points included in the lane line in the target image as an independent variable and the lateral position variable of the pixel points included in the lane line in the target image as a dependent variable; At least some of the pixel points included in the lane line are selected, and based on the longitudinal position and lateral position of the selected pixel points in the target image, equation parameter values ​​in the constructed fitting curve equation are determined, and the fitting curve equation containing the equation parameter values ​​is used to characterize the fitting curve of the position of the lane line in the target image.

6. The method according to claim 5, characterized in that The step of determining the values ​​of equation parameters in the constructed fitting curve equation based on the longitudinal position and the lateral position of the selected pixel point in the target image comprises: Determining an output result of the fitting curve equation based on the constructed fitting curve equation and the longitudinal position of the selected pixel point in the target image; Determining an objective function including equation parameters of the fitting curve equation based on an output result of the fitting curve equation and a lateral position of the selected pixel point in the target image; Determine the values ​​of the equation parameters of the fitting curve equation when the value of the objective function is minimized.

7. The method according to claim 5 or 6, characterized in that: The selecting at least some of the pixel points from the pixel points included in the lane line includes: For each pixel point of the lane line, obtaining a semantic score of the pixel point belonging to the lane line semantic label; The pixels are ranked in descending order according to the semantic scores to obtain a ranking result, and some pixels are selected according to the ranking result.

8. The method according to any one of claims 5 to 7, characterized in that: The step of determining the values ​​of equation parameters in the constructed fitting curve equation based on the longitudinal position and the lateral position of the selected pixel point in the target image comprises: Based on a first conversion relationship between the image coordinate system where the target image is located and the world coordinate system, and a second conversion relationship between the image coordinate system where the bird's-eye view is located and the world coordinate system, the selected pixel point is projected to the image coordinate system where the bird's-eye view is located, to obtain a longitudinal position and a lateral position of the pixel point in the bird's-eye view; Based on the longitudinal position and the lateral position of the selected pixel point in the bird's-eye view, the equation parameter values ​​of the constructed fitting curve equation are determined.

9. A collision warning device, characterized in that: The device comprises: An acquisition module is used to acquire a target image captured by a camera device currently installed on the vehicle; A detection module, used for performing target detection on the target image; A fitting module, for performing curve fitting on the lane lines in the target image based on the position information of each pixel point on the lane line detected, so as to obtain a fitting curve representing the position of each lane line in the target image; the fitting curve is obtained according to a fitting curve equation represented by an equation parameter value; the equation parameter value is determined by the longitudinal position and the lateral position of the pixel point in the target image; an early warning module, configured to determine whether the target object is in the lane where the current vehicle is located based on the position information of the detected target object in the target image and the fitting curve of each lane line; If yes, determining a target curve segment located between the current vehicle and the target object in the fitting curve of two lane lines of the lane based on the position information of the target object in the target image; Calculating the actual travel distance between the target object and the current vehicle based on the target curve segments respectively corresponding to the two lane lines; Based on the actual driving distance, an estimated collision time between the target object and the current vehicle is determined; and when the estimated collision time is within a target time, a collision warning is issued.

10. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the collision warning method as described in any one of claims 1 to 8 are performed.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the collision warning method as claimed in any one of claims 1 to 8 are executed.

Images