Method and device for obtaining road vanishing point position

Abstract

An embodiment of the present invention provides a method and device for obtaining a road vanishing point position, which is applied to a first vehicle device. The method includes: acquiring an image including a second vehicle device; performing lane line detection based on the acquired image to obtain lane line information of a lane in which the first vehicle device is located in the image; performing vehicle detection based on the acquired image to obtain vehicle parameters of the second vehicle device relative to the image; and determining a vanishing point position corresponding to the lane in which the first vehicle device is located based on the lane line information and the vehicle parameters.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving, and in particular to a method and device for acquiring a road vanishing point position. Background Art

[0002] Advanced Driving Assistant System (ADAS) based on visual algorithms is widely used in safe driving. It uses sensors installed on the car to sense the surrounding environment in real time while the car is driving and combines the navigation map data to perform systematic calculations and analysis, so that the driver can be aware of possible dangers in advance, effectively increasing the comfort and safety of car driving. The location of the road vanishing point is the key input information for each of these technologies.

[0003] Related technologies obtain the location of road vanishing points by extracting lane lines from images or using an iterative method. However, such technologies require finding at least two lane lines. When lane lines are very difficult to obtain, such as occlusion caused by congestion, it becomes increasingly difficult to find multiple lane lines, resulting in the inability to obtain the location of the road vanishing point or the accuracy of the obtained road vanishing point location is not high. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for acquiring a road vanishing point position, which can quickly and accurately acquire the road vanishing point position with the help of vehicle information.

[0005] The technical solution of the embodiment of the present invention is achieved as follows:

[0006] An embodiment of the present invention provides a method for obtaining a road vanishing point position, which is applied to a first vehicle device and includes:

[0007] capturing an image including the second vehicle equipment;

[0008] Performing lane line detection based on the acquired image to obtain lane line information of the lane where the first vehicle device is located in the image;

[0009] Performing vehicle detection based on the acquired image to obtain vehicle parameters of the second vehicle device relative to the image;

[0010] Based on the lane line information and the vehicle parameters, a vanishing point position corresponding to the lane where the first vehicle device is located is determined.

[0011] In the above solution, the lane line detection is performed based on the acquired image to obtain the lane line information of the lane where the first vehicle device is located in the image, including:

[0012] Performing a region interception on the image to obtain an image region including the lane where the first vehicle device is located;

[0013] Acquire pixel points belonging to lane lines in the image area;

[0014] The lane line of the lane where the first vehicle device is located in the image area is determined by performing curve fitting on the acquired pixel points.

[0015] In the above solution, the determining the lane line of the lane where the first vehicle device is located in the image area by performing curve fitting on the acquired pixel points includes:

[0016] In response to the lane line at least including a left lane line and a right lane line, determining a lane line with higher integrity between the left lane line and the right lane line based on the acquired pixel points;

[0017] Curve fitting is performed on the pixel points of the lane line with higher integrity to determine the lane line of the lane where the first vehicle device is located in the image area.

[0018] In the above solution, the vehicle detection is performed based on the acquired image to obtain the vehicle parameters of the second vehicle device relative to the image, including:

[0019] Inputting the collected image into a trained neural network model to identify a plurality of borders of the second vehicle equipment;

[0020] Performing non-maximum suppression (NMS) processing on the multiple identified bounding boxes to obtain a target bounding box corresponding to the second vehicle equipment;

[0021] Based on the relative position relationship between the target frame and the image, vehicle parameters of the second vehicle device relative to the image are obtained.

[0022] In the above solution, obtaining the vehicle parameters of the second vehicle device relative to the image based on the relative position relationship between the target frame and the image includes:

[0023] Determining a vehicle height of the second vehicle device relative to the image based on positions of an upper frame and a lower frame of the target frame in the image;

[0024] Based on the position of the upper frame of the target frame in the image, a first distance of the upper frame relative to a first coordinate system direction in the image is determined, and the upper frame is parallel to the first coordinate system direction.

[0025] In the above solution, the step of determining the vanishing point position corresponding to the lane where the first vehicle device is located based on the vehicle parameters and the lane line information includes:

[0026] Determine a second distance of the vanishing point position relative to the first coordinate system direction in the image;

[0027] Draw a first straight line parallel to the direction of the first coordinate system in the image and having a spacing of the second distance;

[0028] A first position of an intersection of the drawn first straight line and the lane line in the image is obtained, and the first position is used as the vanishing point position.

[0029] In the above solution, the step of determining the second distance of the vanishing point relative to the first coordinate system direction in the image includes:

[0030] The second distance of the vanishing point position relative to the first coordinate system direction in the image is determined by the following relationship:

[0031] The second distance is equal to a sum of a constant multiple of the vehicle height and the first distance.

[0032] In the above solution, the step of determining the vanishing point position corresponding to the lane where the first vehicle device is located based on the vehicle parameters and the lane line information includes:

[0033] In response to the vehicle parameters including a vehicle height of the second vehicle device relative to the image, determining the second distance based on a positive correlation between the vehicle height, a second distance of the vanishing point position relative to a first coordinate system direction in the image, and the vehicle height;

[0034] Draw a second straight line parallel to the upper edge of the image and spaced apart by the second distance;

[0035] A second position of an intersection of the drawn second straight line and the lane line in the image is obtained, and the second position is used as the vanishing point position.

[0036] An embodiment of the present invention provides a device for obtaining a road vanishing point position, which is applied to a first vehicle device and includes:

[0037] A collection unit, used for collecting an image including the second vehicle equipment;

[0038] A first detection unit, configured to perform lane line detection based on the acquired image, and obtain lane line information of a lane where the first vehicle device is located in the image;

[0039] A second detection unit, configured to perform vehicle detection based on the acquired image, and obtain vehicle parameters of the second vehicle device relative to the image;

[0040] A determination unit is used to determine a vanishing point position corresponding to the lane where the first vehicle equipment is located based on the lane line information and the vehicle parameters.

[0041] In the above solution, the first detection unit is used to perform region capture on the image to obtain an image region including the lane where the first vehicle device is located;

[0042] Acquire pixel points belonging to lane lines in the image area;

[0043] The lane line of the lane where the first vehicle device is located in the image area is determined by performing curve fitting on the acquired pixel points.

[0044] In the above solution, the first detection unit is used to determine the lane line with higher integrity between the left lane line and the right lane line based on the acquired pixel points in response to the lane line at least including the left lane line and the right lane line;

[0045] Curve fitting is performed on the pixel points of the lane line with higher integrity to determine the lane line of the lane where the first vehicle device is located in the image area.

[0046] In the above solution, the second detection unit is used to input the collected image into the trained neural network model to identify multiple borders of the second vehicle equipment;

[0047] Performing NMS processing on the multiple identified bounding boxes to obtain a target bounding box corresponding to the second vehicle equipment;

[0048] Based on the relative position relationship between the target frame and the image, vehicle parameters of the second vehicle device relative to the image are obtained.

[0049] In the above solution, the second detection unit is used to determine the vehicle height of the second vehicle device relative to the image based on the positions of the upper frame and the lower frame of the target frame in the image;

[0050] Based on the position of the upper frame of the target frame in the image, a first distance of the upper frame relative to a first coordinate system direction in the image is determined, and the upper frame is parallel to the first coordinate system direction.

[0051] In the above solution, the determining unit is used to determine a second distance of the vanishing point position relative to the direction of the first coordinate system in the image;

[0052] Draw a first straight line parallel to the direction of the first coordinate system in the image and having a spacing of the second distance;

[0053] A first position of an intersection of the drawn first straight line and the lane line in the image is obtained, and the first position is used as the vanishing point position.

[0054] In the above solution, the determining unit is used to determine the second distance of the vanishing point position relative to the first coordinate system direction in the image through the following relationship:

[0055] The second distance is equal to a sum of a constant multiple of the vehicle height and the first distance.

[0056] In the above solution, the determination unit is used to determine the second distance in response to the vehicle parameter including the vehicle height of the second vehicle device relative to the image, based on the positive correlation between the vehicle height, the second distance of the vanishing point position relative to the first coordinate system direction in the image, and the vehicle height;

[0057] Draw a second straight line parallel to the upper edge of the image and spaced apart by the second distance;

[0058] A second position of an intersection of the drawn second straight line and the lane line in the image is obtained, and the second position is used as the vanishing point position.

[0059] An embodiment of the present invention provides a device for obtaining a road vanishing point position, comprising:

[0060] A memory for storing executable instructions;

[0061] The processor is configured to implement the method for obtaining the position of a road vanishing point provided by an embodiment of the present invention when executing the executable instructions stored in the memory.

[0062] An embodiment of the present invention provides a storage medium storing executable instructions for causing a processor to execute the instructions to implement the method for obtaining the position of a road vanishing point provided by the embodiment of the present invention.

[0063] The embodiments of the present invention have the following beneficial effects:

[0064] The method for obtaining the road vanishing point provided by the embodiment of the present invention is applied to collect an image including the second vehicle device, respectively determine the lane line information of the lane where the first vehicle device is located, and the vehicle parameters of the second vehicle device relative to the image, determine the horizontal line where the vanishing point is located in the image based on the obtained vehicle parameters, and determine the position of the vanishing point based on the intersection of the lane line and the horizontal line where the vanishing point is located. In this way, the position of the vanishing point can be quickly and accurately determined using auxiliary vehicle information and a lane line, which has a wide range of applications and enhances user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1A schematic diagram of the architecture of a system for acquiring a road vanishing point position provided by an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of the hardware structure of a device for obtaining a road vanishing point position provided by an embodiment of the present invention;

[0067] Figure 3 A schematic diagram of a flow chart of a method for obtaining a road vanishing point position provided by an embodiment of the present invention;

[0068] Figure 4 A binary schematic diagram of lane lines provided in an embodiment of the present invention;

[0069] Figure 5 A schematic diagram of lane line pixel distribution provided by an embodiment of the present invention;

[0070] Figure 6 A schematic diagram of lane line pixel fitting provided by an embodiment of the present invention;

[0071] Figure 7 A schematic diagram of a vehicle border processed by an NMS according to an embodiment of the present invention;

[0072] Figure 8 A schematic diagram of determining vehicle parameters provided by an embodiment of the present invention;

[0073] Fig. 9 A schematic diagram of determining a vanishing point provided by an embodiment of the present invention;

[0074] Fig.10 A schematic diagram of a flow chart of a method for obtaining a road vanishing point position provided by an embodiment of the present invention;

[0075] Fig.11 A schematic diagram of the structure of a device for acquiring road vanishing points provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0077] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0078] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0080] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0081] 1) Lane lines: solid white lines, dashed white lines, solid yellow lines, dashed yellow lines, and road boundary markings that separate different lanes on the road.

[0082] 2) Lane detection refers to finding lane lines in an image and expressing them as a series of points or straight line (curve) equations.

[0083] 3) Vanishing point, also known as the vanishing point, refers to the point on the lane where two or more parallel lines extend toward the distant horizon and converge.

[0084] 4) Bird's-eye view refers to a picture taken from a normal perspective during driving. The parallel lines of the lane lines will intersect at the vanishing point. Through perspective transformation, the lane lines can be transformed into parallel lines. This image after perspective transformation is called a bird's-eye view.

[0085] 5) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0086] The following describes an exemplary application of the device for obtaining the position of the road vanishing point according to the embodiment of the present invention. The device for obtaining the position of the road vanishing point according to the embodiment of the present invention can be implemented by the vehicle device and the server in collaboration, or by the vehicle device itself. Figure 1 Provide explanation.

[0087] Figure 1 The schematic diagram of the architecture of the road vanishing point location acquisition system 100 provided in the embodiment of the present invention is shown in FIG. Figure 1To support an exemplary application, the vehicle device 200 is connected to the server 400 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and a wireless link is used to realize data transmission.

[0088] The vehicle device 200 is provided with an image acquisition device for acquiring images including the second vehicle device. The second vehicle device and the first vehicle device (i.e., the vehicle device 200) may be in the same lane or in different lanes. The vehicle device sends a vanishing point position acquisition request carrying the acquired image to the server 400, and displays the vanishing point position corresponding to the lane where the first vehicle device is located returned by the server on the graphical interface.

[0089] The server 400 is used to receive a request for obtaining a vanishing point position, and perform lane line detection based on the received image to obtain lane line information of the lane where the first vehicle device is located in the image; perform vehicle detection based on the captured image to obtain vehicle parameters of the second vehicle device relative to the image; determine the vanishing point position corresponding to the lane where the first vehicle device is located based on the lane line information and the vehicle parameters, and send the vanishing point position to the vehicle device 200. In this way, the operations of lane line detection, vehicle detection, and determining the road vanishing point position are completed by the server, which reduces the data processing pressure on the vehicle device side and is applicable to the case where the resolution of the source image is relatively large.

[0090] When the device for acquiring the road vanishing point position provided by the embodiment of the present invention is implemented by the vehicle equipment itself, the vehicle equipment 200 is provided with an image acquisition device and a processing chip. The image acquisition device can acquire images around the vehicle equipment during driving, such as acquiring images of a second vehicle equipment located in the same lane or in a different lane as the vehicle logo device 200; the processing chip performs lane line detection based on the acquired image to obtain lane line information of the lane where the vehicle equipment 200 is located in the image, performs vehicle detection based on the acquired image to obtain vehicle parameters of the second vehicle equipment relative to the image, and determines the vanishing point position corresponding to the lane where the vehicle equipment 200 is located based on the lane line information and vehicle parameters. In this way, the determination of the vanishing point position can be realized in real time on the vehicle equipment side, which improves the efficiency of acquiring the vanishing point position and enhances the user experience.

[0091] The device for obtaining the position of a road vanishing point provided in the embodiment of the present invention may be implemented in hardware or in a combination of hardware and software. Various exemplary implementations of the device for obtaining the position of a road vanishing point provided in the embodiment of the present invention are described below.

[0092] Figure 2 The hardware structure diagram of the device for obtaining the road vanishing point position provided by the embodiment of the present invention can be applied to various vehicle devices for running the aforementioned application program in practical applications. Figure 2 , Figure 2The device 200 for obtaining the position of a road vanishing point includes: at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The components in the device 200 for obtaining the position of a road vanishing point are coupled together via a bus system 205. It can be understood that the bus system 205 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 205 is not described in detail. Figure 2 Various buses are labeled as bus system 205 .

[0093] The user interface 203 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0094] It is understood that the memory 202 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a flash memory, etc. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory 202 described in the embodiment of the present invention is intended to include these and any other suitable types of memory.

[0095] The memory 202 in the embodiment of the present invention is used to store various types of data to support the operation of the device 200 for obtaining the road vanishing point position. Examples of such data include: any computer program for operating on the device 200 for obtaining the road vanishing point position, such as an executable program 2021. The program for implementing the method for obtaining the road vanishing point position in the embodiment of the present invention may be included in the executable program 2021.

[0096] The method for obtaining the position of the road vanishing point provided in the embodiment of the present invention can be applied to the processor 201, or implemented by the processor 201. The processor 201 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the method for obtaining the position of the road vanishing point can be completed by the hardware integrated logic circuit in the processor 201 or the instructions in the form of software. The above-mentioned processor 201 can be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 201 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor can be a microprocessor or any conventional processor, etc. The steps of the method provided in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, and the storage medium is located in the memory 202. The processor 201 reads the information in the memory 202 and completes the steps of the method for obtaining the position of the road vanishing point provided in the embodiment of the present invention in combination with its hardware.

[0097] Next, the method for acquiring the road vanishing point position according to the embodiment of the present invention will be described in combination with the exemplary application and implementation of the aforementioned device for acquiring the road vanishing point position according to the embodiment of the present invention.

[0098] Figure 3 A schematic diagram of a method for obtaining a road vanishing point position according to an embodiment of the present invention is shown in FIG. Figure 3 The method for obtaining the road vanishing point position provided by the embodiment of the present invention includes:

[0099] Step 301: A first vehicle device captures an image including a second vehicle device.

[0100] In actual implementation, the vehicle equipment is provided with an image acquisition device and a processing chip. The image acquisition device can acquire images of the surroundings of the vehicle equipment during driving, such as acquiring images including a second vehicle equipment. Here, the second vehicle equipment and the first vehicle equipment may be in the same lane or in different lanes. The processing chip performs subsequent processing based on the acquired images.

[0101] In some embodiments, the first vehicle device and the second vehicle device have the same driving direction, which can be understood as: the first vehicle device is the current vehicle, and the second vehicle device is located in front of the current vehicle. Of course, in other embodiments, the driving directions of the first vehicle device and the second vehicle device may also be different, which is not specifically limited here.

[0102] Step 302: The first vehicle device performs lane line detection based on the collected image to obtain lane line information of the lane where the first vehicle device is located in the image.

[0103] Here, the first vehicle device needs to obtain the lane line information of the lane where the current vehicle is located before determining the road vanishing point position. In actual implementation, the first vehicle device can obtain the lane line information in at least the following ways:

[0104] Performing a region capture on the image to obtain an image region including a lane where the first vehicle device is located; acquiring pixel points belonging to a lane line in the image region; and determining the lane line of the lane where the first vehicle device is located in the image region by performing curve fitting on the acquired pixel points.

[0105] In practical applications, when the image acquisition device of the first vehicle equipment has a large distortion, it is necessary to correct the acquired image by means of a checkerboard or the like before detecting the acquired image, and perform a regional capture on the corrected image to obtain an area of ​​interest including the lane where the current vehicle equipment is located. In this way, only the image area containing the lane is processed, which can improve processing efficiency.

[0106] Since the road environment of the vehicle is constantly changing, for example, the color, lighting and clarity of the lanes will be different at different times and sections. In order to cope with the changing road conditions, it is necessary to combine multiple threshold filtering methods for processing. Here, for lane lines of different colors, lane lines under different lighting conditions, and lane lines of different clarity, different gradient thresholds and color thresholds are used according to different color spaces for different processing, and each processing method is fused to obtain a binary map of the lane lines. Figure 4 A binary schematic diagram of lane lines provided in an embodiment of the present invention extracts pixels belonging to lane lines from the binary image, performs histogram statistics on the pixels of the binary image, and counts the peak points on the left and right sides as the starting point coordinates of the left and right lane lines for curve fitting, thereby determining the lane lines in the image area.

[0107] In practical applications, the binary image may still contain some noisy pixels. In order to accurately detect the lane boundary, it is first necessary to determine which pixels belong to the lane line, so it is necessary to locate the base point of the lane. Figure 5 A schematic diagram of lane line pixel distribution provided by an embodiment of the present invention is shown in FIG. Figure 5 , the base point is Figure 5 The x-axis coordinate where the bottom lane appears. Since the pixels of the lane line are concentrated within a certain range of the x-axis, the thresholded image is divided into two. The pixel distribution peaks on the left and right sides on the x-axis are likely to be the lane line base points.

[0108] After the base points of the left and right lanes are located, the first vehicle device may perform curve fitting on the acquired pixel points in the following manner to determine the lane line of the lane where the first vehicle device is located in the image area:

[0109] In response to the lane lines including at least a left lane line and a right lane line, based on the acquired pixel points, determine the lane line with higher integrity between the left lane line and the right lane line; perform curve fitting on the pixel points of the lane line with higher integrity to determine the lane line of the lane where the first vehicle device is located in the image area.

[0110] In actual implementation, the image acquisition device in the vehicle equipment can collect multiple lane lines. Here, the lane line with better integrity is selected to obtain the vanishing point position. The sliding window polynomial fitting method can be used to locate the pixel of a lane line. Figure 6 A schematic diagram of lane line pixel fitting provided by an embodiment of the present invention is shown in FIG. Figure 6 The pixel points of the left lane line are more complete than those of the right lane line. Therefore, a sliding window polynomial fitting is performed on the left lane line, thereby determining that the lane line of the lane where the current vehicle is located in the image area is the left lane line.

[0111] Step 303: The first vehicle device performs vehicle detection based on the collected image to obtain vehicle parameters of the second vehicle device relative to the image.

[0112] Here, before determining the position of the road vanishing point, the first vehicle device also needs to obtain the vehicle parameters of the second vehicle device in the same lane as the current vehicle relative to the image. In actual implementation, the first vehicle device can obtain the vehicle parameters in at least the following ways:

[0113] The captured image is input into the trained neural network model to identify multiple bounding boxes of the second vehicle equipment; NMS processing is performed on the multiple identified bounding boxes to obtain a target bounding box corresponding to the second vehicle equipment; based on the relative position relationship between the target bounding box and the image, the vehicle parameters of the second vehicle equipment relative to the image are obtained.

[0114] In actual implementation, the first vehicle device identifies multiple bounding boxes of the second vehicle device through a trained neural network model (such as a recurrent neural network (RNN)). Specifically, the first vehicle device inputs the collected image into the input layer of the neural network, passes through the hidden layer, and outputs the multiple bounding boxes of the second vehicle device through the output layer. Here, the training of the neural network model is explained. The first vehicle device can train the above-mentioned neural network model in the following manner: initialize the input layer, intermediate layer and output layer included in the neural network model; construct a training sample set, the training sample set includes vehicle images and non-vehicle images; use the collected image including the second vehicle device as input and the corresponding multiple bounding boxes of the second vehicle device as output, and update the model parameters of the neural network model according to the loss function of the neural network model.

[0115] In actual applications, the borders of the second vehicle equipment output by the above neural network model have many overlapping borders. In order to obtain the target border corresponding to the second vehicle equipment, the multiple identified borders need to be processed by NMS. Specifically: select the border with the highest confidence, calculate its overlap with other borders, and if the overlap is greater than a certain threshold, remove the border and repeat the above process until other borders have lower confidence than the current border. Figure 7 A schematic diagram of a vehicle border processed by NMS according to an embodiment of the present invention, see Figure 7 After multiple bounding boxes are processed by NMS, a target bounding box is obtained. In this way, overlapping bounding boxes are filtered out and the target bounding box is determined.

[0116] Here, after obtaining the target frame of the second vehicle, the vehicle parameters of the second vehicle relative to the image can be determined based on the relative position relationship between the target frame and the image. In some embodiments, the first vehicle device can obtain the vehicle parameters of the second vehicle device relative to the image in the following manner:

[0117] Based on the positions of the upper and lower frames of the target frame in the image, the vehicle height of the second vehicle device relative to the image is determined; based on the position of the upper frame of the target frame in the image, the first distance of the upper frame relative to the first coordinate system direction in the image is determined, and the upper frame is parallel to the first coordinate system direction.

[0118] In some embodiments, the first vehicle device determines the vehicle height of the second vehicle device relative to the image and the distance between the upper frame and a coordinate system parallel to the upper frame based on the position of the target frame in the image.

[0119] For example, Figure 8 A schematic diagram of determining vehicle parameters provided by an embodiment of the present invention, see Figure 8, with the upper left corner of the image as the origin, establish the (x, y) coordinate system, the unit is pixel, the x direction is the first coordinate direction, and the y direction is the second coordinate direction. Assuming that the y direction coordinate of the upper frame of the target frame is y1, and the y direction coordinate of the lower frame is y2, then the height h1 of the second vehicle relative to the image is y2-y1, and the first distance is y1.

[0120] Step 304: The first vehicle device determines a vanishing point position corresponding to the lane where the first vehicle device is located based on the lane line information and the vehicle parameters.

[0121] Here, according to the statistical value, it is found that the horizontal position of the road vanishing point in the image is closely related to the vehicle border. Figure 8 The y-direction coordinate of the vanishing point on the road is contained between the upper and lower borders of the vehicle border, which greatly reduces the search range of the y-direction coordinate of the vanishing point on the road.

[0122] After the lane line information and vehicle parameters are obtained through the above steps, in some embodiments, the first vehicle device can determine the vanishing point position corresponding to the lane where the first vehicle device is located by the following method:

[0123] Determine a second distance of the vanishing point relative to the first coordinate system direction in the image; draw a first straight line parallel to the first coordinate system direction in the image and spaced by the second distance; obtain a first position of an intersection of the drawn first straight line and the lane line in the image, and use the first position as the vanishing point position.

[0124] Here, a first straight line can be drawn according to the vanishing point position relative to the y-direction coordinate in the image, wherein the first straight line can be understood as the horizontal line where the vanishing point is located. The first vehicle device draws the horizontal line where the vanishing point is located and the lane line, and the intersection of the two is the location of the vanishing point. Fig. 9 A schematic diagram of determining a vanishing point provided in an embodiment of the present invention is shown in FIG. Fig. 9 , based on the y-direction coordinate of the vanishing point, only one lane line is needed to obtain the position of the vanishing point.

[0125] In some embodiments, the first vehicle device may determine the second distance of the vanishing point relative to the first coordinate system direction in the image in the following manner:

[0126] The second distance of the vanishing point position relative to the first coordinate system direction in the image is determined by the following relationship: the second distance is equal to the sum of a constant times the vehicle height and the first distance.

[0127] For example, since the horizontal position of the road vanishing point in the image is closely related to the vehicle frame, the vanishing point y-direction coordinate can be defined as a function h=f(y1, h1, vh, a) with the vehicle frame parameters and the camera installation height as variables, where h is the vanishing point y-direction coordinate, y1 is the y-direction coordinate of the upper edge of the vehicle frame, h1 is the vehicle height, vh is the camera installation height (unit: meter), and a is a learnable parameter. The function f for calculating h can be implemented in many ways. For example, a simple formula can be empirically taken as: h=y1+h1*(a / vh-1), where the camera installation height vh can be set to a fixed value according to the height of the vehicle. For example, for ordinary cars, the camera installation height can be set to a uniform value of 1.2 meters. Based on experience or statistics or using a simple fitting algorithm, we can get a reasonable value of a. Corresponding to a camera height of 1.2 meters, a value of about 1.5 can achieve good results. In addition, h=f(y1, h, vh, a) can also be obtained by collecting a large amount of sampling data, that is, the actual different combinations of (h, y1, h1, vh), and using regression fitting methods such as linear fitting, quadratic fitting, support vector machine regression (SVR, Support Vector Regression), etc. to obtain a more accurate function.

[0128] In some embodiments, the first vehicle device may also determine the vanishing point position corresponding to the lane where the first vehicle device is located based on the vehicle parameters and the lane line information in the following manner:

[0129] In response to vehicle parameters including a vehicle height of a second vehicle device relative to the image, a second distance is determined based on a positive correlation between the vehicle height, a second distance of a vanishing point position relative to a first coordinate system direction in the image, and the vehicle height; a second straight line is drawn that is parallel to an upper edge of the image and spaced a second distance apart; a second position in the image of an intersection of the drawn second straight line and the lane line is obtained, and the second position is used as the vanishing point position.

[0130] Here, according to the positive correlation between the height of the second vehicle in the image and the distance of the vanishing point position relative to the x direction, the y direction coordinate of the vanishing point in the image is determined, and the horizontal line where the vanishing point is located in the image is determined accordingly. On the basis of obtaining the horizontal line where the vanishing point is located, only one lane line is needed to obtain the position of the vanishing point.

[0131] By applying the above embodiment of the present invention, the first vehicle device collects an image including the second vehicle device, determines the lane line information of the lane where the first vehicle device is located, and the vehicle parameters of the second vehicle device relative to the image, determines the horizontal line where the vanishing point is located in the image based on the obtained vehicle parameters, and determines the position of the vanishing point based on the intersection of the lane line and the horizontal line where the vanishing point is located. In this way, the position of the vanishing point can be quickly and accurately determined using auxiliary vehicle information and a lane line, which has a wide range of applications and enhances user experience.

[0132] Fig.10 FIG. 1 is a flow chart of a method for obtaining a road vanishing point position provided by an embodiment of the present invention. The method for obtaining a road vanishing point position can be implemented by a vehicle device and a server in collaboration. Fig.10 The method for obtaining the road vanishing point position in an embodiment of the present invention includes:

[0133] Step 1001: A first vehicle device captures an image including a second vehicle device.

[0134] In actual implementation, the first vehicle equipment is provided with an image acquisition device, which can capture the image of the second vehicle equipment through its own image acquisition device (such as a camera). In some embodiments, the first vehicle equipment and the second vehicle equipment have the same driving direction, and the first vehicle equipment and the first vehicle equipment are in the same lane. It can be understood here that the first vehicle equipment is the current vehicle, and the second vehicle equipment is located in front of the current vehicle.

[0135] Of course, in other embodiments, the first vehicle device and the second vehicle device may be in different lanes, or the first vehicle device and the second vehicle device may travel in different directions, which is not specifically limited here.

[0136] Step 1002: The first vehicle device sends a request to obtain a vanishing point position to a server, and the request carries an image including the second vehicle device.

[0137] Here, the first vehicle device sends a request to obtain the vanishing point position to the server, the request carries the first vehicle identification and an image of the second vehicle device, so that the server determines the vanishing point position based on the received image, and returns the determined vanishing point position to the first vehicle device based on the first vehicle identification.

[0138] Step 1003: The server performs lane line detection on the received image based on the vanishing point position acquisition request to obtain lane line information of the lane where the first vehicle device is located in the image.

[0139] Here, the server uses lane line detection technology to perform lane line detection on the received image. In actual implementation, the image acquisition device in the first vehicle equipment captures at least one lane line. Here, the sliding window polynomial fitting method can be used to fit the lane line with better integrity to locate a lane line, and the located lane line is determined as the lane line information of the lane where the first vehicle equipment is located.

[0140] Step 1004: The server performs vehicle detection on the received image based on vehicle detection technology to obtain vehicle parameters of the second vehicle device relative to the image.

[0141] In actual implementation, the server inputs the received image into the trained neural network model to obtain multiple bounding boxes of the second vehicle device, and performs NMS processing on the obtained multiple bounding boxes to obtain a target bounding box corresponding to the second vehicle device. Based on the relative position relationship between the obtained target bounding box and the image, the vehicle parameters of the second vehicle device relative to the image are obtained, see Figure 8 Here, the vehicle parameters include the y-coordinate of the upper frame of the vehicle frame and the vehicle height. Figure 8 The coordinate of the upper frame of the middle vehicle frame in the y direction is y1, and the coordinate of the lower frame in the y direction is y2, then the vehicle height h1 of the second vehicle relative to the image is y2-y1.

[0142] Step 1005: The server obtains the horizontal line where the vanishing point in the image is located based on the vehicle parameters of the second vehicle device relative to the image.

[0143] Here, since the horizontal position of the road vanishing point in the image is closely related to the vehicle frame, after obtaining the vehicle height h1 of the second vehicle device relative to the image and the distance y1 from the upper frame of the vehicle frame to the x-axis, the server calculates the distance horizon of the vanishing point relative to the image according to the formula h=y1+h1*(a / vh-1), where the camera installation height vh can be set to a fixed value according to the height of the vehicle, and a is a learnable parameter. For example, for ordinary cars, the camera installation height can be set to a uniform value of 1.2 meters. Accordingly, based on experience or statistics or a simple fitting algorithm, a is taken as 1.5. See Fig. 9 , after determining h, we can draw a straight line parallel to the x-axis with a spacing of h (such as Fig. 9 Number 1), this straight line is the horizontal line where the vanishing point in the image is located.

[0144] Step 1006: The server determines the location of the vanishing point based on the obtained lane line information and the horizontal line where the vanishing point is located.

[0145] Here, after obtaining the horizontal line where the vanishing point is located, the intersection of the horizontal line where the vanishing point is located and the lane line obtained by the above detection can be drawn (such as Fig. 9 Number 2), the position of the intersection is determined as the position of the vanishing point.

[0146] Step 1007: The server returns the vanishing point position to the first vehicle device.

[0147] In practical applications, the vanishing point position is key input information for front vehicle distance calculation, pedestrian distance estimation, lane line detection and other technologies in ADAS. The first vehicle equipment can perform corresponding driving assistance based on the determined vanishing point position.

[0148] For example, the vanishing point and the lane line affect each other. When the lane line is known, the position of the vanishing point can be determined by the point where the two lane lines intersect. When the position of the vanishing point is known, the lane line can be determined according to the position of the vanishing point. Here, the first vehicle device detects the lane line based on the determined position of the vanishing point, which reduces the difficulty of lane line detection.

[0149] After the above steps, the vehicle equipment sends the collected images to the server for subsequent processing, that is, the operations of lane line detection, vehicle detection and determining the position of the road vanishing point are completed by the server. In this way, the data processing pressure on the vehicle equipment side is reduced, and it is suitable for situations where the resolution of the source image is large.

[0150] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.

[0151] In the assisted driving system, the vehicle equipment is equipped with an image acquisition device and a processing chip. The image acquisition device can acquire the surrounding environment information of the vehicle equipment during driving or stopping, such as acquiring an image including the vehicle in front. Here, the vehicle in front and the current vehicle equipment can be located in the same lane or in different lanes. The processing chip performs lane line detection based on the acquired image to obtain the lane line information of the lane where the current vehicle is located in the image, performs vehicle detection based on the acquired image to obtain the vehicle parameters of the vehicle in front relative to the image, and determines the vanishing point position in the image based on the obtained lane line information and vehicle parameters.

[0152] Specifically, according to the statistical value, it can be found that the y-coordinate of the vanishing point in the image, i.e., the position of the horizontal line, is closely related to the vehicle frame. Figure 8 As shown in the figure, the y coordinate of the vanishing point is contained between the upper and lower edges of the vehicle frame, which greatly reduces the search range of the vanishing point y coordinate. It is only necessary to find the specific position relationship between the frame and the vanishing point y coordinate within this small range. In the assisted driving system, vehicle detection and front vehicle warning are essential functions. If you want to use the vehicle frame information, you do not need additional calculations. You can use vehicle detection technology to detect the collected image and obtain the vehicle frame parameters in the image.

[0153] Here, the vanishing point y-direction coordinate can be defined as a function with the vehicle frame parameters and the camera installation height as variables, that is, h = f (yt, h1, vh, a), where h is the vanishing point y-direction coordinate, yt is the y-direction coordinate of the upper edge of the vehicle frame, h1 is the vehicle height (unit: pixel), vh is the camera installation height (unit: meter), and a is a learnable parameter. The function f for calculating h can be designed in many ways. For example, a simple formula can be empirically taken as: h = yt + h1 * (a / vh-1). The camera installation height vh can be set to a fixed value according to the height of the vehicle. For example, for ordinary cars, the camera installation height can be uniformly set to 1.2 meters, and a reasonable value of a can be obtained based on experience or statistics or a simple fitting algorithm. For example, corresponding to a camera height of 1.2 meters, a value of about 1.5 can achieve good results. In addition, there are many ways to design the formula h=f(yt, h1, vh, a). For example, by collecting a large amount of sampling data, that is, different combinations of (h, yt, h1, vh) that actually appear, various regression fitting methods can be used, such as linear fitting, quadratic fitting and SVR, to obtain a more accurate formula.

[0154] On the basis of determining the horizontal line where the vanishing point is located in the image, the lane line detection technology is used to detect the lane line of the collected image. Only one lane line needs to be detected, and the position of the vanishing point can be obtained according to the horizontal line where the vanishing point is located and one lane line. The position of the vanishing point is the intersection of the horizontal line and the lane line, such as Fig. 9 shown.

[0155] In this way, the above method can quickly and accurately obtain the vanishing point position, and has a wide range of applications, especially in some situations where the existing solutions cannot obtain the vanishing point position, such as in severe situations where lane lines are blocked or blurred due to traffic jams, making it impossible to obtain two lane lines for vanishing point determination. The method provided by the present invention can also accurately obtain the vanishing point position.

[0156] Next, the software implementation of the device for obtaining road vanishing points provided in the embodiment of the present invention is described. Fig.11 A schematic diagram of the structure of a device for obtaining a road vanishing point according to an embodiment of the present invention is shown in FIG. Fig.11 The device 11 for acquiring the road vanishing point in the embodiment of the present invention is applied to a first vehicle device and includes:

[0157] A collection unit 1101, configured to collect an image including a second vehicle device;

[0158] The first detection unit 1102 is used to perform lane line detection based on the collected image to obtain lane line information of the lane where the first vehicle device is located in the image;

[0159] The second detection unit 1103 is used to perform vehicle detection based on the collected image and obtain vehicle parameters of the second vehicle device relative to the image;

[0160] The determination unit 1104 is used to determine the vanishing point position corresponding to the lane where the first vehicle device is located based on the lane line information and the vehicle parameters.

[0161] In some embodiments, the first detection unit 1102 is used to perform region capture on the image to obtain an image region including the lane where the first vehicle device is located;

[0162] Get the pixel points belonging to the lane line in the image area;

[0163] The lane line of the lane where the first vehicle equipment is located in the image area is determined by performing curve fitting on the acquired pixel points.

[0164] In some embodiments, the first detection unit 1102 is configured to determine, in response to the lane line at least including a left lane line and a right lane line, a lane line with higher integrity between the left lane line and the right lane line based on the acquired pixel points;

[0165] Curve fitting is performed on the pixel points of the lane line with higher integrity to determine the lane line of the lane where the first vehicle device is located in the image area.

[0166] In some embodiments, the second detection unit 1103 is used to input the collected image into the trained neural network model to identify multiple borders of the second vehicle equipment;

[0167] Performing NMS processing on the multiple identified bounding boxes to obtain a target bounding box corresponding to the second vehicle device;

[0168] Based on the relative position relationship between the target frame and the image, vehicle parameters of the second vehicle device relative to the image are obtained.

[0169] In some embodiments, the second detection unit 1103 is used to determine the vehicle height of the second vehicle device relative to the image based on the positions of the upper frame and the lower frame of the target frame in the image;

[0170] Based on the position of the upper frame of the target frame in the image, a first distance of the upper frame relative to a first coordinate system direction in the image is determined, and the upper frame is parallel to the first coordinate system direction.

[0171] In some embodiments, the determining unit 1104 is configured to determine a second distance of the vanishing point position relative to the first coordinate system direction in the image;

[0172] Draw a first straight line parallel to the direction of the first coordinate system in the image and spaced by a second distance;

[0173] A first position of an intersection of the first straight line and the lane line in the image is obtained, and the first position is used as the vanishing point position.

[0174] In some embodiments, the determining unit 1104 is configured to determine a second distance of the vanishing point position relative to the first coordinate system direction in the image by using the following relationship:

[0175] The second distance is equal to the sum of a constant multiple of the vehicle height and the first distance.

[0176] In some embodiments, the determining unit 1104 is configured to determine the second distance in response to the vehicle parameter including the vehicle height of the second vehicle device relative to the image, based on the positive correlation between the vehicle height, the second distance of the vanishing point position relative to the first coordinate system direction in the image, and the vehicle height;

[0177] Draw a second straight line parallel to the upper edge of the image and spaced a second distance apart;

[0178] A second position of an intersection of the drawn second straight line and the lane line in the image is obtained, and the second position is used as a vanishing point position.

[0179] The embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method for obtaining the location of a road vanishing point provided by the embodiment of the present invention, for example, Figure 3 The method for obtaining the road vanishing point position is shown.

[0180] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0181] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0182] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0183] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0184] In summary, the method for obtaining the position of the vanishing point of a road provided by the embodiment of the present invention collects an image including the second vehicle device, determines the lane line information of the lane where the first vehicle device is located, and the vehicle parameters of the second vehicle device relative to the image, determines the horizontal line where the vanishing point is located in the image based on the obtained vehicle parameters, and determines the position of the vanishing point based on the intersection of the lane line and the horizontal line where the vanishing point is located. In this way, the position of the vanishing point can be quickly and accurately determined using auxiliary vehicle information and a lane line, which has a wide range of uses and enhances user experience.

[0185] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

Claims

1. A method for obtaining a road vanishing point position, characterized in that: Applied to a first vehicle device, the method comprises: capturing an image including a second vehicle device; Performing lane line detection based on the acquired image to obtain lane line information of the lane where the first vehicle device is located in the image; Selecting a lane line with higher integrity from the lane line information; Performing vehicle detection based on the acquired image to obtain vehicle parameters of the second vehicle device relative to the image, wherein the vehicle parameters include a vehicle height of the second vehicle device relative to the image, and the vehicle height is determined based on positions of an upper frame and a lower frame of a target frame of the second vehicle in the image; Based on the position of the upper frame of the target frame in the image, determining a first distance of the upper frame relative to a first coordinate system direction in the image, wherein the upper frame is parallel to the first coordinate system direction; The sum of the constant times of the vehicle height and the first distance is taken as the second distance; wherein the vehicle height and the vanishing point position relative to the first coordinate system direction in the image are positively correlated with the vehicle height; Draw a second straight line parallel to the upper edge of the image and spaced apart by the second distance; A second position of an intersection of the second straight line and the lane line with higher integrity in the image is obtained, and the second position is used as the vanishing point position.

2. The method according to claim 1, characterized in that The performing lane line detection based on the acquired image to obtain lane line information of the lane where the first vehicle device is located in the image includes: Performing a region interception on the image to obtain an image region including the lane where the first vehicle device is located; Acquire pixel points belonging to lane lines in the image area; The lane line of the lane where the first vehicle device is located in the image area is determined by performing curve fitting on the acquired pixel points.

3. The method according to claim 2, characterized in that The selecting a lane line with higher integrity from the lane line information includes: In response to the lane line at least including a left lane line and a right lane line, based on the acquired pixel points, determining a lane line with higher integrity between the left lane line and the right lane line; The method further comprises: Curve fitting is performed on the pixel points of the lane line with higher integrity to determine the lane line of the lane where the first vehicle device is located in the image area.

4. The method according to claim 1, characterized in that The performing of vehicle detection based on the acquired image to obtain vehicle parameters of the second vehicle device relative to the image includes: Inputting the collected image into a trained neural network model to identify a plurality of borders of the second vehicle equipment; Performing non-maximum suppression (NMS) processing on the multiple identified bounding boxes to obtain a target bounding box corresponding to the second vehicle equipment; Based on the relative position relationship between the target frame and the image, vehicle parameters of the second vehicle device relative to the image are obtained.

5. A device for obtaining the position of a road vanishing point, characterized in that: Applied to a first vehicle device, the device comprises: A collection unit, used for collecting an image including the second vehicle equipment; A first detection unit is used to perform lane line detection based on the acquired image to obtain lane line information of the lane where the first vehicle device is located in the image; and select a lane line with higher integrity from the lane line information; A second detection unit is used to perform vehicle detection based on the acquired image to obtain vehicle parameters of the second vehicle device relative to the image, wherein the vehicle parameters include a vehicle height of the second vehicle device relative to the image, and the vehicle height is determined based on positions of an upper frame and a lower frame of a target frame of the second vehicle in the image; A determination unit is used to determine a first distance of the upper frame of the target frame relative to a first coordinate system direction in the image based on the position of the upper frame of the target frame in the image, wherein the upper frame is parallel to the first coordinate system direction; taking the sum of a constant multiple of the vehicle height and the first distance as a second distance; wherein the vehicle height and the second distance of the vanishing point position relative to the first coordinate system direction in the image are positively correlated with the vehicle height; drawing a second straight line parallel to the upper edge of the image and spaced by the second distance; obtaining a second position in the image of an intersection of the second straight line and a lane line with higher integrity, and taking the second position as the vanishing point position.

6. A device for obtaining the position of a road vanishing point, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method for obtaining the position of a road vanishing point as described in any one of claims 1 to 4 when executing the executable instructions stored in the memory.

7. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the method for obtaining the road vanishing point position according to any one of claims 1 to 4 is implemented.

Images