Lane vanishing point detection method and system, electronic equipment and storage medium

Abstract

The invention relates to the technical field of deep learning, and particularly discloses a lane vanishing point detection method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-detected image containing a target road region; performing vanishing point detection on the image to be detected by using a trained preset lane vanishing point detection model to obtain a vanishing point position of the image to be detected; wherein the preset lane vanishing point detection model comprises a lane segmentation model and a vanishing point detection model; the lane segmentation model is used for extracting a lane line image in the to-be-detected image; the vanishing point detection model is used for determining the vanishing point position according to the lane line image. By effectively combining the lane segmentation result and vanishing point detection, efficient vanishing point detection can be realized, and the accuracy of lane vanishing point detection is improved.

Description

Background Art

[0002] Vanishing Point (VP) is an important geometric concept in computer vision and is widely used in perspective analysis, autonomous driving, robot vision, 3D reconstruction and other fields. A vanishing point means that when a set of parallel lines (two or more lines) are projected in an image, their extension lines will eventually intersect at a point. By detecting the vanishing point in the image, we can obtain information about the spatial layout of the objects in the image. The detection of vanishing points not only helps the system understand the relative position of the vehicle and the road, but also provides important clues for the prediction of lane lines.

[0003] At present, traditional vanishing point detection methods usually rely on low-level feature methods such as feature extraction and line detection, such as Hough Transform. These methods usually need to extract the straight line features in the image first, and then estimate the vanishing point based on these features. However, traditional methods have the following problems: ① They are sensitive to noise, complex scenes and different lighting conditions in the image; ② They need to rely on manually designed features and are difficult to adapt to various complex scenes; ③ The computational efficiency is low and it is difficult to meet real-time requirements.

[0004] Lane segmentation technology is used to accurately segment the lane area from the input road image. Lane line features (such as lanes, solid lines and dashed lines) are important geometric information in the lane segmentation results, and often provide clear clues for vanishing point detection. At present, there are lane segmentation methods based on deep learning, such as ERFNet, which realizes accurate segmentation of lane lines in images. ERFNet is an efficient deep convolutional neural network architecture. ERFNet achieves good computing performance through an efficient residual factorized network structure and is widely used in semantic segmentation and lane segmentation tasks. However, the existing lane segmentation model is not specifically optimized for vanishing point detection, and its ability to extract the location of vanishing points from lane line features is relatively limited.

[0005] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a lane vanishing point detection method, system, electronic device and storage medium.

[0007] In a first aspect, the present invention provides a lane vanishing point detection method, and the technical solution of the method is as follows:

[0008] Acquire an image to be tested that includes a target road area;

[0009] Using the trained preset lane vanishing point detection model, vanishing point detection is performed on the image to be tested to obtain the vanishing point position of the image to be tested; wherein the preset lane vanishing point detection model includes: a lane segmentation model and a vanishing point detection model; the lane segmentation model is used to: extract the lane line image in the image to be tested; the vanishing point detection model is used to: determine the vanishing point position according to the lane line image.

[0010] The beneficial effects of a lane vanishing point detection method of the present invention are as follows:

[0011] The method of the present invention can achieve efficient vanishing point detection by effectively combining lane segmentation results with vanishing point detection, thereby improving the accuracy of lane vanishing point detection.

[0012] Based on the above solution, a lane vanishing point detection method of the present invention can also be improved as follows.

[0013] In an optional manner, the vanishing point detection model includes: a first downsampling layer, a second downsampling layer, a third downsampling layer, a fourth downsampling layer, an average pooling layer and a fully connected layer which are sequentially connected;

[0014] The first downsampling layer is used to downsample the lane line image to obtain a first feature image; the second downsampling layer is used to downsample the first feature image to obtain a second feature image; the third downsampling layer is used to downsample the second feature image to obtain a third feature image; the fourth downsampling layer is used to downsample the third feature image to obtain a fourth feature image; the average pooling layer is used to perform average pooling on the fourth feature image to obtain a fifth feature image; the fully connected layer is used to perform feature mapping on the fifth feature image to obtain a vanishing point detection result including the vanishing point position.

[0015] In an optional manner, the lane line image includes: a lane solid line image and / or a lane dashed line image.

[0016] In an optional manner, the lane segmentation model is an ERFNet model.

[0017] In an optional manner, the training process of the preset lane vanishing point detection model is:

[0018] Acquire a plurality of sample images including a sample road area;

[0019] Inputting any sample image including a sample road area into the lane segmentation model to obtain a sample lane line image in the any sample image, and determining a lane line loss of the any sample image according to a difference between the sample lane line image in the any sample image and the label lane line image;

[0020] Inputting a sample lane line image in any of the sample images into the vanishing point detection model to obtain a sample vanishing point position of the any of the sample images, and determining a vanishing point detection loss of the any of the sample images according to the sample vanishing point position of the any of the sample images and a label vanishing point position;

[0021] Determining a target loss for any of the sample images according to a lane line loss, a vanishing point detection loss, and a loss weight for balancing the lane line loss and the vanishing point detection loss, until a target loss for each sample image is obtained;

[0022] Optimizing the network parameters of the preset lane vanishing point detection model according to the target loss of each sample image to obtain an optimized lane vanishing point detection model, and determining whether the optimized lane vanishing point detection model meets preset training conditions to obtain a determination result;

[0023] When the judgment result is no, the optimized lane vanishing point detection model is used as the preset lane vanishing point detection model, and the step of inputting any sample image containing a sample road area into the lane segmentation model is returned to be executed until the judgment result is yes, and it is determined that the training of the preset lane vanishing point detection model is completed.

[0024] In a second aspect, the present invention provides a lane vanishing point detection system, the technical solution of which is as follows:

[0025] Including: acquisition module and detection module;

[0026] The acquisition module is used to: acquire an image to be tested that includes a target road area;

[0027] The detection module is used to: use the trained preset lane vanishing point detection model to perform vanishing point detection on the image to be tested to obtain the vanishing point position of the image to be tested; wherein the preset lane vanishing point detection model includes: a lane segmentation model and a vanishing point detection model; the lane segmentation model is used to: extract the lane line image in the image to be tested; the vanishing point detection model is used to: determine the vanishing point position according to the lane line image.

[0028] The beneficial effects of a lane vanishing point detection system of the present invention are as follows:

[0029] The system of the present invention can achieve efficient vanishing point detection by effectively combining lane segmentation results with vanishing point detection, thereby improving the accuracy of lane vanishing point detection.

[0030] Based on the above solution, the lane vanishing point detection system of the present invention can also be improved as follows.

[0031] In an optional manner, the vanishing point detection model includes: a first downsampling layer, a second downsampling layer, a third downsampling layer, a fourth downsampling layer, an average pooling layer and a fully connected layer which are sequentially connected;

[0032] The first downsampling layer is used to downsample the lane line image to obtain a first feature image; the second downsampling layer is used to downsample the first feature image to obtain a second feature image; the third downsampling layer is used to downsample the second feature image to obtain a third feature image; the fourth downsampling layer is used to downsample the third feature image to obtain a fourth feature image; the average pooling layer is used to perform average pooling on the fourth feature image to obtain a fifth feature image; the fully connected layer is used to perform feature mapping on the fifth feature image to obtain a vanishing point detection result including the vanishing point position.

[0033] In an optional manner, the lane line image includes: a lane solid line image and / or a lane dashed line image.

[0034] In a third aspect, a technical solution of an electronic device of the present invention is as follows:

[0035] The method comprises a memory, a processor and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the lane vanishing point detection method of the present invention are implemented.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution:

[0037] Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the lane vanishing point detection method of the present invention.

[0038] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0040] Figure 1 A schematic diagram of a process flow of an embodiment of a lane vanishing point detection method of the present invention;

[0041] Figure 2It is a schematic diagram of the overall principle of the preset lane vanishing point detection model;

[0042] Figure 3 It is a schematic diagram of the structure of the downsampling layer;

[0043] Figure 4 It is a structural schematic diagram of an embodiment of a lane vanishing point detection system of the present invention;

[0044] Figure 5 The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0045] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0046] Figure 1 A flow chart of an embodiment of a lane vanishing point detection method provided by the present invention is shown, and the lane vanishing point detection method can be executed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the lane vanishing point detection method by calling computer-readable instructions stored in a memory through a processor. Figure 1 As shown, the following steps are included:

[0047] S1. Acquire an image to be tested that includes a target road area.

[0048] The target road area is: the road area to be detected for lane vanishing points in this embodiment, which can be an urban road area, a town road area, a township road area, or a series of road areas with lane lines. The image to be tested can be set on a gantry on both sides of the road or above the road, and there is no limitation here.

[0049] S2. Using the trained preset lane vanishing point detection model, perform vanishing point detection on the image to be tested to obtain the vanishing point position of the image to be tested.

[0050] Among them, the preset lane vanishing point detection model includes: a lane segmentation model and a vanishing point detection model; the lane segmentation model is used to: extract the lane line image in the image to be tested; the vanishing point detection model is used to: determine the vanishing point position according to the lane line image.

[0051] In an optional manner, the lane segmentation model is an ERFNet model. The input of the ERFNet model is: the image to be tested; the output of the ERFNet model is: the segmentation results of the lane area and lane lines (solid lines and dashed lines), and the segmentation results are specifically divided into seven categories: background image, driving lane image, emergency lane image, lane solid line image, lane dashed line image, ramp area image, guide line area image, that is, the output of the ERFNet model is 1024*512*7, where 7 channels represent 7 1024*512 feature images, and each feature image represents a category.

[0052] In an optional manner, the lane line image includes: a lane solid line image and / or a lane dashed line image.

[0053] Wherein, the lane solid line image and / or the lane dashed line image in the segmentation result of the image to be tested is used as the lane line image.

[0054] It should be noted that when there are multiple lane lines in the image to be tested, the images corresponding to two or more lane lines can be used as lane line images. For example, in a road monitoring image, the images corresponding to all lane lines can be used as lane line images to determine the vanishing point positions corresponding to all lane lines; in a vehicle navigation image, the images corresponding to the lane lines on both sides of the lane where the vehicle is currently located can be used as lane line images to determine the vanishing point position corresponding to the current lane.

[0055] In an alternative approach, Figure 2 As shown, the vanishing point detection model includes: a first downsampling layer, a second downsampling layer, a third downsampling layer, a fourth downsampling layer, an average pooling layer and a fully connected layer which are sequentially connected. Figure 2 The DownsamplerBlock in is the downsampling layer.

[0056] The first downsampling layer is used to downsample the lane line image to obtain a first feature image; the second downsampling layer is used to downsample the first feature image to obtain a second feature image; the third downsampling layer is used to downsample the second feature image to obtain a third feature image; the fourth downsampling layer is used to downsample the third feature image to obtain a fourth feature image; the average pooling layer is used to perform average pooling on the fourth feature image to obtain a fifth feature image; the fully connected layer is used to perform feature mapping on the fifth feature image to obtain a vanishing point detection result including a vanishing point position.

[0057] The lane line image is a 1024*512*2 image, the first feature image is a 512*256*6 image, the second feature image is a 256*128*32 image, the third feature image is a 128*64*64 image, the fourth feature image is a 64*32*128 image, and the fifth feature image is an image after average pooling. The average pooling layer is used to compress the fourth feature image into a smaller space. The vanishing point detection result is a three-dimensional vector (x, y, score); x is the horizontal coordinate of the vanishing point position, y is the vertical coordinate of the vanishing point position, and score is the vanishing point score.

[0058] The structure diagram of the downsampling layers (the first downsampling layer to the fourth downsampling layer) in this embodiment is as follows: Figure 3 As shown. Figure 3 As shown in the figure, the convolution kernel size of the Conv operation is 3x3, padding=1, and stride=2; the pooling window size of the MaxPool operation is 2x2, and stride=2; therefore, the size of the feature image will be reduced by half after the downsampling layer.

[0059] It should be noted that there is only one vanishing point position in this embodiment. Whether it is the vanishing point position for all lane lines or the vanishing point position for the lane line corresponding to a certain lane can be selected and adjusted according to actual conditions and is not limited here.

[0060] In an optional manner, the training process of the preset lane vanishing point detection model is:

[0061] A plurality of sample images containing sample road areas are obtained.

[0062] Input any sample image containing a sample road area into the lane segmentation model to obtain a sample lane line image in the any sample image, and determine the lane line loss of the any sample image based on the difference between the sample lane line image in the any sample image and the label lane line image.

[0063] A sample lane line image in any of the sample images is input into the vanishing point detection model to obtain a sample vanishing point position of the any of the sample images, and a vanishing point detection loss of the any of the sample images is determined according to the sample vanishing point position of the any of the sample images and the label vanishing point position.

[0064] According to the lane line loss, the vanishing point detection loss, and the loss weight for balancing the lane line loss and the vanishing point detection loss of any sample image, the target loss of any sample image is determined until the target loss of each sample image is obtained.

[0065] According to the target loss of each sample image, the network parameters of the preset lane vanishing point detection model are optimized to obtain an optimized lane vanishing point detection model, and it is determined whether the optimized lane vanishing point detection model meets the preset training conditions to obtain a determination result.

[0066] When the judgment result is no, the optimized lane vanishing point detection model is used as the preset lane vanishing point detection model, and the step of inputting any sample image containing a sample road area into the lane segmentation model is returned to be executed until the judgment result is yes, and it is determined that the training of the preset lane vanishing point detection model is completed.

[0067] It should be noted that in order to optimize the lane segmentation and vanishing point detection tasks at the same time, a joint loss function is used, which includes two parts: lane line loss and vanishing point detection loss. The lane line loss uses the cross entropy loss, which is suitable for multi-class classification tasks. The vanishing point detection loss uses the mean square error loss to calculate the difference between the coordinates of the vanishing point and the true vanishing point, and minimize the difference. The final target loss function is: Loss total =Loss lane +λLoss point ; λ is the loss weight between lane line loss and vanishing point detection loss, Loss lane is the lane line loss, Loss point Detect loss for vanishing points.

[0068] The technical solution of this embodiment outputs the vanishing point coordinates by regression, and optimizes them in combination with the geometric information of the lane line; combines the lane line feature loss and the vanishing point regression loss to improve the detection accuracy; and effectively combines the lane segmentation results with the vanishing point detection to achieve efficient vanishing point detection, thereby improving the accuracy of lane vanishing point detection and making up for the shortcomings of the prior art in utilizing the geometric relationship between lane lines and vanishing points.

[0069] Figure 4 FIG. 2 is a schematic diagram showing a structure of an embodiment of a lane vanishing point detection system 200 provided by the present invention. Figure 4 As shown, the system 200 includes: an acquisition module 210 and a detection module 220;

[0070] The acquisition module 210 is used to: acquire an image to be tested that includes a target road area;

[0071] The detection module 220 is used to: use the trained preset lane vanishing point detection model to perform vanishing point detection on the image to be tested to obtain the vanishing point position of the image to be tested; wherein the preset lane vanishing point detection model includes: a lane segmentation model and a vanishing point detection model; the lane segmentation model is used to: extract the lane line image in the image to be tested; the vanishing point detection model is used to: determine the vanishing point position according to the lane line image.

[0072] In an optional manner, the vanishing point detection model includes: a first downsampling layer, a second downsampling layer, a third downsampling layer, a fourth downsampling layer, an average pooling layer and a fully connected layer which are sequentially connected;

[0073] The first downsampling layer is used to downsample the lane line image to obtain a first feature image; the second downsampling layer is used to downsample the first feature image to obtain a second feature image; the third downsampling layer is used to downsample the second feature image to obtain a third feature image; the fourth downsampling layer is used to downsample the third feature image to obtain a fourth feature image; the average pooling layer is used to perform average pooling on the fourth feature image to obtain a fifth feature image; the fully connected layer is used to perform feature mapping on the fifth feature image to obtain a vanishing point detection result including the vanishing point position.

[0074] In an optional manner, the lane line image includes: a lane solid line image and / or a lane dashed line image.

[0075] In an optional manner, the lane segmentation model is an ERFNet model.

[0076] In an optional manner, the method further includes: a training module; the training module is used to:

[0077] Acquire a plurality of sample images including a sample road area;

[0078] Inputting any sample image including a sample road area into the lane segmentation model to obtain a sample lane line image in the any sample image, and determining a lane line loss of the any sample image according to a difference between the sample lane line image in the any sample image and the label lane line image;

[0079] Inputting a sample lane line image in any of the sample images into the vanishing point detection model to obtain a sample vanishing point position of the any of the sample images, and determining a vanishing point detection loss of the any of the sample images according to the sample vanishing point position of the any of the sample images and a label vanishing point position;

[0080] Determining a target loss for any of the sample images according to a lane line loss, a vanishing point detection loss, and a loss weight for balancing the lane line loss and the vanishing point detection loss, until a target loss for each sample image is obtained;

[0081] Optimizing the network parameters of the preset lane vanishing point detection model according to the target loss of each sample image to obtain an optimized lane vanishing point detection model, and determining whether the optimized lane vanishing point detection model meets preset training conditions to obtain a determination result;

[0082] When the judgment result is no, the optimized lane vanishing point detection model is used as the preset lane vanishing point detection model, and the step of inputting any sample image containing a sample road area into the lane segmentation model is returned to be executed until the judgment result is yes, and it is determined that the training of the preset lane vanishing point detection model is completed.

[0083] It should be noted that the beneficial effects of the lane vanishing point detection system provided in the above embodiment are the same as the beneficial effects of the lane vanishing point detection method, which will not be described in detail here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be described in detail here.

[0084] The lane vanishing point detection system of the present invention may be a computer program (including program code) running in a computer device. For example, the lane vanishing point detection system of the present invention is an application software that can be used to execute the corresponding steps in the lane vanishing point detection method of the present invention.

[0085] In some embodiments, the lane vanishing point detection system of the present invention may be implemented in a combination of software and hardware. As an example, the lane vanishing point detection system of the present invention may be a processor in the form of a hardware decoding processor, which is programmed to execute the lane vanishing point detection method of the present invention. For example, the processor in the form of a hardware decoding processor may be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0086] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The name of a module does not limit the module itself in some cases.

[0087] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above lane vanishing point detection methods is implemented. That is, an electronic device according to an embodiment of the present invention may include but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the lane vanishing point detection method shown in any embodiment of the present invention by calling the computer program.

[0088] In an alternative embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0089] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0090] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 In the figure, only one thick line is used to represent the bus 4002, but this does not mean that there is only one bus or one type of bus.

[0091] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0092] The memory 4003 is used to store the application code (computer program) for executing the solution of the present invention, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0093] Among them, the electronic device can also be a terminal device, and the terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, and a smart car device.

[0094] It should be noted that Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0095] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, and when the computer program is executed by a processor, any of the above lane vanishing point detection methods is implemented.

[0096] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0097] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above lane vanishing point detection method.

[0098] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0099] It should be understood that the flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0100] The computer-readable storage medium provided by the embodiment of the present invention may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or component.

[0101] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0102] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the disclosure scope involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.

[0103] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0104] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0105] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A lane vanishing point detection method, characterized in that: include: Acquire an image to be tested that includes a target road area; Using the trained preset lane vanishing point detection model, vanishing point detection is performed on the image to be tested to obtain the vanishing point position of the image to be tested; wherein the preset lane vanishing point detection model includes: a lane segmentation model and a vanishing point detection model; the lane segmentation model is used to: extract the lane line image in the image to be tested; the vanishing point detection model is used to: determine the vanishing point position according to the lane line image.

2. The lane vanishing point detection method according to claim 1, characterized in that: The vanishing point detection model comprises: a first downsampling layer, a second downsampling layer, a third downsampling layer, a fourth downsampling layer, an average pooling layer and a fully connected layer which are sequentially connected; The first downsampling layer is used to downsample the lane line image to obtain a first feature image; the second downsampling layer is used to downsample the first feature image to obtain a second feature image; the third downsampling layer is used to downsample the second feature image to obtain a third feature image; the fourth downsampling layer is used to downsample the third feature image to obtain a fourth feature image; the average pooling layer is used to perform average pooling on the fourth feature image to obtain a fifth feature image; the fully connected layer is used to perform feature mapping on the fifth feature image to obtain a vanishing point detection result including the vanishing point position.

3. The lane vanishing point detection method according to claim 1, characterized in that: The lane line image includes: a lane solid line image and / or a lane dashed line image.

4. The lane vanishing point detection method according to claim 1, characterized in that: The lane segmentation model is an ERFNet model.

5. The lane vanishing point detection method according to any one of claims 1 to 4, characterized in that: The training process of the preset lane vanishing point detection model is as follows: Acquire a plurality of sample images including a sample road area; Inputting any sample image including a sample road area into the lane segmentation model to obtain a sample lane line image in the any sample image, and determining a lane line loss of the any sample image according to a difference between the sample lane line image in the any sample image and the label lane line image; Inputting a sample lane line image in any of the sample images into the vanishing point detection model to obtain a sample vanishing point position of the any of the sample images, and determining a vanishing point detection loss of the any of the sample images according to the sample vanishing point position of the any of the sample images and a label vanishing point position; Determining a target loss for any of the sample images according to a lane line loss, a vanishing point detection loss, and a loss weight for balancing the lane line loss and the vanishing point detection loss, until a target loss for each sample image is obtained; Optimizing the network parameters of the preset lane vanishing point detection model according to the target loss of each sample image to obtain an optimized lane vanishing point detection model, and determining whether the optimized lane vanishing point detection model meets preset training conditions to obtain a determination result; When the judgment result is no, the optimized lane vanishing point detection model is used as the preset lane vanishing point detection model, and the step of inputting any sample image containing a sample road area into the lane segmentation model is returned to be executed until the judgment result is yes, and it is determined that the training of the preset lane vanishing point detection model is completed.

6. A lane vanishing point detection system, characterized in that: include: Acquisition module and detection module; The acquisition module is used to: acquire an image to be tested that includes a target road area; The detection module is used to: use the trained preset lane vanishing point detection model to perform vanishing point detection on the image to be tested to obtain the vanishing point position of the image to be tested; wherein the preset lane vanishing point detection model includes: a lane segmentation model and a vanishing point detection model; the lane segmentation model is used to: extract the lane line image in the image to be tested; the vanishing point detection model is used to: determine the vanishing point position according to the lane line image.

7. The lane vanishing point detection system according to claim 6, characterized in that: The vanishing point detection model comprises: a first downsampling layer, a second downsampling layer, a third downsampling layer, a fourth downsampling layer, an average pooling layer and a fully connected layer which are sequentially connected; The first downsampling layer is used to downsample the lane line image to obtain a first feature image; the second downsampling layer is used to downsample the first feature image to obtain a second feature image; the third downsampling layer is used to downsample the second feature image to obtain a third feature image; the fourth downsampling layer is used to downsample the third feature image to obtain a fourth feature image; the average pooling layer is used to perform average pooling on the fourth feature image to obtain a fifth feature image; the fully connected layer is used to perform feature mapping on the fifth feature image to obtain a vanishing point detection result including the vanishing point position.

8. The lane vanishing point detection system according to claim 6, characterized in that: The lane line image includes: a lane solid line image and / or a lane dashed line image.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the electronic device implements the lane vanishing point detection method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer-readable storage medium implements the lane vanishing point detection method according to any one of claims 1 to 5.

Images