Lane departure detection method, device and computer storage medium

Abstract

The present application discloses a lane departure detection method, device and computer storage medium, the method comprising: obtaining at least two frames of images including lane information of a road; the at least two frames of images are simultaneously shot by at least two image capture devices arranged side by side in a preset direction of a vehicle; horizontally splicing the at least two frames of images in the order of shooting positions, and determining the first lane information corresponding to the target lane of the vehicle in the road based on the obtained straight-stitched image of the road; obtaining the second lane information corresponding to the positioning lane of the vehicle in the road based on the positioning information of the vehicle; detecting whether the target lane and the positioning lane are the same lane based on the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result. In this way, whether the vehicle has lane deviation can be detected in a timely and accurate manner, and the operation is convenient and reliable.

Description

Technical Field

[0001] The present invention relates to the field of vehicle driving technology, and in particular to a lane departure detection method, device and computer storage medium. Background Art

[0002] The traditional lane departure detection method is based on the perception result of a single-frame front view taken by a single camera and the lane-level matching with the high-precision map. Usually, the single-frame front view taken by a single camera can only obtain the perception result of the current lane length of only a dozen meters. The adjacent lane information cannot be accurately detected due to the shooting angle. There are usually several lanes with similar or even identical styles in a multi-lane road. Therefore, once the positioning result of the vehicle is inaccurate, the traditional lane departure detection method will not be able to obtain accurate detection results. However, how to timely and accurately detect whether the vehicle has lane deviation has been under research. Summary of the invention

[0003] In response to the above technical problems, the present application provides a lane departure detection method, device and computer storage medium, which can timely and accurately detect whether a vehicle has lane departure, and the operation is convenient and reliable.

[0004] In order to solve the above technical problems, the present application provides a lane departure detection method, which comprises the following steps:

[0005] Acquire at least two frames of images including lane information of the road; wherein the at least two frames of images are simultaneously captured by at least two image capture devices arranged side by side in a preset direction of the vehicle;

[0006] Transversely stitching the at least two frames of images according to the order of shooting positions, and determining first lane information corresponding to a target lane where the vehicle is located on the road based on the obtained straight-stitched road image;

[0007] Acquiring second lane information corresponding to the positioning lane where the vehicle is located on the road according to the positioning information of the vehicle;

[0008] Whether the target lane and the positioning lane are the same lane is detected according to the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result.

[0009] Optionally, the image capture device includes three image capture areas corresponding to a front area, a left front area, and a right front area of ​​the vehicle respectively, and the acquiring of at least two frames of images including lane information of the road includes:

[0010] The image capture device is controlled to simultaneously capture images of a front area, a left front area, and a right front area of ​​the vehicle, respectively, to obtain at least three frames of images including lane information of the road.

[0011] Optionally, horizontally stitching the at least two frames of images according to the shooting position sequence includes:

[0012] Acquire at least two frames of top-view conversion images corresponding to the at least two frames of images;

[0013] Taking the vehicle driving direction as the longitudinal direction, removing the lateral offset information in the at least two frames of top-view conversion images;

[0014] The at least two frames of overhead conversion images are horizontally spliced ​​according to the order of shooting positions to obtain a straight road splicing image.

[0015] Optionally, acquiring at least two frames of top-view conversion images corresponding to the at least two frames of images includes:

[0016] Acquire internal parameters, external parameters and distortion parameters of the image capture device; wherein the internal parameters are selected from at least one of focal length, optical center and distortion parameter, and / or the external parameters are selected from at least one of pitch angle, yaw angle and ground height;

[0017] According to the internal parameters, external parameters and distortion parameters of the image capture device, the at least two frames of images are subjected to inverse perspective transformation.

[0018] Optionally, the step of horizontally splicing the at least two frames of top-view conversion images in the order of shooting positions to obtain a straight-stitched road image includes:

[0019] Acquiring position information of the image capture device;

[0020] The at least two frames of the overhead conversion images are sequentially staggered, overlapped and spliced ​​according to the posture information, and / or the multiple frames of the overhead conversion images are cut and then spliced ​​according to the posture information to obtain a straight road splicing image.

[0021] Optionally, the determining, based on the obtained road straight-joined image, first lane information corresponding to a target lane where the vehicle is located on the road includes:

[0022] In response to visually perceiving the road straight-joined image, obtaining a road feature point bitmap;

[0023] According to the pose information, optical center and pixel source of each frame of the top-view conversion image, the road feature point bitmap is geometrically restored to obtain a road feature image;

[0024] According to the road feature image and the position information of the vehicle, first lane information corresponding to the target lane where the vehicle is located on the road is obtained.

[0025] Optionally, acquiring, according to the positioning information of the vehicle, second lane information corresponding to the positioning lane where the vehicle is located on the road includes:

[0026] According to the positioning information of the vehicle, obtaining local map information corresponding to the positioning information;

[0027] Determine second lane information corresponding to a positioning lane where the vehicle is located on the road according to the local map information.

[0028] Optionally, the lane information includes the number of surrounding lanes and / or the types of surrounding lane lines, and the detecting whether the target lane and the positioning lane are the same lane according to the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result, includes:

[0029] comparing whether the first lane information and the second lane information are consistent;

[0030] If they are consistent, it is determined that the target lane and the positioning lane are the same lane;

[0031] If they are inconsistent, it is determined that the target lane and the positioning lane are not the same lane.

[0032] Accordingly, the present application provides a lane departure detection device for executing the above method, including: a processor and a memory storing a computer program, and when the processor runs the computer program, the steps of the above lane departure detection method are implemented.

[0033] Accordingly, the present application provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the lane departure detection method described above are implemented.

[0034] As described above, the lane departure detection method, device and computer storage medium of the present application, the lane departure detection method includes: obtaining at least two frames of images including lane information of the road; the at least two frames of images are taken simultaneously by at least two image capture devices arranged side by side in a preset direction of the vehicle; the at least two frames of images are horizontally spliced ​​in the order of shooting positions, and the first lane information corresponding to the target lane of the vehicle in the road is determined based on the obtained road straight-splitting image; according to the positioning information of the vehicle, the second lane information corresponding to the positioning lane of the vehicle in the road is obtained; according to the first lane information and the second lane information, whether the target lane and the positioning lane are the same lane is detected, so as to judge whether the vehicle deviates from the lane based on the detection result. In this way, by splicing multiple frames of images including lane information of the road, and determining the information of the target lane of the vehicle in the road based on the obtained road straight-splitting image, and then combining the positioning information of the vehicle to detect whether the vehicle has lane deviation, it is possible to realize timely and accurate detection of whether the vehicle has lane deviation, and the operation is convenient and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic flow chart of a lane departure detection method provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of a specific process of a lane departure detection method provided by an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of images before stitching in an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of image stitching in an embodiment of the present invention;

[0039] Figure 5 is a schematic diagram of visual perception and geometric restoration in an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of a process for detecting whether a vehicle deviates from a lane in an embodiment of the present invention;

[0041] Figure 7 A schematic structural diagram of a lane departure detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0043] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0044] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0045] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0046] It should be noted that, in this article, step codes such as S101, S102, etc. are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S102 first and then S101, etc., but these should all be within the scope of protection of this application.

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0049] See also Figure 1 , is a lane departure detection method provided in an embodiment of the present application. The lane departure detection method can be executed by a lane departure detection device provided in an embodiment of the present application. The device can be implemented in software and / or hardware. The device can be specifically applied to a cloud server and / or a terminal device. In this embodiment, the lane departure detection method is applied to a vehicle-mounted terminal as an example. The lane departure detection method includes the following steps:

[0050] Step S101: acquiring at least two frames of images including lane information of a road; wherein the at least two frames of images are simultaneously captured by at least two image capture devices arranged side by side in a preset direction of the vehicle;

[0051] Optionally, the lane information includes but is not limited to lane line information, lane sign information, speed limit information, etc. The image capture device may be a camera, a vehicle-mounted camera, etc., and the number of the image capture devices may be equal to or greater than two. The preset direction may be set according to actual needs, for example, it may be the front of the vehicle, or the rear of the vehicle, etc. If there are two image capture devices, the shooting areas of the two image capture devices correspond to the front left area and the front right area of ​​the vehicle respectively, that is, the images shot by the two image capture devices should include the front area of ​​the vehicle after being seamlessly spliced ​​horizontally, and the shooting areas of the two image capture devices may partially overlap. If there are three image capture devices, the shooting areas of the three image capture devices correspond to the front area, the left front area, and the right front area of ​​the vehicle respectively, and the acquisition of at least two frames of images including the lane information of the road includes: controlling the image capture device to simultaneously shoot images of the front area, the left front area, and the right front area of ​​the vehicle respectively, so as to obtain at least three frames of images including the lane information of the road. It can be understood that by using multiple image capture devices to respectively photograph different areas in front of the vehicle, not only the current lane information of the vehicle can be obtained, but also the surrounding lane information of the vehicle can be obtained. Compared with the method based on shooting with a single image capture device, more useful information can be obtained, further improving the accuracy of lane departure detection.

[0052] Step S102: horizontally stitching the at least two frames of images according to the order of shooting positions, and determining first lane information corresponding to the target lane where the vehicle is located on the road based on the obtained straight-stitched road image;

[0053] Optionally, the horizontally stitching of the at least two frames of images in the order of shooting positions includes: obtaining at least two frames of overhead view conversion images corresponding to the at least two frames of images; removing the lateral offset information in the at least two frames of overhead view conversion images with the vehicle driving direction as the longitudinal direction; and horizontally stitching the at least two frames of overhead view conversion images in the order of shooting positions to obtain a straight road image.

[0054] Optionally, the step of obtaining at least two frames of overhead conversion images corresponding to the at least two frames of images comprises: obtaining internal parameters, external parameters and distortion parameters of the image capture device; wherein the internal parameters are selected from at least one of focal length, optical center and distortion parameters, and / or the external parameters are selected from at least one of pitch angle, yaw angle and ground height; and performing inverse perspective conversion on the at least two frames of images according to the internal parameters, external parameters and distortion parameters of the image capture device. It can be understood that in order to capture information such as lane lines of the road, there is an inclination angle between the image capture device and the road surface, rather than directly facing vertically downward (orthographic projection). Sometimes, it is desired to correct the image into the form of orthographic projection, and perspective conversion is required. Inverse perspective conversion can use an IPM (inverse perspective mapping) algorithm to perform inverse perspective conversion on multiple frames of images according to the internal parameters, external parameters and distortion parameters of the image capture device. In one embodiment, the internal parameters are selected from at least one of focal length, optical center and distortion parameters. In another embodiment, the external parameters are selected from at least one of pitch angle, yaw angle and ground height. In this way, by performing bird's-eye view conversion on at least two frames of images taken by different image capture devices at the same time, a bird's-eye view conversion map of the orthographic projection corresponding to at least two frames of images can be obtained. That is, based on the image capture device parameters, the actual road data with a specific scale can be obtained by measuring the feature points in the image.

[0055] Optionally, since the lateral offset information of the road is not needed during visual perception, the at least two frames of the overhead conversion images may be processed laterally before the images are laterally spliced, that is, the lateral offset information in the curved road is removed with the vehicle driving direction as the longitudinal direction.

[0056] Optionally, since different images are taken by different image capture devices and the positions of each image capture device are fixed, that is, the shooting position sequence of the at least two frames of images is fixed, the at least two frames of overhead conversion images can be horizontally spliced ​​in the order of the shooting positions to obtain a straight road splicing image. Optionally, the at least two frames of overhead conversion images are horizontally spliced ​​in the order of the shooting positions to obtain a straight road splicing image, including: obtaining the posture information of the image capture device; splicing the at least two frames of overhead conversion images in sequence according to the posture information, and / or, cutting and splicing the multiple frames of overhead conversion images according to the posture information to obtain a straight road splicing image. Here, the posture information includes position and attitude, the position is three-dimensional information in space, and the attitude is three-dimensional rotation. After calculating the posture information of the image capture device, the corresponding image is copied to a specific position and spliced ​​at a specific angle. The spliced ​​straight road splicing image includes not only the driving lane information, but also the surrounding lane information, and can visually perceive the wider road image included in the multiple frames of images within a larger physical scale.

[0057] Optionally, the determining of the first lane information corresponding to the target lane where the vehicle is located on the road based on the obtained road straight-stitched image includes: obtaining a road feature point bitmap in response to visual perception of the road straight-stitched image; geometrically restoring the road feature point bitmap according to the posture information, optical center and pixel source of each frame of the top-down conversion image to obtain a road feature image; obtaining the first lane information corresponding to the target lane where the vehicle is located on the road according to the road feature image and the posture information of the vehicle. Here, the visual perception of the road straight-stitched image can be based on a preset visual perception model to visually perceive the road straight-stitched image, that is, the road straight-stitched image is used as the input of the visual perception model, and the output of the visual perception model is the road feature point bitmap, and the visual perception model can be obtained by pre-training based on non-real-time images of the same vehicle or different vehicles. By visually perceiving the road straight-stitched image, a road feature point bitmap including information such as lane lines, crosswalks, speed limit signs, and lane numbers will be obtained. Since the road straight-stitching image for visual perception is obtained by horizontally stitching at least two frames of top-view conversion images from which the lateral offset information of the road is removed, the road feature point bitmap perceived visually also lacks lateral offset information. At this time, it is necessary to geometrically restore the road feature point bitmap to restore the lateral offset information of the road using the posture information, optical center and pixel source of each frame of the image, thereby obtaining a road feature image. It can be understood that based on the road feature image, the number of lanes, the number of lane lines, the position and type of each lane line, and whether there are lane signs, speed limit signs and other information contained in the road feature image can be obtained, and then according to the road feature image and the posture information of the vehicle, the position relationship between the target lane and each lane in the road can be obtained, that is, the position relationship between the target lane and each lane in the road, such as whether the target lane of the vehicle in the road is a lane between the first lane line and the second lane line. Specifically, if the number of all lanes included in the road feature image is less than the number of lanes of the road, it means that the at least two frames of images are taken of some lanes in the road. At this time, the first lane information corresponding to the target lane where the vehicle is located in the road includes the number of surrounding lanes and / or the type of surrounding lane lines, for example, the number of lanes on the left and right sides of the target lane, the type of lane lines forming the target lane, the number and type of lane lines on the left and right sides of the target lane, etc.If the number of all lanes included in the road feature image is the same as the number of lanes on the road, it means that the at least two frames of images have captured all lanes on the road. At this time, the lane of the road to which the target lane of the vehicle belongs can be directly determined based on the road feature image and the vehicle's posture information. In other words, the first lane information corresponding to the target lane of the vehicle on the road is the lane identification of the target lane. In this way, the lane information corresponding to the target lane of the vehicle on the road is determined based on the road straight-joined image, which is convenient to operate and has a high recognition accuracy.

[0058] Step S103: acquiring second lane information corresponding to the positioning lane where the vehicle is located on the road according to the positioning information of the vehicle;

[0059] Optionally, the acquiring of the positioning lane information of the vehicle on the road according to the positioning information of the vehicle includes: acquiring local map information corresponding to the positioning information according to the positioning information of the vehicle; and determining the second lane information corresponding to the positioning lane of the vehicle on the road according to the local map information. It can be understood that after determining the positioning information of the vehicle, such as coordinates, the positioning information can be matched with the existing map data to obtain the local map information corresponding to the positioning information, such as the map information with the vehicle as the center and a radius of 50 meters or 100 meters as the local map information, and then determining the second lane information of the positioning lane of the vehicle on the road according to the local map information and the positioning information of the vehicle. It should be noted that the second lane information may include one or more information of lane identification (i.e. the number of lanes), the number of surrounding lanes, and the type of surrounding lane lines, such as the number of lanes on the left and right sides of the positioning lane, the type of lane lines forming the positioning lane, the number and type of lane lines on the left and right sides of the positioning lane, etc.

[0060] Step S104: Detecting whether the target lane and the positioning lane are the same lane according to the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result.

[0061] Optionally, the lane information includes at least one of a lane identification, the number of surrounding lanes and the type of surrounding lane lines, and the detecting whether the target lane and the positioning lane are the same lane according to the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result, includes: comparing whether the first lane information and the second lane information are consistent; if they are consistent, determining that the target lane and the positioning lane are the same lane; if they are inconsistent, determining that the target lane and the positioning lane are not the same lane. Specifically, if the lane information includes a lane mark, whether the first lane information and the second lane information are consistent refers to whether the first lane mark and the second lane mark are the same; if the lane information includes the number of surrounding lanes, whether the first lane information and the second lane information are consistent refers to whether the distribution and quantity of the first surrounding lane number and the second surrounding lane number are the same. For example, in the case where it is known that there are four lanes in total on the road, assuming that there are two lanes on the left side and one lane on the right side of the target lane according to the first lane information, and there are two lanes on the left side and one lane on the right side of the positioning lane, then it is determined that the first lane information and the second lane information are consistent; if the lane information includes the type of surrounding lanes, whether the first lane information and the second lane information are consistent refers to whether the distribution and quantity of the first surrounding lane type and the second surrounding lane type are the same. It can be understood that when the target lane and the positioning lane are the same lane, it means that the vehicle has not deviated from the lane, and it also means that the positioning of the vehicle is accurate; when the target lane and the positioning lane are not the same lane, it means that the vehicle has deviated from the lane, and it also means that the positioning of the vehicle is inaccurate.

[0062] In summary, the lane departure detection method provided by the above embodiment includes: acquiring at least two frames of images including lane information of the road; the at least two frames of images are simultaneously shot by at least two image capture devices arranged side by side in a preset direction of the vehicle; horizontally splicing the at least two frames of images in the order of shooting positions, and determining the first lane information corresponding to the target lane of the vehicle in the road based on the obtained straight-stitched image of the road; acquiring the second lane information corresponding to the positioning lane of the vehicle in the road based on the positioning information of the vehicle; detecting whether the target lane and the positioning lane are the same lane based on the first lane information and the second lane information, so as to judge whether the vehicle deviates from the lane based on the detection result. In this way, by splicing multiple frames of images including lane information of the road, and determining the information of the target lane of the vehicle in the road based on the obtained straight-stitched image of the road, and then detecting whether the vehicle has lane deviation in combination with the positioning information of the vehicle, it is possible to realize timely and accurate detection of whether the vehicle has lane deviation, and the operation is convenient and reliable.

[0063] Based on the same inventive concept as the above embodiments, the above embodiments are specifically described below through a specific example. In this example, the image capture device is taken as a camera.

[0064] See also Figure 2 , is a schematic diagram of a specific process of a lane departure detection method in an embodiment of the present invention, comprising the following steps:

[0065] Step S201: Acquire images of the road surface of the same road collected by multiple cameras;

[0066] See also Figure 3 As shown, in this embodiment, a left camera, a main camera and a right camera are used to collect a frame of image of the road surface of the same road respectively. Figure 3 The middle (a) is taken by the left camera. Figure 3 The middle (b) is taken by the middle camera. Figure 3 The middle (c) is taken by the right camera.

[0067] Step S202: converting the images into a top view and then merging them to obtain a road straight-joined image;

[0068] See also Figure 4 As shown, Figure 4 (a) is for Figure 3 (a) is the overhead image after the overhead conversion. Figure 4 (b) is for Figure 3 (b) is the overhead image after the overhead conversion, and Figure 4 (c) is for Figure 3 (c) is the top view image after the top view conversion. Figure 4 (a), (b) and (c) are horizontally spliced ​​or merged to obtain a straight road splicing image, such as Figure 4 (d) as shown.

[0069] Step S203: visually perceive the road straight-joined image to obtain a road feature point map;

[0070] Here, the road straight-joined image may be input into a preset visual perception model to obtain a road feature point map including information such as lane lines.

[0071] Step S204: geometrically restore the road feature point bitmap to obtain a perception result of the lane lines around the vehicle;

[0072] Here, the road feature point bitmap may be geometrically restored according to the pose information, optical center, and pixel source of each frame of the overhead image to obtain the perception result of the lane lines around the vehicle.

[0073] like Figure 5 As shown, it is a schematic diagram of visual perception and geometric restoration. Figure 5 For a curved road shown in the image (a), multiple frames of images taken by the onboard camera during driving are firstly taken, and the road curvature information is removed. Then, the images are stitched together in sequence to obtain Figure 5 Image (b). Based on Figure 5 The image (b) is visually perceived to obtain the road feature point map, namely Figure 5 Finally, according to the pose information, optical center and pixel source of each frame, the lateral curvature information in the road feature point bitmap is geometrically restored to obtain the feature image of the real curvature angle, that is, Figure 5 As shown in (d).

[0074] Step S205: determining first lane information of the vehicle in the perception result according to the perception result of the lane lines around the vehicle and the position information of the vehicle;

[0075] Step S205: Detecting whether the vehicle deviates from the lane based on the first lane information and the second lane information where the vehicle is located in the map data.

[0076] Here, the second lane information of the vehicle in the map data can be obtained based on the vehicle's positioning data. For example, local high-precision map data of the vehicle's location is first obtained, and then the lane position corresponding to the positioning data is obtained by comparing the coordinates corresponding to the positioning data with the coordinates of the high-precision map data.

[0077] Here, in the process of judging lane departure, it is possible to first judge whether the images containing lanes taken by multiple cameras have included all lanes of the road based on the real-time position information. If all lanes are included, the lane position of the vehicle is directly analyzed using the collected images to obtain the current specific lane information. If not all lanes are included, it is necessary to first analyze which lanes are captured based on the lanes in the collected images or other information on the road, and then further judge the specific lane position of the vehicle.

[0078] Among them, see Figure 6 FIG. 1 is a schematic diagram of a specific process of detecting whether a vehicle deviates from a lane in an embodiment of the present invention, and includes the following steps:

[0079] Step S301: determine whether the lane quantity characteristics corresponding to the first lane information and the second lane information are consistent, if not, execute step S302, otherwise execute step S304;

[0080] For example, assuming that according to the first lane information, it is known that there are two lanes on the left side of the vehicle and one lane on the right side, and according to the first lane information, it is known that there are also two lanes on the left side of the vehicle and one lane on the right side, it can be judged that the lane quantity characteristics corresponding to the first lane information and the second lane information are consistent, otherwise it indicates inconsistency.

[0081] Step S302: determine whether the lane line type features corresponding to the first lane information and the second lane information are consistent, if not, execute step S303, otherwise execute step S304;

[0082] It can be understood that when it is learned from the first lane information that there is a total of 1 lane on both sides of the vehicle, and when it is learned from the second lane information that there are a total of 2 lanes on both sides of the vehicle, it means that the first lane information may only contain partial information of the road. At this time, the lane line type features corresponding to the first lane information and the second lane information can be compared, such as whether the types of the surrounding lane lines are all dotted line-dotted line-solid line in sequence.

[0083] Step S303: determining whether the vehicle deviates from the lane;

[0084] Step S304: Determine whether the vehicle has deviated from the lane.

[0085] In summary, in the lane departure detection method provided in the above embodiment, the images captured by multiple cameras are combined to determine the actual lane position of the vehicle, and the actual lane position of the vehicle is compared with the located lane position of the vehicle to determine whether the positioning is correct. The operation is convenient and accurate.

[0086] Based on the same inventive concept as the above embodiments, the present invention provides a map loading device, such as Figure 7 As shown, the lane departure detection device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 7 The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 7 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the lane departure detection method applied to the lane departure detection device is implemented.

[0087] The lane departure detection device may further include: at least one network interface 312. The various components in the map loading device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 313 is not described in detail. Figure 7 Various buses are labeled as bus system 313.

[0088] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0089] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the lane departure detection device. Examples of these data include: any computer program used to operate on the lane departure detection device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0090] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer storage medium, in which a computer program is stored. The computer storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); or it may be various devices including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer storage medium is executed by the processor, the lane departure detection method applied to the lane departure detection device is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 The description of the illustrated embodiment will not be repeated here.

[0091] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0093] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A lane departure detection method, characterized in that: The method comprises: Acquire at least two frames of images including lane information of the road; wherein the at least two frames of images are simultaneously captured by at least two image capture devices arranged side by side in a preset direction of the vehicle; Transversely stitching the at least two frames of images according to the order of shooting positions, and determining first lane information corresponding to a target lane where the vehicle is located on the road based on the obtained straight-stitched road image; Acquiring second lane information corresponding to the positioning lane where the vehicle is located on the road according to the positioning information of the vehicle; Detecting whether the target lane and the positioning lane are the same lane according to the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result; The determining of the first lane information corresponding to the target lane where the vehicle is located on the road based on the obtained road straight-joined image includes: In response to visually perceiving the road straight-joined image, obtaining a road feature point bitmap; According to the pose information, optical center and pixel source of each frame of the top-view conversion image, the road feature point bitmap is geometrically restored to obtain a road feature image; Acquire first lane information corresponding to a target lane where the vehicle is located on the road according to the road feature image and the position information of the vehicle; The step of horizontally splicing the at least two frames of images according to the shooting position sequence includes: Acquire at least two frames of top-view conversion images corresponding to the at least two frames of images; Taking the vehicle driving direction as the longitudinal direction, removing the lateral offset information in the at least two frames of top-view conversion images; The at least two frames of overhead conversion images are horizontally spliced ​​according to the order of shooting positions to obtain a straight road splicing image.

2. The lane departure detection method according to claim 1, characterized in that: The image capture device includes three image capture areas corresponding to the front area, the left front area and the right front area of ​​the vehicle respectively, and the at least two frames of images including the lane information of the road are obtained, including: The image capture device is controlled to simultaneously capture images of a front area, a left front area, and a right front area of ​​the vehicle, respectively, to obtain at least three frames of images including lane information of the road.

3. The lane departure detection method according to claim 1, characterized in that: The step of acquiring at least two frames of top-view conversion images corresponding to the at least two frames of images includes: Acquiring internal parameters, external parameters and distortion parameters of the image capture device; wherein the internal parameters are selected from at least one of focal length, optical center, and distortion parameter, and / or the external parameters are selected from at least one of pitch angle, yaw angle, and ground height; According to the internal parameters, external parameters and distortion parameters of the image capture device, the at least two frames of images are subjected to inverse perspective transformation.

4. The lane departure detection method according to claim 1, characterized in that: The step of horizontally splicing the at least two frames of the top-view conversion images in the order of the shooting positions to obtain a straight-stitched road image includes: Acquiring position information of the image capture device; The at least two frames of the overhead conversion images are sequentially staggered, overlapped and spliced ​​according to the posture information, and / or the at least two frames of the overhead conversion images are cut and then spliced ​​according to the posture information to obtain a straight road splicing image.

5. The lane departure detection method according to claim 1, characterized in that: The acquiring, according to the positioning information of the vehicle, second lane information corresponding to the positioning lane where the vehicle is located on the road comprises: According to the positioning information of the vehicle, obtaining local map information corresponding to the positioning information; Determine second lane information corresponding to a positioning lane where the vehicle is located on the road according to the local map information.

6. The method according to claim 1, characterized in that The lane information includes at least one of a lane identification, the number of surrounding lanes, and the type of surrounding lane lines. The detecting whether the target lane and the positioning lane are the same lane according to the first lane information and the second lane information, so as to determine whether the vehicle deviates from the lane based on the detection result, includes: comparing whether the first lane information and the second lane information are consistent; If they are consistent, it is determined that the target lane and the positioning lane are the same lane; If they are inconsistent, it is determined that the target lane and the positioning lane are not the same lane.

7. A lane departure detection device, characterized in that: include: A processor and a memory storing a computer program, wherein when the processor runs the computer program, the lane departure detection method according to any one of claims 1 to 6 is implemented.

8. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the lane departure detection method according to any one of claims 1 to 6 is implemented.

Images