Driving area detection method, device and equipment

Abstract

The invention discloses a drivable area detection method, device and equipment, and relates to the technical field of automatic driving. The drivable area detection method comprises the following steps: acquiring a first original image shot by a vehicle camera; according to a first mapping relation, a first angle offset distortion-removed image corresponding to the first original image is determined, and the first mapping relation is a mapping relation from the original image to the angle offset distortion-removed image; taking the first angle offset distortion-removed image as the input of a deep network model, and obtaining a pixel-level classification result of the first angle offset distortion-removed image; and determining a drivable area according to a pixel-level classification result. According to the embodiment of the invention, the drivable area of the vehicle can be detected in time.

Description

Technical Field

[0001] The present application belongs to the field of autonomous driving technology, and in particular, relates to a method, device and equipment for detecting a drivable area. Background Art

[0002] With the continuous development of science and technology and the improvement of living standards, users have higher and higher requirements for the safety performance of vehicles. During the driving process, due to the limited visual range of the cab, rear-end collisions, collisions and other accidents may occur, which poses certain safety hazards to the people in the car and will cause certain property losses.

[0003] In the related art, in order to avoid the above accidents, a fisheye camera is usually used to detect the drivable area. However, in some cases (such as when the vehicle is driving at high speed), the fisheye camera is not used to detect the drivable area in time, and the vehicle cannot avoid danger in an emergency. Summary of the invention

[0004] The embodiments of the present application provide a drivable area detection method, device and equipment, which can solve the problem of being unable to detect the drivable area of ​​a vehicle in a timely manner.

[0005] In a first aspect, an embodiment of the present application provides a drivable area detection method, comprising:

[0006] Acquire a first original image captured by a vehicle camera;

[0007] Determine, according to the first mapping relationship, a first angle-shifted dedistorted image corresponding to the first original image, wherein the first mapping relationship is a mapping relationship from the original image to the angle-shifted dedistorted image;

[0008] The first angle offset dedistorted image is used as an input of the deep network model to obtain a pixel-level classification result of the first angle offset dedistorted image;

[0009] Determine the drivable area based on the pixel-level classification results.

[0010] In a second aspect, an embodiment of the present application provides a drivable area detection device, comprising:

[0011] An acquisition module, used to acquire a first original image captured by a vehicle camera;

[0012] A first determination module is used to determine a first angle-shifted dedistorted image corresponding to the first original image according to a first mapping relationship, wherein the first mapping relationship is a mapping relationship from the original image to the angle-shifted dedistorted image;

[0013] A classification module, used to use the first angle offset dedistorted image as an input of a deep network model to obtain a pixel-level classification result of the first angle offset dedistorted image;

[0014] The second determination module is used to determine the drivable area according to the pixel-level classification result.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the drivable area detection method of the first aspect is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the drivable area detection method of the first aspect is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the drivable area detection method as in the first aspect.

[0018] In a sixth aspect, an embodiment of the present application provides a vehicle, which includes the drivable area detection device provided in the second aspect or the electronic device provided in the third aspect.

[0019] In the embodiment of the present application, the first original image captured by the vehicle camera is obtained; according to the first mapping relationship, the first angle-shifted dedistorted image corresponding to the first original image is determined, wherein the first mapping relationship is the mapping relationship from the original image to the angle-shifted dedistorted image; the first angle-shifted dedistorted image is used as the input of the deep network model to obtain the pixel-level classification result of the first angle-shifted dedistorted image; according to the pixel-level classification result, the drivable area is determined. In this way, by obtaining the first original image captured by the vehicle camera, the drivable area of ​​the vehicle can be detected in time to avoid accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 It is a flowchart of a drivable area detection method provided in an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of the relative position between the camera and the ground before the angle offset processing provided in an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of the relative position between the camera and the ground after the angle offset processing provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a first original image provided in an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a first angle offset dedistorted image provided by an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of a pixel-level classification result provided in an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of a drivable area provided in an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of the structure of a drivable area detection device provided in an embodiment of the present application;

[0029] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0032] The following is a detailed description of the drivable area detection method, device and equipment provided in the embodiments of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0033] Figure 1 is a flow chart of the drivable area detection method provided in the embodiment of the present application. Figure 1 As shown, the drivable area detection method may include:

[0034] Step 101: Acquire a first original image captured by a vehicle camera.

[0035] In some possible implementations of the embodiments of the present application, the vehicle camera in the embodiments of the present application may be a forward-looking monocular wide-angle camera installed on the vehicle.

[0036] Step 102: Determine a first angle-shifted dedistorted image corresponding to the first original image according to a first mapping relationship, wherein the first mapping relationship is a mapping relationship from the original image to the angle-shifted dedistorted image.

[0037] In some possible implementations of the embodiments of the present application, the vehicle camera may be calibrated in advance, wherein the camera calibration is the process of solving the distortion parameters, internal parameters and external parameters of the camera. The embodiments of the present application do not limit the method used for vehicle camera calibration, and any available method may be applied to the embodiments of the present application. For example, active vision camera calibration method, Zhang Zhengyou calibration method, etc.

[0038] After the vehicle camera is calibrated, a first mapping relationship from the original image to the angle-shifted dedistorted image can be determined according to the distortion parameters, intrinsic parameters, and extrinsic parameters of the camera.

[0039] In some possible implementations of the embodiments of the present application, the first mapping relationship in the embodiments of the present application may include: a second mapping relationship from the original image to the dedistorted image and a third mapping relationship from the dedistorted image to the angle-shifted image.

[0040] In some possible implementations of the embodiment of the present application, the second mapping relationship in the embodiment of the present application is as shown in formula (1).

[0041]

[0042] In formula (1), (x m ,y m ) is the coordinate of pixel m in the original image, (x m-corrected ,y m-corrected) are the coordinates of the pixel point corresponding to the pixel point m in the dedistorted image, k1, k2 and k3 are the radial distortion coefficients of the vehicle camera, p1 and p2 are the tangential distortion coefficients of the vehicle camera, pixel point m is any pixel point in the original image, and r is the distance from the pixel point m to the origin of the original image.

[0043] In some possible implementations of the embodiments of the present application, after the vehicle camera is installed, its optical axis is offset from the ground by a pitch angle, and is offset from the vehicle center axis by a roll and yaw angle. In order to make objects in the imaging plane perpendicular to the ground, the imaging plane needs to be offset by a rotation angle. Figure 2 and Figure 3 As shown, Figure 2 The figure shows the position of the vehicle camera relative to the ground before the angle offset processing. Figure 3 The position of the vehicle camera relative to the ground after the angle offset processing is shown. The world coordinate system can be a point where the vertical projection of the intersection of the axle and the center axis of the vehicle body on the ground is the origin of the world coordinate system O, the direction through this point perpendicular to the ground is the positive direction of the world coordinate system Z axis, and the ground is the XOY plane of the world coordinate system, where the direction of the front of the vehicle is the positive direction of the world coordinate system X axis, and the direction toward the left side of the vehicle and perpendicular to the X axis is the positive direction of the world coordinate system Y axis. Get the rotation matrix R of the camera coordinate system relative to the world coordinate system x , R y and R z ,in, θ, and ω are the Euler angles of the camera coordinate system relative to the world coordinate system, that is, θ is the rotation angle of the X-axis of the camera coordinate system relative to the X-axis of the world coordinate system, is the rotation angle of the Y axis of the camera coordinate system relative to the Y axis of the world coordinate system, and ω is the rotation angle of the Z axis of the camera coordinate system relative to the Z axis of the world coordinate system.

[0044] In some possible implementations of the embodiment of the present application, the third mapping relationship in the embodiment of the present application is as shown in formula (2).

[0045]

[0046] In formula (2), (x n ,y n ,z n ) is the coordinate of pixel n in the dedistorted image in the camera coordinate system, (x n ',y n ',z n ') is the coordinate of pixel n in the world coordinate system, R x , R y and R zis the rotation matrix of the camera coordinate system relative to the world coordinate system, λ is the image scaling ratio, (x n ”,y n ”) is the coordinate of the pixel point corresponding to pixel point n in the angle offset image, f x 、c x 、f y and c y is the internal parameter of the vehicle camera, and pixel n is any pixel in the dedistorted image.

[0047] In some possible implementations of the embodiments of the present application, step 102 may include: determining a first dedistorted image corresponding to the first original image according to a second mapping relationship; determining a first angle-shifted image corresponding to the first dedistorted image according to a third mapping relationship; and using the first angle-shifted image as the first angle-shifted dedistorted image.

[0048] After the first original image is obtained by shooting with the vehicle camera, the first dedistorted image corresponding to the first original image can be calculated by the second mapping relationship represented by the above formula (1), and the first angle-shifted image corresponding to the first dedistorted image can be calculated by the third mapping relationship represented by the above formula (2). The first angle-shifted image is the first angle-shifted dedistorted image corresponding to the first original image. Figure 4 and Figure 5 As shown, Figure 4 shows a first original image provided by an embodiment of the present application, Figure 5 FIG. 4 shows a first angle offset dedistorted image provided by an embodiment of the present application. Figure 4 The bent streetlight pole near the vehicle in the first original image shown is distorted, and after angle shifting to remove distortion, the bent streetlight pole near the vehicle is corrected to a straight streetlight pole.

[0049] In the embodiment of the present application, by determining the first angle offset dedistorted image corresponding to the first original image collected, the distortion of the first angle offset dedistorted image is greatly eliminated, and the object is widened, the number of pixels is increased, and the accuracy and overall performance of the drivable area detection are improved. When increasing the pixel points, an interpolation algorithm can be used to interpolate the pixel values ​​of the pixel points.

[0050] Step 103: using the first angle offset dedistorted image as an input of the deep network model to obtain a pixel-level classification result of the first angle offset dedistorted image.

[0051] In some possible implementations of the embodiments of the present application, the deep network model can be pre-trained. The embodiments of the present application do not limit the process of training the deep network model, and any available method can be applied to the embodiments of the present application. For example, a plurality of images are collected by a camera, and the angle offset dedistorted image corresponding to each image is determined respectively, and then the drivable area and the non-drivable area in each angle offset dedistorted image are marked, and the marked images are used as samples to train the deep network model.

[0052] After the first angle-shifted dedistorted image corresponding to the first original image is determined, the first angle-shifted dedistorted image can be input into a pre-trained deep network model, and the deep network model outputs a pixel-level classification result of the first angle-shifted dedistorted image.

[0053] In some possible implementations of the embodiments of the present application, the pixel-level classification results in the embodiments of the present application may include pixels in the drivable area and pixels in the non-drivable area. Figure 6 As shown, Figure 6 is a schematic diagram of the pixel-level classification result provided by the embodiment of the present application. Figure 6 In the figure, the pixels in the drivable area are white, and the pixels in the non-drivable area are black.

[0054] Step 104: Determine the drivable area based on the pixel-level classification result.

[0055] In some possible implementations of the embodiments of the present application, Figure 6 The white area in the middle is determined as the area where the vehicle can drive, and the black area is determined as the area where the vehicle cannot drive. Figure 7 The first angle offset dedistorted image is shown in Figure 7 The area surrounded by the "dots" in the figure and close to the vehicle itself is determined as the vehicle's drivable area.

[0056] In the embodiment of the present application, the first original image captured by the vehicle camera is obtained; according to the first mapping relationship, the first angle-shifted dedistorted image corresponding to the first original image is determined, wherein the first mapping relationship is the mapping relationship from the original image to the angle-shifted dedistorted image; the first angle-shifted dedistorted image is used as the input of the deep network model to obtain the pixel-level classification result of the first angle-shifted dedistorted image; according to the pixel-level classification result, the drivable area is determined. In this way, by obtaining the first original image captured by the vehicle camera, the drivable area of ​​the vehicle can be detected in time to avoid accidents.

[0057] In some possible implementations of the embodiments of the present application, the drivable area detection method provided in the embodiments of the present application may further include: traversing the pixel-level classification results through a boundary search algorithm to obtain multiple boundary points of the drivable area; determining the actual distance between the vehicle and the actual position corresponding to each boundary point according to a fourth mapping relationship, wherein the fourth mapping relationship includes a mapping relationship between the actual position corresponding to each pixel point in the angle-offset de-distorted image and the actual distance between the vehicle.

[0058] In some possible implementations of the embodiments of the present application, traversing the pixel-level classification results through a boundary search algorithm to obtain multiple boundary points of the drivable area may include: taking the lower left corner point in the first angle offset dedistorted image as the search starting point; starting from the search starting point, detecting whether there is a pixel value mutation in the current sliding window; if so, recording the current sliding window information, moving the sliding window to the bottom of the first angle offset dedistorted image and moving it to the right by X pixels, and continuing to perform the step of detecting whether there is a pixel value mutation in the current sliding window, where X is the width of the sliding window; if not, moving the sliding window upward by Y pixels, and continuing to perform the step of detecting whether there is a pixel value mutation in the current sliding window until there is a pixel value mutation in the current sliding window or the sliding window has been moved to the top of the first angle offset dedistorted image, where Y is the height of the sliding window; after traversing the first angle offset dedistorted image, traversing each pixel point in the sliding window corresponding to the recorded sliding window information to obtain multiple boundary points of the drivable area.

[0059] In some possible implementations of the embodiments of the present application, X and Y can be set according to actual needs, for example, both are set to 5. The embodiments of the present application do not limit the method used to detect whether there is a sudden change in pixel value in the sliding window, and any available method can be applied to the embodiments of the present application. For example, a filter for detecting a sudden change in pixel value is used to detect whether there is a sudden change in pixel value in the sliding window.

[0060] In an embodiment of the present application, performing boundary point search through a sliding window can increase the boundary point search speed compared to searching boundary points one pixel at a time, thereby increasing the speed of determining the actual distance between the vehicle and the actual position corresponding to each boundary point.

[0061] In some possible implementations of the embodiments of the present application, traversing each pixel point in the sliding window corresponding to the recorded sliding window information to obtain multiple boundary points of the drivable area can include: for the first sliding window, from bottom to top, comparing whether the pixel-level categories of two adjacent pixel points are the same, and if they are not the same, determining the pixel point below the two adjacent pixel points as a boundary point, wherein the first sliding window is any sliding window in the sliding windows corresponding to the recorded sliding window information.

[0062] In some possible implementations of the embodiments of the present application, determining the actual distance between the vehicle and the actual position corresponding to each boundary point according to the fourth mapping relationship may include: determining the coordinates of the actual position corresponding to each boundary point in the vehicle rear axle coordinate system according to formula (3):

[0063]

[0064] Among them, (u P , v P ) is the coordinate of the pixel point corresponding to the boundary point P in the first angle offset dedistorted image in the first original image, (X c , Y c , Z c ) is the coordinate of the actual position corresponding to the boundary point P in the camera coordinate system, (X w , Y w , Z w ) is the coordinate of the actual position corresponding to the boundary point P in the vehicle rear axle coordinate system, f x 、c x 、f y and c y is the vehicle camera internal parameter, is the rotation matrix of the camera coordinate system relative to the vehicle rear axle coordinate system, r1 to r9 and t1 to t3 are the parameters of the rotation matrix.

[0065] In some possible implementations of the embodiments of the present application, the vehicle rear axle coordinate system in the embodiments of the present application can be a system with the center of the vehicle rear axle as the origin O of the vehicle rear axle coordinate system, the direction passing through this point and perpendicular to the ground as the positive direction of the Z-axis of the vehicle rear axle coordinate system, the direction of the front of the vehicle as the positive direction of the X-axis of the world coordinate system, and the direction toward the left side of the vehicle and perpendicular to the X-axis as the positive direction of the Y-axis of the world coordinate system.

[0066] The embodiment of the present application does not limit the method used to determine the coordinates of the pixel points corresponding to the boundary point P in the first angle offset dedistorted image in the first original image. Any available method can be applied to the embodiment of the present application. For example, the coordinates of the boundary point P in the first angle offset dedistorted image are inversely calculated using the above formulas (1) and (2) to obtain the coordinates of the pixel points corresponding to the boundary point P in the first original image.

[0067] In some possible implementations of the embodiments of the present application, it is assumed that the ground is horizontal and the Z-axis coordinates of each boundary point in the vehicle rear axle coordinate system are all 0, that is, Z w = 0. Solving formula (3), we can get X w and Y w The value of (X w , Yw ) is the coordinate of the actual position corresponding to the boundary point in the vehicle rear axle coordinate system. Based on the coordinate, the distance from the actual position corresponding to the boundary point to the center of the vehicle rear axle can be calculated.

[0068] In the embodiment of the present application, the actual distance between the vehicle and the actual position corresponding to each boundary point can be determined, thereby providing an effective basis for the vehicle to avoid danger and improving safety during driving.

[0069] Corresponding to the above method embodiment, the present application embodiment also provides a drivable area detection device, such as Figure 8 shown. Figure 8 is a schematic diagram of the structure of a drivable area detection device provided in an embodiment of the present application. The drivable area detection device 800 may include:

[0070] An acquisition module 801 is used to acquire a first original image captured by a vehicle camera;

[0071] A first determining module 802 is used to determine a first angle-shifted dedistorted image corresponding to the first original image according to a first mapping relationship, wherein the first mapping relationship is a mapping relationship from the original image to the angle-shifted dedistorted image;

[0072] A classification module 803 is used to use the first angle offset dedistorted image as an input of a deep network model to obtain a pixel-level classification result of the first angle offset dedistorted image;

[0073] The second determination module 804 is used to determine the drivable area according to the pixel-level classification result.

[0074] In the embodiment of the present application, the first original image captured by the vehicle camera is obtained; according to the first mapping relationship, the first angle-shifted dedistorted image corresponding to the first original image is determined, wherein the first mapping relationship is the mapping relationship from the original image to the angle-shifted dedistorted image; the first angle-shifted dedistorted image is used as the input of the deep network model to obtain the pixel-level classification result of the first angle-shifted dedistorted image; according to the pixel-level classification result, the drivable area is determined. In this way, by obtaining the first original image captured by the vehicle camera, the drivable area of ​​the vehicle can be detected in time to avoid accidents.

[0075] In some possible implementations of the embodiments of the present application, the first mapping relationship may include: a second mapping relationship from an original image to a dedistorted image and a third mapping relationship from a dedistorted image to an angle-shifted image;

[0076] Accordingly, the first determining module 802 may specifically:

[0077] Determining a first dedistorted image corresponding to the first original image according to the second mapping relationship;

[0078] Determine, according to the third mapping relationship, a first angle-shifted image corresponding to the first dedistorted image;

[0079] The first angle-shifted image is used as the first angle-shifted dedistorted image.

[0080] In an embodiment of the present application, by determining a first angle offset dedistorted image corresponding to the collected first original image, the distortion of the first angle offset dedistorted image is greatly eliminated, and the object is widened, the number of pixels is increased, and the accuracy and overall performance of the drivable area detection are improved.

[0081] In some possible implementations of the embodiments of the present application, the drivable area detection device 800 provided in the embodiments of the present application may further include:

[0082] A boundary point search module, used to traverse the pixel-level classification results through a boundary search algorithm to obtain multiple boundary points of the drivable area;

[0083] The third determination module is used to determine the actual distance between the vehicle and the actual position corresponding to each of the boundary points according to a fourth mapping relationship, wherein the fourth mapping relationship includes a mapping relationship between the actual position corresponding to each pixel point in the angle-shifted dedistorted image and the actual distance between the vehicle.

[0084] In some possible implementations of the embodiments of the present application, the boundary point search module may include:

[0085] A determination submodule is provided to use the lower left corner point of the first angle offset dedistorted image as a search starting point;

[0086] A detection submodule, used to detect whether there is a sudden change in pixel value in the current sliding window starting from the search starting point;

[0087] a recording submodule, configured to record the current sliding window information when there is a sudden change in pixel value in the current sliding window, move the sliding window to the bottom of the first angle offset dedistorted image and move it to the right by X pixels, and trigger the detection submodule, where X is the width of the sliding window;

[0088] a moving submodule, configured to move the sliding window upward by Y pixels when there is no sudden change in pixel value in the current sliding window, triggering the detection submodule, until there is a sudden change in pixel value in the current sliding window or the sliding window has been moved to the top of the first angle offset dedistorted image, wherein Y is the height of the sliding window;

[0089] The traversal submodule is used to traverse each pixel point in the sliding window corresponding to the recorded sliding window information after traversing the first angle offset dedistorted image, so as to obtain multiple boundary points of the drivable area.

[0090] In an embodiment of the present application, performing boundary point search through a sliding window can increase the boundary point search speed compared to searching boundary points one pixel at a time, thereby increasing the speed of determining the actual distance between the vehicle and the actual position corresponding to each boundary point.

[0091] In some possible implementations of the embodiments of the present application, the traversal submodule may be specifically used for:

[0092] For the first sliding window, from bottom to top, compare whether the pixel-level categories of two adjacent pixel points are the same. If they are not the same, the pixel point below the two adjacent pixel points is determined as a boundary point, wherein the first sliding window is any sliding window in the sliding windows corresponding to the recorded sliding window information.

[0093] In some possible implementations of the embodiments of the present application, the third determination module may be specifically used to:

[0094] According to the above formula (3), the coordinates of the actual position corresponding to each boundary point in the vehicle rear axle coordinate system are determined.

[0095] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0096] The electronic device may include a processor 901 and a memory 902 storing computer program instructions.

[0097] Specifically, the processor 901 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0098] The memory 902 may include a large capacity memory for data or instructions. For example, but not limitation, the memory 902 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 902 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 902 may be inside or outside the electronic device. In some specific embodiments, the memory 902 is a non-volatile solid-state memory.

[0099] In some specific embodiments, the memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the drivable area detection method according to the present application.

[0100] The processor 901 implements the drivable area detection method provided in the embodiment of the present application by reading and executing the computer program instructions stored in the memory 902.

[0101] In one example, the electronic device may further include a communication interface 903 and a bus 910. Fig. 9 As shown, the processor 901, the memory 902, and the communication interface 903 are connected via a bus 910 and communicate with each other.

[0102] The communication interface 903 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0103] The bus 910 includes hardware, software or both, coupling the components of the electronic device to each other. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro channel architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus or other suitable buses or a combination of two or more of these. Where appropriate, the bus 910 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0104] The electronic device can execute the drivable area detection method provided in the embodiment of the present application, thereby achieving the corresponding technical effects of the drivable area detection method provided in the embodiment of the present application.

[0105] In addition, in combination with the drivable area detection method in the above embodiment, the embodiment of the present application also provides a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, the drivable area detection method provided in the embodiment of the present application is implemented. Examples of computer-readable storage media include non-transitory computer-readable media, such as ROM, RAM, magnetic disk or optical disk, etc.

[0106] An embodiment of the present application also provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the drivable area detection method provided in the embodiment of the present application and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0107] An embodiment of the present application also provides a vehicle, which includes a drivable area detection device provided by an embodiment of the present application or an electronic device provided by an embodiment of the present application, so that the electronic device executes the drivable area detection method provided by the embodiment of the present application and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0108] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0109] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable read-only memory (Erasable Read Only Memory, EROM), floppy disks, compact discs (Compact Disc Read-Only Memory, CD-ROM), optical disks, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0110] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0111] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0112] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for detecting a drivable area, characterized in that: The method comprises: Acquire a first original image captured by a vehicle camera; Determine, according to a first mapping relationship, a first angle-shifted dedistorted image corresponding to the first original image, wherein the first mapping relationship is a mapping relationship from the original image to the angle-shifted dedistorted image; Using the first angle-shifted dedistorted image as an input of a deep network model to obtain a pixel-level classification result of the first angle-shifted dedistorted image; A drivable area is determined according to the pixel-level classification result.

2. The method according to claim 1, characterized in that: The first mapping relationship includes: a second mapping relationship from the original image to the dedistorted image and a third mapping relationship from the dedistorted image to the angle-shifted image; The determining, according to the first mapping relationship, a first angle-shifted dedistorted image corresponding to the first original image includes: Determining a first dedistorted image corresponding to the first original image according to the second mapping relationship; Determine, according to the third mapping relationship, a first angle-shifted image corresponding to the first dedistorted image; The first angle-shifted image is used as the first angle-shifted dedistorted image.

3. The method according to claim 2, characterized in that The second mapping relationship includes: Among them, (x m ,y m ) is the coordinate of pixel m in the original image, (x m-corrected ,y m-corrected ) are the coordinates of the pixel point corresponding to the pixel point m in the dedistorted image, k1, k2 and k3 are the radial distortion coefficients of the vehicle camera, p1 and p2 are the tangential distortion coefficients of the vehicle camera, the pixel point m is any pixel point in the original image, and r is the distance from the pixel point m to the origin of the original image.

4. The method according to claim 2, characterized in that: The third mapping relationship includes: Among them, (x n ,y n ,z n ) is the coordinate of pixel n in the dedistorted image in the camera coordinate system, (x n ',y n ',z n ') is the coordinate of the pixel point n in the world coordinate system, R x , R y and R z is the rotation matrix of the camera coordinate system relative to the world coordinate system, λ is the image scaling ratio, (x n ”,y n ”) is the coordinate of the pixel point corresponding to the pixel point n in the angle offset image, f x 、c x 、f y and c y is the internal parameter of the vehicle camera, and the pixel point n is any pixel point in the dedistorted image.

5. The method according to claim 1, characterized in that The method further comprises: The pixel-level classification results are traversed by a boundary search algorithm to obtain multiple boundary points of the drivable area; According to a fourth mapping relationship, the actual distance between the vehicle and the actual position corresponding to each of the boundary points is determined, wherein the fourth mapping relationship includes a mapping relationship between the actual position corresponding to each pixel point in the angle-shifted dedistorted image and the actual distance between the vehicle.

6. The method according to claim 5, characterized in that The pixel-level classification results are traversed by the boundary search algorithm to obtain multiple boundary points of the drivable area, including: Using the lower left corner point of the first angle offset dedistorted image as the search starting point; Starting from the search starting point, detecting whether there is a sudden change in pixel value in the current sliding window; If yes, record the current sliding window information, move the sliding window to the bottom of the first angle offset dedistorted image and move it to the right by X pixels, and continue to perform the step of detecting whether there is a sudden change in pixel value in the current sliding window, where X is the width of the sliding window; If not, the sliding window is moved upward by Y pixels, and the detection of whether there is a sudden change in pixel value in the current sliding window is continued until there is a sudden change in pixel value in the current sliding window or the sliding window is moved to the top of the first angle offset dedistorted image, where Y is the height of the sliding window; After traversing the first angle offset dedistorted image, traverse each pixel point in the sliding window corresponding to the recorded sliding window information to obtain multiple boundary points of the drivable area.

7. The method according to claim 6, characterized in that The traversal records each pixel point in the sliding window corresponding to the sliding window information to obtain multiple boundary points of the drivable area, including: For the first sliding window, from bottom to top, compare whether the pixel-level categories of two adjacent pixel points are the same. If they are not the same, the pixel point below the two adjacent pixel points is determined as a boundary point, wherein the first sliding window is any sliding window in the sliding windows corresponding to the recorded sliding window information.

8. The method according to claim 5, characterized in that Determining the actual distance between the vehicle and the actual position corresponding to each of the boundary points according to the fourth mapping relationship includes: According to the following formula, determine the coordinates of the actual position corresponding to each boundary point in the vehicle rear axle coordinate system: Among them, (u P , v P ) is the coordinate of the pixel point corresponding to the boundary point P in the first angle offset dedistorted image in the first original image, (X c , Y c , Z c ) is the coordinate of the actual position corresponding to the boundary point P in the camera coordinate system, (X w , Y w , Z w ) is the coordinate of the actual position corresponding to the boundary point P in the vehicle rear axle coordinate system, f x 、c x 、f y and c y is the vehicle camera internal parameter, is the rotation matrix of the camera coordinate system relative to the vehicle rear axle coordinate system, and r1 to r9 and t1 to t3 are parameters of the rotation matrix.

9. A drivable area detection device, characterized in that: The device comprises: An acquisition module, used to acquire a first original image captured by a vehicle camera; A first determining module, configured to determine a first angle-shifted dedistorted image corresponding to the first original image according to a first mapping relationship, wherein the first mapping relationship is a mapping relationship from the original image to the angle-shifted dedistorted image; A classification module, used to use the first angle shifted dedistorted image as an input of a deep network model to obtain a pixel-level classification result of the first angle shifted dedistorted image; The second determination module is used to determine the drivable area according to the pixel-level classification result.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the drivable area detection method according to any one of claims 1 to 8 is implemented.

Images