A method and device for identifying ground plane

Through the combination of binocular vision sensor and inertial measurement unit, affine transformation is performed using dense parallax map and pose parameters, the problem of difficulty in ground plane recognition during vehicle vibration is solved, and accurate identification and rapid detection in non-parallel situations are achieved .

CN114648737BActive Publication Date: 2024-12-03ANHUI EYEVOLUTION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011505571.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-18
Publication Date
2024-12-03
Estimated Expiration
2040-12-18

AI Technical Summary

Technical Problem

The existing ground plane detection method is difficult to accurately identify the ground plane when the vehicle vibrates, especially when the baseline of the binocular vision sensor is not parallel to the ground, resulting in detection failure and affecting safe driving.

Method used

Using the ground plane recognition method based on binocular vision sensor and inertial measurement unit, the dense parallax map is obtained by preprocessing the original image, pose parameters are calculated, affine transformation is performed, rotation transformation matrix is ​​generated, disparity map is filtered, and pixel point distance is judged. Determine the ground plane area.

Benefits of technology

When the vehicle vibrates, it can accurately identify the ground plane, simplify detection algorithms, and improve detection speed. It is suitable for ground unmanned platforms and autonomous driving assistance systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114648737B_ABST
    Figure CN114648737B_ABST
Patent Text Reader

Abstract

The present invention discloses a ground plane recognition method and device, including: pre-processing the left and right original images of the current period to obtain a first dense disparity map; obtaining the current period posture parameters; affine transforming the image to be rotated and the first dense disparity map according to the current posture parameters to obtain a rotated image and a second dense disparity map, wherein the image to be rotated is a pre-set left original image or right original image; recognizing the rotated image and the second dense disparity map to obtain a ground plane area. The method has strong applicability and can accurately recognize the ground plane in the case of vehicle body vibration. On this basis, the detection algorithm based on the disparity map is simplified, and the detection speed is further improved. It can be widely used in ground unmanned platforms or automatic driving assistance systems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a ground plane recognition method and device. Background Art

[0002] Ground plane detection is a key technology in fields such as intelligent driving. The following ground plane detection methods are currently proposed:

[0003] 1. Detection methods based on active sensors such as ultrasonic sensors, laser radars, and millimeter-wave radars. Active sensors are used to detect the area ahead and determine the passable ground plane. Ultrasonic sensors are cheap but have a short detection distance, so they are not suitable for situations with fast movement speeds. Laser radars and millimeter-wave radars have a long detection distance and high accuracy, but they are expensive and large in size, so they are not suitable for some low-cost and lightweight vehicles.

[0004] 2. Detection method based on monocular vision. It mainly uses 2D information such as color and texture of monocular images to detect the ground plane. This method is easily affected by lighting, shadows, etc., and cannot achieve good performance in more complex environments.

[0005] However, in actual use, due to vibrations during vehicle movement (wheels entering a pit, running over a stone, etc.), the baseline of the binocular vision sensor is not parallel to the ground (that is, there is a roll angle between the binocular vision sensor and the ground), and the problem of being unable to calculate and detect the ground plane may occur.

[0006] Therefore, in order to ensure safe driving, it is urgent to provide a technical solution for a ground plane recognition method, which can simply and accurately identify the ground plane area when the vehicle vibrates, so as to facilitate safe driving of the vehicle. Summary of the invention

[0007] In view of the above problems in the prior art, the present invention aims to provide a ground plane recognition method, comprising:

[0008] Preprocessing the left and right original images of the current cycle to obtain a first dense disparity map;

[0009] Get the current periodic pose parameters;

[0010] Affine transform the image to be rotated and the first dense disparity map according to the current posture parameters to obtain a rotated image and a second dense disparity map, wherein the image to be rotated is a preset left-eye original image or a right-eye original image;

[0011] The rotated image and the second dense disparity map image are identified to obtain a ground plane area.

[0012] Furthermore, the affine transformation of the image to be rotated and the first dense disparity map according to the current posture parameters to obtain the rotated image and the second dense disparity map includes:

[0013] Obtaining a rolling angle with respect to the ground plane according to the current periodic posture parameter;

[0014] Generate a rotation transformation matrix based on the roll angle and the center point of the image to be rotated;

[0015] The image to be rotated and the first dense disparity map are affine transformed using the rotation transformation matrix to obtain the rotated image and the second dense disparity map.

[0016] Furthermore, the step of recognizing the rotated image and the second dense disparity map image to obtain a ground plane area includes:

[0017] Calculating the second dense disparity map to obtain a first V disparity map;

[0018] filtering the first V-disparity map to obtain a second V-disparity map;

[0019] Obtaining a mapping straight line of a ground plane area according to the second V-disparity map;

[0020] Determining whether the distance between each pixel point in the rotated image and the mapping straight line is less than a preset distance threshold based on the mapping straight line;

[0021] Pixel points whose distance is less than a preset threshold are regarded as the ground plane area.

[0022] Furthermore, filtering the first V-disparity map to obtain the second V-disparity map includes: using one or more of Gaussian blur denoising, gradient filtering, and binarization to filter the first V-disparity map to obtain the second V-disparity map.

[0023] Furthermore, the filtering of the first V-disparity map to obtain a second V-disparity map further includes:

[0024] The disparity range of the second V-disparity map is normalized to between 0 and 255.

[0025] Furthermore, it also includes:

[0026] Reversely rotate the rotated image using the rotation transformation matrix to obtain a final image, wherein the final image includes the ground plane area;

[0027] The ground plane area is taken as the traversable area.

[0028] Furthermore, the left and right original images include a left original image and a right original image;

[0029] The step of obtaining the current periodic posture parameters further includes:

[0030] Determine whether the difference between the current period posture parameter and the previous period posture parameter is greater than a preset posture threshold;

[0031] When the difference between the current period pose parameter and the previous period pose parameter is not greater than a preset pose threshold, affine transforming the image to be rotated and the first dense disparity map according to the previous period pose parameter to obtain a rotated image and a second dense disparity map;

[0032] When the difference between the current period pose parameter and the previous period pose parameter is greater than a preset pose threshold, the image to be rotated and the first dense disparity map are affine transformed according to the current pose parameter to obtain a rotated image and a second dense disparity map.

[0033] In another aspect, the present invention provides a ground plane recognition device, comprising:

[0034] A first dense disparity map determining module is configured to perform preprocessing on the left and right eye original images of the current cycle to obtain a first dense disparity map;

[0035] A parameter acquisition module is configured to execute acquisition of current periodic posture parameters;

[0036] A rotation module is configured to perform an affine transformation on the image to be rotated and the first dense disparity map according to the current posture parameters to obtain a rotated image and a second dense disparity map, wherein the image to be rotated is a preset left-eye original image or a right-eye original image;

[0037] The recognition module is configured to perform recognition on the rotated image and the second dense disparity map image to obtain a ground plane area.

[0038] On the other hand, the present invention provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement a ground plane identification method as described above.

[0039] On the other hand, the present invention provides a ground plane identification device, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements a ground plane identification method as described above by executing the instructions stored in the memory.

[0040] The present invention provides a method and device for identifying a ground plane, which has the following beneficial effects:

[0041] The present invention provides a ground plane recognition method based on a binocular vision sensor and an inertial measurement unit (IMU). The method has strong applicability and can accurately recognize the ground plane when the baseline of the binocular vision sensor is not parallel to the ground due to vehicle vibration. On this basis, the detection algorithm based on the disparity map is simplified, and the detection speed is further improved. The method can be widely used in ground unmanned platforms or automatic driving assistance systems, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A schematic diagram of hardware installation provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a flow chart of a first ground plane identification method provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a left-eye original image provided by an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a dense disparity map provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of a left-eye intermediate image provided by an embodiment of the present invention;

[0048] Figure 6 A schematic diagram of a second dense disparity map provided by an embodiment of the present invention;

[0049] Figure 7 A schematic diagram of a flow chart of a second ground plane identification method provided by an embodiment of the present invention;

[0050] Figure 8 A schematic diagram of a flow chart of a third ground plane identification method provided by an embodiment of the present invention;

[0051] Fig. 9 A schematic diagram of a first V-disparity map provided by an embodiment of the present invention;

[0052] Fig.10 A schematic diagram of a second V-disparity map provided by an embodiment of the present invention;

[0053] Fig.11A schematic diagram of a mapping straight line of a ground plane area provided by an embodiment of the present invention;

[0054] Fig.12 A schematic flow chart of a fourth ground plane identification method provided by an embodiment of the present invention;

[0055] Fig.13 A schematic diagram of the structure of a ground plane identification device provided by an embodiment of the present invention;

[0056] Fig.14 A schematic structural diagram of a ground plane identification device provided in an embodiment of the present invention.

[0057] Among them, 710 - a first dense disparity map determination module, 720 - a parameter acquisition module, 730 - a rotation module, and 740 - a recognition module. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0059] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0060] Figure 1 A hardware installation diagram provided for an embodiment of the present invention is as follows Figure 1 As shown, Figure 1The middle carrier can be a vehicle, a robot, etc., which is an apparatus for traveling on the road. The implementation of the present invention requires collecting the left and right eye original images by a binocular image acquisition device (binocular vision sensor), and obtaining the current cycle posture parameters and the previous cycle posture parameters by an inertial measurement unit. Therefore, before the implementation of the present invention, the binocular image acquisition device and the inertial measurement unit need to be set to the same period, otherwise the detection accuracy will be reduced or deviations will occur, and the binocular image acquisition device and the inertial measurement unit need to be parallel to the ground when installing the binocular image acquisition device and the inertial measurement unit, that is, the installation position of the binocular image acquisition device and the inertial measurement unit has no roll angle (roll angle) with the ground, but can have a heading angle (pitch angle), otherwise, if the installation position of the binocular image acquisition device and the inertial measurement unit has a roll angle with the ground, it is necessary to pre-calibrate the initial roll angle of the two, and in the subsequent rotation transformation matrix calculation, add the initial offset angle of the binocular image acquisition device and the inertial measurement unit.

[0061] like Figure 2 As shown, Figure 2 The present invention provides a flow chart of a first ground plane identification method provided in an embodiment of the present invention. The present invention provides a ground plane identification method, and the method includes:

[0062] S102 , preprocessing the left and right original images of the current period to obtain a first dense disparity map.

[0063] In the specific implementation process, the left and right original images can be captured or collected by a binocular image acquisition device within a specific time period, wherein the time period for shooting or collecting by the binocular image acquisition device is not specifically limited in the embodiment of this specification and can be set according to actual needs.

[0064] The left and right original images may include two original images taken by the binocular image acquisition device, namely the left original image L m and the right original image R m The left and right original images may contain image data such as the ground plane and obstacles. The first dense disparity map is obtained based on the left and right original images, wherein the first dense disparity map may be a first dense disparity map based on the left eye or a first dense disparity map based on the right eye.

[0065] Figure 4 A schematic diagram of a dense disparity map provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, for the left and right original images L m , R m After preprocessing, the first dense disparity map D can be calculated using a stereo matching algorithm (SGM, etc.) m .

[0066] S104. Obtain current periodic posture parameters.

[0067] In the specific implementation process, the current cycle posture parameters and the previous cycle posture parameters are obtained through the inertial measurement unit. The relationship between the difference between the current cycle posture parameters and the previous cycle posture parameters and the preset posture threshold can be used to determine whether the vehicle is vibrating or the binocular vision sensor baseline is not parallel to the ground. Among them, the preset posture threshold is not specifically limited in the embodiments of this specification and can be set according to actual needs.

[0068] For example, the left and right original images L acquired by the binocular image acquisition device are obtained. m , R m and timestamp T m , Figure 3 A schematic diagram of a left-eye original image provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown in the figure, n IMU data and IMU timestamps corresponding to this frame of image are collected at the same time. The amount of IMU data n depends on the quotient of the IMU data acquisition frequency and the image acquisition frequency (generally, the IMU data acquisition frequency is at least 4 times the image acquisition frequency, that is, n>=4); for an IMU that only supports obtaining raw data, the IMU data should at least include the timestamp T of the previous frame of image. m-1 The timestamp T from the beginning to this frame of image m So far, T m-1 With T m The accelerometer data (A m1 , A m2 , ... A mn ), gyroscope data (G m1 , G m2 , ... G mn ) and timestamp data (U m1 , U m2 , ...U mn ); For an IMU with its own pose fusion algorithm, the IMU data at least includes the timestamp T of the previous frame image m-1 The timestamp T from the beginning to this frame of image m So far, the pose parameters between them are quaternion data (Q m1 , Q m2 , ...Q mn ) and timestamp data (U m1 , U m2 , ...U mn ).

[0069] If the IMU type used is the type that only supports obtaining raw data, then for this group of IMU data (accelerometer data A m1 , A m2 , ... Amn , gyroscope data G m1 , G m2 , ... G mn and timestamp data U m1 , U m2 , ...U mn ) to perform pose fusion algorithm calculation and obtain the final timestamp U mn The pose parameters (quaternion, denoted as Q mn ); If the IMU used is the type with its own posture fusion algorithm, the last timestamp U can be directly obtained mn The pose parameters (recorded as quaternion Q mn ).

[0070] S106 , performing affine transformation on the image to be rotated and the first dense disparity map according to the current posture parameters to obtain a rotated image and a second dense disparity map, wherein the image to be rotated is a preset left-eye original image or a right-eye original image.

[0071] In the specific implementation process, the image to be rotated can be one of the left and right original images, which is pre-set. The setting method of the left eye original image or the right eye original image is not specifically limited in the embodiments of this specification, and can be set according to actual needs. For example, the left eye original image is pre-set as the image to be rotated, and then the left eye original image can be directly used as the image to be rotated when calculating the rolling angle. The method of pre-setting the image to be rotated can reduce the calculation process of calculating the rolling angle and improve the calculation time efficiency. The selection of the first dense disparity map can correspond to the selection of the image to be rotated. For example, if the left eye original image is used as the image to be rotated, the first dense disparity map can be based on the first dense disparity map of the left eye.

[0072] The embodiments of this specification take the left original image as an example to be rotated.

[0073] Based on the above embodiments, in one embodiment of this specification, Figure 7 A schematic diagram of a flow chart of a second ground plane identification method provided by an embodiment of the present invention, such as Figure 7 As shown, the affine transformation of the rotated image and the first dense disparity map according to the current posture parameters to obtain the rotated image and the second dense disparity map includes:

[0074] S202, obtaining a rolling angle with respect to the ground plane according to the current periodic posture parameter;

[0075] S204, generating a rotation transformation matrix based on the rolling angle and the center point of the image to be rotated;

[0076] S206 , using the rotation transformation matrix to affine transform the image to be rotated and the first dense disparity map to obtain the rotated image and the second dense disparity map.

[0077] In the specific implementation process, when the difference between the current cycle posture parameters and the previous cycle posture parameters is greater than the preset posture threshold, that is, when vibration occurs or the baseline of the binocular vision sensor is not parallel to the ground, the rotated image and the first dense disparity map can be affine transformed according to the current posture parameters to obtain the rotated image and the second dense disparity map.

[0078] For example, the current periodic pose parameters and the previous periodic pose parameters both include quaternions, so the quaternion Q mn Transformed into a rotation matrix Rm. Among them, it is recorded as

[0079] Q mn =w m +x m i+y m j+z m k

[0080] So,

[0081]

[0082] Then, the rotation matrix R m Expand in the order of YXZ to get the Euler angle E m (E mx ,E my ,E mz ). Then, according to the YXZ sequence:

[0083] R m =R(E my )R(E mx )R(E mz )

[0084] in,

[0085]

[0086]

[0087]

[0088] Solve the equation to find E m The obtained E mz is the roll angle with the ground (i.e. O m ).

[0089] According to the above roll angle combined with the center point of the image (the horizontal and vertical centers are respectively denoted as C mx, C my ) and other information to generate the rotation transformation matrix M m ;M m It is a 2*3 matrix: the center point of the image can be 1 / 2 of the length / width of the left original image or the right original image.

[0090] The process of generating a rotation matrix is ​​actually to obtain an affine transformation matrix: rotating the roll angle around the center of the image.

[0091]

[0092] Treat the rotated image and the first dense disparity map D m , using the rotation transformation matrix M m Perform affine transformation to obtain the rotated image L m’ and the second dense disparity map D m’ , it can be considered that the image L has been rotated m’ and the second dense disparity map D m’ The ground no longer has a roll angle, Figure 5 A schematic diagram of a left-eye intermediate image provided by an embodiment of the present invention; Figure 6 A schematic diagram of a second dense disparity map provided by an embodiment of the present invention is shown in FIG. Figure 5-6 shown.

[0093] S108: Identify the rotated image and the second dense disparity map image to obtain a ground plane area.

[0094] Figure 8 A schematic diagram of a flow chart of a third ground plane identification method provided by an embodiment of the present invention, such as Figure 8 As shown, in some possible embodiments, the step of identifying the rotated image and the second dense disparity map image to obtain a ground plane area includes:

[0095] S302, calculating the second dense disparity map to obtain a first V disparity map;

[0096] S304, filtering the first V-disparity map to obtain a second V-disparity map;

[0097] S306: Obtain a mapping straight line of the ground plane area according to the second V-disparity map.

[0098] In a specific implementation process, the second V disparity map V m’ , using RANSAC and other methods, find the mapping line Z of the ground plane area m , Fig.11 A schematic diagram of a mapping straight line of a ground plane area provided by an embodiment of the present invention, such as Fig.11 As shown;

[0099] S308, judging whether the distance between each pixel point in the rotated image and the mapping straight line is less than a preset distance threshold based on the mapping straight line;

[0100] S310: Pixel points whose distance is less than a preset distance threshold are regarded as the ground plane area.

[0101] In the specific implementation process, the mapping line Z can be used m , to determine the rotated image or the second dense disparity map D m’ Whether each pixel point in is on the ground plane, that is, if it is on the mapping line Z m Any point within a small area around it is considered to be on the ground plane and recorded, and finally the ground plane area is obtained.

[0102] Based on the above embodiment, in one embodiment of the present specification, filtering the first V-disparity map to obtain the second V-disparity map includes:

[0103] The first V-disparity map is filtered using one or more of Gaussian blur denoising, gradient filtering, and binarization to obtain the second V-disparity map.

[0104] In general, the second dense disparity map D m’ In addition to the ground plane, there are other objects on the ground, so we need to further analyze the first V disparity map V m Filtering is performed to improve the accuracy of ground plane detection. One or more of Gaussian blur denoising, gradient filtering, and binarization can be used to filter the first V disparity map V m Filtering can obtain a clearer filtered second V disparity map V m’ , Fig.10 A schematic diagram of a second V-disparity map provided by an embodiment of the present invention is shown in FIG. Fig.10 shown.

[0105] Based on the above embodiment, in one embodiment of the present specification, the filtering of the first V-disparity map to obtain the second V-disparity map further includes:

[0106] The disparity range of the second V-disparity map is normalized to between 0 and 255.

[0107] In a specific implementation process, the disparity range of the second V disparity map may be normalized to 0 to 255. Fig. 9 A schematic diagram of a first V disparity map provided by an embodiment of the present invention is shown in FIG. Fig. 9 shown.

[0108] Based on the above embodiment, in one embodiment of this specification, it further includes:

[0109] Reversely rotate the rotated image using the rotation transformation matrix to obtain a final image, wherein the final image includes the ground plane area;

[0110] The ground plane area is taken as the traversable area.

[0111] In the specific implementation process, it can be understood that since the obtained ground plane area corresponds to the left eye intermediate image, that is, the rotated left eye original image, if it is necessary to correspond to the left eye back to the original image, it is necessary to do a reverse rotation. The method of generating the reverse rotation matrix is ​​the same as the process of generating Mm, except that the roll angle becomes -0. m The traversable area may be a ground level area free of obstacles.

[0112] Based on the above embodiments, in one embodiment of this specification, Fig.12 A schematic diagram of a flow chart of a fourth ground plane identification method provided by an embodiment of the present invention, such as Fig.12 As shown, the left and right original images include a left original image and a right original image;

[0113] The step of obtaining the current periodic posture parameters further includes:

[0114] S702: Determine whether the difference between the current period posture parameter and the previous period posture parameter is greater than a preset posture threshold.

[0115] S704: When the difference between the current cycle pose parameter and the previous cycle pose parameter is not greater than a preset pose threshold, affine transform the image to be rotated and the first dense disparity map according to the previous cycle pose parameter to obtain a rotated image and a second dense disparity map.

[0116] In the specific implementation process, when the difference between the current cycle posture parameters and the previous cycle posture parameters is no greater than the preset posture threshold, it can be characterized as driving on a smooth road surface, that is, when the vehicle is not vibrating, the rolling angle of the previous cycle or the rotation transformation matrix calculated in the previous cycle can be directly used to directly perform affine transformation on the image to be rotated of the current cycle and the first dense disparity map to obtain the rotated image and the second dense disparity map, which can reduce the calculation process.

[0117] S706: When the difference between the current period pose parameter and the previous period pose parameter is greater than a preset pose threshold, affine transform the image to be rotated and the first dense disparity map according to the current pose parameter to obtain a rotated image and a second dense disparity map.

[0118] The present invention relates to a ground plane recognition method, comprising the following steps: obtaining the left and right original images collected by a binocular image acquisition device and the IMU data collected by an IMU acquisition device; preprocessing the left and right original images, and obtaining a first dense disparity map using a stereo matching technology; obtaining a roll angle relative to the ground plane using the IMU data, and performing a corresponding roll angle rotation on the rotated image and the first dense disparity map to obtain a second dense disparity map; performing calculations using the second dense disparity map to obtain a first V disparity map; filtering the first V disparity map, and then fitting to obtain a mapping straight line of the ground plane area; using the second dense disparity map and the mapping straight line to calculate the final ground plane area. The present invention can effectively deal with the problem that the baseline of the binocular vision sensor is not parallel to the ground due to vibration during movement, thereby causing calculation failure.

[0119] Since the roll angle between the ground plane and the ground plane is measured by IMU, the obtained image is approximately equivalent to the image taken when the baseline of the binocular vision sensor is parallel to the ground, reducing the possibility of calculation failure based on the disparity map algorithm; at the same time, when calculating the ground plane later, a simpler ground plane point detection strategy can be used, which improves the calculation speed and is more suitable for scenarios with high real-time requirements.

[0120] On the other hand, an embodiment of the present specification provides a ground plane recognition device, Fig.13 A schematic diagram of the structure of a ground plane recognition device provided by an embodiment of the present invention is shown in FIG. Fig.13 As shown, including:

[0121] A first dense disparity map determining module 710 is configured to perform preprocessing on the left and right eye original images of the current cycle to obtain a first dense disparity map;

[0122] The parameter acquisition module 720 is configured to execute the acquisition of the current periodic posture parameters;

[0123] A rotation module 730 is configured to perform an affine transformation on the image to be rotated and the first dense disparity map according to the current posture parameters to obtain a rotated image and a second dense disparity map, wherein the image to be rotated is a preset left-eye original image or a right-eye original image;

[0124] The recognition module 740 is configured to perform recognition on the rotated image and the second dense disparity map image to obtain a ground plane area.

[0125] On the other hand, an embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement a ground plane identification method as described above.

[0126] On the other hand, an embodiment of the present specification provides a ground plane recognition device, Fig.14 A schematic diagram of the structure of a ground plane recognition device provided by an embodiment of the present invention is shown in FIG. Fig.14 As shown, it includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements a ground plane identification method as described above by executing the instructions stored in the memory.

[0127] Since the technical effects of the ground plane identification device, computer-readable storage medium, and ground plane identification equipment are the same as those of the ground plane identification method, they will not be repeated here.

[0128] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. The test method provided in the embodiment of the present invention has the same implementation principle and technical effects as the aforementioned system embodiment. For the sake of brief description, for the parts not mentioned in the method embodiment, reference can be made to the corresponding contents in the aforementioned system embodiment.

[0129] In several embodiments provided in the present application, it should be understood that the disclosed system and method can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0131] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the above-mentioned claims.

Claims

1. A ground plane recognition method, characterized in that: include: Preprocessing the left and right original images of the current cycle to obtain a first dense disparity map; Get the current periodic pose parameters; Obtaining a rolling angle with respect to the ground plane according to the current periodic posture parameter; Generate a rotation transformation matrix based on the rolling angle and the center point of the image to be rotated, wherein the image to be rotated is a preset left-eye original image or a right-eye original image; Using the rotation transformation matrix to affine transform the image to be rotated and the first dense disparity map to obtain a rotated image and a second dense disparity map; The rotated image and the second dense disparity map image are identified to obtain a ground plane area.

2. The ground plane recognition method according to claim 1, characterized in that: The step of identifying the rotated image and the second dense disparity map image to obtain a ground plane area includes: Calculating the second dense disparity map to obtain a first V disparity map; filtering the first V-disparity map to obtain a second V-disparity map; Obtaining a mapping straight line of a ground plane area according to the second V-disparity map; Determining, based on the mapping straight line, whether the distance between each pixel point in the rotated image and the mapping straight line is less than a preset distance threshold; Pixel points whose distance is less than a preset threshold are regarded as the ground plane area.

3. The ground plane recognition method according to claim 2, characterized in that: The filtering the first V-disparity map to obtain the second V-disparity map includes: filtering the first V-disparity map using one or more of Gaussian blur denoising, gradient filtering, and binarization to obtain the second V-disparity map.

4. The ground plane recognition method according to claim 2, characterized in that: The filtering of the first V disparity map to obtain a second V disparity map further includes: The disparity range of the second V-disparity map is normalized to between 0 and 255.

5. The ground plane recognition method according to claim 1, characterized in that: Also includes: Reversely rotate the rotated image using the rotation transformation matrix to obtain a final image, wherein the final image includes the ground plane area; The ground plane area is taken as the traversable area.

6. The ground plane recognition method according to claim 1, characterized in that: The left and right original images include a left original image and a right original image; The step of obtaining the current periodic posture parameters further includes: Determine whether the difference between the current period posture parameter and the previous period posture parameter is greater than a preset posture threshold; When the difference between the current period pose parameter and the previous period pose parameter is not greater than a preset pose threshold, affine transforming the image to be rotated and the first dense disparity map according to the previous period pose parameter to obtain a rotated image and a second dense disparity map; When the difference between the current period pose parameter and the previous period pose parameter is greater than a preset pose threshold, the image to be rotated and the first dense disparity map are affine transformed according to the current period pose parameter to obtain a rotated image and a second dense disparity map.

7. A ground plane recognition device, characterized in that: include: A first dense disparity map determining module is configured to perform preprocessing on the left and right eye original images of the current cycle to obtain a first dense disparity map; A parameter acquisition module is configured to execute acquisition of current periodic posture parameters; A rotation module is configured to obtain a rolling angle with respect to a ground plane according to the current periodic posture parameter; generate a rotation transformation matrix based on the rolling angle and a center point of an image to be rotated, wherein the image to be rotated is a preset left-eye original image or a right-eye original image; and perform an affine transformation on the image to be rotated and the first dense disparity map using the rotation transformation matrix to obtain a rotated image and a second dense disparity map; The recognition module is configured to perform recognition on the rotated image and the second dense disparity map image to obtain a ground plane area.

8. A ground plane recognition device, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the ground plane identification method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the ground plane recognition method as described in any one of claims 1-6.