Low obstacle detection method, device and terminal equipment based on three cameras

Abstract

The embodiment of the present invention provides a method, device and terminal device for detecting short and low obstacles based on three cameras, the method comprising: obtaining a first image, a second image and a third image synchronously captured by a first camera, a second camera and a third camera; calculating the difference between the first image and the second image to obtain a first difference feature map; calculating the difference between the third image and the second image to obtain a second difference feature map; fusing the first difference feature map and the second difference feature map to obtain a fused feature map; and determining the area where the obstacle is located based on the fused feature map. The difference features generated in two different directions are fused to form a complete obstacle difference feature, effectively identifying short and low obstacles, reducing the influence of interference factors such as markings and shadows on the recognition accuracy, and enhancing the ability to perceive obstacles.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of autonomous driving visual perception, and in particular, to a method, device, and terminal device for detecting low and small obstacles based on three cameras. Background Art

[0002] As vehicles based on assisted driving and autonomous driving are applied to various scenarios, the demand for obstacle detection is increasing. Accurate obstacle detection can improve vehicle safety during driving and ensure the normal driving of the vehicle.

[0003] At present, there are two main methods for identifying obstacles in autonomous driving perception technology, visual perception recognition and laser perception recognition. Visual perception includes monocular vision combined with laser radar, TOF (Time of flight), and binocular vision obstacle avoidance. Obstacles identified by monocular vision combined with laser radar and TOF mainly include limited categories such as people and cars, and the ability to recognize obstacles outside of limited categories or relatively small obstacles on the ground is relatively poor; although binocular vision is not limited to categories, it is very sensitive to lighting requirements, and bright or dark lighting will lead to poor recognition results. As for laser perception recognition, although the recognition of obstacles is not limited to obstacle categories, due to the sparse laser point cloud, the effect is poor when detecting low and small obstacles. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, robot and storage medium for detecting low and small obstacles based on three cameras, so as to solve the problem of improving the detection accuracy of low and small obstacles.

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting small obstacles based on three cameras, comprising:

[0006] Acquire a first image, a second image, and a third image synchronously captured by a first camera, a second camera, and a third camera;

[0007] Calculating the difference between the first image and the second image to obtain a first difference feature map;

[0008] Calculating the difference between the third image and the second image to obtain a second difference feature map;

[0009] fusing the first difference feature map and the second difference feature map to obtain a fused feature map;

[0010] The area where the obstacle is located is determined based on the fused feature map.

[0011] In a second aspect, an embodiment of the present invention further provides a low obstacle detection device, comprising:

[0012] An image acquisition module, used to acquire a first image, a second image and a third image synchronously captured by the first camera, the second camera and the third camera;

[0013] A first difference feature calculation module, used to calculate the difference between the first image and the second image to obtain a first difference feature map;

[0014] A second difference feature calculation module, used for calculating the difference between the third image and the second image to obtain a second difference feature map;

[0015] A difference fusion module, used for fusing the first difference feature map and the second difference feature map to obtain a fused feature map;

[0016] A post-processing module is used to determine the area where the obstacle is located based on the fused feature map.

[0017] In a third aspect, an embodiment of the present invention further provides a low obstacle detection device, comprising:

[0018] at least one processor; and at least one memory storing instructions executable by the at least one processor;

[0019] The instruction is executed by the at least one processor so that the at least one processor implements the low and small obstacle detection method based on three cameras as described in the first aspect.

[0020] In a fourth aspect, an embodiment of the present invention further provides a robot, comprising:

[0021] A first camera, used to capture a first image;

[0022] A second camera, used to capture a second image;

[0023] A third camera, used for collecting a third image;

[0024] And the low obstacle detection device as described in the second aspect or the third aspect.

[0025] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting low and small obstacles based on three cameras as described in the first aspect is implemented.

[0026] In this embodiment, the first image, the second image and the third image synchronously captured by the first camera, the second camera and the third camera are obtained; the difference between the first image and the second image is calculated to obtain the first difference feature map; the difference between the third image and the second image is calculated to obtain the second difference feature map; the first difference feature map and the second difference feature map are fused to obtain the fused feature map; the area where the obstacle is located is determined based on the fused feature map. By combining three cameras and two networks, the difference features generated in two different directions are fused to form a complete obstacle difference feature, which can effectively identify low and small obstacles on indoor and outdoor roads, reduce the impact of interference factors such as road markings and shadows on the recognition accuracy, and can complement general visual perception and laser perception to enhance the ability to perceive obstacles during autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A A flowchart of a method for detecting low and small obstacles based on three cameras provided in Embodiment 1 of the present invention;

[0028] Figure 1B This is a flowchart of an example of data processing of a low obstacle detection method based on three cameras in Embodiment 1 of the present invention;

[0029] Figure 1C The cone-shaped roadblock is a fused feature map obtained by using a three-camera based low obstacle detection method;

[0030] Figure 2 A schematic diagram of the structure of a low obstacle detection device provided in Embodiment 2 of the present invention;

[0031] Figure 3 A schematic diagram of the structure of a low obstacle detection device provided in Embodiment 3 of the present invention;

[0032] Figure 4 This is a schematic diagram of the structure of a robot provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0034] Embodiment 1

[0035] Figure 1A A flowchart of a method for detecting small obstacles based on three cameras provided in Embodiment 1 of the present invention is provided. Figure 1BThis is a flowchart of an example of data processing of a low obstacle detection method based on three cameras in Embodiment 1 of the present invention. This embodiment can be applied to improve the detection accuracy of low obstacles. The method can be performed by a low obstacle detection device based on three cameras. The low obstacle detection device based on three cameras can be implemented by software and / or hardware, and can be configured in a computer device or a robot, for example, an intelligent robot, a server, a personal computer, etc., and specifically includes the following steps:

[0036] Step 101: Acquire a first image, a second image, and a third image synchronously captured by a first camera, a second camera, and a third camera.

[0037] Exemplarily, the first camera, the second camera, and the third camera are all arranged on the same plane, and are all used to obtain the spatial scene in front of the current plane. The first camera and the second camera are aligned and placed at a preset distance in the same horizontal direction, the second camera and the third camera are aligned and placed at a preset distance in the same vertical direction, and the third camera is tilted downward at a preset angle to obtain an image of a larger part of the ground. A three-buffer queue is set, and the images captured by the first camera, the second camera, and the third camera are respectively input into each buffer queue, and images captured at the same time are selected from the three buffer queues as the first image, the second image, and the third image.

[0038] Exemplarily, the first camera and the second camera are symmetrically arranged at an interval of 30 cm in the same horizontal direction, and the angles of the first camera and the second camera are also horizontal to the ground. The second camera and the third camera are arranged at an interval of 40 cm in the same vertical direction, and the third camera is arranged above the second camera and tilted downward by 20°. The first camera, the second camera, and the third camera all continuously shoot the scene in front of the camera to collect images. The first camera, the second camera, and the third camera respectively transmit the collected data to different queues of the three buffer queues, and obtain images at the same time in the three buffer queues as the first image, the second image, and the third image respectively. If there are no three images with completely consistent time, three images with a time difference not exceeding a preset interval threshold, such as 10ms, are selected as the first image, the second image, and the third image respectively. The resolution of the images collected by the three cameras is 1028×720, and the frame rate is 30 frames per second.

[0039] It should be noted that in the above-mentioned embodiment, the first camera and the second camera are in the same horizontal direction, and the second camera and the third camera are in the same vertical direction. This is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, the first camera, the second camera, and the third camera may also have other settings in different positions. For example, the first camera and the second camera are in the same horizontal direction, and the third camera is located directly above the midpoint of the first camera and the second camera. The present invention is not limited here.

[0040] In some embodiments of the present invention, step 101 includes:

[0041] Step 1011: Calculate a first homography matrix according to four pairs of key marker points in the first image and the second image.

[0042] Homography is used to describe the positional mapping relationship between two images. The homography matrix H between the two images is calculated by using the coordinates of the four pairs of key marker points selected from the two images. Then the projection transformation function is called to transform one image into the data of another image. The transformation matrix used for projection transformation is called the homography matrix. The homography matrix can be represented by a 3×3 non-singular matrix H as follows:

[0043]

[0044] The homography matrix H is a homogeneous matrix with 8 unknowns, and the last element is normalized to 1.

[0045] Four pairs of points with the same features are selected from the first image and the second image as key marker point pairs, and each pair of key marker point pairs is a pair of pixels representing the same features in the first image and the second image.

[0046] The coordinate relationship between the four pairs of key marking points in the first image and the second image is used to obtain the value of the first homography matrix H1 for aligning the first image with the second image through the following formula.

[0047]

[0048] In the following embodiments of the present invention, before calculating the first homography matrix, the first image and the second image can be corrected using the intrinsic matrix of the first camera and the second camera, respectively, so as to display the image and provide standard position information for navigation. After correcting the first image and the second image using the intrinsic matrix, the corrected first image and the corrected second image are used to mark key marking point pairs to calculate the first homography matrix.

[0049] It should be noted that the process of calculating the first homography matrix through four pairs of key marker points in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, there may be other processes of calculating the first homography matrix through four pairs of key marker points, and the present invention is not limited here.

[0050] Step 1012: multiply the first homography matrix by the matrix composed of pixel values ​​of the first image to align the first image with the second image.

[0051] For the points in the first image plane, use w=1 to normalize the point value, and use the two coordinates x and y of the image coordinates to form a coordinate matrix [xy 1] -1 , use this coordinate matrix to multiply with the first homography matrix H1 to get the coordinate matrix [x' y' w'] -1 , that is, the first image and the second image are aligned by the following formula.

[0052]

[0053] In some embodiments of the present invention, step 101 includes:

[0054] Step 1013: Calculate a first homography matrix according to four pairs of key marker points in the second image and the third image.

[0055] Four pairs of points with the same features are selected in the second image and the third image as key marker point pairs, and each pair of key marker point pairs is a pair of pixels representing the same features in the second image and the third image.

[0056] The coordinate relationship between the four pairs of key marking points in the second image and the third image is used to obtain the value of the second homography matrix H2 for aligning the third image with the second image through the following formula.

[0057]

[0058] In the following embodiment of the present invention, before calculating the second homography matrix, the second image and the third image can be corrected respectively using the intrinsic matrix of the second camera and the third camera to facilitate display of the image and provide standard position information for navigation. After correcting the second image and the third image using the intrinsic matrix, the corrected second image and the corrected third image are used to mark key marking point pairs to calculate the second homography matrix.

[0059] It should be noted that the process of calculating the second homography matrix through four pairs of key marker points in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, there may be other processes of calculating the second homography matrix through four pairs of key marker points, and the present invention is not limited here.

[0060] Step 1014: multiply the second homography matrix by the matrix composed of pixel values ​​of the third image to align the third image with the second image.

[0061] For the points in the second image plane, use w=1 to normalize the point value, and use the two coordinates x and y of the image coordinates to form a coordinate matrix [xy 1] -1 , use this coordinate matrix to multiply with the second homography matrix H2 to get the coordinate matrix [x' y' w'] -1 , that is, the second image and the third image are aligned by the following formula.

[0062]

[0063] Step 102: Calculate the difference between the first image and the second image to obtain a first difference feature map.

[0064] The first image and the second image include a common part, and the first image and the second image are horizontally aligned. By calculating the difference between the first image and the second image and generating a first difference feature map, the horizontal features in the area in front of the current camera can be obtained. For objects with width, there will be more obvious features.

[0065] It should be noted that the horizontal direction in the above-mentioned embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, according to the settings of the first camera and the second camera, the difference between the first image and the second image can also be calculated to obtain features in other directions, and the present invention is not limited here.

[0066] In some embodiments of the present invention, step 102 includes:

[0067] Step 1021: Input the first image and the second image into the MobileNetV2 network for processing respectively, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the first image and k first feature maps corresponding to the second image.

[0068] The MobileNetV2 network is a network that can significantly reduce model parameters and computational complexity while maintaining similar accuracy. The network now expands the input low-dimensional compressed representation to high dimensions, uses lightweight deep convolution for filtering, and then uses a linear bottleneck to project the features back to the low-dimensional compressed representation. The deep convolution in the MobileNetV2 network uses a 3×3 depthwise separable convolution. The MobileNetV2 network includes multiple sequentially connected convolution blocks, and the output of the previous convolution block is used as the input of the next convolution block.

[0069] After the first image is adjusted to a preset size and input into the MobileNetV2 network, the feature maps output by the first k convolution blocks in the MobileNetV2 network are taken as the k first feature maps corresponding to the first image. The first feature map is a feature map containing image features output by the convolution block of the MobileNetV2 network.

[0070] After the second image is resized to a preset size and input into the MobileNetV2 network, the feature maps output by the first k convolutional blocks in the MobileNetV2 network are taken as the k first feature maps corresponding to the second image. The number of first feature maps extracted from the first image and the second image must be the same.

[0071] For example, k is 3, and after the first image and the second image are adjusted to 512×256, they are respectively input into the MobileNetV2 network, and the feature maps output by the first three convolutional blocks are respectively extracted to obtain the three first feature maps corresponding to the first image and the three first feature maps corresponding to the second image. They respectively include the features of the first image and the second image.

[0072] It should be noted that the value of k in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, k may also be other values, and the present invention is not limited here.

[0073] Step 1022: Calculate the difference between the i-th first feature map corresponding to the first image and the i-th first feature map corresponding to the second image to obtain k first significant feature maps, where i is less than or equal to k.

[0074] The number of first feature maps extracted from the first image and the second image is the same. The value of i varies from 1 to k, and the difference between the first feature map corresponding to the first image and the first feature map corresponding to the second image is calculated layer by layer to make the features in the overlapping part of the first image and the second image more obvious. The feature map obtained after subtraction is called the first significant feature map.

[0075] Exemplarily, the value of k is 3, i is first set to 1, and the difference between the first first feature map corresponding to the first image and the first first feature map corresponding to the second image is calculated to obtain the first first significant feature map; i is set to 2, and the difference between the second first feature map corresponding to the first image and the second first feature map corresponding to the second image is calculated to obtain the second first significant feature map; i is set to 3, and the difference between the third first feature map corresponding to the first image and the third first feature map corresponding to the second image is calculated to obtain the third first significant feature map, so a total of k first significant feature maps are obtained.

[0076] Step 1023: Input the first image and the second image into the mbv2_ca network for processing respectively, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the first image and m second feature maps corresponding to the second image.

[0077] The mbv2_ca network is a MobileNetV2 network with an attention block added. The attention mechanism can be considered as a resource allocation mechanism. The originally evenly distributed resources are redistributed according to the importance of the attention object. More resources are allocated to important units, and less resources are allocated to unimportant or bad units. In the structural design of deep neural networks, the resources to be allocated by attention are basically weights. By adding attention blocks, the convolution feature expression ability is effectively improved. The mbv2_ca network includes multiple convolution blocks connected in sequence, and the output of the previous convolution block is used as the input of the next convolution block.

[0078] After the first image is adjusted to a preset size and input into the mbv2_ca network, the feature maps output by the first m convolution blocks in the mbv2_ca network are taken as the m second feature maps corresponding to the first image. The second feature maps are feature maps containing image features output by the convolution blocks of the mbv2_ca network.

[0079] After the second image is resized to a preset size and input into the mbv2_ca network, the feature maps output by the first m convolutional blocks in the mbv2_ca network are taken as the m second feature maps corresponding to the second image. The number of second feature maps extracted from the first image and the second image must be the same.

[0080] The mbv2_ca network is used to extract features from the first image and the second image to obtain m second feature maps, which can be combined with the k first feature maps obtained by feature extraction using the MobileNetV2 network in step 1021 to form (k+m) features, thereby facilitating the search for more different features in the first image and the second image.

[0081] For example, m is 2, and after the first image and the second image are adjusted to 512×256, they are respectively input into the mbv2_ca network, and the feature maps output by the first two convolutional blocks are respectively extracted to obtain two second feature maps corresponding to the first image and two second feature maps corresponding to the second image. They respectively include the features of the first image and the second image.

[0082] It should be noted that the value of m in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, m may also be other values, and the values ​​of m and k may be equal or unequal, and the present invention is not limited thereto.

[0083] Step 1024: Calculate the difference between the i-th second feature map corresponding to the first image and the i-th second feature map corresponding to the second image to obtain m second significant feature maps, where i is less than or equal to m, and the k first significant feature maps and the m second significant feature maps constitute a first difference feature map.

[0084] The number of second feature maps extracted from the first image and the second image is the same. The value of i varies from 1 to m, and the difference between the second feature map corresponding to the first image and the second feature map corresponding to the second image is calculated layer by layer to make the features in the overlapping part of the first image and the second image more obvious. The feature map obtained after subtraction is called the second significant feature map.

[0085] Exemplarily, the value of m is 2, i is first set to 1, and the difference between the first second feature map corresponding to the first image and the first second feature map corresponding to the second image is calculated to obtain the first second significant feature map; i is set to 2, and the difference between the second second feature map corresponding to the first image and the second second feature map corresponding to the second image is calculated to obtain the second second significant feature map; therefore, a total of 2 second significant feature maps are obtained.

[0086] The k first significant feature maps extracted and calculated using the MobileNetV2 network and the m second significant feature maps extracted and calculated using the mbv2_ca network constitute the first difference feature map. Since the first image and the second image are two images taken by the first camera and the second camera with horizontal visual differences in the same horizontal direction, the first difference feature map generated by processing and calculating the first image and the second image will have relatively obvious horizontal difference features for objects with width.

[0087] The shallow layer of the network extracts the primary features of the image, such as texture, color, corner points, etc., while the deep layer of the network extracts the semantic features that cannot be recognized by the naked eye. The purpose of the present invention is to extract the significant difference features of the image, without extracting the semantic features of the deep layer of the network. In this embodiment, the feature maps of the first k layers of the MobileNetV2 network and the first m layers of the mbv2_ca network are extracted to extract the primary features such as texture and color in each image, which are sufficient for detecting obstacles, thereby improving the efficiency of detecting short obstacles.

[0088] In addition, the MobileNetV2 network and the mbv2_ca network are used to extract the primary features of the image. The combination of the primary features extracted by the two networks can improve the accuracy of detection.

[0089] It should be noted that the MobileNetV2 network and mbv2_ca network used in the above embodiments are exemplary illustrations of the embodiments of the present invention. In some embodiments of the present invention, other networks may also be used to extract features from the first image and the second image, and the present invention is not limited thereto.

[0090] Step 103: Calculate the difference between the third image and the second image to obtain a second difference feature map.

[0091] The second image and the third image include a common part, and the second image and the third image are vertically aligned. By calculating the difference between the second image and the third image, a second difference feature map is generated, and the vertical features in the area in front of the current camera can be obtained. For objects with height, there will be more obvious features.

[0092] In some embodiments of the present invention, step 103 includes:

[0093] Step 1031: input the third image into the MobileNetV2 network for processing, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the third image.

[0094] After adjusting the third image to a preset size and inputting it into the MobileNetV2 network, the feature maps output by the first k convolutional blocks in the MobileNetV2 network are taken as the k first feature maps corresponding to the third image.

[0095] For example, k is 3, and after the third image is resized to 512×256, it is input into the MobileNetV2 network, and the feature maps output by the first three convolutional blocks are extracted to obtain the three first feature maps corresponding to the third image. They include the third image.

[0096] It should be noted that the value of k in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, k may also be other values, and the present invention is not limited here.

[0097] Step 1032: Calculate the difference between the i-th first feature map corresponding to the third image and the i-th first feature map corresponding to the second image, and obtain k third significant feature maps, where i is less than or equal to k.

[0098] The number of first feature maps extracted from the second image and the third image is the same, that is, there are k first feature maps. The value of i varies from 1 to k, and the difference between the first feature map corresponding to the third image and the first feature map corresponding to the second image is calculated layer by layer, so that the features of the overlapping parts of the third image and the second image are more clearly displayed.

[0099] Exemplarily, the value of k is 3, i is first set to 1, and the difference between the first first feature map corresponding to the third image and the first first feature map corresponding to the second image is calculated to obtain the first third significant feature map; i is set to 2, and the difference between the second first feature map corresponding to the third image and the second first feature map corresponding to the second image is calculated to obtain the second third significant feature map; i is set to 3, and the difference between the third first feature map corresponding to the third image and the third first feature map corresponding to the second image is calculated to obtain the third third significant feature map, so a total of k third significant feature maps are obtained.

[0100] Step 1033: input the third image into the mbv2_ca network for processing, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the third image.

[0101] After the third image is adjusted to a preset size and input into the mbv2_ca network, the feature maps output by the first m convolutional blocks in the mbv2_ca network are taken as the m second feature maps corresponding to the third image.

[0102] For example, m is 2, and after the third image is resized to 512×256, it is input into the mbv2_ca network, and the feature maps output by the first two convolutional blocks are extracted to obtain two second feature maps corresponding to the third image. They respectively include the features of the first image and the second image.

[0103] It should be noted that the value of m in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, m may also be other values, and the values ​​of m and k may be equal or unequal, and the present invention is not limited thereto.

[0104] Step 1034: Calculate the difference between the i-th second feature map corresponding to the third image and the i-th second feature map corresponding to the second image to obtain m fourth significant feature maps, where i is less than or equal to m, and the k third significant feature maps and the m fourth significant feature maps constitute a second difference feature map.

[0105] The number of second feature maps extracted from the second image and the third image is the same. The value of i varies from 1 to m, and the difference between the second feature map corresponding to the third image and the second feature map corresponding to the second image is calculated layer by layer to make the features in the overlapping part of the third image and the second image more obvious. The feature map obtained after subtraction is called the fourth significant feature map.

[0106] Exemplarily, the value of m is 2, i is first set to 1, and the difference between the first second feature map corresponding to the third image and the first second feature map corresponding to the second image is calculated to obtain the first fourth significant feature map; i is set to 2, and the difference between the second second feature map corresponding to the third image and the second second feature map corresponding to the second image is calculated to obtain the second fourth significant feature map; therefore, a total of 2 fourth significant feature maps are obtained.

[0107] In step 103, the k third significant feature maps extracted and calculated using the MobileNetV2 network and the m fourth significant feature maps extracted and calculated using the mbv2_ca network constitute the second difference feature map. The second image and the third image are two images taken by the second camera and the third camera that have vertical visual feature differences in the same vertical direction. Therefore, the second difference feature map generated by processing and calculating the second image and the third image will have relatively obvious vertical difference features for objects with height.

[0108] It should be noted that the MobileNetV2 network and mbv2_ca network used in the above embodiments are exemplary illustrations of the embodiments of the present invention. In some embodiments of the present invention, other networks may also be used to extract features from the first image and the second image, and the present invention is not limited thereto.

[0109] Step 104: fuse the first difference feature map and the second difference feature map to obtain a fused feature map.

[0110] The first difference feature map represents the features in the horizontal direction, and the second difference feature map represents the features in the vertical direction.

[0111] The fused feature map generated by fusing the first difference feature map and the second difference feature map can fully reflect the significant features of the obstacle, so that the features of obstacles of various shapes, heights, and widths can all be displayed, thereby improving the accuracy of detection and preventing obstacles from being undetectable due to being too narrow or too short.

[0112] In some embodiments of the present invention, step 104 includes:

[0113] Step 1041: Calculate the sum of the i-th first significant feature map in the first difference feature map and the i-th third significant feature map in the second difference feature map, where i is less than or equal to k, and obtain k first fusion feature maps.

[0114] The first difference feature map includes k first significant feature maps, and the second difference feature map includes k third significant feature maps. They represent the features in the horizontal direction and the vertical direction respectively extracted by the MobileNetV2 network. i changes from 1 to k, and the sum of the first difference feature map and the second difference feature map in the i-th layer is calculated layer by layer, so that the features in the horizontal direction and the vertical direction can constitute the significant features of the complete obstacle.

[0115] Exemplarily, k is 3, i is first set to 1, and the sum of the first first significant feature map and the first third significant feature map is calculated to obtain the first first fused feature map; i becomes 2, and the sum of the second first significant feature map and the second third significant feature map is calculated to obtain the second first fused feature map; i becomes 3, and the sum of the third first significant feature map and the third third significant feature map is calculated to obtain the third first fused feature map. After the above calculation, a total of k first fused feature maps are obtained. The first fused feature map is a complete significant feature of the obstacle obtained by combining the features extracted by the MobileNetV2 network in the horizontal and vertical directions.

[0116] It should be noted that the value of k in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, k may also be other values, and the present invention is not limited here.

[0117] Step 1042: Calculate the sum of the i-th second significant feature map in the first difference feature map and the i-th fourth significant feature map in the second difference feature map, where i is less than or equal to m, to obtain m second fused feature maps.

[0118] The first difference feature map includes m second significant feature maps, and the second difference feature map includes m fourth significant feature maps. They represent the features in the horizontal direction and the vertical direction respectively extracted by the mbv2_ca network. i changes from 1 to m, and the sum of the first difference feature map and the second difference feature map in the i-th layer is calculated layer by layer, so that the features in the horizontal direction and the vertical direction can constitute the significant features of the complete obstacle.

[0119] Exemplarily, m is 2, i is first set to 1, and the sum of the first second significant feature map and the first fourth significant feature map is calculated to obtain the first second fused feature map; i becomes 2, and the sum of the second second significant feature map and the second fourth significant feature map is calculated to obtain the second second fused feature map; after the above calculation, a total of m second fused feature maps are obtained, and the second fused feature map is a complete significant feature of the obstacle obtained by combining the features extracted in the horizontal and vertical directions by the mbv2_ca network.

[0120] It should be noted that the value of m in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, m may also be other values, and the present invention is not limited here.

[0121] Step 1043: Calculate the product of the first fused feature map and the second fused feature map with the corresponding hyperparameters to obtain (k+m) fused hyperparameter feature maps.

[0122] In order to more clearly show the characteristics of obstacles, (k+m) hyperparameters are pre-set to be multiplied with the k first fusion feature maps and the m second fusion feature maps respectively to highlight or weaken some features. The k first fusion feature maps obtained in step 1041 are multiplied with the corresponding k hyperparameters to obtain k fusion hyperparameter feature maps; the m second fusion feature maps obtained in step 1042 are multiplied with the corresponding m hyperparameters to obtain m fusion hyperparameter feature maps. Therefore, a total of (k+m) fusion hyperparameter feature maps are obtained.

[0123] Exemplarily, three first fused feature maps are obtained in step 1041, and two second fused feature maps are obtained in step 1042. The three first fused feature maps are multiplied by three hyperparameters respectively to obtain three fused hyperparameter feature maps; the two second fused feature maps are multiplied by two hyperparameters respectively to obtain two fused hyperparameter feature maps; a total of five fused hyperparameter feature maps are obtained.

[0124] Step 1044: Calculate the sum of (k+m) fusion hyperparameter feature maps to obtain a fusion feature map.

[0125] Calculate the sum of (k+m) fused hyperparameter feature maps, that is, add up the feature maps of all obstacles after hyperparameter adjustment to obtain a fused feature map with better expressiveness. The fused feature map can more accurately reflect the characteristics of the obstacle.

[0126] Step 105: Determine the area where the obstacle is located based on the fused feature map.

[0127] Figure 1C The cone-shaped roadblock is a fusion feature map obtained by using a three-camera based low obstacle detection method, such as Figure 1CAs shown in FIG. 1 , there is an obvious and continuous area with different grayscale values ​​in the fused feature map obtained after calculation and processing, and this area is where the obstacle is located.

[0128] It should be noted that the above embodiment uses the relatively obvious and continuous area with different grayscale values ​​as the area where the obstacle is located for exemplary description of the embodiment of the present invention. In other embodiments of the present invention, other methods can also be used to determine the area where the obstacle is located in the fusion feature map, and the present invention is not limited thereto.

[0129] In some embodiments of the present invention, step 105 includes:

[0130] Step 1051: Compare the grayscale value of each pixel in the fusion feature map with a preset grayscale threshold.

[0131] The preset grayscale threshold is used to determine whether there is an area with obstacles.

[0132] The grayscale value of each pixel in the fusion feature map is compared with the preset grayscale threshold, and the area with obstacles is detected according to the preset grayscale threshold.

[0133] Step 1053: The area formed by pixels whose grayscale values ​​are greater than or equal to the grayscale threshold is taken as the area where the obstacle is located.

[0134] When there is an area formed by several pixels in the fusion feature map whose grayscale values ​​are greater than or equal to the grayscale threshold, the area is regarded as the area where the obstacle is located.

[0135] Exemplarily, the preset grayscale threshold is 200. When the grayscale values ​​of several pixels in the fusion feature map are greater than or equal to 200, the area formed by these pixels is regarded as the area where the obstacle is located, that is, the obstacle has been detected.

[0136] It should be noted that the preset grayscale threshold 200 in the above embodiment is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, the grayscale threshold can also be determined according to the actual application scenario and accuracy requirements, and the present invention is not limited here.

[0137] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0138] In this embodiment, the first image, the second image and the third image synchronously captured by the first camera, the second camera and the third camera are obtained; the difference between the first image and the second image is calculated to obtain the first difference feature map; the difference between the third image and the second image is calculated to obtain the second difference feature map; the first difference feature map and the second difference feature map are fused to obtain the fused feature map; the area where the obstacle is located is determined based on the fused feature map. By combining three cameras and two networks, the difference features generated in two different directions are fused to form a complete obstacle difference feature, which can effectively identify low and small obstacles on indoor and outdoor roads, reduce the impact of interference factors such as road markings and shadows on the recognition accuracy, and can complement general visual perception and laser perception to enhance the ability to perceive obstacles during autonomous driving.

[0139] Embodiment 2

[0140] Figure 2 The structural block diagram of a low obstacle detection device provided in the second embodiment of the present invention may specifically include the following modules:

[0141] An image acquisition module 201 is used to acquire a first image, a second image and a third image synchronously captured by a first camera, a second camera and a third camera;

[0142] A first difference feature calculation module 202, configured to calculate the difference between the first image and the second image to obtain a first difference feature map;

[0143] A second difference feature calculation module 203, used to calculate the difference between the third image and the second image to obtain a second difference feature map;

[0144] A difference fusion module 204 is used to fuse the first difference feature map and the second difference feature map to obtain a fused feature map;

[0145] The post-processing module 205 is used to determine the area where the obstacle is located based on the fused feature map.

[0146] In some embodiments of the present invention, the image acquisition module 201 includes:

[0147] A first homography matrix generation submodule, configured to calculate a first homography matrix according to four pairs of key marker points in the first image and the second image, wherein each pair of the key marker points is a pair of pixels representing the same feature in the first image and the second image;

[0148] The first image alignment module is used to multiply the first homography matrix with a matrix composed of pixel values ​​of the first image to align the first image with the second image.

[0149] In some embodiments of the present invention, the image acquisition module 201 includes:

[0150] A second homography matrix generation submodule, used to calculate a second homography matrix according to four pairs of key marker points in the second image and the third image, wherein each pair of the key marker points is a pair of pixels representing the same feature in the second image and the third image;

[0151] The third image alignment module is used to multiply the second homography matrix by a matrix composed of pixel values ​​of the third image to align the third image with the second image.

[0152] In some embodiments of the present invention, the first difference feature calculation module 202 includes:

[0153] A first feature map generation submodule, used to input the first image and the second image into the MobileNetV2 network for processing respectively, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the first image and k first feature maps corresponding to the second image;

[0154] A first significant feature map generating submodule, used to respectively calculate the difference between the i-th first feature map corresponding to the first image and the i-th first feature map corresponding to the second image, to obtain k first significant feature maps, where i is less than or equal to k;

[0155] A second feature map generation submodule, used to input the first image and the second image into the mbv2_ca network for processing respectively, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the first image and m second feature maps corresponding to the second image;

[0156] The second significant feature map generation submodule is used to calculate the difference between the i-th second feature map corresponding to the first image and the i-th second feature map corresponding to the second image, and obtain m second significant feature maps, where i is less than or equal to m, and k first significant feature maps and m second significant feature maps constitute a first difference feature map.

[0157] In some embodiments of the present invention, the second difference feature calculation module 203 includes:

[0158] A first feature map generation submodule is used to input the third image into the MobileNetV2 network for processing, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the third image;

[0159] A third significant feature map generating submodule, used to respectively calculate the difference between the i-th first feature map corresponding to the third image and the i-th first feature map corresponding to the second image, to obtain k third significant feature maps, where i is less than or equal to k;

[0160] A second feature map generation submodule is used to input the third image into the mbv2_ca network for processing, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the third image;

[0161] The fourth significant feature map generation submodule is used to calculate the difference between the i-th second feature map corresponding to the third image and the i-th second feature map corresponding to the second image, and obtain m fourth significant feature maps, wherein i is less than or equal to m, and k of the third significant feature maps and m of the fourth significant feature maps constitute the second difference feature map.

[0162] In some embodiments of the present invention, the difference fusion module 204 includes:

[0163] A first fused feature map generating submodule, used to calculate the sum of the i-th first significant feature map in the first difference feature map and the i-th third significant feature map in the second difference feature map, where i is less than or equal to k, to obtain k first fused feature maps;

[0164] A second fused feature map generating submodule, used to calculate the sum of the i-th second significant feature map in the first difference feature map and the i-th fourth significant feature map in the second difference feature map, where i is less than or equal to m, to obtain m second fused feature maps;

[0165] A fusion hyperparameter feature map generation submodule, used to calculate the product of the first fusion feature map and the second fusion feature map with the corresponding hyperparameters to obtain (k+m) fusion hyperparameter feature maps;

[0166] The fusion feature map generation submodule is used to calculate the sum of (k+m) fusion hyperparameter feature maps to obtain a fusion feature map.

[0167] In some embodiments of the present invention, the post-processing module 205 includes:

[0168] A grayscale comparison submodule, used to compare the grayscale value of each pixel in the fused feature map with a preset grayscale threshold;

[0169] The obstacle determination submodule is used to take the area formed by pixels whose grayscale values ​​are greater than or equal to the grayscale threshold as the area where the obstacle is located.

[0170] The three-camera-based low and small obstacle detection device provided in the embodiment of the present invention can execute the three-camera-based low and small obstacle detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0171] Embodiment 3

[0172] Figure 3 This is a schematic structural diagram of a low obstacle detection device provided in Embodiment 3 of the present invention. Figure 3 A block diagram of an exemplary low obstacle detection device 12 suitable for implementing an embodiment of the present invention is shown. Figure 3 The low obstacle detection device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0173] like Figure 3 As shown, the low obstacle detection device 12 is in the form of a general-purpose computing device. The components of the low obstacle detection device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0174] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0175] The low-profile obstacle detection device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the low-profile obstacle detection device 12, including volatile and non-volatile media, removable and non-removable media.

[0176] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The low obstacle detection device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 3 not shown, usually called a "hard drive"). Although Figure 3Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0177] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0178] The low obstacle detection device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may also communicate with one or more devices that enable a user to interact with the low obstacle detection device 12, and / or may communicate with any device that enables the low obstacle detection device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the low obstacle detection device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the low obstacle detection device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the low obstacle detection device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0179] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the low obstacle detection method provided in the embodiment of the present invention.

[0180] Embodiment 4

[0181] Figure 4 A schematic diagram of the structure of a robot provided in the fourth embodiment of the present invention may specifically include the following devices:

[0182] The first camera 401 is used to capture a first image;

[0183] The second camera 402 is used to capture a second image;

[0184] The third camera 403 is used to capture a third image;

[0185] A low and small obstacle detection device 12.

[0186] In order to enable those skilled in the art to better understand the embodiments of the present application, an example of the structure of a robot is described in this specification.

[0187] Exemplarily, the first camera 401, the second camera 402, and the third camera 403 are all connected to the low and small obstacle detection device 12, and the collected first image, second image, and third image are transmitted to the low and small obstacle detection device 12 for processing. Figure 3 and Figure 4 , the first camera 401, the second camera 402, and the third camera 403 are equivalent to Figure 3 The external device 14 is connected to the low obstacle device 12 through the I / O interface 22.

[0188] Specifically, the robot may be a robot with different purposes and in different forms, such as a delivery robot, a cleaning robot, a wheeled robot or a mobile robot. This embodiment is only an example and is not limiting.

[0189] The robot provided by the embodiment of the present invention can execute the processing method of the robot provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0190] Embodiment 5

[0191] Embodiment 5 of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned low and small obstacle detection method based on three cameras are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0192] Among them, computer-readable storage media may include, for example, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0193] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for detecting small obstacles based on three cameras, characterized in that: include: Acquire a first image, a second image, and a third image synchronously captured by a first camera, a second camera, and a third camera; wherein the first camera and the second camera are at an angle horizontal to the ground, and the third camera is tilted downward at a preset angle; Calculating the difference between the first image and the second image to obtain a first difference feature map; Calculating the difference between the third image and the second image to obtain a second difference feature map; fusing the first difference feature map and the second difference feature map to obtain a fused feature map; Determine the area where the obstacle is located based on the fused feature map; The step of calculating the difference between the first image and the second image to obtain a first difference feature map includes: Input the first image and the second image into the MobileNetV2 network for processing respectively, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the first image and k first feature maps corresponding to the second image; Calculate the difference between the i-th first feature map corresponding to the first image and the i-th first feature map corresponding to the second image respectively, to obtain k first significant feature maps, where i is less than or equal to k; Input the first image and the second image into the mbv2_ca network for processing respectively, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the first image and m second feature maps corresponding to the second image; Calculate the difference between the i-th second feature map corresponding to the first image and the i-th second feature map corresponding to the second image to obtain m second significant feature maps, where i is less than or equal to m, and k first significant feature maps and m second significant feature maps constitute a first difference feature map.

2. The method according to claim 1, characterized in that The step of acquiring the first image, the second image, and the third image synchronously captured by the first camera, the second camera, and the third camera includes: Calculating a first homography matrix according to four pairs of key marker points in the first image and the second image, wherein each pair of the key marker points is a pair of pixels representing the same feature in the first image and the second image; The first homography matrix is ​​multiplied by a matrix consisting of pixel values ​​of the first image to align the first image with the second image.

3. The method according to claim 1, characterized in that The step of acquiring the first image, the second image, and the third image synchronously captured by the first camera, the second camera, and the third camera includes: Calculating a second homography matrix according to four pairs of key marker points in the second image and the third image, wherein each pair of the key marker points is a pair of pixels representing the same feature in the second image and the third image; The second homography matrix is ​​multiplied by a matrix consisting of pixel values ​​of the third image to align the third image with the second image.

4. The method according to claim 1, characterized in that: The calculating the difference between the third image and the second image to obtain a second difference feature map includes: Input the third image into the MobileNetV2 network for processing, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the third image; respectively calculating the difference between the i-th first feature map corresponding to the third image and the i-th first feature map corresponding to the second image to obtain k third significant feature maps, where i is less than or equal to k; Input the third image into the mbv2_ca network for processing, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the third image; Calculate the difference between the i-th second feature map corresponding to the third image and the i-th second feature map corresponding to the second image to obtain m fourth significant feature maps, where i is less than or equal to m, and k of the third significant feature maps and m of the fourth significant feature maps constitute the second difference feature map.

5. The method according to claim 4, characterized in that The fusing the first difference feature map and the second difference feature map to obtain a fused feature map includes: Calculate the sum of the i-th first significant feature map in the first difference feature map and the i-th third significant feature map in the second difference feature map, where i is less than or equal to k, to obtain k first fused feature maps; Calculate the sum of the i-th second significant feature map in the first difference feature map and the i-th fourth significant feature map in the second difference feature map, where i is less than or equal to m, to obtain m second fused feature maps; Calculate the product of the first fusion feature map and the second fusion feature map with the corresponding hyperparameters to obtain (k+m) fusion hyperparameter feature maps; The sum of the (k+m) fusion hyperparameter feature maps is calculated to obtain a fusion feature map.

6. The method according to claim 1, characterized in that The determining the area where the obstacle is located based on the fused feature map includes: Compare the grayscale value of each pixel in the fused feature map with a preset grayscale threshold; The area formed by pixels whose grayscale values ​​are greater than or equal to the grayscale threshold is taken as the area where the obstacle is located.

7. A low obstacle detection device, characterized in that: include: An image acquisition module, used to acquire a first image, a second image and a third image synchronously captured by a first camera, a second camera and a third camera; wherein the first camera and the second camera are at an angle horizontal to the ground, and the third camera is tilted downward at a preset angle; A first difference feature calculation module, used to calculate the difference between the first image and the second image to obtain a first difference feature map; A second difference feature calculation module, used for calculating the difference between the third image and the second image to obtain a second difference feature map; A difference fusion module, used for fusing the first difference feature map and the second difference feature map to obtain a fused feature map; A post-processing module, used for determining the area where the obstacle is located based on the fused feature map; Wherein, the first difference feature calculation module includes: A first feature map generation submodule, used to input the first image and the second image into the MobileNetV2 network for processing respectively, take the feature maps output by the first k convolution blocks in the MobileNetV2 network, and obtain k first feature maps corresponding to the first image and k first feature maps corresponding to the second image; A first significant feature map generating submodule, used to respectively calculate the difference between the i-th first feature map corresponding to the first image and the i-th first feature map corresponding to the second image, to obtain k first significant feature maps, where i is less than or equal to k; A second feature map generation submodule, used to input the first image and the second image into the mbv2_ca network for processing respectively, take the feature maps output by the first m convolution blocks in the mbv2_ca network, and obtain m second feature maps corresponding to the first image and m second feature maps corresponding to the second image; The second significant feature map generation submodule is used to calculate the difference between the i-th second feature map corresponding to the first image and the i-th second feature map corresponding to the second image, and obtain m second significant feature maps, where i is less than or equal to m, and k first significant feature maps and m second significant feature maps constitute a first difference feature map.

8. A low obstacle detection device, characterized in that: include: at least one processor; and at least one memory storing instructions executable by the at least one processor; The instruction is executed by the at least one processor so that the at least one processor implements the low obstacle detection method based on three cameras as described in any one of claims 1-6.

9. A robot, characterized in that: The robot comprises: A first camera, used to capture a first image; A second camera, used to capture a second image; A third camera is used to capture a third image; wherein the first camera and the second camera are at angles horizontal to the ground, and the third camera is tilted downward at a preset angle; And the low obstacle detection device as described in claim 7 or 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method for detecting low and small obstacles based on three cameras as described in any one of claims 1 to 6 is implemented.

Images