Obstacle identification method and related equipment

Abstract

The invention provides an obstacle recognition method and related equipment, and can be applied to the technical field of mowers. The method comprises the following steps: acquiring a depth map of a target scene; calculating a first brightness feature and a second brightness feature of a candidate obstacle in the depth map; and when the difference between the first brightness feature and the second brightness feature is greater than or equal to a first threshold value, determining that the candidate obstacle is an abnormal obstacle. Wherein the depth map is used for recording depth information from an object in the target scene to the automatic walking equipment. The first brightness feature and the second brightness feature are respectively used for indicating brightness distribution of the candidate obstacle at different visual angles. According to the scheme, whether the obstacle is a real object or a remote light source in a backlight scene can be effectively distinguished by analyzing the brightness characteristics of the candidate obstacle in the depth map of the target scene at different visual angles, so that the situation that the mower mistakenly recognizes the remote light source as a near obstacle and stops running is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of lawn mowers, and in particular to an obstacle recognition method and related equipment. Background Art

[0002] With the rapid development of intelligent society, people's demand for automated lawn maintenance is increasing. Manual lawn mowers can no longer meet the requirements of the era of efficiency and strong controllability. In this situation, smart lawn mowers are constantly being improved. Currently, most smart lawn mowers have automatic driving and autonomous obstacle avoidance functions, which can effectively complete mowing operations in complex environments.

[0003] However, in backlit scenes, the light is strong and uneven, making the light interference complex and diverse. Commonly used image denoising algorithms (such as denoising algorithms based on traditional filtering, denoising algorithms based on frequency domain transformation, denoising algorithms based on deep learning, etc.) are often unable to effectively handle this type of light interference, and it is easy for the lawn mower to mistakenly identify a distant light source as a nearby obstacle and stop moving. Summary of the Invention

[0004] In order to solve the above problems, the embodiments of the present application provide an obstacle recognition method and related equipment, which can enable a lawn mower to operate stably and autonomously in a backlit scene.

[0005] In a first aspect, a method for obstacle identification is provided for use with an autonomous vehicle. The method comprises: obtaining a depth map of a target scene; calculating a first brightness feature and a second brightness feature of a candidate obstacle in the depth map; and determining that the candidate obstacle is an abnormal obstacle if the difference between the first brightness feature and the second brightness feature is greater than or equal to a first threshold. The depth map is used to record the depth information from objects in the target scene to the autonomous vehicle. The first brightness feature and the second brightness feature are used to indicate the brightness distribution of the candidate obstacle at different viewing angles, respectively.

[0006] In the above scheme, by analyzing the brightness characteristics of the candidate obstacles in the depth map of the target scene at different viewing angles, it is possible to effectively distinguish whether the candidate obstacles are real objects (such as trees, stones) or distant light sources in backlit scenes (such as the sun, lights). Once the candidate obstacle is identified as a distant light source, the depth information of the candidate obstacle can be removed from the depth map or the depth information of the candidate obstacle in the depth map can be modified to infinity, so as to prevent the automatic walking device from mistakenly identifying the distant light source as a nearby obstacle and stopping. This not only improves the accuracy of path planning and mowing efficiency, but also enhances the robustness of the environmental perception of the automatic walking device, thereby improving the continuity and stability of the mowing operation.

[0007] In some possible implementations, obtaining a depth map of the target scene includes: obtaining a left image and a right image of the target scene; performing semantic segmentation on the left and right images, respectively, identifying and matching corresponding object regions in the left and right images; for each object region, calculating depth information within each object region based on pixel-level disparity within the corresponding object region in the left and right images; and obtaining a depth map of the target scene based on the depth information within each object region. The left image is generated by a left camera on the autonomous vehicle, and the right image is generated by a right camera on the autonomous vehicle.

[0008] In some possible implementations, before calculating the first brightness feature and the second brightness feature of the candidate obstacle, the above method further includes: if the depth information within the object area is less than or equal to a preset depth threshold, and / or the pixel area of the object area is greater than or equal to a preset pixel area threshold, then the object area is regarded as a candidate obstacle.

[0009] In some possible implementations, the above-mentioned calculation of the first brightness feature and the second brightness feature of the candidate obstacle includes: calculating the circumscribed rectangular box of the candidate obstacle in the depth map; calculating a first projection result of the circumscribed rectangular box in the left image, and performing statistics on the pixel brightness distribution in the first projection result to obtain the first brightness feature; calculating a second projection result of the circumscribed rectangular box in the right image, and performing statistics on the pixel brightness distribution in the second projection result to obtain the second brightness feature.

[0010] In some possible implementations, the method further includes: if the candidate obstacle is an abnormal obstacle, setting the grid state of the position of the candidate obstacle in the occupancy grid map to unknown, and reconstructing the occupancy grid map. If the depth information of the position of the candidate obstacle in the reconstructed occupancy grid map is greater than a preset depth threshold, then defining the position of the candidate obstacle in the reconstructed occupancy grid map as a passable area.

[0011] In some possible implementations, before reconstructing the occupancy grid map, the method further includes: improving the confidence of the lateral grids in the occupancy grid map.

[0012] In some possible implementations, the first brightness feature includes one or more of a luminance histogram, a grayscale histogram, a brightness channel histogram, a cumulative distribution function, a brightness gradient histogram, statistics, and the like.

[0013] In a second aspect, an obstacle recognition device is provided, comprising: an acquisition unit, a calculation unit, and a determination unit. The acquisition unit is used to acquire a depth map of the target scene. The calculation unit is used to calculate a first brightness feature and a second brightness feature of a candidate obstacle in the depth map. The determination unit is used to determine that a candidate obstacle is an abnormal obstacle when the difference between the first brightness feature and the second brightness feature is greater than or equal to a first threshold. The depth map is used to record depth information from an object in the target scene to the automatic walking device. The first brightness feature and the second brightness feature are respectively used to indicate the brightness distribution of the candidate obstacle at different viewing angles.

[0014] In some possible implementations, the acquisition unit is specifically configured to: acquire a left image and a right image of a target scene; perform semantic segmentation on the left and right images, respectively, to identify and match corresponding object regions in the left and right images; for each object region, calculate depth information within each object region based on pixel-level disparity within the corresponding object region in the left and right images; and obtain a depth map of the target scene based on the depth information within each object region. The left image is generated by the left camera of the autonomous vehicle, and the right image is generated by the right camera of the autonomous vehicle.

[0015] In some possible implementations, the above-mentioned calculation unit is further used to, before calculating the first brightness feature and the second brightness feature of the candidate obstacle, if the depth information within the object area is less than or equal to a preset depth threshold, and / or the pixel area of the object area is greater than or equal to a preset pixel area threshold, then treat the object area as a candidate obstacle.

[0016] In some possible implementations, the above-mentioned calculation unit is specifically used to: calculate the circumscribed rectangular box of the candidate obstacle in the depth map; calculate the first projection result of the circumscribed rectangular box in the left image, and perform statistics on the pixel brightness distribution in the first projection result to obtain a first brightness feature; calculate the second projection result of the circumscribed rectangular box in the right image, and perform statistics on the pixel brightness distribution in the second projection result to obtain a second brightness feature.

[0017] In some possible implementations, the determining unit is further configured to: if the candidate obstacle is an abnormal obstacle, set the grid state of the candidate obstacle's location in the occupancy grid map to unknown, and reconstruct the occupancy grid map. If the depth information of the candidate obstacle's location in the reconstructed occupancy grid map is greater than a preset depth threshold, then define the location of the candidate obstacle in the reconstructed occupancy grid map as a passable area.

[0018] In some possible implementations, the determining unit is further configured to improve the confidence of the lateral grids in the occupancy grid map before reconstructing the occupancy grid map.

[0019] In some possible implementations, the first brightness feature includes one or more of a luminance histogram, a grayscale histogram, a brightness channel histogram, a cumulative distribution function, a brightness gradient histogram, statistics, and the like.

[0020] In a third aspect, an obstacle recognition device is provided, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to execute the instructions. When the processor executes the instructions, the method as described in any one of the first aspects is implemented.

[0021] In a fourth aspect, a computer program product comprising instructions is provided, which, when executed by a computing device, causes the computing device to perform any method as described in the first aspect.

[0022] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method according to any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of an obstacle identification method provided in an embodiment of the present application;

[0024] Figure 2A This is a left image in a backlit scene provided by an embodiment of the present application.

[0025] Figure 2B is a right image in a backlit scene provided by an embodiment of the present application;

[0026] Figure 3A is a schematic diagram of a first brightness characteristic provided in an embodiment of the present application;

[0027] Figure 3B is a schematic diagram of a second brightness characteristic provided in an embodiment of the present application;

[0028] Figure 4A is a schematic diagram of another first brightness characteristic provided in an embodiment of the present application;

[0029] Figure 4B is a schematic diagram of another second brightness characteristic provided in an embodiment of the present application;

[0030] Figure 5 This is a schematic structural diagram of an obstacle recognition device provided in an embodiment of the present application;

[0031] Figure 6 is a structural block diagram of a lawn mower in some embodiments of the present application;

[0032] Figure 7 is a schematic diagram of a lawn mower in some embodiments of the present application;

[0033] Figure 8 Schematic diagram of another obstacle recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0035] In order to facilitate understanding of the embodiments of the present application, the devices involved in the present application are first introduced.

[0036] Automatic walking equipment refers to equipment that is easy to move and use, usually with rollers, casters or other devices to facilitate movement and operation as needed. It can be a lawn mower or other inspection robots and other related equipment.

[0037] A lawn mower is a type of autonomous machine. A lawn mower is a mechanical tool used to mow lawns, vegetation, and other objects, such as drum mowers, rotary disc mowers, and wheel mowers. A lawn mower consists of components such as an engine, a circuit board, a cutterhead motor, a cutter disc, wheels, and a drive motor. The engine, typically an internal combustion engine, is the mower's power source. The engine provides power to the drive motor, driving the mower. The circuit board (including the drive board) is the mower's control center, sending commands to the cutterhead motor, drive motor, and other electrical components to ensure they operate accordingly. The circuit board also receives signals from sensors on the mower, monitors its status, and performs corresponding controls. The cutterhead motor raises and lowers the cutterhead, thereby determining the mowing height. The cutterhead houses the blades that enable mowing. The wheels are the moving parts of the mower, enabling movement. Typically, the cutterhead is mounted on wheels, so a drive motor simultaneously drives the rotation of the cutterhead blades and the wheels. The high-speed rotation of the blades removes overlong blades from the lawn, completing the mowing task. Lawn mowers utilize the high-speed rotation of the cutterhead blades to mow the lawn, improving mowing efficiency, saving weeding workers time, and significantly reducing labor costs. Furthermore, using a lawn mower can reduce seedling damage and improve seedling removal efficiency, resulting in a cleaner and more aesthetically pleasing lawn. Consequently, lawn mowers are widely used for lawn care in homes, parks, stadiums, and other locations.

[0038] In order to solve the problem that autonomous walking equipment easily misidentifies a distant light source as a nearby obstacle and stops driving in backlit scenes, the embodiments of the present application provide an obstacle recognition method and related equipment, which can enable the lawn mower to operate stably and autonomously in backlit scenes.

[0039] See also Figure 1 , Figure 1This is a flow chart of an obstacle identification method provided by an embodiment of the present application. Figure 1 As shown, the obstacle recognition method provided by this application includes:

[0040] S101: The autonomous vehicle acquires a left image and a right image of a target scene.

[0041] In some possible implementations, the autonomous vehicle is equipped with a binocular camera to collect real-time image data of the environment ahead (also known as the target scene). The camera can be built into the autonomous vehicle for an integrated design, or installed as a standalone module on the outside of the autonomous vehicle for easy removal and replacement.

[0042] Taking a lawn mower as an example, a binocular camera is set in the lawn mower. The image generated by the left camera on the lawn mower is called the left image, and the image generated by the right camera on the lawn mower is called the right image. For example, the specific contents of the left image and the right image can be referred to respectively. Figure 2A 、 Figure 2B .in, Figure 2A This is a left image in a backlit scene provided by an embodiment of the present application. Figure 2B This is a right image in a backlit scene provided by an embodiment of the present application.

[0043] S102: The autonomous driving device calculates a depth map of the target scene based on the left image and the right image of the target scene.

[0044] In some possible implementations, the autonomous vehicle calculates a depth map of the target scene based on the left and right images of the target scene. This is achieved by performing binocular visual depth estimation on the target scene. The specific process is as follows:

[0045] First, the autonomous vehicle performs semantic segmentation on the left and right images respectively, and identifies and matches the corresponding object areas in the left and right images. Specifically, a pre-trained deep learning semantic segmentation model (such as DeepLab, PSPNet, etc.) is used to perform pixel-level classification on the left and right images respectively. Among them, the goal of semantic segmentation is to label each pixel in the image as a corresponding category (grass, tree, fence, stone, person, animal, etc.). Through the semantic segmentation results, pixels of the same category are aggregated into several connected regions to form object areas. Then, according to the semantic category and spatial position, the corresponding object areas in the left and right images are matched. For example, a "tree" area in the left image corresponds to an object area in the right image that is also a "tree" and has a similar position. Among them, the matching process can also be combined with geometric constraints (such as parallax range, camera calibration parameters) to improve accuracy.

[0046] Next, for each object area, the depth information of each object area is calculated based on the pixel-level disparity in the corresponding object area in the left image and the right image. Specifically, for each pair of matching object areas in the left image and the right image, the disparity map is calculated using the binocular stereo vision principle. Among them, pixel-level disparity refers to the horizontal pixel offset of the same object point in the left image and the right image. The disparity value of each pixel in the object area is calculated by a stereo matching algorithm (such as SGBM, block matching, deep learning disparity estimation network). Then, based on the disparity d of each pixel, the baseline distance B between the left camera and the right camera, and the focal length f of the camera, the depth value Z of each pixel is calculated using the triangulation principle, Z=f×B / d. Finally, the calculated depth value of each pixel is mapped back to the pixel position of the object area to form the depth information of the object area.

[0047] Finally, based on the depth information within each object region, a depth map of the target scene is generated. This depth map records the depth information from the objects in the target scene to the autonomous vehicle. Specifically, the depth information of all object regions is fused according to their positions in the left image to generate a depth map of the target scene. For overlapping regions, depth confidence or semantic priority can be used for fusion processing. The fused depth map can also be smoothed and filtered to remove noise and outliers. Semantic segmentation results can also be combined to apply different depth smoothing strategies to different object regions to improve depth map quality.

[0048] Taking a lawn mower as an example, the left and right images of the lawn working scene obtained by the lawn mower are semantically segmented by a semantic segmentation model, and the obtained image information is segmented into semantics such as grass, trees, fences, stones, etc., and then the corresponding object regions in the left and right images are matched according to the semantic categories and spatial positions. For example, a "tree" region in the left image corresponds to an object region in the right image that is also classified as "tree" and has a similar position. The matching process can also be combined with geometric constraints (such as parallax range and camera calibration parameters) to improve accuracy, and the depth information within each object region is calculated based on the obtained semantic labels, and the depth information within each object region is obtained to obtain the depth information of the target scene. It is understandable that because the scene targeted by the lawn mower is a lawn scene, it will not perform depth calculation on the semantic part of the lawn during semantic recognition, so as to reduce the amount of computation during the operation of the lawn mower.

[0049] S103: For a candidate obstacle in the depth map, the autonomous vehicle calculates a first brightness feature and a second brightness feature of the candidate obstacle, and determines whether the candidate obstacle is an abnormal obstacle based on a difference between the first brightness feature and the second brightness feature.

[0050] In some possible implementations, the autonomous vehicle may determine candidate obstacles from a depth map by referring to the following implementations 1 to 4:

[0051] Implementation method 1: Each object region in the depth map is regarded as a candidate obstacle. In other words, each object region in the acquired target scene that is semantically labeled as a relevant obstacle can be regarded as a candidate obstacle.

[0052] Implementation method 2: Determine candidate obstacles based on the depth information within each object area in the depth map.

[0053] Implementation method 3: Determine candidate obstacles based on the pixel area of each object region in the depth map.

[0054] Implementation method 4: Determine candidate obstacles based on the depth information and pixel area of each object region in the depth map.

[0055] Corresponding to implementation method 2 in step S103, if the depth information within an object area is less than or equal to the preset depth threshold, the autonomous vehicle will treat the object area as a candidate obstacle; if the depth information within an object area is greater than the preset depth threshold, the autonomous vehicle will not treat the object area as a candidate obstacle. The specific value of the preset depth threshold is determined by the user, and can be, for example, 30 cm, 50 cm, 100 cm, etc. Taking a lawn mower as an example, if the visual range of the lawn mower's camera is 3 meters, that is, it can capture image information within 3 meters, but the distance at which it accurately judges obstacles is 1 meter. Therefore, in order to avoid excessive computational effort and resulting in repeated calculation of obstacles, it will present obstacle labels with depth information greater than 1 meter, which will not be considered in this budget.

[0056] Corresponding to Implementation 3 in step S103, if the pixel area of an object region is greater than or equal to a preset pixel area threshold, the autonomous vehicle considers the object region as a candidate obstacle. If the pixel area of an object region is less than the preset pixel area threshold, the autonomous vehicle does not consider the object region as a candidate obstacle. The specific value of the preset pixel area threshold is determined by the user and can be, for example, 30 pixels, 50 pixels, or 60 pixels.

[0057] Corresponding to implementation method 4 in step S103, after the autonomous vehicle calculates the depth information and pixel area within an object region, if the depth information within the object region is less than or equal to a preset depth threshold, and the pixel area of the object region is greater than or equal to the preset pixel area threshold, the autonomous vehicle regards the object region as a candidate obstacle; if the depth information within the object region is greater than the preset depth threshold, or the pixel area of the object region is less than the preset pixel area threshold, the autonomous vehicle does not regard the object region as a candidate obstacle.

[0058] Since the candidate obstacles are reference information used to adjust the travel path of the autonomous vehicle, the number of candidate obstacles obtained through Implementation Method 2, Implementation Method 3, or Implementation Method 4 is less than the number of obstacles in the depth map. Therefore, the candidate obstacles obtained through Implementation Method 2, Implementation Method 3, or Implementation Method 4 can reduce the amount of calculation during path planning, improve real-time performance and response speed, help cope with sudden obstacles and dynamic environmental changes, reduce the risk of misjudgment and over-planning, avoid frequent path adjustments due to long-distance uncertain information, enhance stability, and improve the reliability of the autonomous vehicle.

[0059] It should be understood that the above implementation methods 1 to 4 for determining candidate obstacles are merely examples. In practical applications, any implementation method capable of identifying candidate obstacles from a depth map is within the scope of protection of this application.

[0060] After the autonomous vehicle identifies the candidate obstacle from the depth map, it calculates the bounding rectangle of the candidate obstacle. The bounding rectangle refers to the smallest rectangle that can completely surround the outline of the candidate obstacle. For example, the axis-aligned bounding rectangle, where the sides of the rectangle are parallel to the left image coordinate axis, is simple to calculate and is often used for fast detection. Or the minimum bounding rectangle, which allows the rectangle to rotate and finds the rectangle with the smallest area to more tightly surround the object's outline. Taking the calculation of the axis-aligned bounding rectangle as an example, after the autonomous vehicle identifies the candidate obstacle in the depth map, it extracts the contour point set of the object through image segmentation, edge detection and other methods, calculates the minimum and maximum coordinates of the contour point set, and then determines the coordinates of the upper left corner, width, height and other parameters of the axis-aligned bounding rectangle.

[0061] In some possible implementations, the autonomous driving device calculates a first brightness feature of the candidate obstacle, including: the autonomous driving device calculates a first projection result of the circumscribed rectangular frame of the candidate obstacle in the left image, and performs statistics on the pixel brightness distribution in the first projection result to obtain the first brightness feature.

[0062] The above calculation of the first projection result of the bounding rectangle of the candidate obstacle in the left image requires the intrinsic and extrinsic parameters of the left camera on the autonomous vehicle. Among them, the intrinsic parameters of the left camera are used to describe the internal geometry and optical characteristics of the left camera, usually including focal length, principal point coordinates, distortion parameters, etc. The intrinsic parameters of the left camera are usually represented by an intrinsic parameter matrix. The extrinsic parameters of the left camera are used to describe the rotation and translation relationship between the camera coordinate system and the world coordinate system, usually represented by a rotation matrix and a translation vector.

[0063] The above calculation of the first projection result of the bounding rectangle of the candidate obstacle in the left image is as follows:

[0064] First, obtain the coordinates of all vertices of the circumscribed rectangle, denoted as P world = {Q(i) world}. Among them, Q(i) world Represents the coordinates of the i-th vertex of the bounding rectangle, Q(i) world =(X(i) world ,Y(i) world ,Z(i) world ,1) T , here expressed in homogeneous coordinates.

[0065] Then, use the transformation matrix of the left camera to transform the vertex coordinates from the world coordinate system to the camera coordinate system. The calculation formula is as follows:

[0066] Q(i) cam =[R left |t left ]·Q(i) world

[0067] Among them, Q(i) cam Indicates the specific value of the coordinates of the i-th vertex of the circumscribed rectangular box in the camera coordinate system, Q(i) cam =(X(i) cam ,Y(i) cam ,Z(i) cam ,1) T , Q(i) world Indicates the specific value of the coordinates of the i-th vertex of the circumscribed rectangular box in the world coordinate system, [R left |t left ] represents the transformation matrix of the left camera, R left Represents the rotation matrix in the extrinsic parameters of the left camera, t left Represents the translation vector in the extrinsic parameters of the left camera.

[0068] Next, use the intrinsic parameter matrix of the left camera to project the vertex coordinates of the camera coordinate system to the two-dimensional pixel coordinates. The calculation formula is as follows:

[0069]

[0070] Among them, s left represents the scaling factor, (u(i) left ,v(i) left ) represents the pixel coordinates on the image plane (i.e., two-dimensional pixel coordinates), K left Represents the intrinsic parameter matrix of the left camera.

[0071] The coordinates Q(i) of each vertex of the circumscribed rectangular box world Execute the above steps to get the coordinates of each vertex Q(i) world The corresponding two-dimensional pixel coordinates (u(i) left ,v(i) left ). Connecting these two-dimensional pixel coordinates gives the projection result of the bounding rectangle of the candidate obstacle in the left image (called the first projection result).

[0072] After obtaining the first projection result, statistics are performed on the brightness distribution of pixels in the first projection result, and the statistical results are used as the first brightness feature. In some possible implementations, one or more of a luminance histogram, a grayscale histogram, a luminance channel histogram (L channel histogram), a cumulative distribution function (CDF), a brightness gradient histogram, a statistic, etc. can be used as the first brightness feature to describe the brightness distribution of pixels in the first projection result.

[0073] The luminosity histogram is used to calculate the overall brightness distribution of the first projection result. A luminosity histogram divides the grayscale values of all pixels in the first projection result (typically 0 to 255) into bins and plots the total number of pixels in each bin. Examples of luminosity histograms include RGB histograms and HSV histograms.

[0074] The RGB histogram is used to calculate the distribution of red (R), green (G), and blue (B) pixels in the first projection result. The RGB histogram calculates the distribution of pixel values for each of the three channels (R, G, and B) in the first projection result. Typically, the pixel values for each channel (0 to 255) are divided into bins, and the number of pixels within each bin is counted. The RGB histogram can be used to calculate individual R, G, and B histograms, or to construct a three-dimensional histogram to calculate the frequency of each color combination.

[0075] The HSV histogram is used to count the distribution of hue (H), saturation (S), and value (V) in the first projection result. The HSV histogram counts the distribution of pixel values in the three channels H, S, and V in the first projection result. The H channel represents the color type, usually ranging from 0° to 360° (generally normalized to 0 to 180 or 0 to 255 in computers). The S channel represents the purity or saturation of the color, ranging from 0 to 1 or 0 to 255. The V channel represents the brightness or lightness, ranging from 0 to 1 or 0 to 255. The HSV histogram can count the histograms of H, S, and V separately, or construct a two-dimensional or three-dimensional histogram (such as the HS two-dimensional histogram) to describe the color distribution in more detail.

[0076] The grayscale histogram is used to count the number of pixels corresponding to each grayscale level in the first projection result, and can also reflect the brightness distribution of pixels in the first projection result. The grayscale histogram is similar to the luminance histogram.

[0077] The luminance channel histogram is used to count the pixel value distribution of the luminance channel (such as the V channel in HSV) in the first projection result.

[0078] The cumulative distribution function is a cumulative frequency distribution calculated based on a histogram (such as a luminance histogram, a grayscale histogram, a brightness channel histogram, etc.), which represents the probability that the pixel brightness is less than or equal to a certain value and can reflect the overall trend of the pixel brightness distribution in the first projection result.

[0079] The brightness gradient histogram is used to count the distribution of the brightness gradient of the first projection result, focusing on describing the directional information of the brightness change.

[0080] Statistics such as mean, variance, skewness, and kurtosis comprehensively describe the characteristics of the pixel brightness distribution in the first projection result. The mean describes the central tendency of pixel brightness in the first projection result and reflects the average level of overall brightness. The variance describes the dispersion of pixel brightness in the first projection result and reflects the fluctuation of brightness values around the mean. The skewness describes the symmetry of the pixel brightness distribution in the first projection result and measures the direction and degree of skewness of the distribution relative to the mean. The kurtosis describes the peakedness of the pixel brightness distribution in the first projection result and reflects the steepness or flatness of the distribution curve.

[0081] It should be understood that the above description of the first brightness feature is merely an example and is not specifically limited in this application.

[0082] In some possible implementations, the autonomous vehicle calculates a second brightness feature of the candidate obstacle, including: the autonomous vehicle calculates a second projection result of the circumscribed rectangular frame of the candidate obstacle in the right image, and performs statistics on the pixel brightness distribution in the second projection result to obtain the second brightness feature.

[0083] The above calculation of the second projection result of the circumscribed rectangular box of the candidate obstacle in the right image requires the use of the intrinsic and extrinsic parameters of the right camera on the automatic walking device. Among them, the intrinsic parameters of the right camera are used to describe the internal geometry and optical characteristics of the right camera, usually including focal length, principal point coordinates, distortion parameters, etc. The intrinsic parameters of the right camera are usually represented by an intrinsic parameter matrix. The extrinsic parameters of the right camera are used to describe the rotation and translation relationship between the camera coordinate system and the world coordinate system, usually represented by a rotation matrix and a translation vector. In a stereo camera system, the left camera is usually selected as the reference coordinate system, and the extrinsic parameters of the right camera describe the rotation and translation of the right camera relative to the left camera, that is, the extrinsic parameters of the right camera are obtained based on the extrinsic parameter transformation of the left camera.

[0084] The above calculation of the second projection result of the bounding rectangle of the candidate obstacle in the right image is as follows:

[0085] First, obtain the coordinates of all vertices of the circumscribed rectangle, denoted as P world = {Q(i) world}. Among them, Q(i) world Represents the coordinates of the i-th vertex of the bounding rectangle, Q(i) world =(X(i) world ,Y(i) world ,Z(i) world ,1) T , here expressed in homogeneous coordinates.

[0086] Then, use the transformation matrix of the right camera to transform the vertex coordinates from the world coordinate system to the camera coordinate system. The calculation formula is as follows:

[0087] Q(i) cam =[R right |t right ]·Q(i) world

[0088] Among them, Q(i) cam Indicates the specific value of the coordinates of the i-th vertex of the circumscribed rectangular box in the camera coordinate system, Q(i) cam =(X(i) cam ,Y(i) cam ,z(i) cam ,1) T , Q(i) world Indicates the specific value of the coordinates of the i-th vertex of the circumscribed rectangular box in the world coordinate system, [R right |t right ] represents the transformation matrix of the right camera, R right Represents the rotation matrix in the external parameters of the right camera, t right Represents the translation vector in the extrinsic parameters of the right camera.

[0089] Next, use the intrinsic parameter matrix of the right camera to project the vertex coordinates of the camera coordinate system to the two-dimensional pixel coordinates. The calculation formula is as follows:

[0090]

[0091] Among them, s right represents the scaling factor, (u(i) right ,v(i) right ) represents the pixel coordinates on the image plane (i.e., two-dimensional pixel coordinates), K right Represents the intrinsic parameter matrix of the right camera.

[0092] The coordinates Q(i) of each vertex of the circumscribed rectangular box world Execute the above steps to get the coordinates of each vertex Q(i) world The corresponding two-dimensional pixel coordinates (u(i) right ,v(i) right ). Connecting these two-dimensional pixel coordinates gives the projection result of the bounding rectangle of the candidate obstacle in the right image (called the second projection result).

[0093] After obtaining the second projection result, statistics are collected on the pixel brightness distribution in the second projection result, and the statistical results are used as the second brightness feature. In some possible implementations, one or more of a luminance histogram, a grayscale histogram, a brightness channel histogram, a cumulative distribution function, a brightness gradient histogram, a statistic, etc. can be used as the second brightness feature to describe the pixel brightness distribution in the second projection result. It should be understood that the above description of the second brightness feature is merely an example and is not specifically limited in this application.

[0094] It should be noted that the second brightness feature and the first brightness feature are calculated using the same statistical method. For example, if the luminance histogram of the first projection result is used as the first brightness feature, then the luminance histogram of the second projection result is used as the second brightness feature. Alternatively, if the statistics (such as mean and variance) of the first projection result are used as the first brightness feature, then the statistics (such as mean and variance) of the second projection result are used as the second brightness feature. Alternatively, if the luminance histogram and statistics (such as mean) of the first projection result are used as the first brightness feature, then the luminance histogram and statistics (such as mean) of the second projection result are used as the second brightness feature.

[0095] For example, continue with the above Figure 2A As the left image, Figure 2A The rectangular box in the figure is taken as the first projection result, and the above Figure 2B As the right image, Figure 2B As an example of the second projection result, when the RGB histograms of the first projection result and the second projection result are used as the first brightness feature and the second brightness feature respectively, the first brightness feature and the second brightness feature are respectively as follows: Figure 3A 、 Figure 3B When the grayscale histograms of the first projection result and the second projection result are used as the first brightness feature and the second brightness feature respectively, the first brightness feature and the second brightness feature are respectively as follows Figure 4A 、 Figure 4B shown.

[0096] After calculating the first brightness feature and the second brightness feature, the autonomous vehicle calculates the difference between the first and second brightness features, and determines whether the candidate obstacle is an abnormal obstacle based on the size of the difference. If the difference between the first and second brightness features is greater than or equal to the first threshold, it indicates that the candidate obstacle has a large difference in luminosity at different viewing angles. This is usually a characteristic of distant light sources in backlit scenes, not a characteristic of real objects. Therefore, the candidate obstacle is marked as an abnormal obstacle, indicating that the candidate obstacle has been misjudged. If the difference between the first and second brightness features is less than the first threshold, it indicates that the candidate obstacle has a small difference in luminosity at different viewing angles. This is usually a characteristic of real objects. Therefore, the candidate obstacle is not marked as an abnormal obstacle. The first threshold is determined by the user.

[0097] It should be noted that for real objects (obstacles), their shapes and textures will have parallax at different viewing angles (such as the left image and the right image), but their surface materials and lighting conditions are usually relatively consistent at different viewing angles (such as the left image and the right image). Therefore, the luminosity (brightness) distribution of the corresponding areas is similar, and the overall brightness distribution does not change drastically. Therefore, the brightness difference of the object at different viewing angles is small. In the field of lawn mower technology, a backlit scene refers to an area where the light source is located in the front field of view of the lawn mower, that is, directly opposite or nearly directly opposite the camera or visual sensor on the lawn mower. When the lawn mower is moving towards the light source, strong light directly enters the camera, resulting in highlights, glare or halo phenomena in the image. This situation is called a backlit scene. Distant light sources (such as the sun and lamplight) in backlit scenes appear as bright areas in the image. Due to the long distance, complex light direction and occlusion relationships, the brightness distribution of the light source area varies greatly at different viewing angles (such as the left image and the right image). Specifically, the projection position and brightness distribution of the light source at different viewing angles (such as the left image and the right image) vary significantly. Therefore, the brightness of the distant light source in backlit scenes varies greatly at different viewing angles.

[0098] The method for calculating the difference between the first brightness feature and the second brightness feature by the autonomous vehicle depends on the statistical method of the first brightness feature and the second brightness feature.

[0099] As an example, when the luminosity histogram of the first projection result is used as the first brightness feature and the luminosity histogram of the second projection result is used as the second brightness feature, the cross-entropy between the first brightness feature and the second brightness feature can be used to measure the difference between the first brightness feature and the second brightness feature. The larger the cross-entropy, the greater the difference between the first brightness feature and the second brightness feature; the smaller the cross-entropy, the smaller the difference between the first brightness feature and the second brightness feature. The specific calculation process is as follows:

[0100] First brightness feature: Where n represents the number of bins of the luminosity histogram of the first projection result. Represents the total number of pixels in the i-th bin in the luminosity histogram of the first projection result.

[0101] Second brightness feature: Where n represents the number of bins of the luminosity histogram of the second projection result, Represents the total number of pixels in the i-th bin of the luminosity histogram of the second projection result.

[0102] Normalize the two histograms into probability distribution respectively, and the calculation formula is as follows:

[0103]

[0104] Therefore, the probability distribution of the first brightness feature is P = [p1, p2, ..., p n ], the probability distribution of the second brightness feature is Q = [q1,q2,…,q n ].

[0105] in,

[0106] The cross entropy between the first brightness feature and the second brightness feature is:

[0107]

[0108] As another example, when the luminance histogram of the first projection result is used as the first brightness feature and the luminance histogram of the second projection result is used as the second brightness feature, the Bhattacharyya distance between the first brightness feature and the second brightness feature can be used to measure the difference between the first brightness feature and the second brightness feature. A larger Bhattacharyya distance indicates a greater difference between the first brightness feature and the second brightness feature; a smaller Bhattacharyya distance indicates a smaller difference between the first brightness feature and the second brightness feature.

[0109] Among them, the Bhattacharyya distance between the first brightness feature and the second brightness feature is:

[0110]

[0111] As another example, when the mean pixel brightness in the first projection result is used as the first brightness feature, and the mean pixel brightness in the second projection result is used as the second brightness feature, the difference between the two means can be used to measure the difference between the first and second brightness features. A larger difference indicates a greater difference between the first and second brightness features; a smaller difference indicates a smaller difference between the first and second brightness features.

[0112] S104: When the candidate obstacle is determined to be an abnormal obstacle, the autonomous vehicle updates the travel path.

[0113] In some possible implementations, when the autonomous vehicle determines that a candidate obstacle is an abnormal obstacle, it will set the grid state of the position of the candidate obstacle in the occupancy grid map generated based on the depth map of the target scene to unknown (Unexplored), and reconstruct the occupancy grid map through KinectFusion / TSDF (signed distance function). If the depth information of the position of the candidate obstacle in the reconstructed occupancy grid map is less than the preset depth threshold, indicating that the candidate obstacle has been removed or does not exist, the autonomous vehicle will use the position of the candidate obstacle in the reconstructed occupancy grid map as the passage area. Subsequently, the autonomous vehicle will use a path planning algorithm (such as the A algorithm, Dijkstra algorithm, probabilistic road map (PRM), etc.) to plan a collision-free path (also known as a safe travel path) from the current position to the target area. Finally, the autonomous vehicle will adjust the speed and steering in real time based on the collision-free path in combination with the motion control strategy to ensure accurate tracking of the path, thereby achieving obstacle avoidance and operation.

[0114] Taking a lawn mower as an example, when it determines that a candidate obstacle is an abnormal obstacle, it will reconstruct the abnormal obstacle area. If the depth information of the reconstructed abnormal obstacle area is greater than the preset depth information, it indicates that the candidate obstacle is outside the obstacle judgment range this time. The abnormal obstacle area will not be considered, and the status of the abnormal obstacle area will remain unknown, awaiting subsequent detection; if the candidate obstacle area is less than the preset depth threshold after the reconstructed raster map, it is considered that there is no obstacle in the abnormal obstacle area and it is a passable area.

[0115] In some possible implementations, since distant light sources in backlit scenes often appear on the side of the path of travel, before reconstructing the occupancy grid map, the confidence of the lateral grids in the occupancy grid map is increased to enhance detection in future frames. In some possible implementations, since distant light sources in backlit scenes often appear on the side of the path of travel, before reconstructing the occupancy grid map, the confidence of the lateral grids in the occupancy grid map is increased to enhance detection in future frames. By combining the historical frame Occupancy Grid, using time window filtering to smooth the map update, and using the Bayesian occupancy filter (BOF) to update the grid state frame by frame, the real obstacles are avoided from being accidentally deleted. When the Occupancy Grid is updated, the obstacle avoidance points in the original path of travel are eliminated, and it is found that the original path can be optimized to reduce unnecessary detours.

[0116] Specifically, the grid range of the lateral area is calculated based on the current planned path or driving direction. The centerline of the path can be defined, and a set of grids within a certain width range on both sides of it can be selected as the "lateral area". For example, the angle or lateral offset relative to the robot and the path direction is calculated for each grid, and those exceeding the threshold are considered lateral. Then, the confidence of these grids is adjusted in a preferential manner: reducing the impact of false alarms when fusing sensor data and increasing the weight of free space judgment. The specific method can be to introduce stronger priors for the lateral grids in the Bayesian update process (for example, defaulting to being more free), or using a more conservative occupancy probability increment for its sensor observations. In this way, when a false obstacle is generated by illumination, it will not be easily marked as occupied, and the confidence of the real obstacle is strengthened through multi-frame observations.

[0117] To avoid frequent changes in grid state due to noise or short-term obstacles in single-frame observations, temporal smoothing can be performed on the historical frame map. A common practice is to maintain the occupancy history of each grid in the most recent frames (e.g., sliding window or time decay filtering). One implementation is to apply exponential decay to each grid: if the grid is not observed as an obstacle again for a long time, the occupancy probability decreases by a certain decay factor; and each time a new obstacle is observed, a certain amount of confidence is added. After fusing the temporal information, a Bayesian occupancy filter (BOF) is used to update the grid state between frames to further ensure that real obstacles are not mistakenly deleted.

[0118] In some possible implementations, when the autonomous walking device determines that a candidate obstacle is an abnormal obstacle, it can remove the depth information of the candidate obstacle from the depth map or modify the depth information of the candidate obstacle in the depth map to infinity. It can also transmit information from the perception side to the navigation side to inform the location originally marked with the candidate obstacle that there is actually no obstacle and it can be used as a pass area.

[0119] In some possible implementations, to further improve recognition accuracy, image information from different time periods can be combined to determine whether a candidate obstacle is an abnormal obstacle. In actual applications, the autonomous vehicle reduces its speed and generates multiple images (including multiple left and right images) while driving. It then determines whether the same candidate obstacle is an abnormal obstacle in the images generated at different time periods. Finally, it combines these multiple judgment results to determine whether the candidate obstacle is an abnormal obstacle.

[0120] The following describes an example scenario in which multiple images include the left image and right image in step S101, as well as a new left image and a new right image. The new left image can be generated earlier or later than the left image in step S101. The new right image is generated simultaneously with the new left image.

[0121] Assume that after executing steps S101 to S103 above, the difference between the first brightness feature and the second brightness feature is greater than or equal to the first threshold, and the candidate obstacle in the depth map is marked as an abnormal obstacle.

[0122] For the same candidate obstacle in the new depth map generated based on the new left image and the new right image, the autonomous vehicle calculates the third and fourth brightness features of the candidate obstacle, and if the difference between the third and fourth brightness features is greater than or equal to the first threshold, the autonomous vehicle marks the obstacle in the new depth map as an abnormal obstacle. Since the candidate obstacle is marked as an abnormal obstacle in both the original depth map (referring to the depth map in step S102) and the new depth map, the autonomous vehicle determines that the candidate obstacle is an abnormal obstacle.

[0123] If the candidate obstacle in both the original depth map and the new depth map is not marked as an abnormal obstacle, the autonomous driving device may determine that the candidate obstacle is not an abnormal obstacle.

[0124] If the candidate obstacle is not marked as an abnormal obstacle in one of the depths of the original depth map and the new depth map, the autonomous vehicle may generate a third depth map and determine whether the candidate obstacle in the third depth map is marked as an abnormal obstacle. If the candidate obstacle in the third depth map is marked as an abnormal obstacle, the autonomous vehicle may determine that the candidate obstacle is an abnormal obstacle. If the candidate obstacle in the third depth map is not marked as an abnormal obstacle, the autonomous vehicle may determine that the candidate obstacle is not an abnormal obstacle.

[0125] It should be understood that the above Figure 1Steps S101 to S104 in the obstacle recognition method can also be executed by a computing device that communicates with the automatic walking device. Specifically, after the automatic walking device obtains the left image and the right image of the target scene, the left image and the right image are sent to the computing device. The computing device executes steps S102 to S104 based on the left image and the right image, and after obtaining a new travel path, the new travel path is sent to the automatic walking device, so that the automatic walking device travels according to the new travel path to achieve accurate obstacle avoidance.

[0126] It should be understood that the above Figure 1 The obstacle recognition method can also be applied to other autonomous vehicles, such as vehicles with intelligent driving functions, that is, other autonomous vehicles can perform Figure 1 Steps S101 to S104 in the obstacle recognition method.

[0127] In summary, this technical solution can effectively distinguish whether the candidate obstacle is a real object (such as a tree or stone) or a distant light source (such as the sun or light) in a backlit scene by analyzing the brightness characteristics of the candidate obstacle in the depth map of the target scene at different viewing angles. Once the candidate obstacle is identified as a distant light source, the depth information of the candidate obstacle can be removed from the depth map or the depth information of the candidate obstacle in the depth map can be modified to infinity to prevent the automatic walking device from mistakenly identifying the distant light source as a nearby obstacle and stopping. This not only improves the accuracy of path planning and mowing efficiency, but also enhances the robustness of the environmental perception of the automatic walking device, thereby improving the continuity and stability of the mowing operation.

[0128] In addition, the brightness features of the candidate obstacles are calculated by first calculating the circumscribed rectangular box of the candidate obstacle in the depth map, then calculating the projection result of the circumscribed rectangular box on the corresponding image, and then statistically analyzing the pixel brightness distribution in the projection result. This method is easy to calculate and transform, has high computational efficiency, and is convenient for subsequent processing (such as tracking, projection, and feature extraction). It is suitable for working environments with high real-time requirements and meets the operating requirements of autonomous walking equipment.

[0129] See also Figure 5 , Figure 5 The obstacle recognition device 200 can be used to implement the above-mentioned Figure 1 Obstacle recognition method. Figure 5 As shown, the obstacle recognition device 200 includes: an acquisition unit 201 , a calculation unit 202 and a determination unit 203 .

[0130] The acquisition unit 201 is used to acquire a depth map of a target scene.

[0131] The calculation unit 202 is configured to calculate a first brightness feature and a second brightness feature of a candidate obstacle in the depth map.

[0132] The determining unit 203 is configured to determine that the candidate obstacle is an abnormal obstacle if the difference between the first brightness feature and the second brightness feature is greater than or equal to a first threshold.

[0133] The depth map is used to record the depth information from the object in the target scene to the autonomous vehicle. The first brightness feature and the second brightness feature are used to indicate the brightness distribution of the candidate obstacle at different viewing angles.

[0134] In some possible implementations, the acquisition unit 201 is specifically configured to: acquire a left image and a right image of a target scene; perform semantic segmentation on the left and right images, respectively, and identify and match corresponding object regions in the left and right images; for each object region, calculate depth information within each object region based on pixel-level disparity within the corresponding object region in the left and right images; and obtain a depth map of the target scene based on the depth information within each object region. The left image is generated by the left camera of the autonomous vehicle, and the right image is generated by the right camera of the autonomous vehicle.

[0135] In some possible implementations, the calculation unit 202 is further configured to, before calculating the first brightness feature and the second brightness feature of the candidate obstacle, consider the object area as a candidate obstacle if the depth information within the object area is less than or equal to a preset depth threshold, and / or the pixel area of the object area is greater than or equal to a preset pixel area threshold.

[0136] In some possible implementations, the calculation unit 202 is specifically used to: calculate the bounding rectangular box of the candidate obstacle in the depth map; calculate the first projection result of the bounding rectangular box in the left image, and perform statistics on the pixel brightness distribution in the first projection result to obtain a first brightness feature; calculate the second projection result of the bounding rectangular box in the right image, and perform statistics on the pixel brightness distribution in the second projection result to obtain a second brightness feature.

[0137] In some possible implementations, the determining unit 203 is further configured to: if the candidate obstacle is an abnormal obstacle, set the grid state of the position of the candidate obstacle in the occupancy grid map to unknown, and reconstruct the occupancy grid map. If the depth information of the position of the candidate obstacle in the reconstructed occupancy grid map is greater than a preset depth threshold, then the position of the candidate obstacle in the reconstructed occupancy grid map is used as a passable area.

[0138] In some possible implementations, the determining unit 203 is further configured to improve the confidence of the lateral grids in the occupancy grid map before reconstructing the occupancy grid map.

[0139] In some possible implementations, the first brightness feature includes one or more of a luminance histogram, a grayscale histogram, a brightness channel histogram, a cumulative distribution function, a brightness gradient histogram, statistics, and the like.

[0140] The acquisition unit 201, calculation unit 202, and determination unit 203 can all be implemented in software or hardware. For example, the implementation of the acquisition unit 201 will be described below using the acquisition unit 201 as an example. Similarly, the implementation of the calculation unit 202 and determination unit 203 can refer to the implementation of the acquisition unit 201.

[0141] As an example of a software functional unit, the acquisition unit 201 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the acquisition unit 201 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0142] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0143] As an example of a hardware functional unit, the acquisition unit 201 may include at least one computing device, such as a server. Alternatively, the acquisition unit 201 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0144] The multiple computing devices included in acquisition unit 201 can be distributed in the same region or in different regions. The multiple computing devices included in acquisition unit 201 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in acquisition unit 201 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0145] It should be noted that, in other embodiments, the acquisition unit 201 can be used to execute any step in the obstacle identification method, the calculation unit 202 can be used to execute any step in the identification method, and the determination unit 203 can be used to execute any step in the obstacle identification method. The steps that the acquisition unit 201, the calculation unit 202, and the determination unit 203 are responsible for implementing can be specified as needed. The full functions of the obstacle identification device 200 are realized by the acquisition unit 201, the calculation unit 202, and the determination unit 203 respectively implementing different steps in the obstacle identification method.

[0146] See also Figure 6 , Figure 6 1 is a block diagram of a lawn mower in some embodiments of the present application. In some embodiments, the lawn mower 100 includes a mowing assembly 110, a travel assembly 130, and a control device 120. The mowing assembly 110 is used for mowing grass, and the travel assembly 130 is used to control the travel of the lawn mower 100. The mowing assembly 110 and the travel assembly 130 are connected to the control device 120, and the control device 120 is used to execute the detection method as described in any of the aforementioned embodiments.

[0147] Among them, after the control device 120 executes the detection method, it can accurately identify obstacles such as ground pipes, and promptly control the lawn mower 100 to adjust the travel route through the walking component 130 to avoid the lawn mower 100 from colliding with obstacles. It can not only protect the mechanical structure and components of the lawn mower 100, but also prevent potential consequences caused by damage to pipes caused by collisions.

[0148] In addition, after the control device 120 executes the detection method, the lawn mower 100 can autonomously avoid obstacles without frequent human intervention, reducing the time of downtime to deal with obstacles, ensuring the continuity of mowing operations, improving the efficiency of the lawn mower 100, and enhancing the user experience.

[0149] Please also refer to Figure 7 , Figure 7 Schematic diagram of a lawn mower in some embodiments of the present application. In some embodiments, the lawn mower 100 further includes a camera assembly 140, a sensor module 150, a human-computer interaction module 160, and a collision module 170, all of which are provided on the lawn mower. The camera assembly 140 utilizes a high-definition wide-angle lens that can clearly capture images of the lawn mower 100's surroundings and provide image data for the lawn mower 100. In some embodiments, the camera assembly 140 can also transmit the images in real time to a user terminal via Bluetooth or a mobile phone app.

[0150] Among them, the sensor module 150 integrates multiple sensors that can sense environmental temperature, humidity and other information. The infrared sensor and other sensors can also monitor whether the camera component 140 is blocked and whether there are foreign objects that affect the normal operation of the lawn mower. When an abnormal situation is detected, it will promptly feedback to the control device 120 so that corresponding measures can be taken; the human-computer interaction module 160 is provided with function buttons such as start / stop and mode switching, so that the user can manually intervene in the operation of the lawn mower 100 when necessary.

[0151] Thus, the multiple components on the lawn mower 100 work together, enabling the lawn mower 100 to intelligently plan a path and automatically adjust the working mode based on environmental information, greatly improving mowing efficiency and accuracy, and effectively avoiding collision damage. At the same time, the sensor module 150 monitors the status of the camera component 140 and surrounding foreign objects, ensuring the stability and reliability of the equipment. The user can monitor in real time through the terminal and intervene as needed using the button component, enhancing the user's operating experience and the safety of mowing operations.

[0152] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of an obstacle recognition device provided by an embodiment of the present application. Figure 8As shown, the obstacle recognition device 300 provided in this application includes: a bus 301, a processor 302, a memory 303, and a communication interface 304. The processor 302, the memory 303, and the communication interface 304 communicate with each other via the bus 301. It should be understood that this application does not limit the number of processors and memories in the obstacle recognition device 300.

[0153] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The figure shows only one bus line, but it does not mean that there is only one bus or one type of bus. The bus 301 may include a path for transmitting information between various components of the obstacle recognition device 300 (eg, the memory 303, the processor 302, and the communication interface 304).

[0154] The processor 302 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0155] The memory 303 may include a volatile memory, such as a random access memory (RAM). The memory 303 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0156] The memory 303 stores executable program codes, and the processor 302 executes the executable program codes to respectively implement the functions of the aforementioned acquisition unit 201, calculation unit 202, and determination unit 203, thereby implementing the aforementioned Figure 1 That is, the memory 303 stores instructions for executing the obstacle recognition method.

[0157] The communication interface 304 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the obstacle identification device 300 and other devices or a communication network.

[0158] The present application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device executes the aforementioned Figure 1 obstacle recognition method.

[0159] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the aforementioned Figure 1 obstacle recognition method.

[0160] It should be understood that in the embodiments of the present invention, "when," "when," and "if" all mean that the device will perform corresponding processing under certain objective circumstances, and do not limit the time. Nor do they require that the device must perform a judgment action when implementing it, nor do they mean that there are other limitations.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. An obstacle recognition method for an autonomous vehicle, characterized in that: The method comprises: Acquire a depth map of a target scene, wherein the depth map is used to record depth information from objects in the target scene to the autonomous vehicle; Calculating a first brightness feature and a second brightness feature of the candidate obstacle in the depth map, wherein the first brightness feature and the second brightness feature are respectively used to indicate the brightness distribution of the candidate obstacle at different viewing angles; When the difference between the first brightness feature and the second brightness feature is greater than or equal to a first threshold, the candidate obstacle is determined to be an abnormal obstacle.

2. The obstacle recognition method according to claim 1, characterized in that: The obtaining of the depth map of the target scene includes: Acquire a left image and a right image of the target scene, wherein the left image is generated by a left camera on the autonomous vehicle, and the right image is generated by a right camera on the autonomous vehicle; Performing semantic segmentation on the left image and the right image, respectively, to identify and match corresponding object regions in the left image and the right image; For each object region, calculating depth information within each object region based on pixel-level disparity within the corresponding object region in the left image and the right image; Based on the depth information within each object area, the depth map of the target scene is obtained.

3. The obstacle recognition method according to claim 2, characterized in that: Before calculating the first brightness feature and the second brightness feature of the candidate obstacle, the method further includes: If the depth information in the object region is less than or equal to a preset depth threshold, and / or the pixel area of the object region is greater than or equal to a preset pixel area threshold, the object region is taken as the candidate obstacle.

4. The obstacle recognition method according to claim 3, characterized in that: The calculating the first brightness feature and the second brightness feature of the candidate obstacle includes: Calculate the bounding rectangle of the candidate obstacle in the depth map; Calculating a first projection result of the circumscribed rectangular frame in the left image, and performing statistics on pixel brightness distribution in the first projection result to obtain the first brightness feature; A second projection result of the circumscribed rectangular frame in the right image is calculated, and pixel brightness distribution in the second projection result is statistically analyzed to obtain the second brightness feature.

5. The obstacle recognition method according to claim 4, characterized in that: The method further comprises: In a case where the candidate obstacle is the abnormal obstacle, setting the grid state of the position of the candidate obstacle in the occupied grid map to unknown, and reconstructing the occupied grid map; If the depth information of the position of the candidate obstacle in the reconstructed occupancy grid map is greater than the preset depth threshold, the position of the candidate obstacle in the reconstructed occupancy grid map is used as a passable area.

6. The obstacle recognition method according to claim 5, characterized in that: Before reconstructing the occupancy grid map, the method further includes: Improved confidence in lateral grids in occupancy grid maps.

7. The obstacle recognition method according to any one of claims 1 to 6, characterized in that: The first brightness feature includes one or more of a luminance histogram, a grayscale histogram, a brightness channel histogram, a cumulative distribution function, a brightness gradient histogram, statistics, and the like.

8. An obstacle recognition device, characterized in that: The obstacle recognition device includes a memory and a processor, wherein the memory stores computer program instructions, and the processor executes the computer program instructions to enable the obstacle recognition device to perform the method according to any one of claims 1 to 7.

9. A lawn mower, characterized in that: The lawn mower includes a mowing component, a traveling component and a control device. The mowing component is used for mowing grass, and the traveling component is used for controlling the walking of the lawn mower. The mowing component and the traveling component are connected to the control device, and the control device is used to execute the method according to any one of claims 1 to 7.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method according to any one of claims 1 to 7.

Images