Green belt automatic watering system, device and storage medium based on binocular vision

Abstract

The present invention discloses a binocular vision-based automatic watering system, device and storage medium for a green belt, which is used for an automatic driving watering truck to realize automatic watering and control to jump over pedestrians, vehicles, obstacles, etc. The automatic watering system for the green belt comprises: obtaining a disparity map of a binocular camera in real time, performing segmentation network processing on the disparity map to obtain a segmentation result including green plants and road obstacles; projecting the segmentation result to the Wx-Wz plane of the world coordinate system to calculate and obtain the height of the green plants, and finding the closest distance between the green belts on both sides of the road and the watering truck, and drawing a boundary map; confirming the position where watering is required according to whether there are road obstacles in the horizontal direction of the green belt and the boundary map.

Description

Technical Field

[0001] The present invention relates to the technical field of assisted driving, and in particular to a binocular vision-based green belt automatic watering system, device and storage medium. Background Art

[0002] The green belts of urban roads are mainly composed of hedges, lawns and trees, and they play an irreplaceable and important role in urban road construction: green belts have the safety function of separating traffic, which can separate motor vehicles from non-motor vehicles, and people from non-motor vehicles on the plane, thus avoiding interference with each other to a large extent; green belts have the function of beautifying the city, eliminating the visual fatigue of drivers and reducing the occurrence of traffic accidents; planting trees in green belts can also change the spatial scale of the road, making the road space have a good aspect ratio. In addition, green belts also have the function of purifying the environment, can trap dust, reduce noise, absorb harmful gases, release oxygen, etc.

[0003] Green belts need to be manually watered regularly, but traditional manual irrigation is labor-intensive, spraying is uneven, and the watering effect is affected by extreme weather and operator proficiency.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] The main purpose of the present invention is to disclose a green belt automatic watering system, device and storage medium based on binocular vision, which is used to solve the problems existing in the prior art that traditional manual irrigation has high work intensity, uneven spraying, and the watering effect is affected by extreme weather and operator proficiency.

[0006] To achieve the above object, according to one aspect of the present invention, a green belt automatic watering system based on binocular vision is provided, and the following technical scheme is adopted:

[0007] A green belt automatic watering system based on binocular vision includes: obtaining a disparity map of a binocular camera in real time, performing segmentation network processing on the disparity map to obtain a segmentation result including green plants and road obstacles; projecting the segmentation result to a world coordinate system. The height of green plants is calculated on the plane, and the shortest distance between the green belts on both sides of the road and the sprinkler truck is found, and a boundary map is drawn; the location that needs to be sprinkled is confirmed based on whether there are road obstacles in the horizontal direction of the green belts and the boundary map.

[0008] Furthermore, the real-time acquisition of the disparity map of the binocular camera, the segmentation network processing of the disparity map, and the obtained segmentation results including green plants and road obstacles include: soft segmentation of the target area: dividing the feature map obtained after the disparity map processing into a group of soft object areas {M1, M2, ..., Mk}, where each Mk corresponds to a category k, that is, performing semantic segmentation from the deep network to obtain a coarse segmentation result; target area representation: by aggregating the representations of all pixels in the kth object area, each object area is represented as ;

[0009]

[0010] in, It's pixels The expression, It's pixels The probability of belonging to the kth object area, each layer represents the probability that the pixel belongs to this class ;

[0011] Object context representation: The relationship between each pixel and each object region is:

[0012]

[0013] in, is the relational function, , There are two conversion functions, expressed as 1×1conv→BN→Relu;

[0014] The object context is represented as:

[0015]

[0016] in, and is the transformation function;

[0017] Pixel Aggregated by the object context module and the initial feature map, the final object context representation is as follows:

[0018] ,in, is the conversion function.

[0019] Furthermore, the segmentation result is projected onto the Wx-Wz plane of the world coordinate system to calculate and obtain the height of the green plants, and the shortest distance between the green belts on both sides of the road and the sprinkler truck is found, and the boundary map is drawn, including: extracting the green plant parts in the original image and the disparity map separately to obtain a green plant map; according to the disparity value corresponding to each green plant pixel in the green plant map, the Wz value in the world coordinate system corresponding to each green plant is calculated; according to the Wz value and the coordinates of each green plant pixel in the world coordinate system, the Wx and Wy values ​​are calculated; all points in a preset range within the vehicle's drivable area are selected, and all points are fitted to obtain the road surface.

[0020] Further, the calculation of the Wz value in the world coordinate system corresponding to each green plant according to the disparity value corresponding to each green plant pixel in the green plant image includes:

[0021] The calculation formula is as follows:

[0022]

[0023] Among them, B. are the baseline and focal length of the binocular camera respectively, and disp is the disparity value corresponding to each green plant pixel obtained by stereo matching.

[0024] Further, the calculation of Wx and Wy values ​​according to the Wz value and the coordinates of each green plant pixel in the world coordinate system includes:

[0025]

[0026] in, and are the x and y coordinates of the image coordinate system respectively; and are the x and y coordinates of the optical center in the image coordinate system, and are the x and y coordinates of the world coordinate system respectively.

[0027] Furthermore, calculating the height of the green plant to the road surface includes:

[0028] Take the world coordinate system The range is [-20,20] meters, The range is [0, 60] meters, and the green plants are projected onto In the plane;

[0029] by is the x-axis, is the y-axis, H is the value corresponding to the point (x, y), and H is the height from the green plant to the road surface.

[0030] Further, judging the position where watering is required according to whether there are obstacles in the horizontal direction of the green belt of the fitted road surface and the height of the green plants to the road surface includes:

[0031] Select the largest H in the grid as the grid value;

[0032] Compare the maximum H value with the installation height of the binocular camera;

[0033] When H is less than the installation height and there are no obstacles on the fitted road surface in the horizontal direction of the green belt, the corresponding green plants are determined to be in a position where water can be sprinkled.

[0034] Furthermore, after the green plant parts in the original image and the disparity image are extracted separately to obtain the green plant image, the green belt automatic watering system further includes:

[0035] To project the green plant map, first perform an interpolation filling on the above-obtained projection map. For places where the pixel value is 0, use the mean value of the non-zero points in the 3×3 neighborhood to interpolate and fill. After filling, remove isolated points according to the regional connectivity to obtain the final green plant projection map.

[0036] According to another aspect of the present invention, a green belt automatic watering device based on binocular vision is provided, and the following technical solutions are adopted:

[0037] An automatic watering device for green belts based on binocular vision includes: a segmentation module for acquiring a disparity map of a binocular camera in real time, performing segmentation network processing on the disparity map to obtain a segmentation result including green plants and road obstacles; a calculation module for projecting the segmentation result to a world coordinate system. The plane calculates the height of the green plants, finds the shortest distance between the green belts on both sides of the road and the sprinkler truck, and draws a boundary map; the confirmation module is used to confirm the location where watering is required based on whether there are road obstacles in the horizontal direction of the green belt and the boundary map.

[0038] According to another aspect of the present invention, a storage medium is provided, and the following technical solution is adopted:

[0039] A storage medium includes the above-mentioned green belt automatic watering device based on binocular vision.

[0040] The present invention firstly uses a deep learning network to segment the green belt; secondly, it uses the disparity information to project the green belt from the pixel coordinate system to the In the plane, the real height of the green plant in the world coordinate system is used as the pixel value of the point; then automatic watering is achieved based on the location information and height information of the green plant. The deep network used in the present invention can also segment other common road obstacles, such as cars and pedestrians. If there are road obstacles, the sprinkler truck is controlled to skip this section and not water. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0042] Figure 1 This is a flow chart of a green belt automatic watering system based on binocular vision according to an embodiment of the present invention;

[0043] Figure 2 A diagram showing the structure of a deep learning network according to an embodiment of the present invention;

[0044] Figure 3 The segmentation result of the deep network model described in the embodiment of the present invention and the disparity map corresponding to the image;

[0045] Figure 4 A comparison image of green plants in the original image and the disparity image according to an embodiment of the present invention;

[0046] Figure 5 The selected formable area road surface fitting effect diagram described in the embodiment of the present invention;

[0047] Figure 6 This is the scene 1 described in the embodiment of the present invention where green plants are projected onto Plane renderings;

[0048] Figure 7 The second scene of the embodiment of the present invention projects green plants onto Plane renderings;

[0049] Figure 8 This is the scenario 3 described in the embodiment of the present invention, where green plants are projected onto Plane renderings;

[0050] Fig. 9 This is the effect diagram after the green plant projection diagram is interpolated and filled in the scene 4 described in the embodiment of the present invention;

[0051] Fig.10 This is the effect diagram after the green plant projection diagram is interpolated and filled in the scene 5 described in the embodiment of the present invention;

[0052] Fig.11 This is a rendering of the green belt boundary map according to an embodiment of the present invention;

[0053] Fig.12 This is a rendering of a new boundary map of a green belt in scenario six according to an embodiment of the present invention;

[0054] Fig.13 This is a rendering of a new boundary map of a green belt in scenario seven according to an embodiment of the present invention;

[0055] Fig.14 This is a rendering of a new boundary map of a green belt in scenario eight according to an embodiment of the present invention;

[0056] Fig.15 This is a rendering of a new boundary map of a green belt in scenario nine according to an embodiment of the present invention;

[0057] Fig.16 This is a schematic structural diagram of a green belt automatic watering device based on binocular vision according to an embodiment of the present invention; DETAILED DESCRIPTION

[0058] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0059] Explanation of terms

[0060] 1. Disparity: The positional deviation of pixels in the same scene imaged by the left and right cameras. For example, point X in the scene is the x coordinate in the left camera and the (x+d) coordinate in the right camera. Then d is the value of the x coordinate point in the disparity map. The disparity of each point on the disparity map is denoted as disp.

[0061] 2. Binocular stereo vision: It simulates the principle of human vision. Generally, two left and right cameras shoot the same object from different angles at the same time, or one camera shoots the same object from different angles, and the three-dimensional geometric information of the object can be restored based on the principle of parallax.

[0062] 3. Deep learning: Deep learning is a new field based on machine learning. It originates from the neural network algorithm inspired by the structure of the human brain. In addition, it develops with the increasing depth of the model structure and a series of new algorithms are generated with the improvement of big data and computing power.

[0063] 4. Pixel coordinate system: A uv rectangular coordinate system is established with the upper left corner of the image as the origin and the pixel is in units of pixels. The horizontal coordinate u and vertical coordinate v of the pixel are the column and row numbers in the image array respectively.

[0064] 5. Road obstacles: Different from the general classification of road obstacles, the present invention regards all objects on the road that may affect the normal operation of the sprinkler truck as road obstacles, including fences, pedestrians, cyclists, cars, trucks, buses, motorcycles and bicycles.

[0065] Figure 1 This is a flow chart of a green belt automatic watering system based on binocular vision described in an embodiment of the present invention.

[0066] See also Figure 1 As shown, a green belt automatic watering system based on binocular vision includes:

[0067] S101: Obtain a disparity map of a binocular camera in real time, perform segmentation network processing on the disparity map, and obtain a segmentation result including green plants and road obstacles;

[0068] S103: Project the segmentation result to the world coordinate system Calculate the height of green plants on the plane, find the shortest distance between the green belts on both sides of the road and the sprinkler truck, and draw a boundary map;

[0069] S105: Determine the location where watering is required based on whether there are road obstacles in the horizontal direction of the green belt and the boundary map.

[0070] In the technical solution of this embodiment, step S101 obtains the disparity map of the binocular camera in real time, performs segmentation network processing on the disparity map, and obtains the segmentation result including green plants and road obstacles. The disparity map obtained in real time is input into the deep learning network, and the network structure includes two parts: Backbone and OCR module. The Backbone adopts ResNet or HRNet. The OCR module mainly includes the following parts: soft object regions, object region representation, and object-contextual representations. Figure 2 As shown. After the above network structure processing, the segmentation results including green plants and road obstacles are obtained. In step S103, the segmentation results are projected to the Wx-Wz plane of the world coordinate system to calculate the height of the green plants, find the closest distance between the green belts on both sides of the road and the sprinkler truck, and draw a boundary map. Specifically, let the width of the projected image be d. According to the conversion formula from step 2 to the world coordinate system, it can be known that the optical center of the camera is After projection, the image At, set the initial axis , find the nearest green plant to the camera to the left and right, which is the boundary of the green belt. For special scenes where the road is curved, set a threshold (here we choose a distance of 1 meter, which is represented as 10 pixels wide in the image). If the distance is less than this threshold, the axis will be moved to the left by 1 meter. , v is the v coordinate of the boundary point green plant in the uv coordinate system, then Update the initial axis, and so on. In step S105, determine the location where watering is required based on whether there are road obstacles in the horizontal direction of the green belt and the boundary map. Specifically, determine whether there are road obstacles in the horizontal direction of the green belt based on the boundary map, and drive watering if there are no obstacles.

[0071] This embodiment provides an automatic green belt watering system based on a deep learning network and binocular vision, which combines automatic driving with watering by a sprinkler truck to replace manual watering of the green belt, thereby improving watering efficiency and reducing water consumption.

[0072] As a preferred implementation, step S101 includes: soft segmentation of the target area: dividing the feature map obtained after processing the disparity map into a group of soft object areas {M1, M2, ..., Mk}, where each Mk corresponds to a category k, that is, performing semantic segmentation from the deep network to obtain a coarse segmentation result; target area representation: by aggregating the representations of all pixels in the kth object area, each object area is represented as ;

[0073]

[0074] in, It's pixels The expression, It's pixels The probability of belonging to the kth object area, each layer represents the probability that the pixel belongs to this class ;

[0075] Object context representation: The relationship between each pixel and each object region is:

[0076]

[0077] in, is the relational function, , There are two conversion functions, expressed as 1×1conv→BN→Relu;

[0078] The object context is represented as:

[0079]

[0080] in, and is the transformation function;

[0081] Pixel Aggregated by the object context module and the initial feature map, the final object context representation is as follows:

[0082] ,in, is the conversion function.

[0083] The segmentation results of the deep network model and the disparity map corresponding to the image can be found in Figure 3 ,There are 15 categories in total, and this invention only focuses on the green values ​​and road obstacles on both sides of the road.

[0084] In this embodiment, the OCR process is re-expressed using Transformer. Figure 2 The Transformer encoder-decoder architecture is described in . The decoder cross-attention module is used to extract soft object regions and calculate object region representations, while the encoder cross-attention module and the subsequent FFN are used to aggregate object context representations.

[0085] As a preferred implementation, the segmentation result is projected onto a world coordinate system. The plane is used to calculate the height of the green plants, and the shortest distance between the green belts on both sides of the road and the sprinkler truck is found. The boundary map is drawn by extracting the green plant parts in the original image and the disparity map separately to obtain the green plant map; according to the disparity value corresponding to each green plant pixel in the green plant map, the world coordinate system corresponding to each green plant is calculated. value; according to Value, and the coordinate calculation of each green plant pixel in the world coordinate system as well as value; select all points within the preset range in the vehicle's drivable area and fit all points to obtain the road surface.

[0086] Specifically, the green plant parts in the original image and the disparity image are extracted separately, such as Figure 4 As shown, according to the disparity value corresponding to each green plant pixel in the green plant image, the corresponding depth value is calculated (Z axis of the world coordinate system - reflects the actual distance from the object to the camera in the world coordinate system).

[0087] The calculation formula is as follows:

[0088]

[0089] Among them, B. are the baseline and focal length of the binocular camera respectively, and disp is the disparity value corresponding to each green plant pixel obtained by stereo matching.

[0090] Calculate the world coordinate system and : Given the coordinates of each green plant pixel in the image coordinate system, the green plant is converted to the world coordinate system using the following formula based on the depth value obtained.

[0091]

[0092] in, and are the x and y coordinates of the image coordinate system respectively; and are the x and y coordinates of the optical center in the image coordinate system, and are the x and y coordinates of the world coordinate system respectively.

[0093] Calculate the real height of green plants: Since the road surface may have more or less potholes or speed bumps, the camera will have a certain pitch angle when the sprinkler truck is driving, and it is no longer parallel to the ground. Therefore, the above method calculates the actual height of green plants. It cannot represent the height of objects in the real world. To this end, we select all points in the range of [-1.5, 1.5] × [0,30] meters in the vehicle's drivable area and fit the road surface from these points. The fitting results are as follows: Figure 5 Then, the real height H of the green plant to the road surface is calculated based on the coordinates of the green plant in the world coordinate system.

[0094] Take the world coordinate system The range is [-20,20] meters, The range is [0, 60] meters, and the green plants are projected onto In a plane; is the x-axis, is the y-axis, H is the value corresponding to the point (x, y), and H is the height from the green plant to the road surface.

[0095] Since the selected area is too small, , Expand 10 times, that is, divide 1 meter into 10 grids, and expand the original 40× 60 image to 400× 600. Since there may be more than one pixel point corresponding to the original image in a grid, there is more than one calculated H, so the largest H in this grid is selected as the value of this grid. Considering that green belts are generally dominated by short bushes and lawns, the installation height of the camera is generally about two meters, so the sprinkler can only focus on the green plants below the installation height of the camera when watering. See Figure 6 , Figure 7 and Figure 8 , renderings in different scenarios.

[0096] As a preferred implementation, after the green plant parts in the original image and the disparity image are extracted separately to obtain the green plant image, the green belt automatic watering system also includes: projecting the green plant image, first performing an interpolation filling on the above-obtained projection image, and for places where the pixel value is 0, interpolation filling is performed using the mean of the non-zero points in the 3×3 neighborhood, and after filling, isolated points are removed according to the regional connectivity to obtain the final green plant projection image.

[0097] Specifically, since the points projected in the second step are relatively sparse, some green plants are not mapped. In reality, green belts are generally continuous, and the height of green plants is similar to that of the surrounding areas. Therefore, we need to first interpolate and fill the above-obtained projection map. For places where the pixel value is 0, interpolate and fill with the average value of the non-zero points in the 3×3 neighborhood. After filling, remove isolated points based on regional connectivity to obtain the final green plant projection map, see Fig. 9 and Fig.10 shown.

[0098] More specifically, the method for drawing the boundary map is as follows:

[0099] Assume that the width of the projected image is d. According to the above conversion formula of the world coordinate system, the optical center of the camera is After projection, the image At, set the initial axis , find the nearest green plant to the camera to the left and right, which is the boundary of the green belt. For special scenes where the road is curved, set a threshold (here we choose a distance of 1 meter, which is represented as 10 pixels wide in the image). If the distance is less than this threshold, the axis will be moved to the left by 1 meter. In the image, it is represented as: , v is the v coordinate of the boundary point green plant in the uv coordinate system, then Update the initial axis, and so on.

[0100] More specifically, whether there are road obstacles in the horizontal direction of the green belt can be used to determine whether a sprinkler truck needs to spray water at that location.

[0101] Similarly, the segmentation results of road obstacles are also projected onto plane, and then combined with the boundary map obtained above, the picture is The boundary is divided into left and right sides. If there is a road obstacle in the x direction on the left, the corresponding green plant is ignored. Finally, the results on the left and right sides are combined to obtain a new green plant boundary map. Figure 12-Figure 15 , which are experimental renderings of various scenarios.

[0102] In the above embodiment of the present invention, the road green plant segmentation result map is projected onto the world coordinate system. Plane, because the previous watering trucks are mainly operated manually, people are required to judge whether a certain green belt needs to be watered, and whether there are obstacles on the road that block the watering. The height of the green plant from the ground is used as the pixel value of the point to obtain the location information and height information of the green belt. The computer can be used to determine whether a certain green plant needs to be watered. Since the watering range of the sprinkler truck is limited, combined with the width of the road, the range of 20m wide and 60m away from the camera is selected as the visible area of ​​the sprinkler truck.

[0103] The drivable area will be selected to fit the road surface, and the height from the green plants to the road surface will be calculated

[0104] Since the sprinkler truck may be bumpy during driving, The calculated height of the green plant is different from the actual height. Select a drivable area of ​​[-1.5, 1.5]m×30m (unobstructed road surface) and project each point of the drivable area into the world coordinate system using the formula (2) in step 2. Suppose the road surface equation is , given x(Wx), y(Wy), z(Wz), solve for the coefficients A, B, C, that is, solve

[0105]

[0106] Then, determine whether the location needs watering based on whether there are road obstacles. However, in real life, the road is not always unobstructed. There will always be pedestrians, cyclists, vehicles, etc., which are collectively referred to as road obstacles. Consider the left and right sides of the image separately. If the road obstacle blocks the green plants at a certain horizontal distance, skip the green plants here and do not water them. In the implementation process, find the y coordinate of the road obstacle and skip the pixel value of this row when drawing the boundary map.

[0107] Fig.16 This is a schematic structural diagram of a green belt automatic watering device based on binocular vision according to an embodiment of the present invention;

[0108] See also Fig.16 As shown, a green belt automatic watering device based on binocular vision includes: a segmentation module 20, which is used to obtain a disparity map of a binocular camera in real time, perform segmentation network processing on the disparity map, and obtain a segmentation result including green plants and road obstacles; a calculation module 40, which is used to project the segmentation result to a world coordinate system The plane calculates the height of the green plants, finds the shortest distance between the green belts on both sides of the road and the sprinkler truck, and draws a boundary map; the confirmation module 60 is used to confirm the location where watering is required based on whether there are road obstacles in the horizontal direction of the green belt and the boundary map.

[0109] A storage medium provided according to the present invention includes the above-mentioned green belt automatic watering device based on binocular vision.

[0110] The present invention firstly uses a deep learning network to segment the green belt; secondly, it uses the disparity information to project the green belt from the pixel coordinate system to the In the plane, the real height of the green plant in the world coordinate system is used as the pixel value of the point; then automatic watering is achieved based on the location information and height information of the green plant. The deep network used in the present invention can also segment other common road obstacles, such as cars and pedestrians. If there are road obstacles, the sprinkler truck is controlled to skip this section and not water.

[0111] The above only describes some exemplary embodiments of the present embodiment by way of illustration. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A green belt automatic watering system based on binocular vision, characterized in that: include: Obtain the disparity map of the binocular camera in real time, perform segmentation network processing on the disparity map, and obtain segmentation results including green plants and road obstacles; Project the segmentation result onto the Wx-Wz plane of the world coordinate system to calculate and obtain the height of the green plants, find the shortest distance between the green belts on both sides of the road and the sprinkler truck, and draw a boundary map; Determine the location where watering is required based on whether there are road obstacles in the horizontal direction of the green belt and the boundary map; The real-time acquisition of the disparity map of the binocular camera and the segmentation network processing of the disparity map to obtain the segmentation results including green plants and road obstacles include: Soft segmentation of target area: The feature map obtained after disparity map processing is divided into a set of soft object regions {M1, M2, …, Mk}, where each Mk corresponds to a category k, that is, the coarse segmentation result is obtained by semantic segmentation from the deep network; Object region representation: By aggregating the representations of all pixels in the kth object region, each object region is represented as ; in, It's pixels The expression, It's pixels The probability of belonging to the kth object area, each layer represents the probability that the pixel belongs to the object area ; Object context representation: The relationship between each pixel and each object region is: in, is the relational function, , There are two conversion functions, expressed as 1×1conv→BN→Relu; The object context is represented as: in, and is the transformation function; Pixel Aggregated by the object context module and the initial feature map, the final object context representation is as follows: in, is the conversion function.

2. The green belt automatic watering system according to claim 1 is characterized in that: The segmentation result is projected onto the Wx-Wz plane of the world coordinate system to calculate and obtain the height of the green plants, and the shortest distance between the green belts on both sides of the road and the sprinkler truck is found, and the boundary map is drawn, including: Extract the green plant parts in the original image and the disparity image separately to obtain a green plant image; According to the disparity value corresponding to each green plant pixel in the green plant image, the Wz value in the world coordinate system corresponding to each green plant is calculated; Calculate the Wx and Wy values ​​based on the Wz value and the coordinates of each green plant pixel in the world coordinate system; All points within a preset range in the vehicle's drivable area are selected, and all points are fitted to obtain the road surface.

3. The green belt automatic watering system according to claim 2 is characterized in that: The calculation of the Wz value in the world coordinate system corresponding to each green plant according to the disparity value corresponding to each green plant pixel in the green plant image includes: The calculation formula is as follows: Among them, B. are the baseline and focal length of the binocular camera respectively, and disp is the disparity value corresponding to each green plant pixel obtained by stereo matching.

4. The green belt automatic watering system according to claim 3 is characterized in that: The calculation of Wx and Wy values ​​according to the Wz value and the coordinates of each green plant pixel in the world coordinate system includes: in, and are the x and y coordinates of the image coordinate system respectively; and are the x and y coordinates of the optical center in the image coordinate system, and are the x and y coordinates of the world coordinate system respectively.

5. The green belt automatic watering system according to claim 2, characterized in that: The calculation of the height from the green plant to the road surface includes: Take the world coordinate system The range is [-20,20] meters, The range is [0, 60] meters, and the green plants are projected onto In the plane; by is the x-axis, is the y-axis, H is the value corresponding to the point (x, y), and H is the height from the green plant to the road surface.

6. The green belt automatic watering system according to claim 2, characterized in that: The method of determining the location where watering is required based on whether there are obstacles in the horizontal direction of the green belt of the fitted road surface and the height of the green plants from the road surface includes: Select the largest H in the grid as the grid value; Compare the maximum H value with the installation height of the binocular camera; When H is less than the installation height and there are no obstacles on the fitted road surface in the horizontal direction of the green belt, the corresponding green plants are determined to be in a position where water can be sprinkled.

7. The green belt automatic watering system according to claim 2, characterized in that: After the green plant parts in the original image and the disparity image are extracted separately to obtain the green plant image, the green belt automatic watering system further includes: Project the green plant map, first perform an interpolation filling on the above-obtained projection map, for places where the pixel value is 0, use the mean value of the non-zero points in the 3×3 neighborhood to interpolate and fill, and then remove isolated points according to the regional connectivity after filling to obtain the final green plant projection map.

8. An automatic watering device for green belts based on binocular vision, used for adopting the system according to any one of claims 1 to 7, characterized in that: include: The segmentation module is used to obtain the disparity map of the binocular camera in real time, perform segmentation network processing on the disparity map, and obtain segmentation results including green plants and road obstacles; A calculation module is used to project the segmentation result onto the Wx-Wz plane of the world coordinate system to calculate and obtain the height of the green plants, find the shortest distance between the green belts on both sides of the road and the sprinkler truck, and draw a boundary map; The confirmation module is used to confirm the location where watering is required based on whether there are road obstacles in the horizontal direction of the green belt and the boundary map.

Images