Intelligent mower with three-eye heterogeneous camera

Abstract

The utility model discloses an intelligent mower with three -eyed isomerous camera, through with universal wheel as the rear wheel drive change to front wheel drive, first ensured that the stability of mower front end in the walking process, and then ensured that the field of view stability of camera set in the front end of fuselage, and the image of camera acquisition is more clear because of the reduction of shaking. In addition, by the relative position relation of at least two cameras of constraint setting in the front end of fuselage, to reduce the field of view blind area and increase the common view area, when the subsequent mower is positioned, path planning and obstacle avoidance calculation based on the feature information in the increased common view area, the calculation precision is higher.

Description

Technical Field

[0001] This utility model belongs to the field of garden machinery technology and relates to an intelligent lawnmower with a three-lens heterogeneous camera. Background Technology

[0002] Currently, intelligent lawnmowers with visual recognition capabilities use a rear-wheel drive system with rear-wheel drive and front-mounted omnidirectional wheels. These, along with sensors and cameras at the front, enable obstacle avoidance, positioning, and path planning during operation. However, when encountering uneven surfaces or obstacles, the front-mounted omnidirectional wheels can cause the mower head to sway, making it difficult to maintain a stable field of view for the front-mounted cameras. This reduces the accuracy of the images captured by the cameras, significantly impacting the accuracy of subsequent obstacle avoidance, positioning, and attitude adjustment.

[0003] In addition, existing smart lawnmowers with vision capabilities typically use a monocular camera, such as... Figure 7 As shown, a monocular camera has a large field of view in front of it, but also a large blind spot on its sides. When a monocular camera is used on a larger lawnmower, the blind spot becomes even larger, causing collisions during obstacle avoidance and small-angle attitude adjustments due to the inability to perceive environmental information on the sides of the machine. Therefore, how to reduce the camera's blind spot, lower the probability of collisions, and reduce the likelihood of intelligent lawnmowers getting stuck in confined spaces due to limited visibility and frequent directional adjustments, while ensuring the lawnmower remains stable on undulating terrain or when encountering obstacles, has become a pressing technical problem to be solved in this field. Utility Model Content

[0004] This invention aims to ensure stable field of view for the camera while the lawnmower is in motion, reduce blind spots, improve positioning and path planning accuracy based on physical structure, reduce the probability of getting stuck, and enhance obstacle avoidance capabilities. It provides an intelligent lawnmower with a three-lens heterogeneous camera. By changing the rear-wheel drive (with omnidirectional wheels as front wheels) to front-wheel drive, the stability of the front end of the lawnmower is first ensured during movement, thus ensuring stable field of view for the camera located at the front end. Furthermore, by constraining the relative positional relationship of at least two cameras located at the front end, blind spots are reduced and the shared field of view is increased. Subsequently, when the lawnmower performs positioning, path planning, and obstacle avoidance calculations based on feature information from the increased shared field of view, the calculation accuracy is higher.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A smart lawnmower with a tri-lens heterogeneous camera includes a main body, with a drive wheel and multiple cameras arranged at the front end of the main body along its direction of travel.

[0006] In summary, the advantages of this utility model are as follows: This invention relates to an intelligent lawnmower with a three-lens heterogeneous camera system. By changing the rear-wheel drive (with omnidirectional wheels as the front wheels) to front-wheel drive, the stability of the front end of the lawnmower is ensured during movement, thereby ensuring a stable field of view for the cameras positioned at the front. The images captured by the cameras are clearer due to reduced shaking. Furthermore, by constraining the relative positional relationship of at least two cameras positioned at the front, blind spots are reduced and the shared field of view is increased. Subsequent lawnmower positioning, path planning, and obstacle avoidance calculations based on feature information within this increased shared field of view result in higher calculation accuracy. Attached Figure Description

[0007] Figure 1 This is a structural schematic diagram from the front view of the lawnmower of this utility model.

[0008] Figure 2 This is a structural schematic diagram from the rear view of the lawnmower of this utility model.

[0009] Figure 3 This is a structural schematic diagram of the lawnmower of this utility model from the bottom view.

[0010] Figure 4 This is a top view of the lawnmower of this utility model.

[0011] Figure 5 This is a side view of the lawnmower of this utility model.

[0012] Figure 6 This is a front view of the lawnmower of this utility model.

[0013] Figure 7 This is a diagram illustrating the blind spot of a smart lawnmower equipped with a monocular camera.

[0014] The markings in the diagram are: 11. Upper housing; 121. Front panel; 122. Rear panel; 13. Light strip; 14. Charging port; 15. Emergency stop button; 21. Drive wheel; 22. Swivel wheel; 31. Front-view camera; 32. Secondary-view camera; 4. Mowing disc. Detailed Implementation

[0015] The following specific examples illustrate the implementation of this utility model. Those skilled in the art can easily understand other advantages and effects of this utility model from the content disclosed in this specification. This utility model can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this utility model. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0016] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0017] In this embodiment of the invention, all directional indicators (such as up, down, left, right, front, back, horizontal, vertical, etc.) are only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indicator will also change accordingly.

[0018] Due to installation errors and other reasons, the parallel relationship referred to in the embodiments of this utility model may actually be an approximate parallel relationship, and the perpendicular relationship may actually be an approximate perpendicular relationship.

[0019] This embodiment provides an intelligent lawnmower with a tri-lens heterogeneous camera, such as... Figure 1-3 As shown, it includes the main body and the mowing disc 4 located at the bottom of the main body. The mower performs mowing operations through the mowing disc 4 during self-driving.

[0020] The main body is divided into a front end and a rear end along the direction of travel of the lawnmower. The front end of the main body is provided with a drive wheel 21, and the rear end is provided with a rear wheel. The rear wheel is preferably a driven universal wheel 22. The drive wheel 21 provides the overall driving force for the lawnmower. Therefore, the lawnmower in this embodiment is a front-wheel drive type, driven by the front wheel.

[0021] The front-mounted drive wheel 21 effectively improves the overall stability of the lawnmower when it is on uneven roads or when it is crossing obstacles such as steps and rocks. Compared with traditional rear-drive lawnmowers, the casters mounted on the head are prone to tilting and wobbling when crossing obstacles. The front-drive method greatly improves the stability of the head.

[0022] The main body includes a housing, the bottom of which has an opening for the drive wheels 21, casters 22 and mowing disc 4 to be exposed. The main body also includes a battery pack, control unit, drive unit, equipment mounting bracket, camera module and other components located inside the housing.

[0023] The drive wheel 21 is located at the lower front end of the housing, and slots are made on the left and right sides of the housing to expose the drive wheel 21, thereby improving the flexibility of the drive wheel 21 and optimizing the overall spatial structure.

[0024] The battery pack and mowing disc 4 are located in the middle of the main body in the front-to-back direction, between the drive wheel 21 and the caster wheel 22, so that the overall center of gravity of the mower is in the middle or forward. The lateral width of the front end of the casing is greater than that of the rear end, and the width of the casing gradually decreases from the front end to the rear end of the main body. This can reduce the left and right deviation during movement and improve the overall stability during movement and mowing.

[0025] Furthermore, the spacing between the two drive wheels 21 is approximately equal to the width of the main body, increasing the contact area with the ground and maintaining the stability of the main body when it sways up and down while traveling on bumpy roads.

[0026] The camera module includes an upper shell, a lower shell, a motherboard, a heat sink, and multiple cameras. The upper shell and the lower shell are spliced ​​together to form an assembly structure. The motherboard module is located inside it. The heat sink is located on the upper shell or the lower shell and is in contact with the chip of the motherboard module. The fins of the heat sink face outward to assist the motherboard module in effectively dissipating heat.

[0027] When the upper and lower shells are assembled, they form a fastener for fixing the camera. The camera module is located inside or outside the shell, and the shell has mounting holes for the camera to be exposed. There can be two or more cameras to fully cover the forward and side views of the lawnmower.

[0028] The camera may be an RGB camera, an IR camera, or a combination of multiple cameras.

[0029] Preferably, the camera setup includes three cameras: a front-view camera 31 and two auxiliary cameras 32. The front-view camera 31 is positioned centrally on the main body of the lawnmower, providing the frontal view. The two auxiliary cameras 32 are positioned on the left and right sides of the main body, perpendicular to the direction of travel, respectively. Preferably, the front-view camera 31 is symmetrically distributed to provide the left and right lateral views of the lawnmower. However, this invention does not limit the scope of the setup. In other embodiments, the cameras can be configured as two symmetrically arranged, one in front and one on the left, or one in front and one on the right, or even four or more cameras can be used to provide a more comprehensive field of view.

[0030] The overlapping areas of the fields of view of at least two cameras positioned at the front of the main body form Figure 4 The shared viewing area shown is designed to improve the positioning and path planning accuracy of the smart lawnmower.

[0031] The improvement in positioning accuracy mainly includes two aspects: First, the setting of multi-view cameras increases the overall field of view (FOV) and the number of observable features, thereby improving positioning accuracy; Second, multi-view cameras have a larger baseline and one or more shared viewing areas, which makes the depth calculation of features more accurate and the constraints more sufficient, thereby improving positioning accuracy.

[0032] The improvement in path planning accuracy mainly includes two aspects: First, the realization of the ultra-large FOV (field of view) of multi-view cameras, which allows for a larger observable area and a smaller blind spot, thereby improving the accuracy of path planning; second, the higher positioning accuracy of multi-view cameras, which in turn improves the accuracy of path planning.

[0033] Preferably, the ground clearance of the left-view camera or the right-view camera is less than that of the front-view camera 31, so that the range of the shared viewing area is controlled by the height of the left-view camera or the right-view camera. The upper half of the field of view of the front-view camera 31 has less information involved in path planning. By controlling the height of the left-view camera and the right-view camera, the height range of the shared viewing area can be controlled, which effectively reduces the amount of image processing computation and improves the speed of positioning and planning computation.

[0034] More preferably, the ground clearance of the forward-viewing camera 31 (i.e., the height of the forward-viewing camera 31 from the bottom of the drive wheel 21, refer to...) Figure 6 H1 in the figure is set to 200-300mm; the ground clearance of the left or right camera (i.e., the height of the left or right camera from the bottom of the drive wheel 21) is referenced. Figure 6 The H2 in the figure is preferably set to 200-300 mm.

[0035] More preferably, the ground clearance of the forward-viewing camera 31 is set to 280mm; the ground clearance of the left-viewing camera or the right-viewing camera is set to 280mm.

[0036] Preferably, the center distance between the left-view camera and the front-view camera 31 (refer to...) Figure 4 Within the range of 200-350mm (L3 in the image), the center distance between the right-view camera and the front-view camera 31 (refer to...) Figure 4 The L4 in the middle is in the range of 200-350mm, which makes the positions of the front-view camera 31, the left-view camera and the right-view camera more dispersed, and the observation range is larger.

[0037] More preferably, the center distance between the left-view camera and the front-view camera 31 is 300mm, and the center distance between the right-view camera and the front-view camera 31 is 300mm.

[0038] In the direction of travel, the horizontal distance between the forward-view camera 31 and the drive shaft of the drive wheel 21 is greater than the horizontal distance between the left-view camera and the right-view camera and the drive shaft. The purpose of this arrangement is to make the positions of the forward-view camera 31, the left-view camera, and the right-view camera more dispersed, thereby expanding the observable range.

[0039] More preferably, the horizontal distance between the forward-looking camera 31 and the drive shaft of the drive wheel 21 (refer to...) Figure 5 The L1 in the image is between 200-400mm, and the horizontal distance between the left and right cameras and the drive shaft is (refer to...). Figure 5The L2 in the image is between 100-300mm. The advantage of setting it within the above range is that the positions of the forward-looking camera 31, the left-looking camera, and the right-looking camera are more dispersed, resulting in a larger observation range.

[0040] More preferably, the horizontal distance between the front-view camera 31 and the center of the drive wheel 21 is set to 204.5 mm; the horizontal distance between the secondary camera 32 and the center of the drive wheel 21 is set to 127 mm. Experiments show that the lawnmower has the optimal observation range under the conditions of 204.5 mm and 127 mm distances.

[0041] Additionally, the forward tilt angle of the left-view or right-view camera (refer to...) Figure 4 (b3) Within the range of 30°-70°, the forward tilt angle is the angle between the centerline of the left-view camera and the centerline of the main body, taking an acute angle, or the angle between the centerline of the right-view camera and the centerline of the main body, taking an acute angle.

[0042] By setting the forward tilt angles of the left and right cameras within the aforementioned range, the forward camera 31 can have a better common field of view with the left and right cameras respectively, ensuring that there are no blind spots in the surrounding field of view in front of the lawnmower. To further explain, this means ensuring that there are no blind spots at the junction of the fields of view of the forward camera 31 and the left camera, and that there are no blind spots at the junction of the fields of view of the forward camera 31 and the right camera, thus ensuring the integrity of the field of view.

[0043] Preferably, the forward tilt angle of both the left and right cameras is 57.5°. This angle setting allows the shared field of view generated by the three cameras to achieve higher accuracy in feature point depth estimation, thereby improving positioning accuracy. This improved positioning accuracy can further enhance the accuracy of path planning.

[0044] The horizontal field of view (H angle) and vertical field of view (V angle) of the forward-looking camera 31 are in the range of 90°-150° and 70°-120° respectively, to ensure the effective observation area of ​​the lawnmower, reduce the size of the blind spot, and make both grass and non-grass areas visible in the image.

[0045] More preferably, the horizontal field of view of the forward-looking camera 31 is 130° and the vertical field of view is 95°, so as to ensure that both grassy areas and non-grassy areas can be seen in the image, which helps to perform loopback positioning.

[0046] In this embodiment, the horizontal and vertical field of view of the left-view camera or the right-view camera are within the ranges of 90°-150° and 70°-120°, respectively. The purpose of limiting the horizontal and vertical field of view of the left-view camera or the right-view camera to the above ranges is to ensure the effective observation area of ​​the lawnmower, reduce the size of the blind spot, and ensure that both grassy and non-grassy areas can be seen in the image.

[0047] More preferably, the horizontal field of view of the left-viewing camera or the right-viewing camera is 150° and the vertical field of view is 80°, so as to ensure that both grassy areas and non-grassy areas can be seen in the image, which helps to perform loopback positioning.

[0048] It should be noted that the horizontal field of view of the left-view or right-view camera is 150° (refer to...). Figure 6 (b1 in the text) The vertical field of view is 80° (refer to...) Figure 6 (b2 in the middle), and the horizontal field of view of the forward-looking camera 31 is 130° (refer to b ...). Figure 4 (a1 in the text) Vertical field of view 95° (refer to) Figure 5 The difference in a2) is for the purpose of: To ensure a shared field of view, both the left and right cameras are tilted forward, which increases the blind spot behind the vehicle. Adjusting the horizontal field of view of the left and right cameras to a larger angle compared to the front camera 31 effectively reduces the size of the blind spots on the sides and rear of the vehicle, ensuring a wider observation range and greater safety during lawnmower operation. Adjusting the vertical field of view of the left and right cameras to a smaller angle compared to the front camera 31 avoids obstruction of the field of view by the vehicle body, reducing the FOV while still meeting operational requirements.

[0049] In this embodiment, the downward tilt angle of the forward-looking camera 31 (refer to...) Figure 5 (a3) Within the range of 10°-20°, the downward tilt angle of the left-view camera is within the range of 10°-20°, and the downward tilt angle of the right-view camera is within the range of 10°-20°. The downward tilt angle is the angle between the camera's optical axis and the horizontal line, taking an acute angle, and the downward tilt angles of the left-view camera and the right-view camera are the same.

[0050] Setting the tilt angles of the front-view camera 31, the left-view camera, and the right-view camera within the aforementioned range ensures that the lawnmower has an effective observation area, reduces the size of blind spots, and allows both grassy and non-grassy areas to be seen in the image.

[0051] More preferably, the downward tilt angle of the forward-looking camera 31, the left-looking camera, and the right-looking camera is 15° (refer to...). Figure 6 (b4 in the text). Setting the tilt angle of all three cameras to 15° ensures that the blind spot of the lawnmower is as small as possible.

[0052] Furthermore, for the camera module, the distance from the front-view camera 31 to the edge of the motherboard is 50mm, and the distance from the secondary-view camera 32 to the edge of the motherboard is 130mm. The motherboard is 125*100mm in size, and the heat sink is 125*100*21mm in size. A full-circumference sealing strip is provided at the splicing buckle between the upper and lower shells, a sealing strip is provided at the buckle between the upper and lower shells where the camera is mounted, and a full-circumference sealing strip is provided at the mounting groove between the heat sink and the upper or lower shell. In addition, the motherboard's connecting wires pass through the upper shell and extend outwards, and a sealing ring is provided at the wire exit hole, ensuring the sealing performance of the camera module in all aspects and preventing moisture intrusion.

[0053] Furthermore, a lidar is also installed on the main body of the machine. The lidar is mounted on the casing and is located in the center of the front of the casing. The lidar is higher off the ground than the forward-looking camera 31. The lidar faces the front of the lawnmower and works with the camera module to further improve the accuracy of obstacle recognition and path planning of the lawnmower.

[0054] The front of the main body is provided with a light strip 13, which is installed on the housing and extends laterally to both sides of the housing. Two auxiliary cameras 32 are located at the two extended ends of the light strip 13, with the front-view camera 31 set higher than the light strip 13. This provides sufficient lighting assistance within the field of view of each camera without affecting the acquisition effect of each camera.

[0055] The front of the main body is provided with a charging port 14, which is opened through the casing so that the external charging interface can be connected to the internal battery for charging.

[0056] An emergency stop button 15 is located on the top of the main body. The emergency stop button 15 is connected to the internal control unit and provides an emergency stop and power-off function. The emergency stop button 15 is located behind the forward-looking camera 31 and the lidar.

[0057] In some embodiments, the housing has a split structure, including an upper housing 11 and a lower housing. The lower housing can be connected to the upper housing 11 through an elastic connection structure or a buffer structure, including but not limited to using springs, damping rods and other buffer devices to connect by means of hinges, so that the lower housing can float relative to the upper housing 11 in a certain range of vertical, horizontal and horizontal, front and back directions. By setting the main equipment, including drive wheels 21, casters 22, battery pack, control unit, drive unit, equipment mounting bracket, camera module and so on, when the housing collides with an obstacle, only the lower housing floats, thereby reducing the shaking of the main body.

[0058] Furthermore, the lower housing includes a detachable front panel 121 and a rear panel 122, which are detachably connected to the upper housing 11. The charging port 14 is located on the front panel 121, and a red warning light is also provided on the rear panel 122.

[0059] The intelligent lawnmower provided in this embodiment uses a three-lens camera with the above-mentioned relative positional relationship, which enables the intelligent lawnmower to have higher positioning, path planning, and obstacle avoidance accuracy.

[0060] The method for visual positioning and perception of an intelligent lawnmower based on a tri-lens heterogeneous camera provided in this embodiment includes the following steps: S1, At least two cameras are heterogeneously arranged on the lawnmower, and the common field of view generated by the at least two heterogeneous cameras is obtained; The method of heterogeneously arranging cameras on a lawnmower has been described in detail above and will not be repeated here. It is important to emphasize that in this application, by constraining the different heterogeneous types of the cameras, the types of shared viewing areas also differ. The differences in the types of shared viewing areas mainly depend on the ground clearance of the left and right cameras compared to the front-view camera, the forward tilt angle of the left and right cameras compared to the front-view camera, and the horizontal and vertical field of view angles of the left and right cameras compared to the front-view camera. Different types of shared viewing areas also have varying degrees of impact on improving the accuracy of lawnmower positioning and path planning based on shared viewing areas. In this embodiment, the ground clearance of the front-view camera is greater than that of the left-view and right-view cameras, and the height difference between the front-view camera and the left-view and right-view cameras is 14.2 mm. The forward tilt angle of the front-view camera is 0°, the forward tilt angle of the left-view and right-view cameras is 45°, the horizontal field of view of the front-view camera is 130°, the vertical field of view of the front-view camera is 95°, and the horizontal field of view of the left-view and right-view cameras is 150° and the vertical field of view of the left-view and right-view cameras is 80°. The resulting shared field of view can bring optimal positioning and path planning accuracy to the front-wheel-drive intelligent lawnmower provided in this embodiment.

[0061] The following step S2 details the method for locating and planning the path of a front-wheel-drive lawnmower using the obtained optimal common field of view.

[0062] After obtaining the common viewing area through step S1, the visual positioning and perception method based on a tri-camera heterogeneous camera provided in this embodiment proceeds to the following steps: S2, after adjusting the positioning algorithm parameters and / or path planning parameters based on the common viewing area, performs positioning and / or path planning actions for the lawnmower.

[0063] Specifically, in step S2, the method for locating the lawnmower based on the common viewing area includes the following steps: 1.1 Image Acquisition and Preprocessing The forward-looking camera, left-looking camera, and right-looking camera synchronously acquire image frames at the ground height, forward tilt angle, and field of view angle determined above, and record the synchronously acquired image frames as L, C, and R, respectively. Then, image preprocessing is performed on the synchronously acquired image frames, including distortion removal, brightness normalization, and time synchronization alignment.

[0064] 1.2 Obtain the set of matching points Let the image frames acquired simultaneously by the front-view camera, left-view camera, and right-view camera be image frames C, L, and R, respectively. Then, use algorithms such as FAST, SIFT, and ORB to extract feature points for localization from the image frames after image preprocessing, and let the feature points extracted from image frames C, L, and R be the first feature point, the second feature point, and the third feature point, respectively.

[0065] Then, a second feature point with the highest feature similarity to each first feature point is matched from image frame L, and the first matching point pair consisting of the first feature point and the second feature point with the highest feature similarity is added to the first matching point pair set bound to image frame L. In image frame R, a third feature point with the highest feature similarity to each first feature point is matched, and the second matching point pair formed by the first feature point and the third feature point with the highest feature similarity is added to the second matching point pair set bound to image frame R.

[0066] Taking the matching of point pairs between image frame C and image frame L as an example, the point pair matching process provided in this embodiment is explained as follows: Descriptors are generated for feature points used in point-to-point matching. These descriptors can be floating-point or binary, and are used to quantify the local image characteristics of the feature points.

[0067] For each second feature point in image frame L, calculate the distance between the descriptor of the second feature point and the descriptors corresponding to all first feature points in image frame C, and select the top K first feature points sorted by descriptor distance from smallest to largest as the second nearest neighbor feature list associated with the second feature point. Then, calculate the first nearest neighbor feature list associated with the first feature point currently performing point-to-point matching with the second feature point having the second nearest neighbor feature list. The calculation principle of the first nearest neighbor feature list is the same as that of the second nearest neighbor feature list, except that the first nearest neighbor feature list is selected from image frame L, while the second nearest neighbor feature list is obtained by selecting first feature points from image frame C. The specific process will not be elaborated further.

[0068] The method for calculating the associated third nearest neighbor feature list for each third feature point in image frame R is the same as the method for calculating the associated second nearest neighbor feature list for the second feature point in image frame L, and will not be described again.

[0069] The distance between descriptors is calculated using existing methods, without further explanation.

[0070] Then, feature points are selected from the nearest feature list to form matching point pairs and added to the matching point pair set through one or more of the following: reliable neighbor verification, cross-validation of common-view feature points, and baseline constraint check.

[0071] In this embodiment, the nearest neighbor reliable verification method is as follows: Extract the first nearest neighbor feature point with the smallest descriptor distance and the second nearest neighbor feature point with the second smallest descriptor distance from the feature point that matches the current participating point pair from the nearest neighbor feature list, and determine whether the ratio of the smallest descriptor distance to the second smallest descriptor distance is less than a preset ratio threshold. If so, the nearest neighbor feature list calculated for the feature point is considered reliable. For example, extract the first nearest neighbor feature point (the first feature point ranked first in the first nearest neighbor feature list) that has the smallest descriptor distance to the second feature point that matches the current participating point pair from the first nearest neighbor feature list, and the second nearest neighbor feature point (the first feature point ranked second in the first nearest neighbor feature list) that has the second smallest descriptor distance. Then, calculate the ratio of the first nearest neighbor feature point and the second nearest neighbor feature point, and determine whether the ratio is less than a preset ratio threshold. If so, it is determined that the first nearest neighbor feature list calculated for the second feature point is reliable.

[0072] The method for cross-validation of common feature points is as follows: The feature point with the highest feature similarity matched by the first matching feature point in the point pair matching is the second matching feature point, and the feature point with the highest feature similarity matched by the second matching feature point is the first matching feature point. The matching point pair constitutes a common feature point. Suppose that when performing point-to-point matching on the first feature point in image frame C, the feature point with the highest feature similarity matched with the first feature point in image frame L is the second matching feature point. Then, for this second matching feature point, through the same nearest neighbor feature matching as described above, if the first matching feature point with the highest feature similarity matched with the second matching feature point in image frame C is identified as the first feature point, it is determined that the first feature point and the second matching feature point form a matching point pair, which are co-visible feature points with shared features.

[0073] The method for baseline constraint checking is as follows: The disparity between the first and second matching feature points that constitute a matching point pair is less than a preset disparity threshold. Disparity is the difference between the horizontal coordinates of the first and second matching feature points that constitute a matching point pair in the same coordinate system.

[0074] Finally, the visual depth is estimated for the common-view feature points represented by the matching points. The estimated visual depth is then back-projected onto the 3D space to obtain the spatial point cloud. A sparse map point set is then constructed and the map point IDs are labeled to achieve real-time positioning of the lawnmower.

[0075] In this embodiment, the method for estimating visual depth using co-visual feature points represented by matching points is expressed as follows: This represents the visual depth estimated from the co-visual feature points; The focal length from the camera to the common-view feature point; The baseline distance between cameras (the optical center distance between binocular cameras); The disparity is the shared feature point.

[0076] For example, for a matching point pair consisting of feature points O1 and O2, the focal length from the left-looking camera to O1 is... The focal length from the forward-facing camera to O2 is After aligning the baselines of the left-view camera and the front-view camera, the focal length of the front-view camera is used as the baseline. . The horizontal distance between the left-view camera and the front-view camera. This is the absolute value of the difference between the horizontal coordinates of O1 and O2 in the same coordinate system.

[0077] 1.3 Constructing a sparse map point set The visual depth of the trajectories of each shared feature point is back-projected into 3D space (using the projection relationship between the world coordinate system and the camera coordinate system) to obtain a spatial point cloud. Then, a sparse map point set is constructed based on the spatial point cloud, and each map point is labeled with a feature ID.

[0078] In this embodiment, after obtaining the back-projected spatial point cloud, constructing a sparse map point set includes the following steps: (1) Reprojection error filtering. The generated 3D point cloud is back-projected onto the image plane, and then the back-projection error is calculated, which is the pixel distance between the back-projected point and the actual key point. If the error is greater than the threshold (e.g., 2 pixels), the 3D point is removed.

[0079] (2) Parallax filtering. Filter out co-view feature points whose baseline distance is less than the preset baseline distance to ensure that the baseline between co-view image frames is large enough.

[0080] (3) Observation count filtering. Retain co-visible feature points of the same 3D point in consecutive frames, such as three consecutive frames, to ensure the stability of map points.

[0081] (4) Deduplication and merging. If similar 3D points are generated for each matching point pair (common feature points) (e.g., the Euclidean distance is less than a certain threshold), then these matching point pairs are merged, and the feature point with the most observations is retained.

[0082] (5) Construct a sparse map point set. The final sparse map point set contains several sets of 3D points, each 3D point being associated with the following information: a) 3D coordinates; b) Observation information, i.e., which image frames observed the point and the corresponding 2D pixel coordinates; c) Feature descriptors, used for subsequent matching or relocation.

[0083] The purpose of ID labeling for 3D points is to assign a unique identifier (ID) to each map point and ensure that these map points can be tracked and updated during route planning or localization. The following are the specific steps for ID labeling for 3D points: (1) Assign a unique ID: Assign a globally unique ID to each 3D map point that passes the screening. Usually, an incrementing integer (such as starting from 1) is used. For example, maintain a global counter or hash table and assign a new ID each time a new map point is added.

[0084] (2) Associate keypoints with map points. Associate each 3D map point with its corresponding 2D keypoint. Each 2D keypoint records: a) The unique ID of the image frame; b) Pixel coordinates of 2D keypoints in the image frame; c) Feature descriptor of the 3D key point.

[0085] (3) Maintain the consistency of feature IDs, specifically: a) Adding new points. When a new 2D keypoint is detected and a corresponding 3D map point is generated, a new feature ID is assigned to the 3D map point and it is added to the map point set.

[0086] b) Point update. When an existing map point is observed in a new frame, update the observation information of that map point (e.g., add a new frame ID and 2D coordinates), but keep its feature ID unchanged.

[0087] c) Point removal. If a map point is considered an outlier in subsequent optimizations (e.g., excessive reprojection error), its ID is removed from the set, and it is ensured that the ID is not reassigned.

[0088] 1.4 Based on sparse map point sets, the lawnmower is located and its path is planned. After minimizing the reprojection error on the sparse map point set, the camera's position in the world coordinate system is optimized, thereby achieving the positioning of the lawnmower. The minimization of the reprojection error is expressed by the following formula: in, For camera projection model; for Camera pose at any given moment; For the first 3D map points; for The inverse of the pose of the time-lapse camera; Indicates in Time for the first The observation coordinates of a 3D map point; This is a robust kernel function.

[0089] In this embodiment, the robust kernel function is preferably any one or more of the Huber kernel function, Cauchy kernel function, and Tukey kernel function.

[0090] In summary, the method for locating or planning the path of the lawnmower based on the shared viewing area in step S2 includes the following steps: A1, let the image frames acquired simultaneously by the front-view camera, left-view camera and right-view camera be image frames C, L and R respectively, and extract the feature points for localization from the image frames after image preprocessing; A2 calculates the set of matching point pairs bound to image frames C, L, and R respectively, to identify the common view feature points of each pair of cameras and estimate the visual depth of the common view feature points. A3 backprojects the visual depth estimated from the common-view feature points into 3D space to obtain a spatial point cloud. Then, it constructs a sparse map point set and labels the map point IDs to achieve real-time positioning or path planning for the lawnmower.

[0091] In summary, the intelligent lawnmower provided in this embodiment, by changing the rear-wheel drive (with omnidirectional wheels as the front wheels) to front-wheel drive, firstly ensures the stability of the front end of the lawnmower during movement, thereby ensuring the stable field of view of the camera located at the front end of the lawnmower. The images captured by the camera are clearer due to reduced shaking. In addition, by constraining the relative positional relationship of at least two cameras located at the front end of the lawnmower, blind spots are reduced and the shared field of view is increased. Subsequently, when the lawnmower performs positioning, path planning, and obstacle avoidance calculations based on feature information in the increased shared field of view, the calculation accuracy is higher.

[0092] Obviously, the described embodiments are only some, not all, of the embodiments of this utility model. All other embodiments obtained by those skilled in the art based on the embodiments of this utility model without inventive effort should fall within the protection scope of this utility model.

Claims

1. A smart lawnmower with a tri-lens heterogeneous camera, characterized in that, The device includes a main body, a mowing disc, a battery pack, and a rear wheel. The main body has a drive wheel and multiple cameras at its front end along its direction of travel. The rear wheel is located at the rear end of the main body. The battery pack and the mowing disc are located in the middle of the main body in the front-rear direction and are located between the drive wheel and the rear wheel.

2. The intelligent lawnmower with a tri-lens heterogeneous camera according to claim 1, characterized in that, The camera includes at least two of a front-view camera, a left-view camera, and a right-view camera. The overlapping areas of the fields of view of the at least two cameras located at the front of the main body form a common viewing area. The left-view camera and the right-view camera are respectively located to the left and right of the front-view camera.

3. The intelligent lawnmower with a tri-lens heterogeneous camera according to claim 2, characterized in that, The ground clearance of the left-view camera or the right-view camera is less than the ground clearance of the front-view camera.

4. A smart lawnmower with a tri-lens heterogeneous camera according to claim 2 or 3, characterized in that, The center distance between the left-view camera and the front-view camera is 300mm; the center distance between the right-view camera and the front-view camera is 300mm.

5. A smart lawnmower with a tri-lens heterogeneous camera according to claim 2, characterized in that, The forward tilt angle of the left-view camera is 57.5°, and the forward tilt angle of the right-view camera is 57.5°. The forward tilt angle is the angle between the central axis of the left-view camera and the central axis of the main body, which is an acute angle; or, the angle between the central axis of the right-view camera and the central axis of the main body, which is an acute angle.

6. A smart lawnmower with a tri-lens heterogeneous camera according to any one of claims 2, 3, and 5, characterized in that, The horizontal field of view of the front-view camera is 130° and the vertical field of view is 95°; the horizontal field of view of the left-view camera or the right-view camera is 150° and the vertical field of view is 80°.

7. A smart lawnmower with a tri-lens heterogeneous camera according to any one of claims 2, 3, and 5, characterized in that, The downward tilt angle of the front-view camera, the left-view camera, and the right-view camera is 15°. The downward tilt angle is the angle between the optical axis of the front-view camera, the left-view camera, or the right-view camera and the horizontal line, and is an acute angle.

8. A smart lawnmower with a tri-lens heterogeneous camera according to any one of claims 2, 3, and 5, characterized in that, The horizontal distance between the front-view camera and the drive shaft of the drive wheel is 204.5 mm; the horizontal distance between the left-view camera, the right-view camera and the drive shaft is 127 mm.

9. A smart lawnmower with a tri-lens heterogeneous camera according to claim 2 or 3, characterized in that, It also includes a lidar, which is positioned at a higher altitude than the forward-looking camera.

10. A smart lawnmower with a tri-lens heterogeneous camera according to any one of claims 1-3, characterized in that, It also includes a light strip, which is installed on the main body and extends laterally to both sides of the main body. Two auxiliary cameras are respectively provided at the two extended ends of the light strip, wherein the cameras are set on the main body and above the light strip; the auxiliary cameras include a left-viewing camera and / or a right-viewing camera.

11. A smart lawnmower with a tri-lens heterogeneous camera according to claim 1, characterized in that, The rear wheels are omnidirectional wheels; the camera is an RGB camera or an IR camera.

Images