Map construction method and device, controller and mower

Abstract

This application discloses a map construction method, apparatus, controller, and lawnmower, relating to the field of lawnmower technology. The method includes: acquiring multi-view images collected during the lawnmower's operation and detecting the lawnmower's global pose; performing boundary detection on the multi-view images to obtain boundary line pixel coordinates, and performing feature classification on the boundary regions corresponding to the boundary line pixel coordinates to obtain boundary semantic detection results; and constructing a semantic grid map based on the boundary line pixel coordinates, the boundary semantic detection results, and the lawnmower's global pose. By acquiring multi-view images, the lawnmower's surrounding lawn boundaries can be fully covered, eliminating the perception blind spots present in single-camera solutions; the lawnmower can simultaneously complete the construction of the work area map while autonomously driving along the edge, eliminating the need for manual surveying by the user; and it can also perform semantic classification of lawn boundaries and associate the semantic information of various boundaries into the grid map.

Description

Technical Field

[0001] This application relates to the field of lawnmower technology, and more particularly to a map building method, apparatus, controller and lawnmower. Background Technology

[0002] Currently, the mapping of lawnmower working areas mainly adopts the following three technical solutions: (1) RTK (Real-Time Kinematic)-GNSS (Global Navigation Satellite System) satellite positioning and mapping scheme: The user walks around the lawn boundary with the RTK mobile station by hand holding the RTK mobile station to record the trajectory points to generate a polygon of the working area by using the centimeter-level absolute positioning coordinates output by the RTK-GNSS module.

[0003] (2) SLAM (Simultaneous Localization and Mapping) mapping scheme for lidar: 2D or 3D lidar is mounted on a lawnmower, and an environmental map is constructed through lidar point cloud registration and SLAM algorithm.

[0004] (3) Vision-based SLAM mapping scheme: Based on the images collected by the camera, feature point matching and lawnmower pose calculation are completed to realize environmental mapping.

[0005] However, all three existing mapping schemes mentioned above have inherent flaws: For RTK-GNSS satellite positioning mapping solutions: manual trajectory collection along edges is required, which is cumbersome and results in a poor user experience. For LiDAR-based SLAM mapping solutions: high-precision LiDAR hardware is expensive, increasing the overall cost; additionally, the low installation height of radar mounted on lawnmowers makes it difficult to identify low and subtle boundaries, and it cannot recognize the semantic information of boundaries such as walls, flower beds, and water features. For vision-based SLAM mapping solutions: conventional single forward-looking cameras have limited fields of view, resulting in blind spots during edge operations, easily leading to missed boundary detection and map incompleteness, while also lacking boundary semantic recognition capabilities. Summary of the Invention

[0006] The main purpose of this application is to provide a map building method, device, controller and lawn mower, which aims to solve the technical problems of high mapping cost, perception blind spots in mapping, inability to recognize boundary semantics and poor user experience caused by manual edge-collected trajectory mapping.

[0007] To achieve the above objectives, this application proposes a map construction method, the method comprising: Acquire multi-view images of the lawnmower during its operation and detect the global pose of the lawnmower. Boundary detection is performed on multi-view images to obtain the pixel coordinates of the boundary lines, and feature classification is performed on the boundary regions corresponding to the pixel coordinates of the boundary lines to obtain the boundary semantic detection results. A semantic raster map is constructed based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine.

[0008] Furthermore, to achieve the above objectives, this application also proposes a map building apparatus, which includes: The acquisition module is used to acquire multi-view images collected during the operation of the lawnmower and to detect the global pose of the lawnmower. The detection module is used to perform boundary detection on multi-view images, obtain the pixel coordinates of the boundary lines, and perform feature classification on the boundary regions corresponding to the pixel coordinates of the boundary lines to obtain the boundary semantic detection results. The construction module is used to construct a semantic raster map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine body.

[0009] In addition, to achieve the above objectives, this application also proposes a controller comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the map construction method as described above.

[0010] In addition, to achieve the above objectives, this application also proposes a lawnmower, including a camera array and a controller, wherein the controller is electrically connected to the camera array; the camera array is used to acquire multi-view images, and the camera array consists of at least three independent cameras.

[0011] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the map construction method described above.

[0012] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the map construction method described above.

[0013] This application provides a map building method, apparatus, controller, and lawnmower. The map building method includes: acquiring multi-view images collected during the lawnmower's operation and detecting the lawnmower's global pose; performing boundary detection on the multi-view images to obtain boundary line pixel coordinates, and performing feature classification on the boundary regions corresponding to the boundary line pixel coordinates to obtain boundary semantic detection results; and constructing a semantic grid map based on the boundary line pixel coordinates, the boundary semantic detection results, and the lawnmower's global pose. This application, by acquiring multi-view images, can comprehensively cover the lawn boundary around the lawnmower, eliminating the perception blind spots present in single-camera solutions; the lawnmower can simultaneously complete the map construction of the work area while autonomously driving along the edge, eliminating the need for manual surveying by the user; and it can also perform semantic classification of the lawn boundary and associate the semantic information of various boundaries into the grid map. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating an embodiment of the map construction method of this application. Figure 2 This is a schematic diagram of the module structure of the map building device according to an embodiment of this application; Figure 3 This is a schematic diagram of the equipment structure involved in the map construction method in the embodiments of this application.

[0017] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0019] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0020] This application provides a map construction method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the map construction method of this application.

[0021] Step S11: Acquire multi-view images of the lawnmower during its operation and detect the global pose of the lawnmower. It should be noted that the lawnmower is equipped with a camera array for capturing multi-view images. Optionally, the camera array consists of three independent cameras: a front-view camera, a left-side camera, and a right-side camera.

[0022] In one specific example, the forward-facing camera is mounted on the central axis directly in front of the lawnmower, with the entire machine fixed at a height of 18cm from the ground. The lens's principal optical axis is tilted downwards relative to the horizontal plane, with a pitch angle of 15°, suitable for close-range observation of the lawn's boundaries. This forward-facing camera has a horizontal field of view of 120°, completely covering the area directly in front of the lawnmower, as well as 60° of physical space to the left and right of the front of the machine. Its main function is to acquire images of the lawn over a large area in front of the machine and identify frontal boundaries and obstacles.

[0023] The left-side camera is fixedly mounted on the outer left side of the lawnmower. In one specific example, the lens's principal optical axis is deflected 45° towards the front left of the machine, while maintaining a 20° downward tilt angle relative to the horizontal plane, aligning with the observation perspective of low-lying edges on the ground. This camera has a horizontal field of view of 110° and is specifically designed for capturing and sensing images of subtle boundaries such as the edges of low-lying lawns, road surfaces, and flower beds on the left side of the lawnmower.

[0024] The right-side camera and the left-side camera are mounted symmetrically on the outer right side of the lawnmower body. In a specific example, the main optical axis of the lens is deflected 45° towards the front right side of the body, the downward tilt angle of the lens is also set to 20°, and the horizontal field of view is 110°, completing the perception and acquisition of the low-lying boundary on the right side of the lawnmower.

[0025] The three cameras, designed with differentiated combinations of tilt, pitch, and field of view angles, construct a continuous and complete visual perception field around the lawnmower. The overall equivalent horizontal perception coverage angle reaches 270°, covering the entire area from the left rear to the right rear of the mower, eliminating the blind spots on the left and right sides caused by a single front-facing camera. Simultaneously, the edge fields of view on the left and right sides of the front-facing camera overlap with the front edge fields of view of the left and right cameras, respectively, with an overlap angle of approximately 30° on each side. This means there is a 30° image overlap observation area on both the left and right front sides of the mower. This hardware-level field-of-view overlap effectively suppresses false detections due to noise in single-frame images and interference from single-camera feeds, significantly improving the robustness of lawn boundary detection and overall mapping accuracy.

[0026] In this embodiment, during the autonomous edge-moving operation of the lawnmower, images of the surrounding environment are simultaneously and continuously acquired by the front-view camera, the left-side camera, and the right-side camera. In addition, the images acquired by each camera are processed by timestamp alignment, distortion correction, and adaptive exposure compensation to obtain multi-view images.

[0027] Furthermore, it should be noted that an RTK (Real-Time Kinematic)-GNSS (Global Navigation Satellite System) satellite positioning module is installed on the unobstructed area of ​​the lawnmower's top. This module receives signals from multiple navigation satellites to calculate the lawnmower's absolute coordinates in the global coordinate system. Additionally, a high-resolution wheel encoder is coaxially connected to the drive wheel axle of the lawnmower's chassis. As the drive wheels rotate, the encoder synchronously collects the axle rotation pulse signals, and based on these signals, calculates the lawnmower's relative travel distance, linear velocity, and heading change in real time.

[0028] In this embodiment, the RTK-GNS satellite positioning module receives satellite signals and calculates the lawnmower's real-time global absolute coordinates as global positioning observations. Wheel rotation pulses are collected by a wheel encoder, and relative mileage information such as the machine's travel distance and speed are calculated as machine motion observations. Time-series images are continuously acquired by front-view and left / right-side multi-view cameras, and the pixel feature changes between adjacent frames are compared to calculate the machine's displacement and rotation angle in adjacent frames, obtaining visual mileage observations. By fusing global positioning observations, visual mileage observations, and machine motion observations, the lawnmower's global pose is accurately calculated.

[0029] In one feasible embodiment, the multi-view image includes a front view image; the detection of the lawnmower's global pose includes: Step S111: Based on the acquired front view image, compare the pixel grayscale between two adjacent frames to obtain the visual mileage observation value. Step S112: Obtain the positioning observation value detected by the satellite positioning module and the body motion mileage observation value detected by the wheel encoder; Step S113: Based on the positioning observation value, the visual mileage observation value, and the body motion mileage observation value, the global pose of the lawnmower is obtained by fusing the positioning.

[0030] It should be noted that the global pose of the machine includes the lawnmower's position in the global map coordinate system, the machine's facing angle, linear velocity, and angular velocity.

[0031] In this embodiment, based on the acquired front view image, the pixel grayscale changes between two adjacent frames are compared to calculate the translational displacement and rotation angle between adjacent frames of the lawnmower, obtaining visual odometer observations. These visual odometer observations can reflect the lawnmower's displacement and attitude changes in a short period of time in real time. In addition, positioning observations output in real time by the satellite positioning module are acquired; these are absolute position coordinates in the global coordinate system. Wheel rotation pulse signals are collected in real time by the chassis wheel encoder, and the lawnmower's travel distance, rotation speed, and other body motion odometer observations are calculated. Furthermore, the positioning observations, visual odometer observations, and body motion odometer observations are weighted and fused using an extended Kalman filter to obtain the lawnmower's global pose. The extended Kalman filter (EKF) is a mature and widely used data fusion algorithm in the field of robot localization, and will not be elaborated upon here.

[0032] Optionally, for open and unobstructed working conditions, such as scenarios with no tree canopy or buildings obstructing the view and sufficient satellite signal, the weight of positioning observations can be increased. This allows the global pose of the lawnmower to be determined primarily by satellite absolute positioning data, supplemented by visual odometer observations and wheel odometer observations, thus ensuring absolute accuracy of global positioning.

[0033] For situations where signal obstruction causes failure, such as when a lawnmower enters an area blocked by trees or building fences, the satellite positioning module monitors its own signal quality indicators in real time, including positioning calculation status and the number of valid on-orbit satellites. If the signal quality indicators fail to meet the preset signal validity threshold for a preset number of consecutive preset cycles, satellite positioning is deemed to have failed. In this case, reducing the weight of positioning observations can also mask the failed satellite observation data. A tightly coupled positioning mode combining visual odometry observations and chassis wheel odometry observations is then switched to. Based on the visual odometry observations and the machine's motion odometry observations, the machine's travel distance and real-time attitude are continuously estimated, ensuring stable and uninterrupted pose output during short-term travel. Furthermore, once the lawnmower leaves the obstructed area, if the signal quality indicators detected by the satellite positioning module meet the preset signal validity threshold, the weight of the positioning observations is increased again. The current positioning observations are used to perform global closed-loop correction on the estimated global pose of the machine during the signal obstruction failure period, correcting the pose drift caused by the previous visual odometry observations and machine motion odometry observations, thus obtaining the final global pose of the lawnmower. This embodiment effectively reduces the dependence of the positioning system on satellite signals, significantly improves the positioning stability in complex courtyard and shady lawn scenarios, and thus improves the overall map construction accuracy.

[0034] Step S12: Perform boundary detection on the multi-view image to obtain the pixel coordinates of the boundary lines, and perform feature classification on the boundary regions corresponding to the pixel coordinates of the boundary lines to obtain the boundary semantic detection results. In this embodiment, pixel-level boundary segmentation is performed on the multi-view image to obtain a binary segmentation map. This binary segmentation map includes pixel labels: lawn pixels and non-lawn pixels. Then, based on the binary segmentation map, the pixel locations at the boundary between lawn and non-lawn areas are extracted to obtain precise boundary line pixel coordinates. Further, semantic classification and recognition are performed on the local image boundary regions corresponding to the boundary line pixel coordinates to obtain boundary semantic detection results. Optionally, considering the actual working environment of the courtyard lawn, the boundary semantic detection results are categorized into boundaries between lawn and paved roads or driveways, lawn and walls or fences, and lawn and flower beds or planting beds, etc.

[0035] In one feasible embodiment, step S12 includes: Step S121: Input the multi-view image into a dual-head neural network model, and extract features from the multi-view image through the backbone network in the dual-head neural network model to obtain multi-scale image features; Step S122: The multi-scale image features are fused through the neck network in the dual-head neural network model to obtain fused features; Step S123: The fused features are segmented by the boundary segmentation head in the dual-head neural network model to obtain a binary segmentation map, and the pixel coordinates of the boundary line are extracted from the binary segmentation map; Step S124: The semantic classification head in the dual-head neural network model is used to perform feature classification on the boundary region corresponding to the pixel coordinates of the boundary line to obtain the boundary semantic detection result.

[0036] In this embodiment, the dual-head neural network model includes a backbone network, a neck network, a boundary segmentation head, and a semantic classification head. Optionally, the backbone network uses MobileNetV3-Large (Mobile Network Version 3-Large, a third-generation large-scale lightweight network for mobile devices), and the neck network uses a lightweight BiFPN (Bidirectional Feature Pyramid Network). Multi-view images are input into the dual-head neural network model, and the backbone network in the model performs global feature extraction on the multi-view images to obtain multi-scale image features. Optionally, differentiated input resolutions can be set for images captured by different cameras. The input resolution for images captured by the front-view camera is 640×480, while the input resolution for images captured by the left and right cameras is reduced to 480×360. This effectively reduces the computational load of network inference and improves the real-time inference speed at the lawnmower end, while ensuring that near-ground low-lying boundary details are not lost.

[0037] Furthermore, the multi-scale image features are bidirectionally weighted and fused using a neck network to obtain fused features that simultaneously perceive both near and far boundaries, addressing the problem that single-scale features cannot adapt to boundary detection at different distances. Then, a boundary segmentation head performs pixel-level segmentation on the fused features, outputting a binary segmentation map with the same resolution as the input multi-view image. The binary segmentation map contains two types of pixel labels: grass pixels and non-grass pixels. Optionally, post-processing can be performed on the binary segmentation map to remove image noise interference, ultimately extracting the contour pixels at the boundary between grass and non-grass areas, which are the boundary line pixel coordinates. In addition, a semantic classification head performs feature classification on the boundary regions corresponding to the boundary line pixel coordinates to obtain boundary semantic detection results, which include boundary semantic category and boundary semantic category confidence.

[0038] The dual-head neural network in this embodiment adopts a structure design with a shared backbone network and a shared neck network, and parallel output from dual detection heads. It eliminates the need for repeated extraction of image features, and while ensuring the accuracy of boundary detection and semantic classification, it further reduces the overall network inference time, making it suitable for the low-computing-power operating environment of lawnmower embedded devices.

[0039] Step S13: Construct a semantic grid map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine body.

[0040] In this embodiment, the pixel coordinates of the boundary line are projected onto the ground plane coordinate system using a pre-constructed homography transformation matrix to obtain the ground coordinates. The homography transformation matrix is ​​determined based on the camera's installation height and pitch angle, enabling rapid conversion from pixel coordinates to local physical coordinates on the real ground. Furthermore, using the lawnmower's real-time global pose as the conversion reference, the ground coordinates are further converted to physical boundary coordinates in the global map coordinate system. The ground plane coordinate system is a local coordinate system with the lawnmower's center as its origin. The X-axis corresponds to the machine's lateral direction, and the Y-axis corresponds to the machine's forward direction, representing the local position of the boundary relative to the machine body, and changing in real-time with the machine's movement. The global map coordinate system is an absolute geodetic coordinate system established based on the satellite positioning module; the origin and coordinate axis directions of the global map coordinate system do not shift with the machine's movement or changes in the camera's viewing angle.

[0041] Furthermore, the entire lawn operation area is pre-divided into uniformly discrete grids according to a preset grid unit size. Based on the physical boundary coordinates, the corresponding grids are calculated and matched point by point. Then, the boundary semantic category and boundary semantic category confidence score from the boundary semantic detection results are associated and bound to the corresponding matching grid, so that the grid simultaneously possesses semantic information such as boundary type and classification confidence score. Further, the lawnmower continuously operates along the lawn boundary, repeating the image acquisition, coordinate transformation, grid matching, and semantic binding process in real time, incrementally iteratively updating the grid data, gradually stitching together and integrating all valid grids, and finally generating a semantic grid map covering the entire operation area of ​​the machine.

[0042] Furthermore, since there is overlap between adjacent cameras such as front-view and side-view cameras, the same real physical boundary can be observed and captured by multiple cameras. Therefore, the angle of incidence of light is introduced as a weighting criterion to perform weighted fusion and update of the semantic information of the same grid output by multiple cameras. The angle of incidence of light refers to the angle at which ambient natural light strikes the camera lens. The larger the angle of incidence, the more severe the interference from direct sunlight, backlighting, and glare on the lens, resulting in lower detection accuracy for a single camera and a smaller weight. Conversely, the smaller the angle of incidence, the clearer the image without light interference, the more reliable the detection result, and the larger the weight. The angle of incidence of light for each camera is calculated in real time and normalized to generate the fusion weight for each camera. Then, based on the fusion weight, the confidence scores of the boundary semantic categories within the same grid are weighted and summed to obtain the final boundary semantic detection result for that grid. This weakens the detection error under adverse lighting conditions such as direct sunlight and backlighting, improving the accuracy of semantic classification.

[0043] Furthermore, it should be noted that to address the issues of long single-machine mapping time and low efficiency in mapping large outdoor lawn areas, multiple lawnmowers can be simultaneously dispatched. Each lawnmower starts from a different initial point within the work area, independently performing local edge mapping tasks. After completing local mapping, each lawnmower uploads a local semantic raster map with global absolute coordinate information to a cloud server in real time, based on its satellite positioning module. The cloud server uses a unified global map coordinate system as a reference to stitch together the local semantic raster maps uploaded by all lawnmowers, generating a global work area map covering the entire lawn work area. This final unified global work area map is then synchronously distributed to all lawnmowers, significantly improving the overall efficiency of mowing large lawns.

[0044] This embodiment can fully cover the lawn boundary around the lawn mower by acquiring multi-view images, eliminating the perception blind spots of the single-camera solution; when the lawn mower moves autonomously along the edge, it can simultaneously complete the construction of the work area map without the need for manual surveying by the user; at the same time, it can perform semantic classification of the lawn boundary and write the semantic information of various boundaries into the raster map.

[0045] In one feasible embodiment, after step S13, the method further includes: Step A11: Determine the actual physical distance from the lawnmower to the boundary based on the ground coordinates; Step A12: Based on the actual physical distance and the boundary semantic detection results, perform edge-travel control on the lawnmower.

[0046] In this embodiment, the horizontal coordinate value in the ground coordinate system is used as the actual physical distance from the lawnmower to the boundary. The target edge distance associated with the boundary semantic detection result is obtained by querying a preset strategy table. Furthermore, based on the actual physical distance and the target edge distance, the lawnmower's heading and speed are dynamically adjusted through a closed-loop control algorithm to achieve adaptive edge-following.

[0047] In a specific example, the preset strategy table is as follows:

[0048] The formula for the closed-loop control algorithm is as follows:

[0049] in, K is the control output of the closed-loop control algorithm, used to adjust the lawnmower's heading and speed; p K is a preset proportionality coefficient. i K is the preset integral coefficient. d t is the preset differential coefficient; e(t) is the distance error value.

[0050] The distance error value is calculated as follows: e(t) = d_target - d_measured, where d_target is the distance along the edge of the target and d_measured is the actual physical distance.

[0051] During operation, a distance error value is generated based on the target edge distance and the actual physical distance. The lawnmower's driving posture is continuously corrected through a closed-loop control algorithm, ensuring that the lawnmower always travels stably along the edge at a safe distance from the corresponding semantic boundary, avoiding collisions with various boundary obstacles and improving the overall quality of edge operation.

[0052] Furthermore, this embodiment can also add boundary corner prediction and adaptive corner control logic to adapt to the complex and ever-changing working conditions of lawn boundary corners. Specifically: during the lawnmower's movement along the edge, if the front-view image captured by the front-view camera detects a boundary ahead, and the left or right image simultaneously detects a lateral boundary, it can be determined that the lawnmower has reached the boundary corner position. Combining the boundary contour shape detected by multi-view images, the corner type is predicted in advance. Corner types include right-angle corners, obtuse-angle corners, and concave obstacle corners, etc., and then differentiated edge-moving control is performed according to different corner types. Optionally, for right-angle corners: the system controls the lawnmower to reduce its speed to 30% of the rated speed in advance, and performs a stationary turning action after reaching the corner position. After the turning is completed, it continues to follow the preset target edge distance and resumes normal edge-moving state. For obtuse-angle corners: there is no need to stop in place; the lawnmower is controlled to decelerate and turn in a gentle arc, maintaining continuous movement of the machine without stopping throughout the process, ensuring the continuity of edge-moving and improving work efficiency. For concave obstacle corners: the system immediately exits the regular edge-tracing path, performs a local obstacle avoidance maneuver, avoids protruding obstacles, and quickly returns to the original edge-tracing path to continue subsequent edge-tracing operations. This embodiment, by implementing differentiated edge-tracing control based on different corner types, effectively solves problems such as the inability of a single visual perspective to identify boundary corners in advance, the ease of hitting edges at corners, path discontinuity, and abrupt changes in edge-tracing trajectory, further improving the smoothness of edge-tracing and operational safety under complex irregular lawn boundaries.

[0053] In one feasible embodiment, after step S13, the method further includes: Step B11: Obtain the real-time boundary observation values ​​detected by the lawnmower during operation; Step B12: Determine the spatial deviation rate between the real-time boundary observation and the boundary position in the semantic raster map; Step B13: Update the semantic raster map according to the spatial deviation rate.

[0054] In this embodiment, throughout the entire lawnmower operation, the camera continuously captures environmental images. Based on these images, a dual-head neural network continuously outputs real-time boundary pixel coordinates, which are then uniformly converted to the global map coordinate system to obtain real-time boundary observations. This enables uninterrupted dynamic perception of environmental boundaries throughout the operation. Furthermore, the real-time boundary observations are matched and compared with historical physical boundary coordinates within the semantic raster map. The spatial deviation rate between the two is calculated to quantify the degree of offset of the lawn boundaries, obstacles, and other environmental features relative to the semantic raster map. If the spatial deviation rate is less than a preset redraw threshold, the corresponding raster area is located based on the offset physical boundary coordinates, and the raster in that area is updated without clearing the entire semantic raster map and reconstructing the entire map.

[0055] This embodiment acquires real-time boundary observations detected by the lawnmower during operation; determines the spatial deviation rate between the real-time boundary observations and the boundary positions in the semantic raster map; and updates the semantic raster map based on the spatial deviation rate, thereby achieving long-term adaptive evolution of the semantic raster map and ensuring that the map always remains consistent with the actual working environment, thus guaranteeing the accuracy of subsequent path planning and edge driving.

[0056] In one feasible implementation, the dual-head neural network model is trained according to the following steps: Step C11: Obtain the original dataset; In this embodiment, several original images are captured by a camera in a real lawn scene. These original images are then input into a pre-trained visual model, which can be a model such as BiSeNet (Bilateral Segmentation Network) or U-NetU (U-shaped segmentation network). Using the pre-trained visual model's existing target prior features, an initial coarse annotation result is automatically generated for each original image. Subsequently, professional annotators refine the coarse annotation results output by the model to obtain the annotation result corresponding to each original salient. The annotation result includes segmentation mask map labels and boundary category labels.

[0057] Step C12: Perform data augmentation on the original dataset to obtain a training sample set; In this embodiment, the original dataset undergoes perspective transformation, color jittering, lens smudge simulation, motion blur, random cropping and flipping, and shadow simulation to obtain several enhanced images. Furthermore, supervised augmentation processing is performed on original images with insufficient data in categories such as water feature boundaries to obtain several augmented images. Supervised augmentation processing includes image synthesis and style transfer. For example, based on labeled lawn segmentation maps, a generative adversarial network is used to synthesize different types of boundary textures; existing lawn boundary samples are transferred to different lawn textures and lighting styles to expand data diversity. Further, a training sample set is formed based on each original image, each enhanced image, and each synthesized image.

[0058] Step C13: Iteratively train the training samples in the training sample set and the segmentation mask map labels associated with the training samples to obtain the initial model; It should be noted that the initial model includes a backbone network, a neck network, and a boundary segmentation head.

[0059] In this embodiment, training samples from the training sample set are input into the initial model. The initial model detects and outputs a segmentation mask. Based on the segmentation mask and the segmentation mask labels associated with the training samples, a target loss value is calculated, for example, using the cross-entropy loss function. The parameters of the initial model are then optimized based on the target loss value until a preset number of iterations is reached, resulting in the initial model at the end of training. It should be noted that during training, all network parameters of the four inverted residual modules at the front end of the backbone network in the initial model are frozen, while the parameters of all remaining network layers at the back end of the backbone network are unfrozen. Specifically, this includes the remaining post-inverted residual modules, the global average pooling layer, the channel attention layer, and the convolutional layer at the end of the backbone network.

[0060] Step S24: The training samples in the training sample set and the segmentation labels and boundary category labels associated with the training samples are used to perform dual-task joint training on the target model to be trained, thereby obtaining the dual-head neural network model. The target model to be trained includes a semantic classification head and an initial model that has been preliminarily trained.

[0061] In this embodiment, all network parameters of the backbone network are unfrozen, and a semantic classification head is added to the original network structure of the model to conduct joint training of boundary segmentation and boundary classification tasks. The specific training process is as follows: Training samples from the training sample set are input into the target model to be trained. The backbone network extracts features from the training samples to obtain multi-scale feature extraction results. The neck network fuses the multi-scale feature extraction results to obtain fused feature samples. The boundary segmentation head performs boundary segmentation on the fused feature samples to obtain a segmentation mask image, and extracts the boundary line pixel coordinates from the segmentation mask image. The semantic classification head performs feature classification on the boundary regions corresponding to the boundary line pixel coordinates to obtain the boundary type. The classification loss is calculated based on the boundary type and boundary category label, and the segmentation loss is calculated based on the segmentation mask image and segmentation label. The total loss value is calculated by weighting the classification loss and segmentation loss according to the preset weight coefficients. Based on the total loss value, the target model to be trained is subjected to dual-task joint training to obtain the dual-head neural network model. The total loss value is calculated as follows: total loss value = λ1 × segmentation loss + λ2 × classification loss. λ1 and λ2 can be set according to the actual situation, for example, λ1 = 1.0 and λ2 = 0.5.

[0062] In another optional embodiment, for paired image samples of the same scene simultaneously acquired by the left and right cameras in the three-camera setup, an additional multi-view consistency loss is introduced to further optimize the model's generalization ability. The multi-view consistency loss uses the Kullback-Leibler (KL) divergence loss function to implement consistency constraints. KL divergence measures the degree of difference between two probability distributions, accurately quantifying the magnitude of the deviation in the model's prediction results under both left and right views. The larger the deviation, the higher the loss value, and the stronger the penalty imposed on the model. The core constraint logic of this loss function is: to force the left and right camera branch networks to output completely identical boundary categories for the same physical boundary with consistent spatial locations; if there is a deviation in the predicted probability distribution of the boundary categories corresponding to the left and right images, the KL divergence loss function outputs a non-zero loss value, applying a reverse penalty to the model parameters. This allows the model to learn general boundary semantic features unaffected by shooting perspective, imaging angle, or subtle differences in the image, eliminating the prediction inconsistency problem caused by binocular vision imaging angle deviation, and further improving the stability and robustness of model predictions in complex outdoor lawn scenes.

[0063] This embodiment utilizes training samples from the training sample set and the segmentation mask map labels associated with the training samples to conduct single-task iterative training to obtain an initial model. This initial model consists of a backbone network, a neck network, and a boundary segmentation head, enabling the model to perform basic lawn boundary pixel segmentation. After completing the initial model pre-training, a semantic classification head is added to the initial model to construct the target model to be trained. Then, based on the training samples, segmentation mask map labels, and boundary category labels, the target model to be trained undergoes joint training for boundary segmentation and boundary classification tasks. This ultimately yields a dual-head neural network model with both segmentation and classification capabilities, effectively reducing the fitting difficulty of simultaneous training of dual-branch networks, avoiding mutual interference during multi-task training, and significantly improving the stability and environmental robustness of the model's boundary segmentation and classification predictions.

[0064] In one feasible implementation, step S13 includes: Step S131: According to the pre-constructed homography transformation matrix, the pixel coordinates of the boundary line are projected onto the ground plane coordinate system to obtain the ground coordinates; In this embodiment, the boundary pixel coordinates are transformed by coordinate projection according to the pre-calibrated homography transformation matrix, and the boundary pixel coordinates in the image pixel coordinate system are projected and transformed into the robot ground plane coordinate system to obtain the ground coordinates of the corresponding boundary.

[0065] It should be noted that the homography transformation matrix is ​​the transformation matrix between the image pixel coordinate system and the lawnmower's ground plane coordinate system. This homography transformation matrix is ​​determined based on the camera's installation height and pitch angle. The horizontal coordinate of the ground plane corresponds to the lawnmower's lateral direction, and the vertical coordinate corresponds to the lawnmower's forward direction.

[0066] In a feasible embodiment, the homography transformation matrix is ​​constructed according to the following steps: Step D11: Obtain the calibration image corresponding to the calibration chessboard grid; In this embodiment, the calibration chessboard is placed at a known location on the ground, and the calibration image corresponding to the calibration chessboard is acquired by a camera.

[0067] Step D12: Based on the calibration image, solve for the intrinsic parameter matrix of the camera; In this embodiment, corner detection is performed on the calibration image to extract the pixel corner coordinates. Based on the pixel corner coordinates and the preset physical size parameters of the checkerboard pattern, the camera calibration calculation is completed using the Zhang Zhengyou calibration algorithm to obtain the intrinsic parameter matrix of the camera. The Zhang Zhengyou calibration algorithm is an existing mature algorithm and will not be described in detail here.

[0068] Step D13: Determine the extrinsic parameter matrix from the camera coordinate system to the ground plane coordinate system based on the camera's installation height and tilt angle. In this embodiment, the rotation matrix and translation vector of the camera relative to the ground plane are calculated by combining the camera's installation height and pitch angle. Based on these parameters, the extrinsic parameter matrix from the camera coordinate system to the ground plane coordinate system is determined. The rotation matrix, calculated from the camera's pitch angle, characterizes the camera's attitude deflection relative to the ground; the translation vector, calculated from the camera's installation height above the ground, characterizes the spatial position offset of the camera's optical center relative to the ground.

[0069] Step D14: Construct the homography transformation matrix based on the intrinsic parameter matrix and the extrinsic parameter matrix.

[0070] In this embodiment, the homography transformation matrix is ​​constructed based on the intrinsic parameter matrix and the extrinsic parameter matrix. The formula for calculating the homography transformation matrix is: H_img2ground = K×[R|t], where K is the intrinsic parameter matrix, R is the rotation matrix, and t is the translation vector.

[0071] Furthermore, considering the unevenness of outdoor lawn surfaces and the impact of lawnmower movement on the machine's posture, this embodiment incorporates a dynamic correction mechanism. Specifically, it uses a pre-set inertial measurement unit to detect the lawnmower's pitch and / or roll angles. If a tilt is detected, the homography transformation matrix is ​​dynamically updated based on the lawnmower's pitch and / or roll angles. For example, when the lawnmower is detected to be tilting or its pitch or roll angles deviate from the initial calibrated posture thresholds, the extrinsic parameter matrix of the homography transformation matrix is ​​updated using the real-time feedback of the pitch and roll angles from the inertial measurement unit. This eliminates coordinate transformation errors caused by machine posture swaying, ensuring that the ground coordinate solution remains accurate and reliable even under bumpy conditions.

[0072] Step S132: Perform discrete grid matching on the physical boundary coordinates to obtain a matching grid, and associate the boundary semantic detection result corresponding to the boundary line pixel coordinates with the matching grid; In this embodiment, the entire working area of ​​the lawnmower is uniformly and discretely divided into grids according to a preset grid unit size. The grid unit size can be flexibly set according to the lawnmower's path planning accuracy and operation control accuracy. For example, in this embodiment, the grid unit size is set to 2cm×2cm.

[0073] Based on the physical boundary coordinates and the preset grid cell size, the grid matching calculation is completed for each boundary point, and the matching grid corresponding to each physical boundary coordinate is quickly located. After completing the matching grid traversal search for all boundary points, the boundary semantic detection results are bound to the corresponding matching grid one by one.

[0074] Step S133: Construct the semantic raster map based on the matching raster of the associated boundary semantic detection results.

[0075] It should be noted that the graticule in the semantic raster map is associated with multiple parameters, including the raster occupancy status, boundary semantic category, semantic category confidence, cumulative number of observations and detections of the raster, and the latest update timestamp of the raster. Optionally, the raster occupancy status is specifically defined as follows: 0 represents a free cutable area, 1 represents a lawn boundary area, 2 represents an obstacle area, and 3 represents an unexplored area.

[0076] In this embodiment, a complete semantic raster map is gradually built by incrementally iterating frame by frame based on the matching rasters of the associated boundary semantic detection results. That is, as the lawnmower travels along the edge, it continuously collects environmental images and iteratively updates the raster information frame by frame. As the travel range continues to expand, effective matching rasters are continuously accumulated, and finally stitched together to form a semantic raster map covering the entire work area.

[0077] In one feasible embodiment, step S133 includes: Step S1331: Perform Bayesian probability update on the matching grid of the associated boundary semantic detection results, and simultaneously accumulate the observation and detection count of the updated matching grid; It should be noted that the boundary semantic detection results of each frame of image output are easily affected by environmental interference such as sudden changes in outdoor lighting, image noise, and weed obstruction, resulting in instantaneous false detections. Directly modifying the raster state based on a single frame detection result can easily lead to map boundary distortion and frequent map jitter. Therefore, in this embodiment, a Bayesian probabilistic iterative update mechanism is introduced to achieve the fusion of multi-frame temporal observation information. Specifically: the historical boundary occupancy probability of the raster in the previous update is used as the prior probability, and the current semantic category confidence is used as the likelihood probability of this observation. The updated posterior boundary occupancy probability is calculated, and this posterior probability is used to cover the original semantic category confidence of the matching raster. Each time a raster probability update is completed, the observation detection count corresponding to the matching raster is incremented by 1.

[0078] Step S1332: If the number of observations and detections of the matching grid is greater than the preset number and the boundary semantic detection results of each detection are consistent, then the matching grid is regarded as the boundary grid. In this embodiment, when the cumulative number of observations of the same matching grid is greater than the preset number, and the boundary semantic detection results of the matching grid output in multiple consecutive frames are consistent, it is determined that the grid does not have any instantaneous false detections, and the matching grid is marked as a confirmed boundary grid. The preset number can be set according to the actual situation, for example, it can be set to 3 times.

[0079] Step S1333: Traverse all boundary grids to form the contour curve; In this embodiment, all marked and confirmed boundary grids are traversed, and neighborhood connectivity retrieval and curve fitting are performed on the discretely distributed boundary grids based on the global coordinates of each boundary grid, so as to connect the discrete grid points to obtain the boundary contour curve.

[0080] Step S1334: If the closure degree of the contour curve is greater than the preset closure degree threshold, then the semantic raster map construction is determined to be complete.

[0081] It should be noted that the preset closure threshold can be set according to the actual situation, for example, set to 95%. In this embodiment, the closure of the fitted contour curve is detected: when the closure of the contour curve is greater than the preset closure threshold, it indicates that the boundary of the lawn operation area is complete, and the semantic raster map is determined to be completed. In addition, when the closure of the contour curve is less than or equal to the preset closure threshold, it indicates that there are gaps in the boundary and local areas have not been explored. Then, the coordinates of the gaps are identified, and the lawnmower is controlled to move to the coordinates of the gaps for supplementary sweeping and observation, thereby constructing a complete semantic raster map.

[0082] Furthermore, it should be noted that the lawn operation area will continuously change due to factors such as vegetation growth and the placement of obstacles by humans. Therefore, the semantic raster map needs to be updated. Specifically: all valid boundary rasters within the current lawnmower's coverage area are traversed, and the current boundary raster's occupancy status, boundary semantic category, and other attribute information are compared with the attribute information of historical boundary rasters in the semantic raster map. If the two rasters have inconsistent occupancy statuses or different semantic category detection results, the raster is determined to be a conflict anomaly. The total number of all conflict anomaly rasters is then counted, and combined with the preset raster unit size, the total conflict area is calculated. Additionally, the total area of ​​all boundary rasters traversed during the current operation is counted, and further, the conflict area ratio is calculated based on the total conflict area and the total area of ​​all boundary rasters. Furthermore, if the conflict area ratio is greater than a preset conflict area ratio threshold, it indicates that the on-site lawn environment and boundary layout have undergone significant changes, and the original semantic raster map can no longer match the actual on-site environment. At this point, the system proactively interrupts the lawnmower's operation and sends an environmental change notification to the user's mobile device, indicating significant environmental changes and requesting a partial map redraw. After the user confirms the redraw command via the mobile app, the lawnmower autonomously plans its route and automatically heads to the area of ​​environmental anomaly. This area refers to the region containing conflicting raster cells. The lawnmower switches to edge-mapping mode, overlaying and fusing the raster occupancy status and boundary semantic categories of the environmental anomaly area into the original semantic raster map, ultimately achieving seamless and highly efficient autonomous map repair and iterative updates.

[0083] This embodiment projects the boundary line pixel coordinates onto the ground plane coordinate system according to a pre-constructed homography transformation matrix to obtain ground coordinates. Then, based on the global pose of the machine, the ground coordinates are converted into physical boundary coordinates in the global map coordinate system. Discrete grid matching is performed on the physical boundary coordinates to obtain matching grids, and the boundary semantic detection results corresponding to the boundary line pixel coordinates are associated with the matching grids. Based on the matching grids with associated boundary semantic detection results, a semantic grid map is constructed. The map construction of the entire work area can be completed simultaneously while the lawnmower is autonomously traveling along the edge, without the need for manual surveying by the user. At the same time, this solution can accurately distinguish different types of lawn boundaries and complete semantic classification, writing the subdivided semantic information of each type of boundary into the grid map. This gives the map not only spatial location information but also rich environmental semantic perception capabilities, providing reliable data support for subsequent accurate edge-traveling, obstacle avoidance, and full-coverage mowing path planning.

[0084] In one feasible embodiment, the entire process of autonomously constructing a semantic raster map is illustrated by way of example, taking into account a complete courtyard operation scenario, as follows: Users can issue autonomous mapping commands through a mobile app. After clicking to start the mapping task, the lawnmower will automatically drive out of the charging station without manual remote control guidance. It will then automatically cruise along the outermost contour of the yard operation area. It achieves full-area environmental collaborative semantic perception based on three cameras: front, left, and right. At the same time, it is equipped with a satellite positioning module and an extended Kalman filter module to complete the global pose calculation of the machine.

[0085] During the edge mapping cruise, the three cameras work together to complete differentiated boundary perception and edge control. For example, the left camera continuously perceives the boundary between the lawn and the flower bed on the left side of the aircraft, outputting the semantic category as lawn-flower bed boundary with a semantic classification confidence of 0.94. It uses a tracking distance of 3cm and a driving speed of 50% to carry out high-precision and refined edge driving. When the aircraft travels to the right corner of the courtyard, the front camera detects the boundary of the hard wall in front in real time, outputting the semantic category as lawn-hard wall boundary. The aircraft immediately decelerates autonomously and completes the heading change. Then, the right camera takes over to complete the real-time perception and edge tracking of the right wall boundary, adapting to the complex boundary shape of the corner.

[0086] During the mapping patrol, assuming the left-side camera is interfered with by accumulated dead leaves on the ground, it misclassifies the area of ​​dead leaves as a lawn-flower bed boundary from a single perspective, outputting a classification result of lawn-flower bed boundary with a semantic confidence of only 0.61. However, in the same area, the front-side camera, through multi-angle observation from the side, outputs the correct classification result of lawn-hardscape boundary with a semantic confidence of 0.78. When the two cameras output conflicting semantic classification results for the same spatial grid, a multi-view perception weighted arbitration mechanism is immediately activated. The semantic confidence of the two detection results is then weighted and fused by voting to determine the boundary semantic category based on the overall confidence of the fused results.

[0087] Furthermore, during edge-movement, if the front-facing camera detects a lawn-water feature boundary 1.5m directly in front of the device from a distance, with a semantic confidence level as high as 0.96, the system determines this area to be a high-risk restricted area. The edge-movement control module immediately executes a safety emergency strategy, instantly locking the walking drive motor to achieve an emergency stop. Simultaneously, it pushes a safety alarm message to the user's mobile app via wireless communication: a dangerous water feature boundary has been detected, and the device has stopped urgently. At the same time, it can automatically generate a 30cm radius safety restricted area in the global grid map around the water feature boundary, completing the grid attribute labeling of the dangerous area.

[0088] After the user confirms the alarm information and issues a command to continue mapping via the app, the lawnmower autonomously plans an obstacle avoidance path, bypassing the restricted areas around the water feature, and continues to complete the edge detection and raster mapping work for the remaining areas. After the entire machine travels a full circle along the boundary of the entire area, the closure degree of the fitted boundary contour is calculated. The measured boundary contour closure degree is 98.5%, which is higher than the preset closure degree threshold of 95%, and the system automatically determines that the semantic raster map of the entire area has been completed. When the lawnmower performs full-coverage mowing operations in the future, it can automatically avoid the restricted buffer zone based on this semantic raster map. The entire process does not require manual surveying, manual point marking, or map correction by the user, realizing one-click fully automatic semantic mapping and intelligent operation.

[0089] It should be noted that the examples in the figure are only for understanding this application and do not constitute a limitation on the map construction method of this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0090] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0091] This application also provides a map building apparatus, please refer to... Figure 2 , Figure 2 This is a schematic diagram of the module structure of the map building device according to an embodiment of this application; the map building device includes: The acquisition module 21 is used to acquire multi-view images collected during the operation of the lawnmower and to detect the global pose of the lawnmower. The detection module 22 is used to perform boundary detection on multi-view images, obtain the pixel coordinates of the boundary lines, and perform feature classification on the boundary regions corresponding to the pixel coordinates of the boundary lines to obtain the boundary semantic detection results. The construction module 23 is used to construct a semantic raster map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine body.

[0092] The detection module 22 is also used for: The multi-view images are input into a dual-head neural network model, and the backbone network in the dual-head neural network model is used to extract features from the multi-view images to obtain multi-scale image features. The multi-scale image features are fused using the neck network in the dual-head neural network model to obtain fused features; The fused features are segmented by the boundary segmentation head in the dual-head neural network model to obtain a binary segmentation map, and the pixel coordinates of the boundary line are extracted from the binary segmentation map. The semantic classification head in the dual-head neural network model performs feature classification on the boundary region corresponding to the pixel coordinates of the boundary line to obtain the boundary semantic detection result.

[0093] The map building device also includes: The dataset acquisition module is used to acquire the original dataset; The data augmentation module is used to perform data augmentation processing on the original dataset to obtain a training sample set; The first training module is used to iteratively train the training samples in the training sample set and the segmentation mask map labels associated with the training samples to obtain an initial model, wherein the initial model includes a backbone network, a neck network, and a boundary segmentation head; The second training module is used to perform dual-task joint training on the training samples in the training sample set and the segmentation mask map labels and boundary category labels associated with the training samples to obtain the dual-head neural network model. The target model to be trained includes a semantic classification head and an initial model that has been preliminarily trained.

[0094] The multi-view images include a front view image; the map building device also includes: The comparison module is used to compare the pixel grayscale between two adjacent frames of the acquired front view image to obtain the visual distance observation value. The observation acquisition module is used to acquire the positioning observations detected by the satellite positioning module and the body motion mileage observations detected by the wheel encoder; The fusion positioning module is used to obtain the global pose of the lawnmower by fusing the positioning observation value, the visual mileage observation value and the body motion mileage observation value.

[0095] Module 23 is also used for: According to the pre-constructed homography transformation matrix, the pixel coordinates of the boundary line are projected onto the ground plane coordinate system to obtain the ground coordinates. The homography transformation matrix is ​​determined based on the camera's installation height and pitch angle. Based on the global pose of the machine body, the ground coordinates are converted into physical boundary coordinates of the global map coordinate system; Discrete grid matching is performed on the physical boundary coordinates to obtain a matching grid, and the boundary semantic detection results corresponding to the boundary line pixel coordinates are associated with the matching grid. The semantic raster map is constructed based on the matching raster of the associated boundary semantic detection results.

[0096] Module 23 is also used for: Perform Bayesian probability update on the matching grid of the associated boundary semantic detection results, and simultaneously accumulate the observation and detection count of the updated matching grid; If the number of observations and detections of the matching grid is greater than the preset number and the boundary semantic detection results of each detection are consistent, then the matching grid is regarded as the boundary grid. Traverse all boundary grids to form the outline curve; If the closure degree of the contour curve is greater than the preset closure degree threshold, then the semantic raster map construction is determined to be complete.

[0097] The map building device also includes: The calibration image module is used to acquire the calibration image corresponding to the calibration chessboard grid; The solution module is used to solve for the intrinsic parameter matrix of the camera based on the calibration image; The extrinsic parameter matrix determination module is used to determine the extrinsic parameter matrix from the camera coordinate system to the ground plane coordinate system based on the camera's installation height and pitch angle. The transformation matrix construction module is used to construct the homography transformation matrix based on the intrinsic parameter matrix and the extrinsic parameter matrix.

[0098] The map building device also includes: An angle detection module is used to detect the pitch angle and / or roll angle of the lawnmower through a preset inertial measurement unit; The first update module is used to dynamically update the homography transformation matrix based on the pitch angle and / or roll angle of the lawnmower if the lawnmower is detected to be tilting.

[0099] The map building device also includes: The physical distance determination module is used to determine the actual physical distance from the lawnmower to the boundary based on the ground coordinates. The driving control module is used to perform edge-travel control on the lawnmower based on the actual physical distance and the boundary semantic detection results.

[0100] The map building device also includes: The real-time boundary observation acquisition module is used to acquire the real-time boundary observation values ​​detected by the lawnmower during operation. A spatial deviation rate determination module is used to determine the spatial deviation rate between the real-time boundary observation value and the boundary position in the semantic raster map; The second update module is used to update the semantic raster map according to the spatial deviation rate.

[0101] The map building apparatus provided in this application, employing the map building method in the above embodiments, can solve the technical problems mentioned in the background art. Compared with the prior art, the beneficial effects of the map building apparatus provided in this application are the same as those of the map building method provided in the above embodiments, and other technical features in the map building apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0102] This application provides a controller, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps of the map construction method described above.

[0103] This application also proposes a lawnmower, referring to... Figure 3 , Figure 3 This is a schematic diagram of the equipment structure involved in the map construction method in this application embodiment. The lawnmower includes a camera array and a controller. The controller is electrically connected to the camera array. The camera array is used to acquire multi-view images and consists of at least three independent cameras.

[0104] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the map construction method in the above embodiments.

[0105] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0106] The aforementioned computer-readable storage medium may be included in the lawnmower or may exist independently and not assembled into the lawnmower.

[0107] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the lawnmower, cause the lawnmower to: acquire multi-view images collected during its operation and detect the global pose of the lawnmower; perform boundary detection on the multi-view images to obtain boundary line pixel coordinates, and perform feature classification on the boundary regions corresponding to the boundary line pixel coordinates to obtain boundary semantic detection results; and construct a semantic grid map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the lawnmower.

[0108] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0110] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0111] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described map construction method, and is capable of solving the technical problems described in the background art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the map construction method provided in the above embodiments, and will not be repeated here.

[0112] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the map construction method described above.

[0113] The computer program product provided in this application can solve the technical problems described in the background section. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of this application are the same as the beneficial effects of the map construction method provided in the above embodiments, and will not be repeated here.

[0114] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A map construction method, characterized in that, include: Acquire multi-view images of the lawnmower during its operation and detect the global pose of the lawnmower. Boundary detection is performed on multi-view images to obtain the pixel coordinates of the boundary lines, and feature classification is performed on the boundary regions corresponding to the pixel coordinates of the boundary lines to obtain the boundary semantic detection results. A semantic raster map is constructed based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine.

2. The map construction method as described in claim 1, characterized in that, The process of performing boundary detection on multi-view images to obtain boundary line pixel coordinates, and then classifying the boundary regions corresponding to the boundary line pixel coordinates to obtain boundary semantic detection results includes: The multi-view images are input into a dual-head neural network model, and the backbone network in the dual-head neural network model is used to extract features from the multi-view images to obtain multi-scale image features. The multi-scale image features are fused using the neck network in the dual-head neural network model to obtain fused features; The fused features are segmented by the boundary segmentation head in the dual-head neural network model to obtain a binary segmentation map, and the pixel coordinates of the boundary line are extracted from the binary segmentation map. The semantic classification head in the dual-head neural network model performs feature classification on the boundary region corresponding to the pixel coordinates of the boundary line to obtain the boundary semantic detection result.

3. The map construction method as described in claim 2, characterized in that, The dual-head neural network model is trained according to the following steps: Obtain the original dataset; The original dataset is augmented to obtain a training sample set; The initial model is obtained by iteratively training the training samples in the training sample set and the segmentation mask map labels associated with the training samples. The initial model includes a backbone network, a neck network, and a boundary segmentation head. The training samples in the training sample set, along with the segmentation mask map labels and boundary category labels associated with the training samples, are used to perform dual-task joint training on the target model to be trained, thereby obtaining the dual-head neural network model. The target model to be trained includes a semantic classification head and an initial model that has been preliminarily trained.

4. The map construction method as described in claim 1, characterized in that, The multi-view images include a front view image; the detection of the lawnmower's global pose includes: Based on the acquired front view images, the pixel grayscale values ​​between two adjacent frames are compared to obtain the visual distance observation values. Acquire positioning observations detected by the satellite positioning module and body motion mileage observations detected by the wheel encoder; The global pose of the lawnmower is obtained by fusing the positioning observations, visual mileage observations, and body motion mileage observations.

5. The map construction method as described in claim 1, characterized in that, The step of constructing a semantic raster map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine includes: According to the pre-constructed homography transformation matrix, the pixel coordinates of the boundary line are projected onto the ground plane coordinate system to obtain the ground coordinates. The homography transformation matrix is ​​determined based on the camera's installation height and pitch angle. Based on the global pose of the machine body, the ground coordinates are converted into physical boundary coordinates of the global map coordinate system; Discrete grid matching is performed on the physical boundary coordinates to obtain a matching grid, and the boundary semantic detection results corresponding to the boundary line pixel coordinates are associated with the matching grid. The semantic raster map is constructed based on the matching raster of the associated boundary semantic detection results.

6. The map construction method as described in claim 5, characterized in that, The step of constructing the semantic raster map based on the matching raster of the associated boundary semantic detection results includes: Perform Bayesian probability update on the matching grid of the associated boundary semantic detection results, and simultaneously accumulate the observation and detection count of the updated matching grid; If the number of observations and detections of the matching grid is greater than the preset number and the boundary semantic detection results of each detection are consistent, then the matching grid is regarded as the boundary grid. Traverse all boundary grids to form the outline curve; If the closure degree of the contour curve is greater than the preset closure degree threshold, then the semantic raster map construction is determined to be complete.

7. The map construction method as described in claim 5, characterized in that, The homography transformation matrix is ​​constructed according to the following steps: Obtain the calibration image corresponding to the calibration chessboard grid; Based on the calibration image, the intrinsic parameter matrix of the camera is obtained; Based on the camera's installation height and pitch angle, determine the extrinsic parameter matrix from the camera coordinate system to the ground plane coordinate system; The homography transformation matrix is ​​constructed based on the intrinsic parameter matrix and the extrinsic parameter matrix.

8. The map construction method as described in claim 7, characterized in that, After constructing the homography transformation matrix based on the intrinsic parameter matrix and the extrinsic parameter matrix, the process further includes: The pitch and / or roll angles of the lawnmower are detected by a preset inertial measurement unit. If a tilting of the lawnmower is detected, the homography transformation matrix is ​​dynamically updated based on the lawnmower's pitch angle and / or roll angle.

9. The map construction method as described in claim 5, characterized in that, After projecting the boundary line pixel coordinates onto the ground plane coordinate system according to the pre-constructed homography transformation matrix to obtain the ground coordinates, the method further includes: Based on the ground coordinates, determine the actual physical distance from the lawnmower to the boundary; Based on the actual physical distance and the boundary semantic detection results, the lawnmower is controlled to travel along the edge.

10. The map construction method as described in claim 1, characterized in that, After constructing the semantic raster map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine, the method further includes: Acquire real-time boundary observation values ​​detected by the lawnmower during its operation; Determine the spatial deviation rate between the real-time boundary observations and the boundary locations in the semantic raster map; The semantic raster map is updated based on the spatial deviation rate.

11. A map building device, characterized in that, The map building device includes: The acquisition module is used to acquire multi-view images collected during the operation of the lawnmower and to detect the global pose of the lawnmower. The detection module is used to perform boundary detection on multi-view images, obtain the pixel coordinates of the boundary lines, and perform feature classification on the boundary regions corresponding to the pixel coordinates of the boundary lines to obtain the boundary semantic detection results. The construction module is used to construct a semantic raster map based on the boundary line pixel coordinates, the boundary semantic detection results, and the global pose of the machine body.

12. A controller comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the map construction method according to any one of claims 1 to 10.

13. A lawnmower, characterized in that, Includes a camera array and a controller as described in claim 11, wherein the controller is electrically connected to the camera array. The camera array is used to acquire multi-view images, and the camera array consists of at least three independent cameras.

Images