Mapping method and device, equipment and storage medium

Abstract

The invention discloses a mapping method, device and equipment and a storage medium, relates to the technical field of mapping, is applied to mowers, and comprises the following steps: processing an environment image of a currently collected lawn area through a target student model to obtain a lattice diagram; the target student model is a model obtained by performing knowledge distillation operation based on a target loss function in advance, and the target loss function is a loss function constructed based on cross entropy loss, distillation loss and boundary perception loss; planning a current path according to the boundary characteristics corresponding to the lattice diagram, advancing according to the current path, and then skipping to a step of processing the environment image of the currently collected lawn area through the target student model; and constructing a global map on the basis of the lattice diagrams corresponding to the mower until a preset termination condition is met. In this way, through the knowledge distillation processing model, on the premise that the mapping effect of the mower is not affected, the parameter quantity of the model can be obviously reduced, and the requirement for the hardware cost of the mower is lowered.

Description

Technical Field

[0001] The present invention relates to the field of mapping technology, and in particular to a mapping method, device, equipment and storage medium. Background Art

[0002] Current mapping technologies for lawn mowing robots are mainly divided into the following three categories: Preset boundary line technology: The working area is defined by burying physical wires or magnetic boundary lines, but the installation and maintenance are complicated, and it is difficult to adapt to terrain changes (such as adding flower beds, planting trees, etc.). Visual marker guidance technology: Visual markers (such as ribbons and poles) need to be laid out on the lawn, and guide lines are formed by identifying markers to connect multiple areas. This method relies on manual deployment, has poor flexibility, and the markers are easily affected by environmental interference (such as occlusion and fading). Historical path learning technology: Common routes are generated based on multiple driving data, but they cannot cope with dynamic environments (such as temporary obstacles) and require long-term data accumulation, or the user needs to remotely control the machine to walk along the boundary to record the boundary as a reference for subsequent automatic mowing tasks.

[0003] In addition, existing visual mapping solutions mostly rely on switching between guide lines between internal and external boundaries. This lacks accuracy in recognizing complex vegetation boundaries (such as mixed areas of lawn and gravel roads) and requires manual intervention to handle equipment. Furthermore, cameras for vision-based positioning and mapping solutions such as Vslam (Visual Simultaneous Localization and Mapping) and VIO (Visual Inertial Odometry) require a more advanced hardware platform, which increases product costs.

[0004] It can be seen that how to reduce the cost of lawn mower mapping while ensuring the mapping effect of the lawn mower is a problem to be solved in this field. Summary of the Invention

[0005] In view of this, the present invention aims to provide a mapping method, apparatus, device, and storage medium that can significantly reduce the number of model parameters and the hardware cost requirements for the lawn mower without affecting the mapping effect of the lawn mower. The specific solution is as follows:

[0006] In a first aspect, the present application provides a mapping method applied to a lawn mower, comprising:

[0007] The target student model is used to process the currently captured environmental image of the lawn area to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by performing a knowledge distillation operation based on a target loss function in advance, the target loss function is a loss function constructed based on cross entropy loss, distillation loss, and boundary perception loss, and the distillation loss is a loss function constructed based on KL divergence loss and normalization loss between model feature maps;

[0008] Planning a current path according to the boundary features corresponding to the dot matrix, and moving along the current path, and then jumping to the step of processing the environmental image currently collected of the lawn area by the target student model;

[0009] Until the current path meets a preset termination condition, a global map corresponding to the lawn area is constructed based on the dot maps corresponding to the lawn mower.

[0010] Optionally, performing the knowledge distillation operation based on the target loss function includes:

[0011] Transfer the knowledge of the pre-set teacher model to the initial student model through the knowledge distillation operation;

[0012] The initial student model is trained in a first phase based on the cross entropy loss and the distillation loss to obtain a trained student model; the first phase is the first several training cycles in the preset training cycle;

[0013] Based on KL divergence loss and boundary perception loss, combined with the teacher model, the trained student model is trained in the second stage to obtain the target student model; the second stage is the training cycle in the preset training cycle except the first stage.

[0014] Optionally, the step of processing the currently captured environmental image of the lawn area by the target student model to obtain a dot matrix representing the grass area and the non-grass area includes:

[0015] The lawn area is imaged through its own camera to obtain the corresponding environmental image;

[0016] Inputting the environment image into a target student model to generate a semantic segmentation mask representing grass areas and non-grass areas;

[0017] According to the positions of the grass area and the non-grass area in the semantic segmentation mask, a dot map representing the grass area and the non-grass area is generated through a camera projection transformation operation.

[0018] Optionally, after generating the semantic segmentation mask representing the grass area and the non-grass area, the method further includes:

[0019] performing binarization processing on the semantic segmentation mask based on the grass area and the non-grass area to obtain a first processed mask;

[0020] The first processed mask is processed by morphological operations to obtain a second processed mask, so as to generate a bitmap representing the grass area and the non-grass area through a camera projection transformation operation according to the positions of the grass area and the non-grass area in the second processed mask.

[0021] Optionally, planning the current path according to the boundary features corresponding to the dot matrix includes:

[0022] If the dot matrix indicates that there is a single non-grass area, a target boundary line between the grass area and the non-grass area is selected by a greedy algorithm based on its own real-time position;

[0023] Planning a current path based on the target boundary line and a preset direction of travel; the preset direction of travel is clockwise or counterclockwise;

[0024] If the dot map indicates that there are multiple continuous non-grass areas, the shortest path from the real-time position to the next boundary line is determined as the current path;

[0025] If the dot matrix indicates the presence of multiple non-continuous non-grass areas, the current path is planned to jump from its own real-time position to the next boundary line based on a preset path planning algorithm; the preset path planning algorithm is an algorithm constructed based on the ratio of grass areas and non-grass areas.

[0026] Optionally, until the current path meets a preset termination condition, constructing a global map corresponding to the lawn area based on each dot map corresponding to the lawn mower includes:

[0027] If the current path coincides with the starting point of the lawn mower entering the lawn area, it is determined that the current path meets the preset termination condition;

[0028] The dot patterns obtained by the lawn mower during its movement are determined, and a global map corresponding to the lawn area is constructed using boundary features corresponding to the dot patterns.

[0029] Optionally, after constructing the global map corresponding to the lawn area, the method further includes:

[0030] If the automatic operation function corresponding to the lawn mower is set to no, then return to the charging pile location corresponding to the lawn mower and enter the standby state;

[0031] If the automatic operation function corresponding to the lawn mower is set to yes, the lawn mowing task is performed on the lawn area according to the global map, and after completing the mowing task corresponding to the lawn area, the lawn mower returns to the charging pile position and enters the standby state.

[0032] In a second aspect, the present application provides a mapping device for use with a lawn mower, comprising:

[0033] An image processing module is configured to process the currently captured environmental image of the lawn area using a target student model to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by pre-performing a knowledge distillation operation based on a target loss function, wherein the target loss function is a loss function constructed based on cross entropy loss, distillation loss, and boundary perception loss, and the distillation loss is a loss function constructed based on KL divergence loss and a normalized loss between the model feature map;

[0034] a path planning module, configured to plan a current path according to the boundary features corresponding to the dot matrix, travel along the current path, and then jump to the step of processing the environmental image currently collected of the lawn area by the target student model;

[0035] The mapping module is used to construct a global map corresponding to the lawn area based on the dot maps corresponding to the lawn mower until the current path meets the preset termination condition.

[0036] In a third aspect, the present application provides an electronic device, comprising:

[0037] Memory, used to store computer programs;

[0038] A processor is used to execute the computer program to implement the mapping method as described above.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which implements the mapping method described above when executed by a processor.

[0040] It can be seen that in this application, the lawn mower first processes the environmental image currently collected in the lawn area through the target student model to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by performing a knowledge distillation operation based on the target loss function in advance, and the target loss function is a loss function constructed based on cross entropy loss, distillation loss and boundary perception loss, and the distillation loss is a loss function constructed based on the normalization loss between the KL divergence loss and the model feature map; then the current path is planned according to the boundary features corresponding to the dot map, and the current path is followed, and then the step of processing the environmental image currently collected in the lawn area through the target student model is jumped to; until the current path meets the preset termination condition, a global map corresponding to the lawn area is constructed based on the dot maps corresponding to the lawn mower. In this way, this solution reduces the number of model parameters through knowledge distillation, and combined with the loss function related to the knowledge distillation operation, it can ensure that the performance of the student model used is almost consistent with that of the corresponding teacher model. By processing the collected environmental images through the student model, grass areas and non-grass areas can be effectively identified. It can be seen that without affecting the mapping effect of the lawn mower, this solution can significantly reduce the number of model parameters and reduce the hardware cost requirements for the lawn mower. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 This is a flow chart of a mapping method disclosed in this application;

[0043] Figure 2 This is a specific knowledge distillation operation flow chart disclosed in this application;

[0044] Figure 3 This is a flowchart of a specific mapping method disclosed in this application;

[0045] Figure 4 This is a schematic diagram of a specific lawn mower structure disclosed in this application;

[0046] Figure 5 This is a schematic diagram of a specific visual area of ​​a lawn mower disclosed in this application;

[0047] Figure 6 A schematic diagram of a specific matrix of grass areas and non-grass areas disclosed in this application;

[0048] Figure 7 This is a schematic diagram of a specific grass area and non-grass area effect disclosed in this application;

[0049] Figure 8 This is a schematic diagram of the structure of a mapping device disclosed in this application;

[0050] Figure 9 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] See also Figure 1 As shown, an embodiment of the present invention discloses a mapping method applied to a lawn mower, comprising:

[0053] Step S11: Process the environmental image of the lawn area currently collected through the target student model to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by performing a knowledge distillation operation based on the target loss function in advance, and the target loss function is a loss function constructed based on cross entropy loss, distillation loss and boundary perception loss, and the distillation loss is a loss function constructed based on KL divergence loss and normalized loss between the model feature map.

[0054] In this application, the model used to recognize environmental images within the lawn mower is a student model obtained through pre-processed knowledge distillation. Compared to the original teacher model, the student model has significantly fewer parameters and can be adapted to lightweight hardware. It should be noted that during the knowledge distillation process, task supervision is specifically performed based on an objective loss function, involving cross-entropy loss, boundary-aware loss, and distillation loss. The distillation loss is a loss function constructed based on the KL (Kullback-Leibler) divergence loss and the normalized loss between the model feature maps. This ensures that the performance of the student model used is nearly identical to that of the corresponding teacher model, maintaining model performance as much as possible while significantly reducing the number of parameters. In a specific embodiment, when the model is deployed to the lawn mower, online fine-tuning can be performed to adapt to different lawn environments.

[0055] In a specific embodiment, processing the currently captured environmental image of the lawn area by the target student model to obtain a dot matrix representing the grass and non-grass areas may include: performing an image acquisition operation on the lawn area using its own camera to obtain a corresponding environmental image; inputting the environmental image into the target student model to generate a semantic segmentation mask representing the grass and non-grass areas; and generating a dot matrix representing the grass and non-grass areas through a camera projection transformation operation based on the positions of the grass and non-grass areas in the semantic segmentation mask. Specifically, the lawn mower may perform an image acquisition operation on the lawn area using its own camera to obtain the current environmental image, which may then be submitted to the target student model for image recognition, ultimately outputting a semantic segmentation mask representing the grass and non-grass areas. Furthermore, based on the positions of elements such as grass and obstacles in the semantic segmentation mask, a camera projection transformation operation may be performed to generate a corresponding dot matrix representing the grass and non-grass areas. It is understood that the segmentation model involved in this solution may be DeeplabV3+ (a deep learning model for semantic segmentation).

[0056] In another specific embodiment, after generating a semantic segmentation mask representing grass and non-grass regions, the method may further include: binarizing the semantic segmentation mask based on the grass and non-grass regions to obtain a first processed mask; processing the first processed mask through morphological operations to obtain a second processed mask, thereby generating a bitmap representing the grass and non-grass regions through a camera projection transformation operation based on the locations of the grass and non-grass regions in the second processed mask. Specifically, after processing the environment image through the target student model to obtain the corresponding semantic segmentation mask, the grass and non-grass regions in the semantic segmentation mask may be marked (a 0 / 1 matrix), and then binarizing the semantic segmentation mask based on the grass and non-grass regions to obtain the corresponding first processed mask. Furthermore, to improve the mask quality, the first processed mask may be processed through morphological operations to eliminate isolated noise points to obtain a second processed mask. It is understood that the above operations can improve the mask quality. Subsequently, based on the locations of the grass and non-grass regions in the second processed mask, a bitmap representing the grass and non-grass regions is generated through a camera projection transformation operation.

[0057] Step S12: planning a current path according to the boundary features corresponding to the dot map, and moving along the current path, and then jumping to the step of processing the environmental image currently collected of the lawn area by the target student model.

[0058] In this application, the above steps allow the lawn mower's own target student model to process the environmental image captured by the camera, generating a corresponding dot matrix representing the grass and non-grass areas. Furthermore, based on the boundary features corresponding to the dot matrix, the lawn mower's path can be planned. The mower then travels along the planned current path, collects environmental images of the lawn area, and continues to process the newly collected environmental images using the target student model.

[0059] In a specific embodiment, the process of planning the current path according to the boundary features corresponding to the dot matrix may include: if the dot matrix represents the existence of a single non-grass area, then according to its own real-time position, a target boundary line between the grass area and the non-grass area is selected through a greedy algorithm; based on the target boundary line and a preset direction of travel, the current path is planned; the preset direction of travel is clockwise or counterclockwise; if the dot matrix represents the existence of multiple continuous non-grass areas, the shortest path from its own real-time position to the next boundary line is determined as the current path; if the dot matrix represents the existence of multiple non-continuous non-grass areas, then based on a preset path planning algorithm, the current path from its own real-time position to the next boundary line is planned; the preset path planning algorithm is an algorithm constructed based on the proportion of grass areas and non-grass areas. Specifically, when planning the mower's current path based on the boundary features corresponding to the dot matrix, the mower's real-time position, the number of non-grass areas in the dot matrix, and the continuity of the non-grass areas must be considered. Furthermore, if the dot matrix indicates a single non-grass area, the mower can use a greedy algorithm to select the boundary line (which can be the longest boundary line) between the grass and non-grass areas, combining its real-time position, to determine a target boundary line. Based on this target boundary line and the mower's pre-set direction of travel, the mower's current path is planned. It is understood that the mower's pre-set direction of travel can be clockwise or counterclockwise, i.e., from the right or from the left. Accordingly, if the dot matrix indicates multiple continuous non-grass areas, the shortest path between the mower's real-time position and the next boundary line is considered, and this path is determined as the current path, with the environment imagery continuing to be captured along the boundary line. Furthermore, if the dot matrix indicates multiple non-grass areas, a pre-set path planning algorithm is used. This algorithm takes into account the relative proportions of grass and non-grass areas and plans a path from the mower's current position to the next boundary line, recording this path as the current path and continuing along the boundary line. This allows the mower to continuously capture images of the lawn area and build a map.

[0060] Step S13: until the current path meets a preset termination condition, a global map corresponding to the lawn area is constructed based on the dot maps corresponding to the lawn mower.

[0061] In the present application, the above steps can be used to plan the travel path of the lawn mower using the collected environmental images to continuously collect environmental images; when the planned path meets the termination conditions, the collection of environmental images can be stopped, and the dot matrix corresponding to the environmental images collected during the previous movement can be used to construct a global map of the lawn area.

[0062] In a specific embodiment, until the current path meets a preset termination condition, constructing a global map corresponding to the lawn area based on the bitmaps corresponding to the lawn mower may include: determining that the current path meets the preset termination condition if the current path coincides with the starting point of the lawn mower entering the lawn area; determining the bitmaps obtained during the lawn mower's movement, and constructing a global map corresponding to the lawn area using the boundary features corresponding to each bitmap. Specifically, the termination condition is a pre-set condition for determining whether the lawn mower has completed the acquisition of all environmental images of the lawn area. If the lawn mower's planned path coincides with the starting point of the lawn mower entering the lawn area, indicating that the lawn mower has completed a full circle of the lawn area, the planned current path may be determined to meet the termination condition. At the end of the movement, the bitmaps of the entire lawn area may be determined, and the boundary features corresponding to each bitmap may be used to construct a global map corresponding to the lawn area. It will be understood that when constructing the global map, it is necessary to combine the bitmaps and the real-time location of the lawn mower corresponding to each bitmap to generate a complete global map.

[0063] In another specific embodiment, after constructing the global map corresponding to the lawn area, the process may further include: if the automatic operation function corresponding to the lawn mower is set to "no," the lawn mower returns to the location of the charging station corresponding to the lawn mower and enters a standby state; if the automatic operation function corresponding to the lawn mower is set to "yes," the lawn mower performs a mowing task on the lawn area according to the global map, and after completing the mowing task corresponding to the lawn area, returns to the location of the charging station and enters a standby state. Specifically, before constructing the map, the lawn mower's automatic operation function can be set by relevant staff to determine whether the mowing task needs to be performed directly after the map is constructed; further, if the automatic operation function is set to "no," the lawn mower can directly return to the corresponding charging station location after the map is constructed and enter a standby state; correspondingly, if the automatic operation function is set to "yes," the lawn mower can directly use the constructed global map to perform the mowing task on the lawn area after the map is constructed, and after completing the mowing task, return to the charging station location and enter a standby state. This allows for flexible setting of the lawn mower's task mode.

[0064] It can be seen that this scheme reduces the amount of model parameters through knowledge distillation, and combined with the loss function related to the knowledge distillation operation, it can ensure that the performance of the student model used is almost consistent with that of the corresponding teacher model. By processing the collected environmental images through the student model, grass areas and non-grass areas can be effectively identified; it can be seen that without affecting the mapping effect of the lawn mower, this scheme can significantly reduce the amount of model parameters and reduce the hardware cost requirements for the lawn mower; moreover, there is no need to set markers in the lawn area or record the boundary lines of the lawn area in advance. The lawn mower can be started directly to realize automatic mapping, which simplifies the operation process and improves the adaptability of the lawn mower.

[0065] The following example will specifically introduce the knowledge distillation process. Figure 2 As shown in Figure 2, the process of performing knowledge distillation based on the target loss function specifically includes:

[0066] Step S21: Transfer the knowledge of the preset teacher model to the initial student model through knowledge distillation operation.

[0067] In this embodiment, DeepLabV3+ (a deep learning model for semantic segmentation) is used as the base segmentation model. Parameter reduction is achieved based on a hybrid architecture of MobileNetV3 (a lightweight computer vision model) and DeepLabV3+. During model structure optimization, the backbone network is first replaced, from the teacher model (DeepLabV3 + ResNet-50 (deep convolutional network)) to the student model (MobileNetV3-Large + lightweight FPN (feature pyramid network)). Next, ASPP (Atrous Spatial Pyramid Pooling) is simplified, replacing the original ASPP's four-branch dilated convolution (growth rates = [6, 12, 18]) with a two-branch (rates = [6, 12]) + global pooling branch. Furthermore, all 3×3 standard convolutions are replaced with depthwise separable convolutions, reducing a significant number of model parameters. Furthermore, during model training, pruning can be performed gradually based on channel importance (assessed using the L1 norm). Specifically, 50% of the teacher's channels are initially retained, with 10% pruned every 10 epochs (training cycles), ultimately retaining 20% ​​of the channels (i.e., 2048 in the teacher model → 512 in the student model). 1×1 convolutions + skip connections can also be used to maintain feature expressiveness. In specific embodiments, Post-Training Quantization (PTQ) can be applied to quantize model parameters, converting floating-point numbers (FP32) to integers (INT8), reducing model size and improving speed.

[0068] It should be pointed out that in order to perform knowledge transfer, the following loss function is constructed to control knowledge distillation:

[0069] L total = L seg + L distill + L aux ;

[0070] Among them, L seg Use weighted cross entropy loss (Weighted CE) to solve category imbalance (such as the difference in the proportion of grass and background); use normalization loss to compare the multi-scale features of its teacher model and student model, and distillation loss L distill is the weighted sum of KL divergence loss and normalization loss; L auxThis is a boundary-aware loss that uses the Sobel operator (an edge detection algorithm) to extract edges and penalize blurry segmentation mask boundaries. The weight coefficients can initially be set to α = 1.0, β = 0.5, and γ = 0.2. In a specific embodiment, the weight coefficients can be dynamically adjusted based on the validation set's Intersection of Union (IoU). For example, if the student model underfits the teacher's knowledge, β can be gradually increased to 0.8. If edge segmentation is unclear, γ can be increased to 0.5.

[0071] Furthermore, in a specific embodiment, ablation experiments demonstrated the following loss contributions: using only cross-entropy loss, the model achieved an average IoU of 87.3%. Adding distillation loss increased the average IoU to 89.6%, and adding boundary-aware loss increased the average IoU to 90.8%. This demonstrates that the combined loss function improves the accuracy of lawn segmentation using the student model derived from knowledge distillation. Furthermore, using seasonal simulation (generating snow and dead grass using a generative adversarial network) for data augmentation further improved the model's average IoU in winter scenes.

[0072] Step S22: performing a first-stage training on the initial student model based on the cross entropy loss and the distillation loss to obtain a trained student model; the first stage is the first several training cycles in the preset training cycle.

[0073] Furthermore, distillation training can be performed in stages, with the first few training cycles of a preset training cycle serving as the first stage. To achieve feature alignment, only the student model can be trained, with the optimization objective being to minimize the cross-entropy loss + distillation loss. Data augmentation techniques can also be used to improve feature robustness. As can be seen, the initial student model ultimately yields the corresponding post-training student model after the first stage of training.

[0074] Step S23: Based on the KL divergence loss and the boundary perception loss, combined with the teacher model, the trained student model is trained in the second stage to obtain the target student model; the second stage is the training cycle in the preset training cycle except the first stage.

[0075] Correspondingly, the second stage of model training involves jointly optimizing the teacher and student models, refining the raw scores generated by the teacher model for the input data. The optimization targets focus on KL divergence loss and boundary-aware loss, while weak data augmentation techniques are used to stabilize the learning of the raw scores. As can be seen, after two stages of training, the target student model can be ultimately obtained based on the teacher model.

[0076] It can be seen that in the process of knowledge distillation, this application constructs a loss function based on cross entropy loss, distillation loss and boundary perception loss to optimize the performance of the student model; and combined with phased training, the student model can be targetedly optimized according to the corresponding cross entropy loss and distillation loss, and the teacher model and student model can be jointly optimized according to the KL divergence loss and boundary perception loss to ensure that the final student model reaches the performance level of the teacher model as much as possible.

[0077] like Figure 3 As shown, the embodiment of the present application discloses a mapping method, which specifically includes:

[0078] In the embodiment of this application, Figure 4 As shown in the figure, the lawn mower is equipped with a monocular camera 11, an RTK (Real-time kinematic) positioning antenna 12, a mainboard and a processor 13. The camera is used to collect environmental images, the RTK antenna locates the real-time position information of the lawn mower, and the mainboard and processor are used to carry the model and implement operations such as image analysis, path planning, and mapping. Figure 5 As shown, a top view 21 of the visible area, a side view 22 of the visible area, a camera blind spot 23, and a camera blind spot 24. In a specific embodiment, for edge devices (such as Raspberry Pi), the TensorRT (a high-performance deep learning inference optimizer and runtime library) engine can be used to optimize the computation graph, which can effectively reduce latency.

[0079] First, you need to start the lawn mower, and control the machine from the charging pile to the lawn area, where the recharging path will be recorded, and then the lawn mower enters the automatic mapping mode; in the automatic mapping mode, the lawn mower collects environmental images through the (2D) camera, and the collected images are sent to the lightweight segmentation model (student model) to generate semantic segmentation masks of grass and obstacles, and according to the position of the grass and obstacles in the mask in the image, the camera projection transformation is used to generate a dot map of the grass and obstacles. In other words, the image coordinate system can be converted into a world coordinate system in combination with the internal and external parameters of the camera to obtain a top view of the lawn, obstacles, etc. in the camera relative to the lawn mower in the world coordinate system. Furthermore, the binarization result of the top view is rasterized to obtain the corresponding raster map of grass and non-grass, that is, the corresponding dot map. As shown in the figure, Figure 6 The diagram shows a matrix diagram of grass area and non-grass area, non-grass area 31, blind area 32 and grass area 33. Figure 7 The figure shows the comparison of the recognition effects of grass area and non-grass area.

[0080] The mower's path can then be planned using this dot matrix, allowing for continuous image acquisition and grass / non-grass analysis. Finally, a global map of the lawn area can be constructed using the dot matrix associated with the movement process and the mower's specific location. Specifically, based on the boundary features of the dot matrix, a greedy algorithm prioritizes the longest continuous boundary line, moving in a clockwise or counterclockwise direction. If multiple areas exist (such as flower beds), the shortest jump path is planned to the starting point of the next boundary line, and automatic edge mapping is restarted. A non-grass boundary is found, and the mower automatically moves along the extracted grass / non-grass boundary line (in a clockwise or counterclockwise direction). The RTK trajectory coordinates are recorded simultaneously during the movement. When multiple discontinuous non-grass areas (such as flower beds) are detected, an improved A* algorithm can be used to plan the jump path. The improved A* algorithm considers distance weighting and non-grass area coverage weighting, such as a distance weight of 0.6 and a non-grass area coverage weight of 0.4. (Non-grass area coverage refers to the size of the non-grass area in the dot map. The mower builds and improves the global map as it moves, performing a weighted search within the global map.) The specific formula is: Cost = 0.6 × Distance + 0.4 × (1 − Coverage). It can be dynamically updated, and environmental images are continuously collected during path changes, updating the dot map in real time to address temporary obstacles. In specific embodiments, RTK positioning can be fused with visual positioning data through Kalman filtering to improve coordinate accuracy.

[0081] Furthermore, when the mower's path is closed to its starting point, all boundary lines and RTK data collected during the journey can be integrated to generate a complete global map. It is understood that if the mower is pre-set to automatic operation, it can switch to automatic cutting after generating the map data. After the mowing task is completed, it will automatically return to the charging station to charge. If automatic operation is not pre-set, the mower will return to the charging station and enter standby mode after completing the mapping data.

[0082] It can be seen that the lawn mower in this application can perform fully automatic mapping, and no human intervention is required during the mapping process. It relies on a pre-set lightweight student model to recognize the environmental image captured by the camera to obtain the corresponding dot map. The processor load is low, suitable for embedded device deployment, and no preset markers are required. No lidar or multi-sensor fusion is required, which reduces the cost of lawn mower mapping deployment. In addition, the recognized dot map and the RTK position information of the lawn mower are combined to accurately construct a map, and dynamic correction is performed in combination with visual data to further reduce map accuracy errors. During the mapping process, the lawn mower can detect grass areas and non-grass areas, automatically plan the travel path, jump to the next lawn boundary, and dynamically update the global map in real time to ensure the mapping effect.

[0083] like Figure 8As shown, the embodiment of the present application discloses a mapping device applied to a lawn mower, comprising:

[0084] An image processing module 41 is configured to process the currently captured environmental image of the lawn area using a target student model to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by pre-performing a knowledge distillation operation based on a target loss function, wherein the target loss function is a loss function constructed based on cross entropy loss, distillation loss, and boundary perception loss, and the distillation loss is a loss function constructed based on KL divergence loss and a normalized loss between the model feature map;

[0085] a path planning module 42 for planning a current path according to the boundary features corresponding to the dot matrix, and traveling along the current path, and then jumping to the step of processing the environmental image currently collected of the lawn area by the target student model;

[0086] The mapping module 43 is configured to construct a global map corresponding to the lawn area based on the dot maps corresponding to the lawn mower until the current path meets a preset termination condition.

[0087] It can be seen that this solution reduces the number of model parameters through knowledge distillation, and combined with the loss function related to the knowledge distillation operation, it can ensure that the performance of the student model used is almost consistent with that of the corresponding teacher model. By processing the collected environmental images through the student model, grass areas and non-grass areas can be effectively identified. It can be seen that without affecting the mapping effect of the lawn mower, this solution can significantly reduce the number of model parameters and reduce the hardware cost requirements for the lawn mower.

[0088] In a specific embodiment, the device may include:

[0089] The knowledge transfer module is used to transfer the knowledge of the pre-set teacher model to the initial student model through the knowledge distillation operation;

[0090] A first training module is configured to perform a first phase of training on the initial student model based on a cross entropy loss and a distillation loss to obtain a trained student model; the first phase is the first several training cycles in a preset training cycle;

[0091] The second training module is used to perform a second stage of training on the trained student model based on KL divergence loss and boundary perception loss in combination with the teacher model to obtain a target student model; the second stage is a training cycle in the preset training cycle except the first stage.

[0092] In a specific embodiment, the image processing module 41 may include:

[0093] An image acquisition unit is used to acquire images of the lawn area through its own camera to obtain corresponding environmental images;

[0094] a mask generation unit, configured to input the environment image into a target student model to generate a semantic segmentation mask representing grass areas and non-grass areas;

[0095] The bitmap generation unit is used to generate a bitmap representing the grass area and the non-grass area through a camera projection transformation operation according to the positions of the grass area and the non-grass area in the semantic segmentation mask.

[0096] In another specific embodiment, the device may further include:

[0097] a first mask processing module, configured to perform a binarization process on the semantic segmentation mask based on the grass area and the non-grass area to obtain a first processed mask;

[0098] The second mask processing module is used to process the first processed mask through morphological operations to obtain a second processed mask, so as to generate a bitmap representing the grass area and the non-grass area through a camera projection transformation operation according to the positions of the grass area and the non-grass area in the second processed mask.

[0099] In a specific embodiment, the path planning module 42 may include:

[0100] A boundary line selection unit is configured to select a target boundary line between a grass area and a non-grass area using a greedy algorithm based on its own real-time position when the dot matrix indicates the presence of a single non-grass area;

[0101] A first path planning unit is configured to plan a current path based on the target boundary line and a preset moving direction; the preset moving direction is a clockwise direction or a counterclockwise direction;

[0102] a second path planning unit, configured to determine, when the dot matrix indicates the presence of a plurality of continuous non-grass areas, a shortest path from the real-time position to the next boundary line as a current path;

[0103] The third path planning unit is used to plan the current path from its own real-time position to the next boundary line based on a preset path planning algorithm when the dot matrix represents the existence of multiple non-continuous non-grass areas; the preset path planning algorithm is an algorithm constructed based on the ratio of grass areas and non-grass areas.

[0104] In a specific embodiment, the mapping module 43 may include:

[0105] a termination determination unit, configured to determine that the current path meets a preset termination condition when the current path coincides with a starting point where the lawn mower enters the lawn area;

[0106] The map construction unit is used to determine each dot map obtained by the lawn mower during its movement, and to construct a global map corresponding to the lawn area by using boundary features corresponding to each dot map.

[0107] In a specific embodiment, the device may further include:

[0108] A first functional module is configured to return to the charging pile location corresponding to the lawn mower and enter a standby state when the automatic operation function corresponding to the lawn mower is set to no;

[0109] The second functional module is used to perform a mowing task on the lawn area according to the global map when the automatic operation function corresponding to the lawn mower is set to yes, and after completing the mowing task corresponding to the lawn area, return to the charging pile position and enter a standby state.

[0110] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 9 This is a structural diagram of an electronic device 50 according to an exemplary embodiment. The content in the diagram cannot be considered as any limitation to the scope of use of the present application.

[0111] Figure 9 This is a schematic diagram of the structure of an electronic device 50 provided in an embodiment of the present application. The electronic device 50 may specifically include: at least one processor 51, at least one memory 52, a power supply 53, a communication interface 54, an input / output interface 55, and a communication bus 56. The memory 52 is used to store a computer program, which is loaded and executed by the processor 51 to implement the relevant steps of the mapping method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 50 in this embodiment may specifically be an electronic computer.

[0112] In this embodiment, the power supply 53 is used to provide operating voltage for each hardware device on the electronic device 50; the communication interface 54 can create a data transmission channel between the electronic device 50 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 55 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0113] In addition, the memory 52, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 521, a computer program 522, etc., and the storage method can be temporary storage or permanent storage.

[0114] The operating system 521 is used to manage and control the hardware devices on the electronic device 50 and the computer program 522. It can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the mapping method performed by the electronic device 50 disclosed in any of the aforementioned embodiments, the computer program 522 can further include computer programs capable of performing other specific tasks.

[0115] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned mapping method. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0116] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0117] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0119] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0120] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A mapping method, characterized in that: Applications in lawn mowers, including: The target student model is used to process the currently captured environmental image of the lawn area to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by performing a knowledge distillation operation based on a target loss function in advance, the target loss function is a loss function constructed based on cross entropy loss, distillation loss, and boundary perception loss, and the distillation loss is a loss function constructed based on KL divergence loss and normalization loss between model feature maps; Planning a current path according to the boundary features corresponding to the dot matrix, and moving along the current path, and then jumping to the step of processing the environmental image currently collected of the lawn area by the target student model; Until the current path meets a preset termination condition, a global map corresponding to the lawn area is constructed based on the dot maps corresponding to the lawn mower.

2. The mapping method according to claim 1, wherein: The knowledge distillation operation based on the target loss function includes: Transfer the knowledge of the pre-set teacher model to the initial student model through the knowledge distillation operation; The initial student model is trained in a first phase based on the cross entropy loss and the distillation loss to obtain a trained student model; the first phase is the first several training cycles in the preset training cycle; Based on KL divergence loss and boundary perception loss, combined with the teacher model, the trained student model is trained in the second stage to obtain the target student model; the second stage is the training cycle in the preset training cycle except the first stage.

3. The mapping method according to claim 1, wherein: The target student model is used to process the environmental image of the lawn area currently collected to obtain a dot matrix representing the grass area and the non-grass area, including: The lawn area is imaged through its own camera to obtain the corresponding environmental image; Inputting the environment image into a target student model to generate a semantic segmentation mask representing grass areas and non-grass areas; According to the positions of the grass area and the non-grass area in the semantic segmentation mask, a dot map representing the grass area and the non-grass area is generated through a camera projection transformation operation.

4. The mapping method according to claim 3, wherein: After generating the semantic segmentation mask representing the grass area and the non-grass area, the method further includes: performing binarization processing on the semantic segmentation mask based on the grass area and the non-grass area to obtain a first processed mask; The first processed mask is processed by morphological operations to obtain a second processed mask, so as to generate a bitmap representing the grass area and the non-grass area through a camera projection transformation operation according to the positions of the grass area and the non-grass area in the second processed mask.

5. The mapping method according to claim 1, wherein: The planning of the current path according to the boundary features corresponding to the dot matrix includes: If the dot matrix indicates that there is a single non-grass area, a target boundary line between the grass area and the non-grass area is selected by a greedy algorithm based on its own real-time position; Planning a current path based on the target boundary line and a preset direction of travel; the preset direction of travel is clockwise or counterclockwise; If the dot map indicates that there are multiple continuous non-grass areas, the shortest path from the real-time position to the next boundary line is determined as the current path; If the dot matrix indicates the presence of multiple non-continuous non-grass areas, the current path is planned to jump from its own real-time position to the next boundary line based on a preset path planning algorithm; the preset path planning algorithm is an algorithm constructed based on the ratio of grass areas and non-grass areas.

6. The mapping method according to claim 1, wherein: The process of constructing a global map corresponding to the lawn area based on each dot map corresponding to the lawn mower until the current path meets a preset termination condition includes: If the current path coincides with the starting point of the lawn mower entering the lawn area, it is determined that the current path meets the preset termination condition; The dot patterns obtained by the lawn mower during its movement are determined, and a global map corresponding to the lawn area is constructed using boundary features corresponding to the dot patterns.

7. The mapping method according to any one of claims 1 to 6, characterized in that: After constructing the global map corresponding to the lawn area, the method further includes: If the automatic operation function corresponding to the lawn mower is set to no, then return to the charging pile location corresponding to the lawn mower and enter the standby state; If the automatic operation function corresponding to the lawn mower is set to yes, the lawn mowing task is performed on the lawn area according to the global map, and after completing the mowing task corresponding to the lawn area, the lawn mower returns to the charging pile position and enters the standby state.

8. A mapping device, characterized in that: Applications in lawn mowers, including: An image processing module is configured to process the currently captured environmental image of the lawn area using a target student model to obtain a dot map representing the grass area and the non-grass area; the target student model is a model obtained by pre-performing a knowledge distillation operation based on a target loss function, wherein the target loss function is a loss function constructed based on cross entropy loss, distillation loss, and boundary perception loss, and the distillation loss is a loss function constructed based on KL divergence loss and a normalized loss between the model feature map; a path planning module, configured to plan a current path according to the boundary features corresponding to the dot matrix, travel along the current path, and then jump to the step of processing the environmental image currently collected of the lawn area by the target student model; The mapping module is used to construct a global map corresponding to the lawn area based on the dot maps corresponding to the lawn mower until the current path meets the preset termination condition.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the mapping method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the mapping method according to any one of claims 1 to 7.

Images