Mower ground type identification algorithm based on millimeter wave radar and application method thereof

Abstract

The invention relates to a lawn mower ground type identification algorithm based on millimeter wave radar and an application method. The algorithm comprises the following steps: A1, carrying out preprocessing and distance dimension fast Fourier transform processing on millimeter wave radar data, calculating an amplitude value of each distance unit based on obtained multi-channel spectrum data, obtaining distance information of a ground type by calculating an amplitude mean value of the multi-channel spectrum data, and constructing a distance amplitude spectrogram; a2, taking the distance amplitude spectrogram as input, and respectively extracting local features and global features of the distance amplitude spectrogram through a convolutional neural network and a Mama network; a3, carrying out feature fusion on the local features and the global features through a channel attention block, carrying out deep semantic information combination by using a semantic token learning device and a semantic token fusion device, carrying out sequential attention mechanism processing to generate a feature weighted graph, and finally mapping the features into probability distribution through a full connection layer and a softmax function, and identifying the ground type and outputting a prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent ground recognition, and in particular to a ground type recognition algorithm for a lawn mower based on millimeter wave radar and an application method thereof. Background Art

[0002] In the fields of modern automation and intelligent technology, accurate identification of surface types is crucial for autonomous robotics, particularly in applications such as lawn mowers. By collecting data on different surface types and analyzing their unique signal characteristics, effective differentiation can be achieved. However, existing technologies still face numerous challenges in achieving this goal. For example, surface types are complex and variable. On rainy days, radar echo signals can vary depending on factors such as grass permeability, air permeability, length, and humidity. Furthermore, the similarities between different surface types complicate feature extraction, making it crucial to extract distinct, unaffected features from complex environments.

[0003] CN119027649A discloses a target detection method and device, electronic device and storage medium for a lawn mower, wherein the method comprises: obtaining a target frame set obtained by detecting a sensor shell surface of the target sensor through a target sensor, wherein each target frame in the target frame set is a single-frame point cloud obtained by a single detection by the target sensor, and the target sensor is a sensor configured on the lawn mower for generating a point cloud; performing a projection operation on the target frame set to obtain a target projection image, wherein the projection operation is an operation of projecting onto a specified plane; inputting the target projection image into a trained target classification model to obtain a target classification result, wherein the target classification model is a classification model for distinguishing multiple preset categories, and the target classification result is used to indicate a preset category among the multiple preset categories that matches the target frame set.

[0004] CN115494837A discloses a path tracking control method for a lawn mower that integrates virtual radar and multi-level neural networks, including: 1: deriving a kinematic model of a crawler mower; 2: obtaining virtual path boundaries and virtual radar maps, and performing batch normalization; 3: building a first-level deep neural network; 4: training the first-level deep neural network; 5: building a second-level deep neural network; 6: training the second-level deep neural network; 7: an upper computer obtains a virtual radar map generated by a crawler mower at the current position through a virtual radar, and obtains the corresponding control speed of crawler drive motors on both sides after two-stage series neural network calculations. The upper computer then sends the drive motor speed obtained by real-time calculation to the lower computer, and the lower computer controls the rotation of the crawler drive motor, thereby realizing motion control of the crawler mower.

[0005] Traditional lawn mowers require laying cables to define their working areas, which is labor-intensive and capital-intensive. It also struggles to adapt to the needs of different plots of land, leading to low efficiency and increased costs. Furthermore, the sensors used are susceptible to weather and lighting conditions and can infringe on the privacy of others. In severe weather, the performance of equipment like LiDAR can significantly decline, impacting detection accuracy and reliability. Furthermore, existing classification algorithms are incomplete in feature extraction and are prone to misjudgment when faced with complex and diverse ground types. Furthermore, due to the sheer volume of data, inference speed is slow, making them unable to meet real-time requirements.

[0006] In addition, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making the present invention, but due to space limitations, not all details and contents are listed in detail. However, this does not mean that the present invention does not have the characteristics of these prior arts. On the contrary, the present invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background technology. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the present invention provides a millimeter-wave radar-based lawn mower ground type recognition algorithm and its application method, addressing at least some of the aforementioned technical issues. The proposed algorithm utilizes millimeter-wave radar for data acquisition, enabling all-weather operation regardless of environmental influences. Furthermore, because millimeter-wave radar collects data through echoes, the proposed algorithm can rapidly extract a multi-scale semantic feature that fuses global and local features, thereby improving classification accuracy and speed.

[0008] The present invention discloses a lawn mower ground type recognition algorithm based on millimeter wave radar, which includes:

[0009] A1. Perform preprocessing and range-dimensional fast Fourier transform on the collected millimeter-wave radar data to generate multi-channel spectrum data. Based on this multi-channel spectrum data, the amplitude value of each range bin is calculated. The amplitude mean of this multi-channel spectrum data is then calculated to obtain distance information about the ground type. This information is then used to construct a range-amplitude spectrum.

[0010] A2. Take the range-magnitude spectrogram as input, extract local features of the range-magnitude spectrogram using a convolutional neural network, and extract global features of the range-magnitude spectrogram using a Mamba network.

[0011] A3. Feature fusion of local and global features is performed through the channel attention block. Deep semantic information is combined using the semantic token learner and semantic token fuser. After processing through the sequential attention mechanism, a feature weighted graph is generated. Finally, the features are mapped to a probability distribution through a fully connected layer and a softmax function to identify the ground type and output the prediction result.

[0012] Preferably, the distance-dimensional fast Fourier transform processing refers to converting the time domain signal collected by the millimeter-wave radar into a frequency domain signal through a fast Fourier transform algorithm, so as to analyze the reflection intensity information at different distances.

[0013] According to a preferred embodiment, when preprocessing millimeter-wave radar data, each frame of the signal is windowed using a Hanning window, which reduces spectrum leakage by applying a smooth transition amplitude attenuation at both ends of the signal.

[0014] According to a preferred embodiment, based on the data in the distance amplitude diagram matrix and the distance information of the ground type, a distance amplitude spectrum diagram is drawn with the distance information as the horizontal coordinate and the amplitude value as the vertical coordinate to intuitively display the distribution of target reflection intensity at different distances.

[0015] According to a preferred embodiment, the Mamba network uses a dilated convolution scanning strategy to form a simpler sequence structure by skipping sampling points. The scanning process of the dilated convolution scanning strategy includes both horizontal and vertical skip scanning. Furthermore, the sequence structure formed by skipping sampling points is 2×112×224=2×25088, which is simpler than the sequence structure after serpentine scanning (4×224×224=4×50176).

[0016] According to a preferred embodiment, a patch sequence of an image is obtained by hole scanning, and then the patch sequence of the image is linearly projected, discretized and forward propagated through the S6 block of the standard Mamba network to convert the input into a hidden state, and the hidden state and output are continuously updated.

[0017] According to a preferred embodiment, the parameter matrices A, B, and C in the state space equation of the Mamba network are dynamically updated learnable parameters, and the state space equation is converted into:

[0018]

[0019] According to a preferred embodiment, the channel attention block performs feature fusion by element-wise summation, and the formula is as follows:

[0020] X l+1 =SE(SSM(X l ))+SE(Conv(X l )).

[0021] According to a preferred embodiment, when the semantic token learner executes the sequential attention mechanism, the weighted processing of the input feature S includes: calculating the maximum value S max and the mean value Save , after one-dimensional convolution and sigmoid activation, the feature weighted graph m is generated and applied to the original input to obtain the weighted feature

[0022] According to a preferred embodiment, the process of converting the weighted feature n into a semantic token is as follows:

[0023] A=n⊙U1,

[0024] V=n⊙U2,

[0025] u=Softmax(A T ⊙V).

[0026] The present invention also discloses an application method of a millimeter-wave radar-based lawn mower ground type recognition algorithm, which includes one or more of the following steps:

[0027] A0. Configure a millimeter-wave radar and use it to collect millimeter-wave radar data.

[0028] A4. Adjust the working state of the lawn mower based on the prediction results output by the recognition algorithm.

[0029] Compared with existing research technologies, the millimeter-wave radar-based lawn mower ground recognition technology of the present invention has the following significant advantages:

[0030] First, millimeter-wave radar exhibits greater environmental adaptability. Compared to lidar, millimeter-wave radar is less sensitive to environmental interference sources such as raindrops and dust, enabling more stable surface identification in adverse weather conditions. In contrast, lidar's performance is significantly affected under similar conditions, with its light propagation distance shortened, resulting in reduced detection accuracy and reliability. Furthermore, millimeter-wave radar costs significantly less than lidar, typically ranging from a few hundred to a few thousand yuan, compared to lidar prices of several thousand to tens of thousands of yuan. Therefore, adopting millimeter-wave radar not only effectively reduces overall costs but also enhances product market competitiveness. Furthermore, millimeter-wave radar boasts a longer detection range, enabling it to proactively detect surface types even at a distance, further improving system responsiveness and operational efficiency.

[0031] Second, the multi-scale Mamba network used in the present invention combines the local feature extraction capabilities of convolutional neural networks (CNNs) with the global feature extraction capabilities of Mamba networks, and integrates the extraction process of deep semantic features. This design makes the resulting feature representation more comprehensive and representative than other methods. In addition, the selective scanning strategy (i.e., hole scanning) adopted reduces the computational complexity in state-space model sequence modeling while ensuring the global receptive field, thereby improving processing speed.

[0032] Combining the above features, the present invention not only improves the accuracy and efficiency of ground type recognition, but also demonstrates excellent performance in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the overall technical route provided by the present invention;

[0034] Figure 2 It is a data processing flow chart of the range-dimensional fast Fourier transform (Range-FFT) provided by the present invention;

[0035] Figure 3.1 、 Figure 3.2 、 Figure 3.3 is the range-amplitude spectrum of different ground types provided by the present invention, wherein, Figure 3.1 (a) is grassland, Figure 3.2 (b) is ceramic tile, Figure 3.2 (c) is floor tiles, Figure 3.3 (d) is an asphalt road surface, Figure 3.3 (e) wooden floor;

[0036] Figure 4 It is a comparative diagram of the two scanning strategies;

[0037] Figure 5 It is a schematic diagram of the STL and STF structures provided by the present invention;

[0038] Figure 6 Schematic diagram of the architecture of the multi-scale Mamba network provided by the present invention;

[0039] Figure 7 The training accuracy graph and loss graph after 100 rounds of training provided by the present invention are shown in FIG.

[0040] Figure 8 The present invention provides wooden floor data collected in daytime and nighttime environments. DETAILED DESCRIPTION

[0041] The following is a detailed description with reference to the accompanying drawings.

[0042] In the fields of modern automation and intelligent technology, accurate surface identification is crucial for autonomous robotic operations. Millimeter-wave radar, as an advanced sensor technology, is widely used for surface identification due to its high resolution, strong anti-interference capabilities, and all-weather operation. By collecting data on different surface types, their unique signal characteristics can be leveraged to distinguish different surface materials and structures.

[0043] Multi-channel spectrum data refers to the frequency distribution data corresponding to different receiving antenna channels obtained after distance-dimensional fast Fourier transform. Each channel represents the echo signal from a specific direction.

[0044] The amplitude mean calculates the average of the absolute values of the amplitudes of each frequency component in the spectrum data of each channel, which is used to quantify the overall strength of all frequency components in the channel.

[0045] The frequency domain signal is the representation of the original time domain signal after Fourier transform, which shows the intensity distribution of the signal at different frequencies and helps to identify and analyze the target characteristics at a specific distance.

[0046] Ground type distance information refers to the characteristic parameters of different ground materials or structures at specific distances, such as reflection intensity, derived from millimeter-wave radar echo signal analysis, which is used to distinguish different types of ground.

[0047] The range-magnitude map matrix is a two-dimensional array where each row or column represents a specific range cell, and the corresponding value represents the received signal strength at that range.

[0048] The range-amplitude spectrum is a visualization tool that graphically displays the data of the range-amplitude spectrum matrix, where the x-axis represents distance information and the y-axis represents amplitude value (i.e., signal strength). This chart can be used to intuitively observe the reflection characteristics of targets at different distances and help identify different ground types.

[0049] Convolutional neural network (CNN) is a deep learning model specifically designed to process data with grid structures such as images, and effectively extract local features from input data through local connections and weight sharing.

[0050] Local features refer to the detailed information within a specific area of an image, such as edges, textures, or local specific shapes. These features help to distinguish different local patterns.

[0051] The Mamba network is an improved neural network architecture designed to extract broader contextual information, namely global features, from the input data.

[0052] Global features refer to large-scale properties that cover the entire image, such as overall shape or distribution pattern, and are crucial for understanding the overall content of the image.

[0053] Example 1

[0054] Figure 1 The overall technical route diagram of the present invention is shown. Specifically, the present invention discloses a lawn mower ground type recognition algorithm based on millimeter wave radar, which includes the following steps:

[0055] A1. Perform preprocessing and range-dimensional fast Fourier transform on the collected millimeter-wave radar data to generate multi-channel spectrum data. Based on this multi-channel spectrum data, the amplitude value of each range bin is calculated. The amplitude mean of this multi-channel spectrum data is then calculated to obtain distance information about the ground type. This information is then used to construct a range-amplitude spectrum.

[0056] A2. Take the range-magnitude spectrogram as input, extract local features of the range-magnitude spectrogram using a convolutional neural network, and extract global features of the range-magnitude spectrogram using a Mamba network.

[0057] A3. Feature fusion of local and global features is performed through the channel attention block. Deep semantic information is combined using the semantic token learner and semantic token fuser. After processing through the sequential attention mechanism, a feature weighted graph is generated. Finally, the features are mapped to a probability distribution through a fully connected layer and a softmax function to identify the ground type and output the prediction result.

[0058] Preferably, in step A1, the millimeter wave radar data collected by the millimeter wave radar can be defined as:

[0059] S=[(X1,y1),(X2,y2),...,(X n ,y n )],

[0060] Among them, X i =[x i1 ,x i2 ,…,x ic ], X i is the data sample of each frame, a total of n frames, i=1,2,…,n, x ij is the sample of the jth channel of the i-th frame, the total number of channels per frame is C, the data sample length is L, y i The ground type label.

[0061] Preferably, a Hanning window is used for preprocessing each frame of millimeter-wave radar data. The Hanning window can effectively reduce spectrum leakage by gradually reducing the amplitude to zero at both ends of the signal, thereby obtaining clearer spectrum lines in the frequency domain.

[0062] Furthermore, the Hanning window is a cosine window function that can effectively suppress spectrum leakage by applying a smooth transition amplitude attenuation (gradually decreasing to zero from the center to the two ends) to the sampled data at both ends of the time domain signal. Its core principle is to reduce the interference energy of non-mainlobe frequencies in spectrum analysis by reducing the mutation during signal truncation, thereby obtaining a more concentrated main lobe and clearer spectrum lines in the frequency domain. In the present invention, each frame of millimeter-wave radar data is pre-processed with a Hanning window and then a range-dimensional FFT (Range-FFT) is performed, so that the spectrum energy of the ground reflection signal is more concentrated at the frequency point corresponding to the actual distance, thereby improving the resolution of the subsequent range-amplitude spectrum and the accuracy of target detection.

[0063] Preferably, if Figure 2 As shown in FIG, the millimeter-wave radar data after Hanning window preprocessing is processed by distance-dimensional fast Fourier transform to obtain multi-channel spectrum data, where the spectrum corresponding to the multi-channel spectrum data is defined as:

[0064]

[0065] Preferably, the amplitude value (i.e., signal strength) for each range bin is calculated based on the acquired multi-channel spectrum data. These amplitude values form a range-magnitude map matrix, where each row or column represents a specific range bin, and the corresponding value represents the received signal strength at that distance. Therefore, the range-magnitude map matrix is a form of organization for the data after FFT processing.

[0066] Preferably, key features are extracted from the distance amplitude map matrix, wherein the average value (or peak mean) of the amplitude value in each distance unit can be calculated by amplitude weighted averaging to obtain the distance information of the ground type, wherein the calculation formula is:

[0067]

[0068] Preferably, the amplitude value of each frame of the range-dimensional fast Fourier transform output represents the reflection intensity of the target at that distance, where a larger amplitude indicates the presence of a strong reflecting target (such as the ground or other objects) at that distance. Using the amplitude as a weight means that distance points with higher reflection intensity contribute more to the final synthesized distance information, thereby more significantly reflecting the distance characteristics of the real target and suppressing the interference of weak reflection noise or clutter.

[0069] Preferably, based on the data in the range-amplitude map matrix and the distance information of the ground type, a range-amplitude spectrum is plotted with the distance information as the horizontal axis and the amplitude value (i.e., signal strength) as the vertical axis. The range-amplitude spectrum can intuitively display the distribution of target reflection intensity at different distances, improving the system's range resolution and making target detection more direct and accurate. Each peak corresponds to a target at a specific distance, facilitating the system's identification and analysis of ground type.

[0070] Step A1 of the present invention reduces the dimensionality of multi-dimensional radar data to a two-dimensional visual spectrum (range amplitude spectrum), simplifies the complexity of data analysis, and retains key distance-intensity correlation information. In addition, the present invention realizes the step-by-step strengthening and multiplication of the technical effects of each link by constructing a collaborative processing link of pre-processing (Hanning window)-transformation (FFT)-feature extraction (amplitude weighting)-visualization (spectrogram). Specifically, the spectrum leakage introduced by time domain signal truncation is first suppressed by the Hanning window, providing a main lobe distribution basis for energy concentration for subsequent FFT; then, the range dimension FFT converts the optimized time domain signal into a frequency domain spectrum, and under the dual effects of main lobe width compression and sidelobe suppression, significantly improves the distance resolution; then, amplitude weighted averaging is performed on the multi-channel spectrum data to further weaken the interference of random noise and clutter and enhance the signal-to-noise ratio of the target reflection intensity; finally, the high-purity spectrum data output by the above steps is integrated to generate a range amplitude spectrum, so that the target peak position and amplitude can be clearly presented in the visual interface. This interlocking design enables forward coupling of spectrum optimization, noise suppression, and feature enhancement, systematically resolving the detection ambiguity problem caused by the performance bottleneck of a single link in traditional methods. While improving resolution, anti-interference capability, and target resolution accuracy, it ensures the robustness and practicality of the technical solution in complex dynamic scenarios.

[0071] Preferably, in step A2, the distance amplitude spectrum generated in step A1 can be used as the input of the neural network, such as Figure 3.1 to Figure 3.3 As shown, the distance amplitude spectrum may include a variety of different ground types such as grass, ceramic tiles, floor tiles, asphalt pavement, and wooden planks.

[0072] Convolutional neural networks (CNNs) can capture local information from radar signal images and extract local features of various spectral characteristics, such as peaks, extreme points, slopes, and local time-frequency signatures. However, due to their limited local receptive field (LRF), CNNs struggle to capture global contextual information across vast geographic areas. A LRF refers to the fact that each convolution kernel in a CNN only covers a local region of the input image (e.g., a 3×3 or 5×5 pixel range) during a single computation. This design makes CNNs adept at extracting local features (such as edges, textures, and small peaks), but they cannot directly model the correlation between distant pixels. The Mamba-based image classification algorithm, on the other hand, is highly efficient when processing long sequences, which greatly helps the model better map spectral images into a common feature space and accurately identify spectral changes. The Mamba model primarily extracts global image features, such as the overall waveform shape, peak distribution, entropy, peak mean, and variance.

[0073] Based on this, the neural network used in the present invention is a multi-scale Mamba network. Multi-scale means that different network layers have different input image pixels, and different features can be observed at different scales. This can be understood as the fact that people perceive different features at different distances from a fixed object. That is, when an object is imaged at different sizes in the field of view (i.e., at different scales), the features it exhibits are also different.

[0074] Preferably, in step A2, the size of the processed distance amplitude spectrogram is reshaped to 224 pixels × 224 pixels, a 3 × 3 convolution kernel is used in the convolutional neural network, and a 4 × 4 convolution kernel is used in the S6 block of the Mamba network. Combining the state space structure of the convolutional neural network and the Mamba, a dual-path module design is performed, wherein the dual-path modules are respectively: extracting local features using a convolutional neural network and extracting global features using a Mamba network. Specifically, the present invention can extract local features of the distance amplitude spectrogram, such as peak points, extreme points, slope, etc. of the spectrum waveform, through a convolutional neural network, and can extract global features of the distance amplitude spectrogram, such as the overall shape of the waveform, peak distribution, entropy, peak mean and variance, etc., through a Mamba network.

[0075] Preferably, the peak mean value is calculated as follows:

[0076]

[0077] Among them, x i Represents the amplitude value corresponding to the i-th point, where N represents the label of the last point of the horizontal axis in the figure (according to Figure 3.1 to Figure 3.3 In other words, the peak mean is obtained by summing all the amplitude values corresponding to the horizontal axis and dividing it by the total number of points in the five frames.

[0078] Preferably, the peak variance is calculated as follows:

[0079]

[0080] Preferably, the steps for calculating the sample entropy are as follows: first define the parameters r = 0.1*σ, m = 1 or 2 (here 2 is selected), then reconstruct the template vector X m (i) = {x(i), x(i+1), ..., x(i+m-1)}; then calculate the distance and define the vector X m (i) With X m The distance d[Xm(i),Xm(j)] between the two elements is the absolute value of the maximum difference between the two corresponding elements, that is,

[0081] d[X m (i),X m (j)]=max k＝0,…,m-1 (|x(i+k)-x(j+k)|);

[0082] For a given X m (i), statistics X m (i) With X m (j) The number of j whose distance between them is less than or equal to r (i.e. 1≤j≤Nm and j≠i), and is recorded as B i For 1≤i≤Nm, define:

[0083]

[0084] Then increase the template vector length to m+1 and repeat the above operation to get Then the sample entropy is calculated by the following formula:

[0085]

[0086] By using a dual-path approach to extract global and local features from the range-amplitude spectra of different ground types, the model can be enhanced to capture features at different scales, thereby improving model performance. This approach can more comprehensively capture signal characteristics, helping to improve classification and recognition accuracy.

[0087] Preferably, the present invention uses the Atrous convolution (hole convolution) scanning strategy in the Mamba network, which can be referred to as hole scanning. Atrous convolution is a special convolution operation that expands the receptive field by inserting holes in the convolution kernel (i.e., skipping some pixels) while keeping the number of parameters unchanged. The formula is as follows:

[0088]

[0089] Among them, f is the input feature map, K r is the expanded convolution kernel, (x, y) is the position of the output feature map, and r is the step size.

[0090] By adjusting the step size r, the size of the receptive field can be controlled and visualized. If the original scanning method (such as snake scanning) is used, the image is divided into 16 patches. For example, a 4×16 sequence will be obtained. If the hole scanning is used, a 2×8 sequence will be obtained. Figure 4 This operation not only saves multiple computational costs in state-space model sequence modeling, but also captures more extensive contextual information.

[0091] Furthermore, the specific process of scanning is as follows:

[0092] like Figure 4 As shown, the large square on the left is used as a reference standard. The large square is divided into 16 patches as a data image. The 16 patches are marked in order from 1 to 16. The two arrows on the right side of the large square point to two scanning processes respectively, among which the arrow on the upper side points to the serpentine scanning process, and the arrow on the lower side points to the hole scanning process.

[0093] Preferably, there are four directions for serpentine scanning. For example, taking 16 patches (4×4) as an example, the 16 patches of the large square are traversed and scanned in four directions. The resulting sequence is sorted according to the patch block labels corresponding to the scanning direction, resulting in a sequence length of 4×16. Hole scanning uses a method of skipping a patch block for scanning. There are two directions in total, distinguished by vertical dotted lines. For the first direction, patch blocks 1, 3, 9, and 11 are obtained by horizontal skip scanning, and patch blocks 6, 14, 8, and 16 are obtained by vertical skip scanning. After scanning in the first direction, the sequence of 1, 3, 9, 11, 6, 14, 8, and 16 will be obtained; for the second direction, patch blocks 2, 4, 10, and 12 are obtained by horizontal skip scanning, and patch blocks 5, 13, 7, and 15 are obtained by vertical skip scanning. Finally, the sequence of 2, 4, 10, 12, 5, 13, 7, and 15 will be obtained, with a total sequence length of 2×8.

[0094] From the above comparison, we can see that hole scanning significantly reduces the amount of data to be processed by skipping some sampling points on the feature map, while maintaining the global receptive field, thereby avoiding the loss of spatial resolution caused by traditional pooling or strided convolution.

[0095] Preferably, a patch sequence of the image is obtained by selective scanning (i.e., hole scanning). The scanned image patch sequence is linearly projected, discretized, and forward propagated through the S6 block of the standard Mamba network, which converts the input into a hidden state and continuously updates the hidden state and output. The state space model (SSM) of the Mamba network is expressed as follows:

[0096] h′(t)=Ah(t)+B x (t),

[0097] y(t)=Ch(t),

[0098] Among them, x(t) represents the input sequence, h(t) represents the hidden state of the current time step, y(t) represents the predicted output sequence, h'(t) represents the rate of change of the hidden state of the current time step, and A, B, and C are all learnable parameter matrices.

[0099] Next, the discretization operation is performed. The purpose of discretization is to convert the continuous differential equation into a discrete function so that the model is aligned with the sampling frequency of the input signal, thereby improving computational efficiency. Discretization is defined as follows:

[0100]

[0101] in, where represents the discretized A, B, and C matrices, respectively; I is the identity matrix; and △ represents the sample time scale. Furthermore, the discretization formula for the B matrix includes an exact expression and an approximate expression. The approximate expression improves efficiency by simplifying computations. This example process is a visual task requiring processing high-resolution inputs with extremely long sequences. Using an exact expression would dramatically increase computational complexity. Therefore, the present invention uses an approximate expression when discretizing the B matrix, sacrificing some accuracy for computational simplification.

[0102] To simplify the computation, a global convolution method is used to perform repeated applications of the above discretization process simultaneously, as follows:

[0103]

[0104] in, represents convolution calculation, Represents the SSM kernel, and the discretized SSM formula is as follows:

[0105]

[0106] Furthermore, the processed outputs are combined to reconstruct the global structure of the feature map.

[0107] This paper uses the Mamba network architecture to extract global features, which significantly improves the adaptability, computational efficiency, and lightweightness of the model compared to the traditional SSM architecture. Specifically, the Mamba network architecture has the following three advantages:

[0108] (1) In the original SSM architecture, the trained A, B, and C matrices do not change once training is completed, which limits the model's adaptability to different inputs. The Mamba network parameterizes the A, B, and C matrices, allowing them to be dynamically adjusted and updated based on different inputs. This flexibility greatly enhances the model's adaptability and robustness to variable inputs, enabling it to better cope with complex and diverse data environments.

[0109] (2) Mamba network defines a new hardware scanning method and introduces a new calculation process, such as the following demonstration formula:

[0110]

[0111] This formula not only demonstrates the new computational process but also demonstrates that parallel operations can be implemented during both training and inference, significantly increasing the overall model speed. Through this efficient parallel computing mechanism, the Mamba network significantly reduces processing time while maintaining accuracy, improving real-time performance.

[0112] (3) The Mamba network adopts a simpler SSM architecture design. This simplified structure not only reduces the number of model parameters and computational complexity, but also ensures the lightweight nature of the model. Compared with the traditional SSM, the Mamba network effectively reduces resource consumption while maintaining powerful feature extraction capabilities, making it more suitable for resource-constrained scenarios such as mobile devices or embedded systems.

[0113] Preferably, the local features (map) and the global features (map) are fused through the channel attention block (SE block) to obtain multi-scale features. The outputs of each SE block are combined by element-wise summation and expressed as a dual path. The formula is as follows:

[0114] X l+1 =SE(SSM(X l ))+SE(Conv(X l )),

[0115] Among them, X l denotes the feature map of layer l, SE(·) denotes the SE block operation. Each path uses one SE block to ensure that the respective features of global and local information are dynamically rebalanced to emphasize the most salient features. This fusion aims to preserve the integrity of the broad global view and the intricate local view to achieve comprehensive feature representation.

[0116] Furthermore, the semantic token learner (STL) and semantic token fuser (STF) are used to combine deep semantic information to obtain features with deep semantic information. Among them, STL is specifically used to abstract and compress sequence input into a set of representative semantic tags, and the sequential attention mechanism is used to enhance the focus and relevance of features.

[0117] The specific process is as follows Figure 5 As shown: First, a sequential attention module is used to highlight the salient features of the input feature S using one-dimensional convolution and find the maximum value S of S max and the mean value S ave ,Right now:

[0118] S max =max(S),

[0119] S ave =mean(S).

[0120] Secondly, these features are connected, 1D convolution is performed, and then sigmoid function activation is performed to generate the feature weighted map m:

[0121] m=Sigmoid(Conv1d(Concat(S max, S ave ))).

[0122] The weight map is then applied to the original input:

[0123]

[0124] in, represents the Kronecker product, and n represents the weighted feature.

[0125] Furthermore, the process of converting the weighted feature n into semantic tokens is as follows:

[0126] A=n⊙U1,

[0127] V=n⊙U2,

[0128] u=Softmax(A T ⊙V),

[0129] Among them, U1 and U2 are weight matrices initialized using Gaussian distribution, ⊙ represents the inner product, A and V represent the length-level and dimension-level semantic feature matrices, and u represents the final output feature.

[0130] STL and STF operations help downsampling and capture key high-level features for downstream tasks.

[0131] The final output features obtained after the last processing are converted into probability distribution through the fully connected layer and softmax function to obtain the final prediction result (grass, asphalt road, wooden floor, etc.).

[0132] Preferably, the fully connected layer is a key component in the neural network, used to map the extracted features to the final classification results. The specific steps are as follows:

[0133] (1) Feature flattening: Flatten the multi-dimensional feature map output by the convolutional layer into a one-dimensional vector.

[0134] (2) Linear transformation: The flattened feature vector is input into the fully connected layer. Each neuron in the fully connected layer is connected to each element of the input vector, and a linear transformation is performed through the weight matrix and the bias vector. Assume that the fully connected layer has N neurons and the length of the input vector is M. The size of the weight matrix is N×M, the length of the bias vector is N, and the calculation formula for the linear transformation is:

[0135] z=Wx+b,

[0136] Where W is the weight matrix, x is the input vector, b is the bias vector, and z is the output vector after linear transformation.

[0137] (3) Nonlinear activation: The activation function (ReLU) is used in the fully connected layer to introduce nonlinearity, enabling the neural network to learn more complex features. The ReLU function is defined as:

[0138] ReLU(z)=max(0,z).

[0139] The output vector after the activation function is:

[0140] a=ReLU(z),

[0141] Among them, a is the output vector after activation.

[0142] Preferably, the softmax function is used to convert the output of the fully connected layer into a probability distribution. The specific steps are as follows:

[0143] (1) Calculate the exponent: Calculate the exponent (power of e) for each element in the output vector a of the fully connected layer. Assume that the length of the output vector a is N, then calculate the exponent of each element:

[0144]

[0145] Among them, a i is the i-th element of the output vector.

[0146] (2) Normalization: Calculate the sum of all indices:

[0147]

[0148] Divide each index by the sum S to get the probability of each class:

[0149]

[0150] Based on this, each element in the output vector represents the probability that the sample belongs to the corresponding category, and the sum of all probabilities is 1. The category with the highest probability corresponding to the sample is the final predicted category of the sample.

[0151] For example, during the training phase, labels are typically represented using a one-hot encoding. In this example, a class is represented by a five-dimensional vector, where (1,0,0,0,0) indicates a probability of 1 for the first class and a probability of 0 for the second, third, fourth, and fifth classes. For a five-class classification problem, the labels can be represented as follows: Class 0: Grass: (1,0,0,0,0); Class 1: Ceramic Tile: (0,1,0,0,0); Class 2: Floor Tile: (0,0,1,0,0); Class 3: Asphalt Pavement: (0,0,0,1,0); Class 4: Wooden Floor: (0,0,0,0,1).

[0152] Taking the wooden floor as an example, the weight matrix dimension in the experiment is 5×384, the input feature vector length is 384, and the data volume is relatively large. Here, a compressed version of the W weight matrix is used to illustrate the feature x = (0.5, -0.3, 0.2, 0.1, -0.4)' obtained by the multi-scale module and the semantic module. The weight matrices W and b of the fully connected layer are as follows:

[0153] b=(0.1,0.2,0.3,0.4,0.5).

[0154] From the formula z=Wx+b', we can calculate z=(-0.01, 0.14, 0.29, 0.44, 0.59). Substituting the activation function ReLU(z)=max(0, z) and a=ReLU(z), we can get a=(0, 0.14, 0.29, 0.44, 0.59).

[0155] We further calculate the exponent of each element, which is (1, 1.15, 1.34, 1.55, 1.8), and then calculate softmax(a) using the following formula:

[0156]

[0157] After calculation, softmax(a) = (0.1462, 0.1681, 0.1959, 0.2266, 0.2632). It can be seen that the probability of this data belonging to the first category is 14.62%, the probability of belonging to the second category is 16.81%, the probability of belonging to the third category is 19.59%, the probability of belonging to the fourth category is 22.66%, and the probability of belonging to the fifth category is 26.32%. Because the probability of the fifth category is the highest, the classification result of this embodiment is the fifth category, and the corresponding one-hot encoding is (0, 0, 0, 0, 1), that is, the predicted result is wooden floor.

[0158] If the data is collected in a normal environment, the peak values, peak means and peak variances of these types of data are quite different. Then the peak values, peak means and peak variances can be used as features to distinguish them. If the data is collected in a night environment, taking the wooden floor as an example, Figure 8 (a) is the wooden floor data collected in daytime environment. Figure 8 (b) is the wooden floor data collected at night. By comparison, it can be seen that the peak value, peak mean, and peak variance of the two similar data are very small, indicating that the data collected by the millimeter-wave radar during the day and at night are relatively small. Therefore, the peak value, peak mean, and peak variance can be used as characteristic features to distinguish data categories.

[0159] Preferably, the overall internal structure diagram of the multi-scale Mamba network used in the present invention is as follows: Figure 6 As shown. For example, all the models of the above network are merged, and the distance amplitude spectra of different ground types after Range-FFT processing are input, and the image is reshaped to 224×224 size. In the data enhancement and processing steps, common enhancement methods and normalization such as random 10-degree rotation, Gaussian blur, changes in color saturation and brightness, and Mixup are added. The optimizer is set to adamW5, and the learning rate adjustment strategy is selected as cosine annealing. After setting the above steps, the number of training rounds is defined as 100. After training, it can be seen from the accuracy graph and the Loss graph that the network can eventually achieve a recognition accuracy of 96% to 97%, and the Loss is finally reduced to between 0.4 and 0.5. The results are shown as follows. Figure 7 shown.

[0160] Example 2

[0161] This embodiment is a further improvement of embodiment 1, and repeated contents will not be repeated here.

[0162] The present invention also discloses a method for applying a millimeter-wave radar-based ground type recognition algorithm to a lawn mower. This algorithm identifies ground type and automatically adjusts the lawn mower's operating state based on the identified ground type. In other words, the present invention also provides a technique for applying millimeter-wave radar to ground type recognition and classification in a lawn mower system.

[0163] Preferably, the application method of the present invention may further include one or more of the following steps based on the algorithm described in Example 1:

[0164] A0. Configure a millimeter-wave radar and use it to collect millimeter-wave radar data.

[0165] A4. Adjust the working state of the lawn mower based on the prediction results output by the recognition algorithm.

[0166] Preferably, step A0 can be performed before step A1, so that the millimeter-wave radar data collected by the millimeter-wave radar configured in step A0 can be pre-processed in step A1.

[0167] Exemplarily, the present invention can use a millimeter-wave radar sensor with a 60GHz operating frequency band to collect ground type data. The sensor is configured with a two-transmit and three-receive antenna layout to ensure the accuracy and reliability of data collection. The installation height of the millimeter-wave radar is set to 22cm from the ground, and it is tilted 30° toward the ground perpendicular to the ground to optimize the recognition ability of different ground types. The sampling frame rate of the millimeter-wave radar can be set to 10Hz, with 1 chirp per frame and L=26 points per frame. In order to ensure the accuracy of the ground type data and avoid interference from irrelevant signals, the radar data is intercepted in the distance dimension within the set distance range according to the pre-set distance range (0.1m~4m), without retaining the data of the entire distance.

[0168] Existing radar technologies mostly use lidar to collect ground type information. Compared to existing radar technologies, millimeter-wave radar is relatively insensitive to environmental interference sources, such as raindrops and dust particles, which do not significantly interfere with its detection performance. Therefore, it can more stably identify ground types. In contrast, lidar is susceptible to interference from these environmental factors, affecting its accurate perception of ground features. These factors are crucial to the accuracy of the collected data. Furthermore, millimeter-wave radar has a longer detection range, allowing it to detect ground types at greater distances in advance.

[0169] Preferably, step A4 can be performed after step A3, so that the prediction result outputted in step A3 can be used as the basis and basis for regulating the working state of the lawn mower. Furthermore, if the output prediction result is grass, the lawn mower is driven to work; otherwise, it is not driven to work.

[0170] It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention specification and its drawings are illustrative and do not constitute a limitation on the claims. The scope of protection of the present invention is defined by the claims and their equivalents. The present invention specification contains multiple inventive concepts, such as "preferably" or "according to a preferred embodiment", which means that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application based on each inventive concept. Throughout the text, the features guided by "preferably" are only an optional method and should not be understood as having to be set. Therefore, the applicant reserves the right to abandon or delete the relevant preferred features at any time.

Claims

1. The millimeter wave radar-based lawn mower ground type recognition algorithm is characterized by: It includes: A1. Perform preprocessing and range-dimensional fast Fourier transform on the collected millimeter-wave radar data to generate multi-channel spectrum data. Based on this multi-channel spectrum data, the amplitude value of each range bin is calculated. The amplitude mean of this multi-channel spectrum data is then calculated to obtain distance information about the ground type. This information is then used to construct a range-amplitude spectrum. A2. Take the range-magnitude spectrogram as input, extract local features of the range-magnitude spectrogram using a convolutional neural network, and extract global features of the range-magnitude spectrogram using a Mamba network. A3. Feature fusion of local and global features is performed through the channel attention block. Deep semantic information is combined using the semantic token learner and semantic token fuser. After processing through the sequential attention mechanism, a feature weighted graph is generated. Finally, the features are mapped to a probability distribution through a fully connected layer and a softmax function to identify the ground type and output the prediction result.

2. The algorithm according to claim 1, characterized in that When preprocessing millimeter-wave radar data, each frame of the signal is windowed using a Hanning window. The Hanning window reduces spectrum leakage by applying a smooth transition amplitude attenuation at both ends of the signal.

3. The algorithm according to claim 1 or 2, characterized in that Based on the data in the range-amplitude diagram matrix and the distance information of the ground type, a range-amplitude spectrum diagram is drawn with the distance information as the horizontal axis and the amplitude value as the vertical axis to intuitively display the distribution of target reflection intensity at different distances.

4. The algorithm according to any one of claims 1 to 3, characterized in that: The Mamba network adopts a scanning strategy of dilated convolution to form a simpler sequence structure by skipping sampling points, wherein the scanning process of the dilated convolution scanning strategy includes two directions: horizontal skip scanning and vertical skip scanning.

5. The algorithm according to any one of claims 1 to 4, characterized in that: The patch sequence of the image is obtained by hole scanning, and then the patch sequence of the image is linearly projected, discretized and forward propagated through the S6 block of the standard Mamba network to convert the input into a hidden state, and continuously update the hidden state and output.

6. The algorithm according to any one of claims 1 to 5, characterized in that: The parameter matrices A, B, and C in the state space equation of the Mamba network are dynamically updated learnable parameters, and the state space equation is converted into:

7. The algorithm according to any one of claims 1 to 6, characterized in that: The channel attention block performs feature fusion by element-wise summation, and the formula is as follows: X l+1 =SE(SSM(X l ))+SE(Conv(X l ))。 8. The algorithm according to any one of claims 1 to 7, characterized in that: When the semantic token learner executes the sequential attention mechanism, the weighted processing of the input feature S includes: calculating the maximum value S max and the mean value S ave , after one-dimensional convolution and sigmoid activation, the feature weighted graph m is generated and applied to the original input to obtain the weighted feature 9. The algorithm according to any one of claims 1 to 8, characterized in that: The process of converting weighted features n into semantic tokens is as follows: A=n⊙U1,V=n⊙U2,u=Softmax(A T ⊙V)。 10. The application method of the millimeter wave radar-based lawn mower ground type recognition algorithm according to any one of claims 1 to 9, characterized in that: It includes one or more of the following steps: A0. Configure a millimeter-wave radar and use it to collect millimeter-wave radar data. A4. Adjust the working state of the lawn mower based on the prediction results output by the recognition algorithm.

Images