Greenhouse inspection method based on multi-target fusion

Abstract

The invention discloses a greenhouse inspection method based on multi-target fusion, and the method comprises the following steps: carrying a laser radar by an inspection robot to collect the internal environment information of a greenhouse, and constructing a three-dimensional grid map through employing a LeGO-LOAM algorithm; s2, according to the grid map in the S1, finishing path planning of the inspection robot by adopting a TTAO (Triangular Topology Aggregation Optimization) algorithm; the inspection robot is provided with a sensor for measuring the temperature and humidity, the illuminance and the CO2 value of the surrounding environment, a ZED binocular camera is provided for collecting image information of the plant, an improved TransNeXt model is adopted for recognizing the collected image information, and the growth condition of the plant is analyzed. According to the invention, the growth condition of the plants in the greenhouse can be detected more accurately and more efficiently, the labor cost is reduced, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a greenhouse inspection method based on multi-target fusion. Background Technique

[0002] With the growth of the global population and the acceleration of the urbanization process, the demand for efficient agriculture is increasing day by day. As an efficient agricultural production facility, greenhouse can provide a stable growth environment and improve the yield and quality of planted crops. However, the traditional greenhouse management mainly relies on manual labor, which has the disadvantages of low efficiency, high cost, and being easily affected by subjective factors. In recent years, significant progress in sensor technology, robot technology, data processing and analysis technology, etc. has provided a solid foundation for the development of the greenhouse rail inspection system. Therefore, an automated and intelligent greenhouse rail inspection system has emerged to meet the high-efficiency management requirements of modern agricultural production.

[0003] The techniques of plant image acquisition and internal path planning in greenhouse are important technical means for inspection robots to achieve more accurate and efficient detection of the growth conditions of plants. In reality, greenhouses often cover a large area. By relying on radar to sense the surrounding specific environment, obtain image data information, and draw a two-dimensional map, the inspection robot can drive along the best path, shorten the driving time, improve the harvesting efficiency, and reduce costs in all aspects. Different types of plants have significant differences in morphology, texture, color, etc., and have different requirements for environmental parameters. It is necessary to combine multiple sensors to collect environmental information and models for image recognition, and to realize the real-time monitoring of the growth conditions of plants according to multiple indicators, resulting in low efficiency. Summary of the Invention

[0004] Object of the Invention: In order to overcome the deficiencies of the background technique, the present invention requests to disclose a greenhouse inspection method based on multi-target fusion, which can improve the efficiency of detecting the growth conditions of plants inside the greenhouse.

[0005] Technical Solution: The greenhouse inspection method based on multi-target fusion disclosed by the present invention includes the following steps:

[0006] S1. The inspection robot is equipped with a lidar to collect the internal environmental information of the greenhouse, and the LeGO-LOAM algorithm is used to construct a three-dimensional grid map;

[0007] S2. According to the grid map in S1, the TTAO algorithm of the triangular topology aggregator optimizer is used to complete the path planning of the inspection robot;

[0008] S3. The inspection robot is equipped with sensors to measure the temperature, humidity, illuminance, and CO2 values of the surrounding environment, and is equipped with a ZED binocular camera to collect image information of the planted plants. The improved TransNeXt model is used to identify the collected image information and analyze the growth of the planted plants.

[0009] Further, the following method is used to collect the internal environment information of the greenhouse in S1:

[0010] The lidar is controlled by a DC motor to operate. The lidar laser emits light and projects it onto the entire FOV area, and the receiver receives the emitted light. The distance between the lidar and the obstacle is the product of the time interval between laser emission and reception and the speed of light. The expression is referred to the following formula:

[0011]

[0012] Among them, P r is the power received by the receiver, P0 is the transmitted power, A r is the area of the receiver receiving hole, γ is the loss of the air to the laser beam, ρ is the reflectivity of the obstacle, η0 is the optical system efficiency, and R is the distance between the obstacle and the lidar;

[0013] The single-beam laser emitter of the lidar rotates at a constant speed. During the rotation, a laser beam is emitted every certain time. After the lidar rotates one week, a complete point cloud is obtained to construct a map. The encoder receives the rotational speed data of the DC motor, and the speed and pose data are obtained from the motor speed, and finally the odometer data is obtained.

[0014] Further, the LeGO-LOAM algorithm is used to construct a three-dimensional grid map in S1, which specifically includes the following steps:

[0015] Use the point cloud segmentation module to separate the ground point cloud and at the same time cluster the remaining point clouds, filtering out the point cloud clusters with fewer numbers. Specifically: First, project the original point cloud scanned by the lidar into a depth map, then judge the ground points by calculating the pitch angle between the point clouds, and finally use the image-based segmentation method to cluster the depth map;

[0016] Use the feature extraction module to extract the edge point and plane point features of the segmented point cloud. Specifically: For a continuous point cloud formed by the lidar scan line, based on the coordinates of several point clouds before and after each point cloud, calculate the curvature of each point cloud, and distinguish the plane feature points and edge feature points through the curvature. The curvature calculation in the algorithm is measured by the following formula, and it is defined as the feature degree c:

[0017]

[0018] Among them, S represents the number of point clouds participating in curvature calculation, and S / 2 point clouds are selected from before and after this point; r represents the distance measurement value of the point cloud;

[0019] The lidar odometry module is used to find the pose transformation matrix between consecutive frames through feature matching, which mainly includes two parts: feature matching and pose estimation. Specifically: for a certain point A in the current frame edge feature point set in the previous frame edge feature point set the two closest points are selected. Points B and C are in different scan lines. Assuming the coordinates of points A, B, and C are respectively denoted as then the distance calculation formula from the point to the feature line is:

[0020]

[0021] For a certain point A in the current plane feature point set in the previous frame plane feature point set three adjacent points are selected. Assuming the coordinates of points A, B, C, and D are respectively denoted as Similarly based on the equal area method, the distance calculation formula from the point to the feature plane is:

[0022]

[0023] For the pose estimation problem, the LeGO-LOAM algorithm first constructs the first optimization problem using the constraint relationship between plane points to optimize t z , θ roll , θ pitch Then, it constructs the second optimization problem using edge point constraints and, based on the t z , θ roll , θ pitch obtained from the first step of optimization, estimates t y , y z , θ yaw Finally, through two-step L-M optimization, a 6-degree-of-freedom pose transformation matrix [t z , t y , t z , θ roll , θ pitch , θ yaw is obtained;

[0024] The lidar mapping module is used to further process the features and then register them in the global point cloud map. The global map is the final output result of SLAM. In LeGO-LOAM, the point cloud and pose of each frame are associated and saved separately, which is represented as a set in the code. That is, at time t, the map created for the previous t - 1 moments is:

[0025]

[0026] Adopt a pose transformation fusion module to fuse the results of lidar odometry and lidar mapping and output the final pose estimation.

[0027] Further, S2 specifically includes the following steps:

[0028] S2-1. Population initialization:

[0029] Each vertex in the triangular topology unit represents a search agent. Let the population size, that is, the planted area, be N, and the variable dimension be D. In the TTAO algorithm, the number of individuals N can be divided into N / 3 triangular topology units, where [·] represents rounding down, and the extra individuals are randomly generated in the search space. In the initialization stage, N / 3 agents are randomly generated in the feasible region. The mathematical expression for each individual generation is:

[0030]

[0031] In the formula represents the first search individual in the i-th triangular topology unit, and i is a positive integer between 1 and N / 3; r0 represents a random number between [0, 1]; and are the upper and lower bounds of the variable, and their mathematical expressions are defined as:

[0032]

[0033] S2-2. Formation of triangular topology units:

[0034] Use the first vertex as the starting vertex in the spherical coordinate system to locate a length of The coordinate system forms the second vertex, and the generated length is The direction vector rotates counterclockwise by π / 3, and then the third vertex is obtained through coordinate transformation. The expressions of these vertices are:

[0035]

[0036] where l represents the size of the triangular topology unit, and is mathematically expressed as:

[0037]

[0038] In the formula, t represents the current iteration number; T represents the maximum number of iterations; l decreases as the iteration number increases but does not decrease to 0; and represent the direction vectors of the other two sides guided by the first point, and the formula is:

[0039]

[0040] wherein and θ j (j = 1,..., D) are random numbers between [0, π].

[0041] Each group of triangular topological units aggregates internally into a fourth vertex, which is formed in a linearly weighted manner, thus using individual information. The fourth vertex is defined as:

[0042]

[0043] where r1, r2, and r3 are random numbers between [0, 1], and r1 + r2 + r3 = 1. Therefore, the fourth search agent is located inside each triangular topological unit;

[0044] S2 - 3. General aggregation:

[0045] Utilize the geometric properties of triangles to aggregate information in the population and update the individual positions;

[0046] Collect the information of excellent individuals in different triangular units and generate new feasible solutions. Information interaction occurs between the best individual in each triangular topological unit and the best individual in any randomly selected group of units. There is a linear combination with different weights between each dimensional variable of the two positive individuals. The new individual is generated in the better connection of the two vertices, which is mathematically expressed as:

[0047]

[0048] where r4 is a random number between [0, 1]; and represent the best individual of unit i and the unit randomly selected at the t - th iteration. Meanwhile, compare the fitness value of with the optimal or sub - optimal search agent and update the optimal agent. Assuming a minimization problem, the mathematical expressions for the optimal and sub - optimal individuals updated in the (t + 1) th iteration are:

[0049]

[0050] wherein represents the sub - optimal individual at the i - th iteration; f(·) is the function of the given problem;

[0051] S2 - 4. Local aggregation:

[0052] Generate new vertices inside the triangle through the method of constructing similar triangles to further optimize the individual positions;

[0053] Each group is re-searched within a certain local area to utilize each topological triangular unit. The calculation of the new vertex is as follows:

[0054]

[0055] In the formula, α decreases to adjust the size of the aggregation range. The calculation formula of α is:

[0056]

[0057] If the new individual is better than the original individual, update the position; otherwise, do not update. The corresponding mathematical expression is as follows:

[0058]

[0059] Furthermore, in S3, an STM32 single-chip microcomputer is selected as the control core of the inspection robot, and it is composed of a DHT11 temperature and humidity sensor module, a BH1750 light sensor module, a T6004 carbon dioxide sensor module, and a ZED binocular camera. The single-chip microcomputer transmits the collected environmental parameters to the upper computer module through the data transmission module, so as to achieve intelligent monitoring of the greenhouse environment. At the same time, temperature and humidity, illuminance, and CO2 concentration value alarms are set on the inspection robot. When the effective parameter value exceeds the warning value, the buzzer continuously alarms.

[0060] Furthermore, in S3, an improved TransNeXt model is used to identify the image information collected by the ZED binocular camera. The specific steps for analyzing the growth of plants are as follows:

[0061] For a given input [C, H, W], a set of pixels within a sliding window centered on the pixel at (i, j) is defined as ρ(i, j). The fixed window size is k×k, ||ρ(i,j)|| = k 2 , and the pixel set obtained through feature map pooling is defined as σ(X). The given pooling size is H p ×W p , ||σ(X)|| = H p W p , and the pixel focus attention is described as follows:

[0062]

[0063] In the formula, S represents the attention score; Q represents the query vector; K represents the key vector;

[0064]

[0065] A (i,j)～ρ(i,j) , A (i,j)～σ(X) =Split(A (i,j))with size[k 2 ,H p W p

[0066] PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(X)

[0067] where A represents the attention weight; Concat represents the operation of concatenating two vectors; d represents the dimension of the vector; B represents the position bias; Split represents splitting the attention weight A (i，j) into two parts, corresponding to the attention weights of the local window and the global features respectively; X represents the position input feature; V represents the value vector;

[0068] Before feature map pooling, a single-layer neural network is used for projection and activation to compress and extract useful information in advance. After pooling, layer normalization is used again to normalize the output to ensure the variance consistency of X and σ(X). The downsampling operator is called the activation pool and is expressed as:

[0069] σ(X)=LayerNorm(AvgPool(GELU(Linear(X))))

[0070] When using the TransNeXt model for image recognition, first preprocess the collected plant images, such as adjusting the image size and normalization. Then input the processed images into the model, and extract multi-scale features of the images through convolutional layers and Transformer encoder components. The multi-scale token generation and self-attention mechanism enable the model to focus on key regions of the image, such as the leaves and fruits of plants. These features are fused at multiple scales, and finally the image is classified or recognized by a classifier to obtain the final classification result or corresponding information of the image, realizing the efficient and accurate recognition of the growth state of plants.

[0071] Furthermore, dynamic convolution is introduced into the convolutional GLU module of TransNeXt to further improve the performance and robustness of the model in the plant image recognition task. The specific steps are as follows:

[0072] Define a dynamic convolution module, which contains an attention mechanism and multiple convolutional kernels. The attention mechanism is used to generate the weights for each input feature map, and these weights are used to dynamically combine the convolutional kernels;

[0073] Integrate the dynamic convolution module into the convolutional GLU module, replacing the original static convolution part, that is, using dynamic convolution in the convolutional GLU module to generate channel attention weights; ​

[0074] Integrate the modified convolutional GLU module into the TransNeXt model, replacing the original convolutional GLU module, that is, use the new convolutional GLU module in each block of TransNeXt.

[0075] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows:

[0076] 1. The present invention combines the inspection robot and the lidar, giving full play to the respective roles of the robot and the lidar, and improving the reliability and robustness of detection.

[0077] 2. The present invention uses the LeGO-LOAM algorithm to construct a three-dimensional grid map, and realizes more precise positioning and map construction by combining a point cloud segmentation module, a feature extraction module, a lidar odometry module, a lidar mapping module, and a pose transformation fusion module.

[0078] 3. The present invention uses the triangular topology aggregation optimizer TTAO algorithm to complete the path planning of the inspection robot, enabling the inspection robot to complete the detection of planted crops according to the optimal path. This algorithm is a novel mathematics-based metaheuristic algorithm that gradually optimizes the search space through continuous iteration and finally finds the global optimal solution.

[0079] 4. The present invention uses the STM32 single-chip microcomputer as the control core of the inspection robot, and sets multiple sensors to achieve precise detection of parameter changes such as temperature, humidity, light, and CO2 in different areas, so as to timely obtain the growth conditions of planted crops and provide a good growth environment for planted crops.

[0080] 5. The present invention uses the improved TransNeXt model to identify the collected image information and judge the growth conditions of planted crops. TransNeXt is a hierarchical vision Backbone network mainly used for image classification and similar vision tasks, which includes aggregation attention as a Token mixer and convolutional GLU as a Channel mixer.

[0081] 6. The present invention introduces dynamic convolution into the convolutional GLU module of the TransNeXt model, thereby further improving the performance and robustness of the model in the task of planted crop image recognition. Description of the drawings

[0082] Figure 1 It is the general flow chart of the method of the present invention;

[0083] Figure 2 It is the structural block diagram of the inspection robot equipped with sensors;

[0084] Figure 3 It is the structural block diagram of the improved TransNeXt model after introducing dynamic convolution. Detailed implementation mode

[0085] A greenhouse inspection method based on multi-objective fusion, as Figures 1-3 shown, the implementation of this solution is based on an orbital inspection robot.

[0086] To achieve functions such as walking, power taking, communication, and positioning, the walking components of the inspection robot are mainly divided into 3 parts.

[0087] (1) Main walking components: The main body is driven by a motor. First, a spring is installed inside the structure to ensure that the walking components can closely fit the track surface; in addition, guide wheel devices are installed at both ends and on both sides of the walking components to ensure that the inspection robot can accurately and smoothly move along the track during walking and turning, avoiding deviation. The mutual cooperation between the two components ensures that the walking components will not shake violently during walking.

[0088] (2) Fixed bracket for power taking and communication device: A sliding contact arm made of copper alloy is installed on this bracket, which cooperates with the power supply and communication sliding contact lines of the track to achieve the purpose of power taking and communication.

[0089] (3) Fixed bracket for laser barcode scanner: By installing a laser barcode scanner on this bracket, the barcode on the track surface is scanned to achieve precise positioning of the inspection robot.

[0090] Step 1: The lidar configured on the inspection robot's arm measures the distance of an object by emitting a laser beam and receiving the reflected signal to obtain the internal environmental information of the greenhouse for functions such as environmental perception, obstacle detection, and map construction. The lidar starts working through a DC motor to collect information around the inspection robot. The lidar laser emits light rays that project onto the entire FOV area, and the receiver receives the emitted light rays. The distance between the lidar and the obstacle is the product of the time interval between laser emission and reception (also known as the flight time of the laser beam) and the speed of light. The expression can refer to the following formula:

[0091]

[0092] where, P r is the power received by the receiver, P0 is the transmitted power, A r is the area of the receiver's receiving hole, γ is the loss of the air to the laser beam, ρ is the reflectivity of the obstacle, η0 is the optical system efficiency, and R is the distance between the obstacle and the lidar.

[0093] The single-beam laser emitter of the lidar rotates at a constant speed. During the rotation, a laser beam is emitted every certain period of time. After the lidar rotates one week, a complete frame of point cloud can be obtained to construct a map. The encoder receives the rotational speed data of the DC motor, and from the motor speed, the speed and pose data are obtained, and finally the odometer data is obtained.

[0094] Before the data collected by the lidar can obtain a complete three-dimensional grid map, it needs to be optimized through the LeGO-LOAM algorithm before mapping can be finally achieved.

[0095] LeGO-LOAM is a new algorithm derived from the LOAM framework. It is optimized based on the ground on the basis of LOAM and a simple loop detection is added. Its main improvements are in two aspects: lightweight and ground optimization, with loop detection capabilities. It can receive the input from a 3D lidar and output a 6-degree-of-freedom pose estimate. The entire software system is divided into 5 parts:

[0096] The first part is the point cloud segmentation module (Segmentation): The main operation of this part is to separate the ground point cloud; at the same time, the remaining point cloud is clustered to filter out the point cloud clusters with a small number of points.

[0097] In the LeGO-LOAM algorithm, point cloud segmentation is implemented using a range image. The range image is obtained by projecting the original point cloud. The resolution of the range image is related to the parameters of the lidar. Assuming that the horizontal angular resolution of the lidar is δ and the number of vertical lines (laser transceiver modules) is n, the resolution of its range image should be Therefore, the range image represents the scanning result of the lidar, and each pixel of the range image corresponds to an original point cloud.

[0098] The main steps of point cloud segmentation based on the range image in the LeGO-LOAM algorithm are as follows: First, project the original point cloud obtained by lidar scanning into a range image, then judge the ground points by calculating the pitch angle between point clouds, and finally use an image-based segmentation method to cluster the range image.

[0099] The second part is the feature extraction module (Feature Extraction): Edge point and plane point features are extracted from the segmented point cloud (the ground point cloud has been separated).

[0100] The specific feature extraction idea is as follows: For a continuous point cloud formed by the lidar scanning line, based on the coordinates of several point clouds before and after each point cloud, the curvature of each point cloud is calculated, and the plane feature points and edge feature points are distinguished by the curvature. The curvature calculation in the algorithm does not use the curvature calculation formula in mathematics, but measures the curvature of the point cloud through the following formula, defining it as the feature degree c:

[0101]

[0102] Among them, S represents the number of point clouds participating in the curvature calculation, and S / 2 point clouds are selected from before and after this point; r represents the distance measurement value of the point cloud. It can be seen that for planar points, the distance measurement values r of their adjacent point clouds are similar, and the calculated curvature value c will be very small; on the contrary, for edge points, the distance measurement values r of their adjacent point clouds vary greatly, and the curvature result calculated by the above formula will also be very large. Therefore, by setting a curvature threshold, the point cloud can be divided according to the calculation result of the above formula.

[0103] The third part is the lidar odometry module (Lidar odometry): The pose transformation matrix between consecutive frames is found by performing (edge point and plane point) feature matching between consecutive frames, which mainly includes two parts: feature matching and pose estimation.

[0104] The principle of feature matching is to use the scan-to-scan method to achieve the registration of point clouds between two consecutive frames, and find the corresponding features in the feature set from the previous scan in the set. is the set of edge feature points in the previous frame, i.e., the feature line, is the set of plane feature points in the previous frame, i.e., the feature plane. Therefore, the matching of edge feature points between two adjacent frames is transformed into the problem of calculating the shortest distance from a point to a line; the matching of plane feature points is transformed into the problem of calculating the shortest distance from a point to a plane. The matching principles of these two types of feature points are analyzed separately below.

[0105] For a certain point A in the current frame edge feature point set , select the two closest points in the previous frame edge feature point set , and require that point B and point C are in different scan lines, so as to ensure that the three points can definitely form a triangle. Assume that the coordinates of points A, B, and C are respectively denoted as Then the distance calculation formula from a point to the feature line is:

[0106]

[0107] For a certain point A in the current plane feature point set , select three adjacent points in the previous frame plane feature point set . Assume that the coordinates of points A, B, C, and D are respectively denoted as Similarly based on the equal area method, the distance calculation formula from a point to the feature plane is:

[0108]

[0109] For the pose estimation problem, the LeGO-LOAM algorithm adopts a two-step L-M (Levenberg-Marquardt) optimization operation. The LeGO-LOAM algorithm first constructs the first optimization problem using the constraint relationship between planar points and optimizes t z , θ roll , θ pitch . Then, it constructs the second optimization problem using edge point constraints and, based on t z , θ roll , θ pitch obtained from the first step of optimization, estimates t y , y z , θ yaw . Through two-step L-M optimization, a 6-degree-of-freedom pose transformation matrix [t z , t y , y z , θ roll , θ pitch , θ yaw is finally obtained.

[0110] Part 4: Lidar Mapping module: Further process the features and then perform registration in the global point cloud map.

[0111] The global map is the final output result of SLAM. In LeGO-LOAM, the point cloud and pose of each frame are associated and saved separately, which is represented as a set in the code. That is, at time t, the map created for the previous t - 1 moments is:

[0112]

[0113] The local map mainly serves for further optimization of inter-frame motion estimation and is composed of the point cloud near the current frame. The selection method of the point cloud is as follows:

[0114] (1) Loop closure detection: Then add the past 50 key frames and splice them into a point cloud, which is equivalent to a sliding window.

[0115] (2) Pose optimization: Then add the 50 key frames with the closest Euclidean distance to the current one and splice them into a point cloud.

[0116] The matching optimization of scan-2-map based on the local map is essentially the same as scan-2-scan in lidar odometry, except that the matching object with the current frame is changed from the previous frame point cloud to the local map point cloud.

[0117] Part 5 Pose Transformation Fusion Module (Transform Integration): Fuse the results of lidar odometry and lidar mapping to output the final pose estimation.

[0118] The process of constructing a three-dimensional grid map inside the greenhouse using the LeGO-LOAM algorithm is as follows: First, the point cloud segmentation module projects the original point cloud data onto the range image, separates the ground and non-ground point clouds, and removes noise clusters; then the feature extraction module extracts edge point and plane point features from the segmented point cloud respectively; subsequently, the lidar odometry module uses these features to match between consecutive frames and estimate the lidar pose transformation matrix; at the same time, the lidar mapping module converts the feature points into the global map for cumulative update to construct a three-dimensional grid map; finally, the pose transformation fusion module fuses the pose estimation results of odometry and mapping, and outputs the accurate pose through an optimization algorithm.

[0119] Step 2: According to the two-dimensional grid map drawn in Step 1, use the Triangular Topology Aggregation Optimizer (TTAO) algorithm to complete the path planning of the inspection robot. Specifically as follows:

[0120] The TTAO algorithm is a novel mathematics-based metaheuristic algorithm used to solve single-objective optimization problems and achieve continuous optimization and engineering applications. The core of this algorithm is based on the similar triangle topology in mathematics. By dividing the problem space into multiple triangles, the topological relationship of the triangles is used to represent the search direction in the solution space. In each generation of the algorithm, TTAO calculates the fitness value of each solution, selects individuals with higher fitness, and uses the topological relationship for crossover and mutation operations to generate new solutions. In this way, TTAO can gradually optimize the search space through continuous iteration and finally find the global optimal solution. The TTAO algorithm includes two strategies: general aggregation and local aggregation, which help to iteratively construct multiple similar triangle topology units to balance exploration and exploitation. The former generates new vertices through forward information exchange between different triangle topology units, and the latter constructs new units at promising positions according to the local optimal vertices of each unit.

[0121] (1) Population initialization: Each vertex in the triangle topology unit represents a search agent. Let the population size, that is, the planting area, be N, and the variable dimension be D. In the TTAO algorithm, the number of individuals N can be divided into N / 3 triangle topology units, where [·] represents rounding down, and the extra individuals are randomly generated in the search space. In the initialization stage, N / 3 agents are randomly generated in the feasible region. The mathematical expression for each individual generation is:

[0122]

[0123] In the formula denote the first search individual in the \(i\)-th triangular topological unit, and \(i\) is a positive integer between 1 and \(N / 3\); \(r_0\) denotes a random number between \([0, 1]\); and are the upper and lower bounds of the variables, and their mathematical expressions can be defined as:

[0124]

[0125] (2) Formation of triangular topological units: The formation of triangular topological units utilizes the conversion between polar coordinates and ordinary coordinate systems. Using the first vertex as the starting vertex in the spherical coordinate system, locate a length of coordinate system to form the second vertex. The generated length of direction vector is rotated counterclockwise by \(\pi / 3\), and then the third vertex is obtained through coordinate transformation. The expressions of these vertices can be written as:

[0126]

[0127] where \(l\) represents the size of the triangular topological unit, which is mathematically expressed as:

[0128]

[0129] In the formula, \(t\) represents the current iteration number; \(T\) represents the maximum number of iterations; \(l\) decreases as the number of iterations increases but does not decrease to 0; and represent the direction vectors of the other two sides guided by the first point, and the formula is:

[0130]

[0131] In the formula and \(\theta\) j (\(j = 1, \ldots, D\)) are random numbers between \([0, \pi]\).

[0132] Each group of triangular topological units aggregates internally into a fourth vertex, which is formed in a linearly weighted manner, thus using individual information. The fourth vertex is defined as:

[0133]

[0134] where \(r_1\), \(r_2\), and \(r_3\) are random numbers between \([0, 1]\), and \(r_1 + r_2 + r_3 = 1\). Therefore, the fourth search agent is located inside each triangular topological unit.

[0135] At the beginning of each iteration, new similar triangle topological units are obtained from vertices and two edges of the same length, and during the iteration process, the side lengths are dynamically changing. The best vertex in each unit can be considered as the leading vertex in each iteration, which guides the evolution of other individuals in each unit. In addition, other vertices are formed based on the best vertex.

[0136] Due to the randomness of θ, each triangle topological unit shows the same size and direction at time t, that is, congruence. For different times, the rotated directions of the triangle topological units are consistent. If the vertices of the triangle time unit t are transferred to the triangle time unit t + 1, the two are similar in rotation.

[0137] (III) General aggregation: Collect information on excellent individuals in different triangle units and generate new feasible solutions. Information interaction occurs between the best individual in each triangle topological unit and the best individuals in any randomly selected group of units. There is a linear combination with different weights between each dimension variable of the two positive individuals. The new individual is generated in the connection of the better two vertices, which is mathematically expressed as:

[0138]

[0139] where r4 is a random number between [0, 1]; and represent the best individual of unit i and the unit randomly selected at the t-th iteration. At the same time, compare with the fitness value of the optimal or sub-optimal search agent and update the optimal agent. Assuming a minimization problem, the mathematical expressions for the optimal and sub-optimal individuals updated in the (t + 1) th iteration are:

[0140]

[0141] where represents the sub-optimal individual at the i-th iteration; f(·) is the function of the given problem.

[0142] The number of each vertex in the unit represents the ranking of its fitness value, including three different movement methods. The first is where the crossed individual is better than the original position, and the second is where the crossed individual is better than the sub-optimal position. Information exchange between groups helps increase population diversity and enables individuals to fully explore.

[0143] (4) Local aggregation: A triangular topology is temporarily formed between the updated optimal or sub-optimal individual and two vertices in the group with good fitness values. In this case, the topology is not necessarily an equilateral triangle. Based on the difference in motion vectors composed of the optimal and sub-optimal individuals, the position of the optimal individual is perturbed in the local area (in terms of direction and step size). Therefore, each group is re-searched within a certain local area to utilize each topological triangular unit. The calculation of the new vertex is as follows:

[0144]

[0145] where α decreases to adjust the size of the aggregation range. The calculation formula of α is:

[0146]

[0147] The purpose of using the sub-optimal individual information is to prevent the optimal individual from falling into local extrema. After aggregation, it should be ensured that the guiding point of the temporary triangular unit is the optimal within the unit. To make the convergence develop in a promising direction, the fitness values of the two vertices before and after local mining are compared to determine the update of the position. If the new individual is better than the original individual, the position is updated; otherwise, no update is performed. The corresponding mathematical expressions are as follows:

[0148]

[0149] The purpose of local aggregation is to aggregate the positive information inside each topological triangular unit, mine each topological triangular unit as precisely as possible, so that the range of subsequent iterative mutation gradually becomes smaller.

[0150] When applying the Triangular Topology Aggregation Optimizer (TTAO) algorithm to plan the path of the greenhouse inspection robot, first, the population is initialized to randomly generate individuals. Then, triangular topological units are formed by randomly selecting three individuals as vertices and then randomly selecting an internal point within the triangle. Next, general aggregation is performed to aggregate information in the population using the geometric properties of the triangle and update the individual positions. After that is local aggregation, where new vertices are generated within the triangle through the method of constructing similar triangles to further optimize the individual positions. These steps work together to gradually optimize the path, enabling the robot to effectively avoid obstacles and move towards the target position.

[0151] After multiple iterations, the optimal path planning of the inspection robot is obtained based on the TTAO algorithm.

[0152] Step 3: Inside the greenhouse, by setting up multiple sensors, it is possible to achieve precise detection of the changes in parameters such as temperature, humidity, light, and CO2 in different areas, thereby promptly obtaining the growth status of the planted crops and providing a good growth environment for them. In this paper, an STM32 single-chip microcomputer is used as the control core of the inspection robot, which consists of a DHT11 temperature and humidity sensor module, a BH1750 light sensor module, a T6004 carbon dioxide sensor module, and a ZED binocular camera. The single-chip microcomputer transmits the collected environmental parameters to the host computer module through the data transmission module, so as to achieve intelligent monitoring of the greenhouse environment. At the same time, temperature, humidity, illuminance, and CO2 concentration value alarms are set on the inspection robot. When the effective parameter value exceeds the warning value, the buzzer on the robot will continuously alarm.

[0153] The STM32 single-chip microcomputer is a 32-bit microcontroller developed based on the ARM Cortex-M core. It is widely used in various fields due to its high performance, low power consumption, and strong reliability. In the greenhouse, the STM32 single-chip microcomputer can integrate a variety of intelligent sensors and peripheral control devices to achieve real-time monitoring and automated management of environmental parameters such as temperature, humidity, light, and CO2 concentration.

[0154] The DHT11 temperature and humidity sensor uses digital signal output and a single-wire communication protocol, featuring low cost and simplicity in use. It converts temperature and humidity into digital signals and transmits the data to the master device through the single-wire communication protocol. The BH1750 is a digital light intensity sensor that can obtain the light intensity of the surrounding environment without distinguishing the environmental light source. It has the characteristics of high precision and fast response, and is also suitable for low-power environments such as battery-powered due to its low-power design. The T6004 carbon dioxide sensor uses non-dispersive infrared technology, with a small size, high sensitivity, low humidity dependence, long lifespan, and low power consumption, and performs excellently in monitoring CO2 concentration in environments such as greenhouses.

[0155] The inspection robot is equipped with a ZED binocular camera to collect image information of the planted crops, and uses an improved TransNeXt model to identify the collected image information to judge the growth status of the planted crops.

[0156] ZED is a binocular camera produced by STEREOLABS, which is widely used in unmanned vehicles and collaborative robotic arms in research institutions; its 3D resolution can reach the resolution of RGB in Ultra mode, with a physical size of 175×30×33mm, a frame rate of up to 100FPS, a depth range of 0.3 - 25m, and a maximum field of view of 90°(H)×60°(V)×100°(D), and the interface is USB3.0.

[0157] TransNeXt is a hierarchical vision Backbone network mainly used for image classification and similar vision tasks, which includes Aggregated Pixel-focused Attention as the Token mixer and Convolutional GLU as the Channel mixer.

[0158] Aggregated Pixel-focused Attention includes Pixel-focused Attention, incorporate query embedding, positional attention mechanisms, and length-scaled cosine attention.

[0159] The Pixel-focused Attention mechanism (PFA) is a pixel-centered attention mechanism that has fine-grained perception near each query while maintaining coarse-grained perception of global information. For a given input [C, H, W], a set of pixels within a sliding window centered on the pixel at (i, j) is defined as ρ(i, j), with a fixed window size of k×k, ||ρ(i, j)|| = k 2 , and the set of pixels obtained through feature map pooling is defined as σ(X), with a given pooling size of H p ×W p , ||σ(X)|| = H p W p . Therefore, the Pixel-focused Attention is described as follows:

[0160]

[0161] where S represents the attention score; Q represents the query vector; K represents the key vector.

[0162]

[0163] A (i，j)～ρ(i，j) , A (i，j)～σ(X) = Split(A (i，j) ) with size[k 2 , H p , W p )]

[0164] PFA(X (i，j) ) = A (i，j)～ρ(i，j) V ρ(i，j) + A (i，j)～σ(X) V σ(X)

[0165] Where A represents the attention weight; Concat represents the operation of concatenating two vectors; d represents the dimension of the vector; B represents the positional bias; Split represents splitting the attention weight A (i，j) into two parts, corresponding to the attention weights of the local window and the global feature respectively; X represents the positional input feature; V represents the value vector.

[0166] To utilize the linear complexity pattern of PFA for large-scale image inference in subsequent applications, parameter-free global average pooling is adopted for downsampling in the spatial dimension. However, the average pooling operator seriously loses information. Therefore, a single-layer neural network is used for projection and activation before feature map pooling to compress and extract useful information in advance, thereby improving the information compression rate after downsampling. After pooling, layer normalization is used again to normalize the output to ensure the variance consistency of X and σ(X). The downsampling operator is called "activation pool" and can be expressed as:

[0167] σ(X) = LayerNorm(AvgPool(GELU(Linear(X))))

[0168] query embeding: Integrate the learnable QueryToken into the attention mechanism of the Backbone network. Instead of using the Query in the input, it learns a Query according to the current task to perform cross-attention. This method is classified as learnable Key-Value (LKV) attention.

[0169]

[0170] Positional attention: Use a set of learnable Keys for information aggregation. These Keys interact with the Query from the input to obtain the attention weight, that is, Query-Learnable-Value (QLV) attention. Each value corresponds to a different Query, which can enable the Key to receive information from multiple Queries during the training process and learn more implicit relative position information for the current Query.

[0171] length-scaled cosine attention: The length-scaled cosine attention uses the cosine similarity, which can generate more moderate attention weights and effectively enhance the training stability of large visual models. The length-scaled cosine attention multiplies an additional learnable coefficient λ to the cosine similarity result of the Query and the Key, enabling the attention mechanism to effectively ignore unimportant Tokens. The entropy of the dot product attention is:

[0172]

[0173] Where n is the sequence length; d represents the dimension of the model; σ(1) describes the asymptotic complexity of the function, representing the ratio of n and d.

[0174] Set λ as λ = τlogn, where τ is a learnable variable and n is the sequence length. Then the length-scaled cosine attention is:

[0175]

[0176] Where Q is the query matrix; K is the key matrix; V is the value matrix; and represent the normalized versions of the query and the key respectively; N represents the number of effective keys that each query interacts with; T is a learnable parameter used to adjust the scaling of the attention scores.

[0177] When using the TransNeXt model for image recognition, first preprocess the collected plant images, such as resizing and normalizing the images. Then input the processed images into the model. The convolutional layer and the Transformer encoder components are used to extract the multi-scale features of the images. The multi-scale token generation and self-attention mechanism enable the model to focus on the key regions of the images, such as the leaves and fruits of the plants. These features are fused at multiple scales, and finally, the classifier classifies or recognizes the images to obtain the final classification results or corresponding information of the images, achieving efficient and accurate recognition of the growth status of the plants.

[0178] Convolutional GLU is an innovative channel mixer that combines the gated linear unit (GLU) and convolutional operations for Vision Transformers (ViTs) to enhance the model's ability to capture local features and improve robustness. This design uses convolutional layers to capture local features and provide position information, similar to conditional position encoding (CPE). At the same time, the gating mechanism allows the model to assign unique attention weights to each channel, thereby achieving more refined channel attention control. While maintaining computational efficiency, convolutional GLU provides enhanced model performance through its gating activation function. In the TransNeXt model, convolutional GLU is combined with aggregated attention to form a powerful visual backbone network that demonstrates excellent performance in multiple visual tasks, effectively solving the depth degradation problem in traditional ViTs and achieving more natural visual perception by simulating the characteristics of the biological visual system.

[0179] Although the convolutional GLU module in the TransNeXt model has avoided the overly coarse-grained drawback of global average pooling in the SE mechanism through a channel attention mechanism based on nearest-neighbor image features, there are still some limitations when dealing with different input patterns:

[0180] (1) Static convolutional kernels: Traditional convolutional GLU modules use fixed convolutional kernel weights and cannot dynamically adjust according to the features of the input data, resulting in insufficient adaptability of the model when dealing with different types of image features. For example, there are significant differences in the morphology, texture, and color of different types of plants, and it is difficult for fixed convolutional kernels to capture these diverse features.

[0181] (2) Lack of flexibility: The use of global average pooling in the SE mechanism makes all Tokens share the same gating signal. This design is too rough and lacks the ability to finely model local features. Especially in the task of plant image recognition, the local features of plants (such as the texture of leaves and the shape of fruits) need to be accurately recognized.

[0182] Therefore, dynamic convolution is introduced into the convolutional GLU module of TransNeXt to further improve the performance and robustness of the model in the task of plant image recognition.

[0183] Dynamic Convolution is a convolutional method that can dynamically adjust the weights of convolutional kernels according to the features of the input data. Its main part consists of an attention mechanism and multiple convolutional kernels. The attention mechanism is used to generate the weights for each input feature map, and these weights are used to dynamically combine the convolutional kernels. The formula is expressed as:

[0184]

[0185] In the formula, y is the output feature map; x is the input feature map; θ k is the k-th convolutional kernel; ω k is the weight of the k-th convolutional kernel, generated by the attention mechanism; K is the number of convolutional kernels.

[0186] Dynamic convolution can dynamically adjust the weights of convolutional kernels according to the features of the input data, thus better adapting to different input patterns and enhancing the feature extraction ability of the model. Introducing dynamic convolution into the convolutional GLU module of TransNeXt makes the channel attention weights of each Token change dynamically according to its local features, further improving the local modeling ability. The specific steps are as follows:

[0187] First, define a dynamic convolution module that contains an attention mechanism and multiple convolutional kernels. The attention mechanism is used to generate weights for each input feature map, and these weights are used to dynamically combine the convolutional kernels. Then, integrate the dynamic convolution module into the convolutional GLU module, replacing the original static convolution part, that is, using dynamic convolution in the convolutional GLU module to generate channel attention weights. Finally, integrate the modified convolutional GLU module into the TransNeXt model, replacing the original convolutional GLU module, that is, using the new convolutional GLU module in each block of TransNeXt.

Claims

1. A greenhouse inspection method based on multi-objective fusion, characterized in that, It includes the following steps: S1. The inspection robot is equipped with a lidar to collect the internal environmental information of the greenhouse, and the LeGO-LOAM algorithm is used to construct a three-dimensional grid map; S2. According to the grid map in S1, the path planning of the inspection robot is completed by using the Triangular Topology Aggregation Optimizer (TTAO) algorithm; S3. The inspection robot is equipped with sensors to measure the temperature, humidity, illuminance and CO2 values of the surrounding environment, and a ZED binocular camera is equipped to collect the image information of the planted plants. An improved TransNeXt model is used to identify the collected image information and analyze the growth of the planted plants.

2. The greenhouse patrol inspection method based on multi-objective fusion according to claim 1, characterized in that: The following method is used to collect the internal environmental information of the greenhouse in S1: The lidar is controlled by a DC motor to operate. The lidar laser emits light and projects it onto the entire FOV area. The receiver receives the emitted light. The distance between the lidar and the obstacle is the product of the time interval between laser emission and reception and the speed of light. The expression is as follows: Among them, P r is the power received by the receiver, P0 is the transmitted power, A r is the area of the receiver's receiving aperture, γ is the loss of air to the laser beam, ρ is the reflectivity of the obstacle, η0 is the efficiency of the optical system, and R is the distance between the obstacle and the lidar; The single-beam laser emitter of the lidar rotates at a constant speed. During the rotation, a laser beam is emitted every certain time. After the lidar rotates one week, a complete frame of point cloud is obtained to construct a map. The encoder receives the rotational speed data of the DC motor, and the speed and pose data are obtained from the motor speed, and finally the odometer data is obtained.

3. The greenhouse inspection method based on multi-objective fusion according to claim 1, wherein: In S1, the LeGO-LOAM algorithm is used to construct a three-dimensional grid map, which specifically includes the following steps: The point cloud segmentation module is used to separate the ground point cloud, and at the same time cluster the remaining point clouds and filter out the point cloud clusters with a small number. Specifically: first project the original point cloud scanned by the lidar into a depth map, then judge the ground points by calculating the pitch angle between the point clouds, and finally use an image-based segmentation method to cluster the depth map; The feature extraction module is used to extract the edge point and plane point features of the segmented point cloud. Specifically: for a continuous point cloud formed by the lidar scan line, based on the coordinates of several point clouds before and after each point cloud, calculate the curvature of each point cloud, and distinguish the plane feature points and edge feature points through the curvature. The curvature calculation in the algorithm is measured by the following formula, and it is defined as the feature degree c: Among them, S represents the number of point clouds participating in the curvature calculation, and S / 2 point clouds are selected before and after this point; r represents the distance measurement value of the point cloud; Using a lidar odometry module, a pose transformation matrix between consecutive frames is found through feature matching, which mainly includes two parts: feature matching and pose estimation. Specifically: for a set of edge feature points in the current frame For a certain point A in it, two closest points are selected from the set of edge feature points in the previous frame Points B and C are in different scan lines. Assuming the coordinates of points A, B, and C are denoted as Then the distance calculation formula between a point and a feature line is: For the current set of planar feature points For a certain point A in it, select three adjacent points in the previous frame of planar feature points Suppose the coordinates of the four points A, B, C, and D are respectively denoted as Similarly, based on the equal area method, the distance calculation formula between the point and the feature plane is: For the pose estimation problem, the LeGO-LOAM algorithm first constructs the first optimization problem using the constraint relationship between planar points, and optimizes \(t\) z , \(\theta\) roll , \(\theta\) pitch . Then it constructs the second optimization problem using edge point constraints, and based on \(t\) z , \(\theta\) roll , \(\theta\) pitch obtained from the first-step optimization, estimates \(t\) y , \(y\) z , \(\theta\) yaw . Through two-step L-M optimization, a 6-degree-of-freedom pose transformation matrix \([t\) z , \(t\) y , \(y\) z \(\theta\) roll , \(\theta\) pitch , \(\theta\) yaw is finally obtained; The lidar mapping module is used to further process the features, and then register them in the global point cloud map. The global map is the final output result of SLAM. In LeGO-LOAM, each frame of point cloud and pose are associated and saved separately. In the code, it is represented as a set. That is, at time t, the map created for the previous t-1 moments is: The pose transformation fusion module is used to fuse the results of the lidar odometer and the lidar mapping to output the final pose estimation.

4. The greenhouse inspection method based on multi-objective fusion according to claim 1, characterized in that, S2 specifically includes the following steps: S2-1. Population initialization: Each vertex in the triangular topological unit represents a search agent. Let the population size, i.e., the planted area, be N, and the variable dimension be D. In the TTAO algorithm, the number of individuals N can be divided into N / 3 triangular topological units, where [·] represents rounding down, and the extra individuals are randomly generated in the search space. In the initialization stage, N / 3 agents are randomly generated in the feasible region. The mathematical expression for each individual generation is: In the formula represents the first search individual in the i-th triangular topological unit, and i is a positive integer between 1 and N / 3; r0 represents a random number between [0, 1]; and are the upper and lower bounds of the variable, and their mathematical expressions are defined as: S2-2. Formation of triangular topological units: Using the first vertex as the starting vertex in the spherical coordinate system, locate a length of The coordinate system forms the second vertex, and the generated length is The direction vector of is rotated counterclockwise by π / 3, and then the third vertex is obtained through coordinate system transformation. The expressions of these vertices are: where l represents the size of the triangular topological unit, which is mathematically expressed as: where \(t\) represents the current iteration number; \(T\) represents the maximum number of iterations; \(l\) decreases as the number of iterations increases but does not decrease to 0; and represent the direction vectors of the other two sides guided by the first point, and the formula is: wherein and θ j (j = 1, … D} are random numbers between [0, π]. Each group of triangular topological units aggregates internally into a fourth vertex, which is formed in a linearly weighted manner to utilize individual information. The fourth vertex is defined as: where r1, r2, and r3 are random numbers between [0, 1], and r1 + r2 + r3 = 1. Therefore, the fourth search agent is located inside each triangular topological unit; S2-3. General aggregation: Utilize the geometric characteristics of triangles to aggregate information in the population and update the individual positions; Collect the information of excellent individuals in different triangular units and generate new feasible solutions. Information interaction occurs between the best individual in each triangular topological unit and the best individual in any randomly selected group of units. There is a linear combination with different weights between each dimension variable of the two positive individuals. The new individual is generated in the connection between the better two vertices, which is mathematically expressed as: where r4 is a random number between [0, 1]; and represent the best individual of unit i and the unit randomly selected at the t-th iteration. At the same time, compare fitness values of the optimal or sub-optimal search agents and update the optimal agent. Assuming a minimization problem, the mathematical expressions for the optimal and sub-optimal individuals updated at the (t + 1) th iteration are as follows: where represents the sub-optimal individual at the i-th iteration; f(·) is the function of the given problem; S2-4. Local aggregation: Generate new vertices inside the triangle through the method of constructing similar triangles to further optimize the individual positions; [[ID=IO]]Research each group again within a certain local area to utilize each topological triangular unit. The calculation of the new vertex is: In the formula, α decreases to adjust the size of the aggregation range. The calculation formula of α is: If the new individual is better than the original individual, update the position; otherwise, do not update. The corresponding mathematical expression is as follows:

5. The greenhouse inspection method based on multi-objective fusion according to claim 1, wherein; In S3, the STM32 single-chip microcomputer is selected as the control core of the inspection robot, which consists of a DHT11 temperature and humidity sensor module, a BH1750 light sensor module, a T6004 carbon dioxide sensor module, and a ZED binocular camera. The single-chip microcomputer transmits the collected environmental parameters to the upper computer module through the data transmission module to achieve intelligent monitoring of the greenhouse environment. At the same time, temperature and humidity, illuminance, and CO2 concentration value alarms are set on the inspection robot. When the effective parameter value exceeds the warning value, the buzzer continuously alarms.

6. The greenhouse patrol inspection method based on multi-objective fusion according to claim 5, wherein, In S3, the improved TransNeXt model is used to identify the image information collected by the ZED binocular camera, and the analysis of the growth situation of the planted plants specifically includes the following steps: For a given input [C, H, W], a set of pixels within a sliding window centered at the pixel at (i, j) is defined as ρ(i, j), with a fixed window size of k×k, ||ρ(i,j)|| = k 2 , the pixel set obtained through feature map pooling is defined as σ(X), with a given pooling size of H p ×W p , ||σ(X)|| = H p W p , the pixel focusing attention is described as follows: In the formula, S represents the attention score; Q represents the query vector; K represents the key vector; A (i,j)～ρ(i,j) ,A (i,)～σ(X) = Split(A (i，j) ) with size [k 2 ,H p W p ​ PFA(X (i，j) ) = A (i,j)～ρ(i,j) V ρ(i,j) + A (i,j)～σ(X) V σ(X) Where A represents the attention weight; Concat represents the operation of concatenating two vectors; d represents the dimension of the vector; B represents the position bias; Split represents splitting the attention weight A (i，j) into two parts, corresponding to the attention weights of the local window and the global feature respectively; X represents the position input feature; v represents the value vector; Before feature map pooling, a single-layer neural network is used for projection and activation to compress and extract useful information in advance. After pooling, layer normalization is used again to normalize the output to ensure the variance consistency of X and σ(X). Among them, the downsampling operator is called the activation pool, which is expressed as: σ(X) = LayerNormm(AvgPool(GELU(Linear(X)))) When using the TransNeXt model for image recognition, first preprocess the collected plant images, such as resizing and normalizing the images. Then, input the processed images into the model, and extract multi-scale features of the images through convolutional layers and Transformer encoder components. Among them, multi-scale token generation and self-attention mechanism enable the model to focus on key regions of the image, such as the leaves and fruits of plants. These features are fused at multiple scales, and finally, the image is classified or recognized through a classifier to obtain the final classification result or corresponding information of the image, realizing the efficient and accurate recognition of the growth state of plants.

7. The greenhouse inspection method based on multi-objective fusion according to claim 6, characterized in that: Introduce dynamic convolution in the convolutional GLU module of TransNeXt to further improve the performance and robustness of the model in the plant image recognition task. The specific steps are as follows: Define a dynamic convolution module that contains an attention mechanism and multiple convolutional kernels. The attention mechanism is used to generate weights for each input feature map, and these weights are used to dynamically combine the convolutional kernels. Integrate the dynamic convolution module into the convolutional GLU module to replace the original static convolution part, that is, use dynamic convolution in the convolutional GLU module to generate channel attention weights. Integrate the modified convolutional GLU module into the TransNeXt model to replace the original convolutional GLU module, that is, use the new convolutional GLU module in each block of TransNeXt.

Images