Agricultural trajectory point identification method, apparatus, and system

Abstract

Embodiments of the present application provide an agricultural trajectory point identification method, apparatus, and system, which are applied in the field of agriculture. The identification method comprises: obtaining a first image, the first image being a four-channel image, and the four channels comprising a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a corresponding trajectory channel in the first image; and inputting the first image into an agricultural trajectory point identification model, and outputting the type of each corresponding trajectory point in the first image, the trajectory point type comprising field type or road type. In this way, the trajectory generated by an agricultural vehicle is comprehensively considered in combination with environmental features of a satellite map corresponding to the trajectory points, so as to obtain the type of each trajectory point, and the obtained trajectory point type is more accurate.

Description

Agricultural trajectory point identification method, device and system

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 1, 2023, with application number 202311127925.8 and application name “Agricultural trajectory point identification method, device and system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of artificial intelligence, and more specifically to a method, device, and system for identifying agricultural trajectory points. Background Art

[0003] With the development of electronic information technology, intelligent and information-based agricultural machinery has gradually become the future development direction. At present, many agricultural tasks can be carried out in the field by agricultural vehicles, which greatly reduces labor costs and improves work efficiency.

[0004] Agricultural vehicles generate a lot of track information during use, including field tracks during field operations and road tracks between fields. Classifying and identifying agricultural vehicle tracks is crucial for calculating the area being worked, studying crop distribution, and planning routes.

[0005] Therefore, how to identify the type of agricultural vehicle's driving trajectory is an urgent problem to be solved.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a method, device, and system for identifying agricultural trajectory points, which can identify the type of driving trajectory of agricultural vehicles.

[0008] In a first aspect, a method for identifying agricultural track points is provided. The method includes: acquiring a first image, wherein the first image is a four-channel image, comprising a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a track channel corresponding to the first image. The first image is input into an agricultural track point recognition model, and the model outputs a type of each track point corresponding to the first image, wherein the track point type includes either a field type or a road type.

[0009] In the above technical solution, the type of each track point is determined by a trained agricultural track point recognition model, taking into account the tracks generated by agricultural vehicles and the environmental characteristics of the satellite map corresponding to the track point. In this way, compared to simply considering the difference between field operation tracks and road driving tracks, taking the environmental factors of the tracks into account can improve the accuracy of agricultural track point type recognition. For example, for single and double ridge field operation tracks, the solution of the embodiment of the present application is more accurate in determining the type of track point than the solution of determining the type of track point based on the parallel arrangement of multiple ridge lines of the traditional field operation track.

[0010] In conjunction with the first aspect, in certain implementations of the first aspect, the agricultural trajectory point recognition model includes M feature extraction layers and an information exchange layer. The output of the i-th feature extraction layer serves as the input to the i+1-th feature extraction layer, where M is a positive integer greater than or equal to 2, i is a positive integer, and 1≤i≤M-1. The downsampling factor of the M feature extraction layers increases layer by layer, and the output of the M-th feature extraction layer serves as the input to the information exchange layer. Inputting a first image into the agricultural trajectory point recognition model and outputting the type of each trajectory point corresponding to N first images includes: inputting the first image into the first feature extraction layer of the M feature extraction layers, and obtaining M local feature information layer by layer. Inputting the output of the M-th feature extraction layer into the information exchange layer, outputting global feature information for the N first images, where the global feature information includes environmental features of the satellite image. The type of each trajectory point is determined based on the M local feature information and the global feature information.

[0011] In the above technical solution, the agricultural trajectory point recognition model based on multi-level feature extraction can obtain local feature information and global feature information under different downsampling scales. When extracting features, the receptive field increases from small to large, which can take into account both detailed features and global features, and can extract more hidden features, further improving the accuracy of agricultural trajectory point type recognition.

[0012] In conjunction with the first aspect, in certain implementations of the first aspect, the agricultural trajectory point recognition model further includes M decoding layers, wherein a portion of the input to the i-th decoding layer is the output of the i+1-th decoding layer, and another portion of the input to the i-th decoding layer is the i-th local feature information corresponding to the i-th feature extraction layer, and the upsampling multiple of the i-th decoding layer is the same as the downsampling multiple of the i-th feature extraction layer. Determining the type of each trajectory point based on the M local feature information and the global feature information includes: inputting the global feature information and the M-th local feature information into the M-th decoding layer to obtain the M-th classification feature map. Inputting the M-th classification feature map and the M-1-th local feature information into the M-1-th decoding layer, and obtaining, layer by layer, M-1 classification feature maps other than the M-th classification feature map, wherein the M classification feature maps include the M-th classification feature map and the other M-1 classification feature maps, and the i+1-th classification feature map contains less detailed information than the i-th classification feature map. Determining the type of each trajectory point based on the M classification feature maps.

[0013] In the above technical solution, the front feature extraction layer can extract local details, while the back feature extraction layer has a wider receptive field, which is used to predict the type of a long track. Through this overall first, then local feature extraction and fusion method, that is, by first constructing the overall classification of the entire track in the last feature extraction layer and information interaction layer, "which large area is a field, which large section is a road", and then passing it through higher resolution decoding layers layer by layer, gradually optimizing the details, and obtaining classification feature maps with different levels of detail. In this way, the accuracy of track point classification can be further improved.

[0014] In conjunction with the first aspect, in certain implementations of the first aspect, the agricultural trajectory point recognition model further includes M output layers. Determining the type of each trajectory point based on the M classification feature maps includes: inputting the i-th classification feature map into the i-th output layer to obtain the i-th first classification result, where the i-th first classification result is the probability of the type of each pixel point at the downsampling scale of the i-th feature extraction layer. The type of each pixel point includes background type, field type, and road type. Determining the type of each trajectory point based on the M first classification results, the M first output results including the i-th first classification result.

[0015] In the above technical solution, the classification feature maps obtained at different sampling scales are input into their respective corresponding output layers, so that a more accurate first classification result can be obtained.

[0016] In conjunction with the first aspect, in certain implementations of the first aspect, the agricultural trajectory point recognition model further includes a post-processing layer configured to separate background pixels from non-background pixels to determine whether each pixel corresponding to a trajectory point in the first classification result is a field type or a road type. The M first classification results are input into the post-processing layer to determine the type of each trajectory point.

[0017] In the above technical solution, the model will only classify the parts with tracks as fields or roads, and will only recognize the background in the parts without tracks, without the common boundary blur and scattered point problems in semantic segmentation.

[0018] In conjunction with the first aspect, in certain implementations of the first aspect, acquiring the first image includes acquiring trajectory points within a first time period, and determining a second image based on the trajectory points and the satellite map corresponding to the trajectory points, wherein the first image is a partial image or a full image of the second image.

[0019] In combination with the first aspect, in some implementations of the first aspect, the satellite map level corresponding to the first image is level 18.

[0020] In the above technical solution, after a large number of tests at different map levels, the map level of the above-mentioned first image is determined to be a tile map at level 18. This can avoid the level being too low, which may cause the trajectory to be too thin and lose many details, and can also avoid the level being too high, which may cause the image to be trapped in the local area and unable to obtain more global information.

[0021] In a second aspect, a device for identifying agricultural trajectory points is provided. The device includes an acquisition unit and a processing unit. The acquisition unit is configured to acquire a first image, wherein the first image is a four-channel image, comprising a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a trajectory channel corresponding to the first image. The processing unit is configured to input the first image into an agricultural trajectory point recognition model and output the type of each trajectory point corresponding to the first image, wherein the trajectory point type may include a field type or a road type.

[0022] It should be understood that the technical effects corresponding to the technical solutions of the identification device of the second aspect can refer to the first aspect and will not be elaborated here.

[0023] In conjunction with the second aspect, in certain implementations of the second aspect, the agricultural trajectory point recognition model includes M feature extraction layers and an information exchange layer. The output of the i-th feature extraction layer serves as the input to the i+1-th feature extraction layer, where M is a positive integer greater than or equal to 2, i is a positive integer, and 1≤i≤M-1. The downsampling multiples of the M feature extraction layers increase layer by layer, and the output of the M-th feature extraction layer serves as the input to the information exchange layer. The processing unit is specifically configured to: input a first image into the first feature extraction layer of the M feature extraction layers, and obtain M local feature information layer by layer. The output of the M-th feature extraction layer is input into the information exchange layer, and global feature information is output for the N first images. The global feature information includes environmental features of the satellite image. The type of each trajectory point is determined based on the M local feature information and the global feature information.

[0024] In combination with the second aspect, in certain implementations of the second aspect, the agricultural trajectory point recognition model further includes M decoding layers, a portion of the input to the i-th decoding layer is the output of the i+1-th decoding layer, and another portion of the input to the i-th decoding layer is the i-th local feature information corresponding to the i-th feature extraction layer, and the upsampling multiple of the i-th decoding layer is the same as the downsampling multiple of the i-th feature extraction layer. The processing unit is specifically configured to: input the global feature information and the M-th local feature information into the M-th decoding layer to obtain the M-th classification feature map. Input the M-th classification feature map and the M-1-th local feature information into the M-1-th decoding layer, and obtain M-1 classification feature maps other than the M-th classification feature map layer by layer, wherein the M classification feature maps include the M-th classification feature map and the other M-1 classification feature maps, and the i+1-th classification feature map has less detailed information than the i-th classification feature map. Based on the M classification feature maps, the type of each trajectory point is obtained.

[0025] In conjunction with the second aspect, in certain implementations of the second aspect, the agricultural trajectory point recognition model further includes M output layers. The processing unit is specifically configured to: input the i-th classification feature map into the i-th output layer to obtain the i-th first classification result, where the i-th first classification result is the probability of the type of each pixel point at the downsampling scale of the i-th feature extraction layer. The type of each pixel point includes background type, field type, and road type. Based on the M first classification results, the type of each trajectory point is obtained, and the M first output results include the i-th first classification result.

[0026] In conjunction with the second aspect, in certain implementations of the second aspect, the agricultural trajectory point recognition model further includes a post-processing layer configured to separate background pixels from non-background pixels to determine whether each pixel corresponding to a trajectory point in the first classification result is a field type or a road type. The processing unit is specifically configured to input the M first classification results into the post-processing layer to determine the type of each trajectory point.

[0027] In conjunction with the second aspect, in certain implementations of the second aspect, the acquiring unit is specifically configured to: acquire trajectory points within a first time period, and determine a second image based on the trajectory points and the satellite map corresponding to the trajectory points, where the first image is a partial image or a full image of the second image.

[0028] In combination with the second aspect, in some implementations of the second aspect, the satellite map level corresponding to the first image is level 18.

[0029] In a third aspect, an agricultural trajectory point identification device is provided, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to perform any possible method in the first aspect.

[0030] Optionally, the above-mentioned processing unit may be a processor, and the above-mentioned storage unit may be a memory, wherein the memory may be a storage unit within the chip (for example, a register, a cache, etc.), or a storage unit located outside the chip within the smart device (for example, a read-only memory, a random access memory, etc.).

[0031] In a fourth aspect, a smart device is provided, comprising the agricultural trajectory point recognition device described in the second aspect, or comprising the recognition device described in the third aspect.

[0032] In some possible implementations, the smart device may include an agricultural vehicle, etc.

[0033] In a fifth aspect, a system for identifying agricultural trajectory points is provided, which includes a trajectory collection device and an agricultural trajectory point identification device, wherein the agricultural trajectory point identification device can be the agricultural trajectory point identification device described in any one of the second aspect or the third aspect above.

[0034] In combination with the fifth aspect, in certain implementations of the fifth aspect, the agricultural trajectory point identification device may be located in a cloud server.

[0035] In combination with the fifth aspect, in certain implementations of the fifth aspect, the system further includes a device for receiving instructions sent by the cloud server.

[0036] In a sixth aspect, a computer-readable medium is provided, wherein the computer-readable medium stores a program code, and when the computer program code is run on a computer, the computer executes the method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of an agricultural trajectory point recognition system provided by an embodiment of the present application;

[0038] FIG2 is a schematic diagram of an agricultural trajectory point provided in an embodiment of the present application;

[0039] FIG3 is a flow chart of an agricultural trajectory point recognition method provided in an embodiment of the present application;

[0040] FIG4 is a schematic diagram of a data preprocessing process provided by an embodiment of the present application;

[0041] FIG5 is a schematic diagram of the structure of an agricultural trajectory point recognition model provided in an embodiment of the present application;

[0042] FIG6 is a schematic diagram of the structure of an information interaction layer provided in an embodiment of the present application;

[0043] FIG7 is a schematic diagram of a model training convergence speed provided in an embodiment of the present application;

[0044] FIG8 is a rendering of an agricultural trajectory recognition result provided by an embodiment of the present application;

[0045] FIG9 is a schematic diagram of an agricultural trajectory point recognition device provided by an embodiment of the present application;

[0046] FIG10 is a schematic diagram of the hardware structure of an agricultural trajectory point recognition device provided by the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0048] The embodiments of the present application can be applied to scenarios such as counting the operating area, studying the distribution of crops, and planning the operating path in agriculture.

[0049] In order to better understand the solution of the embodiment of the present application, the system schematic diagram of the embodiment of the present application is briefly introduced below with reference to FIG1 .

[0050] FIG1 is a schematic diagram of an agricultural trajectory point recognition system provided in an embodiment of the present application.

[0051] As shown in Figure 1, the agricultural trajectory point identification system includes a trajectory collection device 111 and a data processing device 120. The trajectory collection device 111 sends the collected trajectory data of the agricultural vehicle 110 to the data processing device 120 for processing to obtain the type of the trajectory point. It should be understood that the data processing device 120 can be located in the agricultural vehicle 110 or a processing device independent of the agricultural vehicle 110. For example, the data processing device 120 can be a cloud server, in which case the collected trajectory data needs to be uploaded to the cloud server for processing. This embodiment of the present application is not limited to this.

[0052] The trajectory collection device 111 may include a positioning system and an inertial measurement unit (IMU). The positioning system is used to estimate the geographic location of the agricultural vehicle 110. The positioning system may be, for example, a global positioning system (GPS), the Beidou system, or other positioning systems. The IMU is used to sense the position and orientation changes of the agricultural vehicle 110 based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope.

[0053] The data processing device 120 can be a device or server with data processing capabilities, such as a cloud server, network server, application server, or management server. The data processing device receives trajectory data and trajectory-related map data from the trajectory acquisition device 111 via a communication interface. The data processing device then processes the data using the agricultural trajectory point identification method provided in the embodiments of this application through a memory for storing data and a processor for processing data. The memory in the data processing device can be a general term that includes local storage and a database for storing historical data. The database can be hosted on the data processing device or on another network server. This embodiment of the application does not impose any restrictions on this.

[0054] It should be understood that the data processing device may include at least one processor, which is a circuit with signal processing capabilities. In one implementation, the processor may be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit. The logical relationship of the hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0055] In order to facilitate understanding of the solutions of the embodiments of the present application, a brief description of the technical terms involved in the embodiments of the present application is first given.

[0056] (1) Deep Neural Networks

[0057] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. The term "many" here doesn't necessarily refer to a specific metric. Based on the location of different layers within a DNN, the neural network can be divided into three categories: input layer, hidden layer, and output layer.

[0058] (2) Convolutional neural network (CNN)

[0059] A convolutional neural network is a neural network with convolution kernels. Convolution kernels can be thought of as filters. A convolution layer is the layer of neurons in a convolutional neural network that performs convolution processing on the input signal. A convolution layer typically contains several feature planes, each of which can be composed of a number of neural units arranged in a rectangular pattern.

[0060] (3) Loss function

[0061] In the process of training deep neural networks, because we hope that the output of the deep neural network will be as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the network's predicted value is high, adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0062] (4) RGB channels

[0063] The pixel value of an image can be a red, green, and blue (RGB) color value. In other words, the image is composed of three RGB channels, representing the three basic hues of red, green, and blue in the image. The value of each channel is between 0 and 255. These three colors with different values ​​can almost form all colors.

[0064] At present, the trajectory type recognition methods in the field of autonomous driving are mostly used to identify the trajectories of non-agricultural vehicles. For example, in smart transportation, it is necessary to identify whether the trajectory type is a vehicle trajectory or a non-vehicle trajectory (for example, a cyclist trajectory). This is the identification of different trajectory generators. For another example, in autonomous driving, it is necessary to identify whether the vehicle's trajectory is a turning, U-turn, or straight-ahead type. This is the type identification of the trajectory itself of the same trajectory generator. For another example, in autonomous driving, there is also a solution to determine the road type to which the trajectory belongs based on the trajectory position and the unique ID of the road type (for example, highways, national roads, provincial roads, and county roads, etc.).

[0065] In terms of agricultural vehicle trajectory identification, there is currently a method that filters out trajectories in non-agricultural areas by driving speed. However, this method uses the exclusion method to filter out trajectories in non-agricultural areas, and the accuracy of the results is low. Alternatively, there is a method of identifying farmland plots through clustering algorithms. Although the processing speed is good, the processing accuracy is still relatively poor, and field operation trajectories in non-dense cells may be mistakenly deleted. In addition, clustering parameters need to be manually set according to different agricultural vehicle operation types, and the degree of automation is low. Alternatively, the improved DBSCAN clustering algorithm is used to filter out road driving points and field transfer points in the operation trajectories of agricultural vehicles. Although this method has improved accuracy, it still uses the exclusion method to filter out the collected trajectory points, and does not strictly distinguish whether the trajectory points are field trajectory types or road driving types. Therefore, in this method, it is still possible to filter out trajectory points in the farmland area.

[0066] Another approach distinguishes track point types based on the more regular nature of field tracks compared to road tracks. For example, field tracks are often composed of multiple parallel ridges stacked together, while road tracks are often single or double tracks. However, exceptions to this rule exist.

[0067] FIG2 is a schematic diagram of an agricultural trajectory point provided in an embodiment of the present application.

[0068] Case 1: The field operation track is single or double ridge.

[0069] As shown in FIG2(a), if it is judged based on the rule that field operation tracks are mostly composed of multiple ridges arranged in parallel, while road driving tracks are often single tracks or double tracks, FIG2(a) is likely to be determined as a road driving track. However, the track of FIG2(a) may also be a field operation track in which only two ridges are operated.

[0070] As shown in Figure 2(b), the upper trajectory is a double ridge, and the lower trajectory parallel to the upper trajectory is a single ridge. Based solely on the above rules, it is impossible to determine whether the trajectory point is a field operation trajectory or a road driving trajectory.

[0071] As shown in FIG2(c), according to the above rules, the rightmost track is very likely to be judged as a road driving track, but the rightmost track may also be a single-row field operation track close to the road.

[0072] Case 2: Trajectory of urban road sections

[0073] As shown in Figure 2(d), when the trajectory point falls within the range of the urban road section, no matter whether it is a single line, double line or other shape, it cannot be a field operation trajectory. However, the type of the trajectory point in Figure 2(b) cannot be determined by the above rules alone.

[0074] To address the above issues, the embodiments of the present application propose an agricultural trajectory point recognition method, device, and system, which will be described in detail below with reference to Figures 3 to 10.

[0075] FIG3 is a flow chart of an agricultural trajectory point recognition method provided in an embodiment of the present application.

[0076] S310 , obtaining a first image, where the first image is a four-channel image, including a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a trajectory channel corresponding to the first image.

[0077] It should be understood that the track corresponding to the first image may be a part or all of the track formed by the agricultural vehicle within the first time.

[0078] The first time is preset. For example, the formed tracks may be all tracks of the agricultural vehicle within one day, or all tracks of the agricultural vehicle within half a day.

[0079] It should also be understood that the red channel, green channel, and blue channel of the first image represent the environmental features of the corresponding trajectory point on the satellite map. The pre-processing process of obtaining the first image in S310 will be described in detail below with reference to FIG.

[0080] S320: Input the first image into an agricultural trajectory point recognition model, and output the type of each trajectory point corresponding to the first image, where the type of the trajectory point includes a field type or a road type.

[0081] It should be understood that the agricultural trajectory point recognition model is trained in advance, and the training data used includes four-channel training images and corresponding trajectory point type labels in the training images, and the type labels include field type and road type.

[0082] For example, an embodiment of the present application can be to complete the code in an Ubuntu system, in a Tensorflow environment, and train the model on an RTX3090-GPU with 24G of GPU memory.

[0083] For example, the dataset is the actual operation trajectories of farmers in Heilongjiang, Liaoning and other regions, with 1,032 trajectories accurately marked, including various operation modes such as sowing, rotary tillage, and harvesting. There are an average of about 40 pictures per trajectory, and a total of about 42,000 pictures.

[0084] During training, each batch_size can be set to 8, for a total of 48 batches. The learning rate can be set to 5e-3, first warming up from low to high, and then gradually decreasing after reaching the maximum learning rate.

[0085] FIG7 is a schematic diagram of a model training convergence speed provided in an embodiment of the present application.

[0086] The horizontal axis of Figure 7 represents the training step size, and the vertical axis represents the model accuracy. As shown in Figure 7, the training model of the present embodiment converges quickly in the first 18,000 steps, then slowly increases in the middle. After 34,000 steps of training, the accuracy stops increasing. The optimal accuracy of the model is 97.5%.

[0087] As a possible implementation method, when the corresponding trajectory in the first image is a partial trajectory formed by an agricultural vehicle in the first time, multiple first images can be input into the agricultural trajectory point recognition model separately to obtain multiple classification results corresponding to different trajectories, and then the multiple classification results are spliced ​​to obtain the trajectory classification result within the first time.

[0088] In this implementation, the data information of the RGB channels in the first image is richer, which enhances the influence of the environmental factors of the trajectory on the trajectory classification judgment, thereby improving the accuracy of trajectory classification.

[0089] As a possible implementation, the agricultural trajectory point model can include M feature extraction layers, information exchange layers, M decoding layers, M output layers, and post-processing layers. The M feature extraction layers are used to obtain M local feature information at different downsampling scales. The information exchange layer is used to interact with the M local feature information to obtain global feature information. The M decoding layers are used to obtain classification feature maps at different upsampling scales. The M output layers are used to obtain the probability values ​​of different classifications at different sampling scales. The post-processing layer fuses the probabilities of different classifications at different sampling scales based on prior knowledge to determine the type of each trajectory point.

[0090] It should be understood that the agricultural trajectory point recognition model can also be other neural network structures, and the embodiments of the present application are not limited to this.

[0091] In the above technical solution, the type of each track point is determined by a trained agricultural track point recognition model, taking into account the tracks generated by agricultural vehicles and the environmental characteristics of the satellite map corresponding to the track point. In this way, compared to simply considering the difference between field operation tracks and road driving tracks, taking the environmental factors of the tracks into account can improve the accuracy of agricultural track point type recognition. For example, for single and double ridge field operation tracks, the solution of the embodiment of the present application is more accurate in determining the type of track point than the solution of determining the type of track point based on the parallel arrangement of multiple ridge lines of the traditional field operation track.

[0092] In addition, the agricultural trajectory point recognition model based on multi-level feature extraction can obtain local feature information and global feature information at different downsampling scales. When extracting features, the receptive field increases from small to large, which can take into account both detailed features and global features, and can extract more hidden features, further improving the accuracy of agricultural trajectory point type recognition.

[0093] The following will describe in detail the agricultural trajectory point recognition method, device and system provided by the embodiments of the present application in conjunction with Figures 4 to 8.

[0094] FIG4 is a flow chart of a data preprocessing process provided in an embodiment of the present application.

[0095] S410: Obtain trajectory points within the first time.

[0096] As a possible implementation, all track points of the agricultural vehicle 110 within the first time are acquired by the track acquisition device 111. The track points include position information of latitude and longitude.

[0097] The specific form of the trajectory collection device 111 has been described in detail above and will not be repeated here.

[0098] S420: Determine a second image based on the trajectory points and the satellite map corresponding to the trajectory points, where the first image is a partial image of the second image.

[0099] It should be understood that the satellite map may be a publicly available downloadable satellite map.

[0100] As a possible implementation manner, a satellite map corresponding to the track point is determined based on the position information of the track point, and the track point information is added to the satellite map to obtain a second image.

[0101] The second image is a four-channel image, including a red channel, a green channel, a blue channel, and a trajectory channel. The first image can be a partial image or the entire image of the second image. The red channel, the green channel, and the blue channel represent scene information around the trajectory.

[0102] Specifically, the range of the driving trajectory map is determined according to the latitude and longitude positions of the starting trajectory point and the ending trajectory point in the trajectory points within the first time. Accordingly, the range of the first satellite map can also be determined by the latitude and longitude positions of the starting trajectory point and the ending trajectory point.

[0103] For example, the corresponding tiles under the 18-level satellite map are found according to the latitude and longitude coordinates of some trajectory points in the first time, and 4*4 256*256 tiles are combined into a large 1024*1024 tile (that is, the first image).

[0104] In this way, the first image can include more image information.

[0105] For another example, a large tile of 1024*1024 is overlapped to add 128 pixels of information to the edge, and the pixels of the first image are expanded to 1280*1280.

[0106] In this way, the edge information of the first image input into the agricultural trajectory point recognition model is richer, which helps to improve the accuracy of trajectory point recognition.

[0107] As a possible implementation, when the first image is a partial image of the second image, other partial images of the second image can be respectively input into the agricultural trajectory point recognition model to obtain the type of the corresponding trajectory point.

[0108] For example, the second image corresponding to the driving trajectory within a day is divided into multiple 1280*1280 tiles, each tile is individually identified using the agricultural trajectory point recognition model, and the results are combined to obtain the type results of the trajectory points within a day.

[0109] As a possible implementation manner, the satellite map level corresponding to the first image is level 18.

[0110] In the above technical solution, after a large number of tests at different map levels, the map level of the above-mentioned first image is determined to be a tile map at level 18. This can avoid the level being too low, which may cause the trajectory to be too thin and lose many details, and can also avoid the level being too high, which may cause the image to be trapped in the local area and unable to obtain more global information.

[0111] The pre-processed first image is input into the agricultural trajectory point recognition model to obtain the type of each trajectory point. The steps of FIG3 will be described in detail below in conjunction with FIG5 and FIG6.

[0112] For ease of understanding, the meanings of the two rectangles in Figure 5 are explained here, where the "rectangular rectangle" represents input and output data, and the "rounded rectangle" represents the processing model of the input data, for example, processing the first image through the agricultural trajectory point recognition model.

[0113] FIG5 is a schematic structural diagram of an agricultural trajectory point recognition model provided in an embodiment of the present application.

[0114] The agricultural trajectory point model may include M feature extraction layers 520, an information interaction layer 530, M decoding layers 550, M output layers 530, and a post-processing layer 570. The M feature extraction layers are used to obtain M local feature information at different downsampling scales.

[0115] Optionally, the agricultural trajectory point recognition model may further include an input layer 510 .

[0116] Optionally, S1, input the acquired first image as an input image into the input layer 510 to obtain a first output.

[0117] For example, a first image of size 1280*1280*4 is input to the input layer 510, and the obtained first output includes a plurality of patches, wherein the number of channels of the first output is expanded.

[0118] Specifically, the input layer 510 may include two convolutional layers, the convolution kernel size of convolution layer #1 may be 3*3, the sliding step may be 2, and the number of channels may be 12, and the convolution kernel size of convolution layer #2 may be 3*3, the sliding step may be 2, and the number of channels may be 24.

[0119] In the above technical solution, the image size of the first image can be reduced and the number of features can be enriched through the input layer.

[0120] S2: Input the first output into the first feature extraction layer of M feature extraction layers, and output M local feature information layer by layer.

[0121] Among them, the embodiment of the present application does not limit the number of feature extraction layers. The number M of feature extraction layers can be 4. The following is a detailed description taking 4 feature extraction layers as an example.

[0122] Since the first image in the embodiment of the present application has a large number of channels, that is, a four-channel first image, which includes not only the RGB channels of the satellite map but also the trajectory channel, the capability of the feature extraction layer is required to be high. Therefore, the extraction quality of local feature information can be guaranteed by using multiple feature extraction layers.

[0123] It should also be understood that the downsampling factor of each feature extraction layer can be increased layer by layer.

[0124] For example, the downsampling multiples of the first feature extraction layer, the second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer may be 4 times, 8 times, 16 times, and 32 times, respectively.

[0125] As the feature extraction layers go deeper, the size of the feature map decreases, but the number of channels increases. The feature extraction layers closer to the front have smaller receptive fields and are used to extract detailed features, while the feature extraction layers closer to the back have larger receptive fields and are used to extract comprehensive features within a region.

[0126] For example, the deep-wise convolution structure of the first feature extraction layer uses two blocks, with 24 input channels and 32 output channels; the deep-wise convolution structure of the second feature extraction layer uses two blocks, with 32 input channels and 64 output channels; the deep-wise convolution structure of the third feature extraction layer uses six blocks, with 64 input channels and 128 output channels; and the deep-wise convolution structure of the fourth feature extraction layer uses two blocks, with 128 input channels and 256 output channels.

[0127] Specifically, taking M=4 as an example, the first input is input into the first feature extraction layer to obtain the first local feature information, which is then input into the second feature extraction layer, and so on, to obtain the first local feature information, the second local feature information, the third local feature information and the fourth local feature information respectively.

[0128] S3: Input the output of the M-th feature extraction layer 520 to the information interaction layer 530 to output global feature information of the N first images, where the global feature information includes environmental features of the satellite images.

[0129] For example, taking M=4 as an example, the output of the fourth feature extraction layer 524 is input into the information interaction layer 530 to obtain global feature information, wherein the downsampling multiple of the fourth feature extraction layer 524 is 32 times.

[0130] For example, after 32x downsampling, the size of the feature map is converted from the original 1280*1280*4 to 40*40*256, where 40*40 represents the height and width of the feature map, and each feature point corresponds to 32*32 pixels in the first image.

[0131] FIG6 is a schematic diagram of the structure of an information interaction layer provided in an embodiment of the present application.

[0132] As shown in FIG6 , the output of the fourth feature extraction layer 524 is input into three pooling convolution layers to obtain three global feature maps of different degrees. Subsequently, the three global feature maps of different degrees are input into the upsampling convolution layer 534 to obtain global feature information. The size of the global feature information is the same as the size of the output of the fourth feature extraction layer 524.

[0133] For example, the output of the fourth feature extraction layer 524 is passed through a first pooling convolution layer 531 to obtain a first-level global feature map. The output of the fourth feature extraction layer 524 is passed through a second pooling convolution layer 532 to obtain a second-level global feature map. The output of the fourth feature extraction layer 524 is passed through a third pooling convolution layer 533 to obtain a third-level global feature map. Global feature maps of varying degrees indicate that each feature point can utilize different levels of global information.

[0134] For example, the output size of the 4th feature extraction layer 524 is 40*40*256, the kernel size of the pooling layer #1 of the first pooling convolution layer 531 is 2, the sliding step size is 2, and the obtained feature map size is 20*20*256. After passing through the first pooling convolution layer 531, the convolution kernel size of the convolution layer #1 is 5, the sliding step size is 1, and the number of channels is 256. Each feature point can comprehensively utilize 5 / 20=25% of the global information, which is the first degree of global feature map.

[0135] For example, the output size of the 4th feature extraction layer 524 is 40*40*256, the kernel size of the pooling layer #2 of the second pooling convolution layer 532 is 4, the sliding step size is 4, and the obtained feature map size is 10*10*256. After passing through the second pooling convolution layer 532, the convolution kernel size of the convolution layer #2 is 5, the sliding step size is 1, and the number of channels is 256. Each feature point can comprehensively utilize 5 / 10=50% of the global information, which is the second-degree global feature map.

[0136] For example, the output size of the 4th feature extraction layer 524 is 40*40*256, the kernel size of the pooling layer #3 of the third pooling convolution layer 533 is 8, the sliding step size is 8, and the obtained feature map size is 5*5*256. After passing through the convolution kernel size of the convolution layer #3 of the third pooling convolution layer 533, the sliding step size is 1, and the number of channels is 256. Each feature point can comprehensively utilize 5 / 5=100% of the global information, which is the third degree of global feature map.

[0137] The three global feature maps of different levels are input into the upsampling convolution layer 534 to obtain global feature information.

[0138] Specifically, the first-degree global feature map, the second-degree global feature map, and the third-degree global feature map are all upsampled, and the size of the global feature map is restored to the same size as the output of the fourth feature extraction layer 524 through a convolution layer.

[0139] For example, the size of the first-level global feature map is 20*20*256, the size of the second-level global feature map is 10*10*256, and the size of the third-level global feature map is 5*5*256. The three global feature maps are upsampled using different multiples to obtain three global feature maps of size 40*40*256. These three global feature maps are concatenated to obtain a global feature map of size 40*40*768. Subsequently, the convolutional layer is used to regress the size of the global feature map to 40*40*256.

[0140] S4: Obtain the type of each trajectory point based on the M local feature information and the global feature information.

[0141] The agricultural trajectory point recognition model further includes M decoding layers 550, wherein a portion of the input of the i-th decoding layer is the output of the i+1-th decoding layer, and another portion of the input of the i-th decoding layer is the i-th local feature information corresponding to the i-th feature extraction layer. The upsampling multiple of the i-th decoding layer is the same as the downsampling multiple of the i-th feature extraction layer.

[0142] As a possible implementation method, the global feature information and the Mth local feature information are input into the Mth decoding layer to obtain the Mth classification feature map. The Mth classification feature map and the M-1th local feature information are input into the M-1th decoding layer, and the other M-1 classification feature maps except the Mth classification feature map are obtained layer by layer. Among them, the M classification feature maps include the Mth classification feature map and the other M-1 classification feature maps, and the i+1th classification feature map has less detailed information than the ith classification feature map. Based on the M classification feature maps, the type of each trajectory point is obtained.

[0143] For example, taking M=4, the global feature information and the fourth local feature information corresponding to the fourth feature extraction layer are input into the fourth decoding layer 554 to obtain the fourth classification feature map. The upsampling factor of the fourth decoding layer 554 is the same as the downsampling factor of the fourth feature extraction layer. The size of the fourth local feature information is the same as the size of the global feature information.

[0144] The fourth classification feature map and the third local feature information are input into the third decoding layer 553 to obtain the third classification feature map. The fourth classification feature map and the third local feature information have the same size.

[0145] The third classification feature map and the second local feature information are input into the second decoding layer 552 to obtain the second classification feature map. The third classification feature map and the second local feature information have the same size.

[0146] The second classification feature map and the first local feature information are input into the first decoding layer 551 to obtain the first classification feature map. The second classification feature map and the first local feature information have the same size.

[0147] The detailed information in the above classification feature maps, from more to less, are the first classification feature map, the second classification feature map, the third classification feature map, and the fourth classification feature map.

[0148] In short, M local feature information and global feature information are restored to the original resolution scale layer by layer through M decoding layers, and the features of various channels are fused and converted into classification feature maps.

[0149] In the above technical solution, the front feature extraction layer can extract local details, while the back feature extraction layer has a wider receptive field, which is used to predict the type of a long track. Through this overall first, then local feature extraction and fusion method, that is, by first constructing the overall classification of the entire track in the last feature extraction layer and information interaction layer, "which large area is a field, which large section is a road", and then passing it through higher resolution decoding layers layer by layer, gradually optimizing the details, and obtaining classification feature maps with different levels of detail. In this way, the accuracy of track point classification can be further improved.

[0150] Optionally, the agricultural trajectory point recognition model further includes M connection jump layers 540, where the i-th connection jump layer is used to directly map the i-th local feature information and the i+1-th classification feature map output by the i+1-th decoding layer to the i-th decoding layer.

[0151] The Mth connection skip layer directly maps the Mth local feature information and global feature information to the Mth decoding layer. For example, the fourth connection skip layer 544 directly maps the fourth local feature information and global feature information to the fourth decoding layer.

[0152] In this way, the defects of information change or loss caused by downsampling of each feature extraction layer and upsampling of each decoding layer can be compensated, thereby improving the accuracy of trajectory point classification.

[0153] Furthermore, the agricultural trajectory point recognition model also includes M output layers 560 .

[0154] According to the M classification feature maps, the type of each trajectory point can be:

[0155] The i-th classification feature map is input into the i-th output layer to obtain the i-th first classification result. The i-th first classification result is the probability of each pixel's type at the downsampling scale of the i-th feature extraction layer. Each pixel's type includes background type, field type, and road type. Based on the M first classification results, the type of each trajectory point is obtained. The M first output results include the i-th first classification result.

[0156] Specifically, a convolutional layer with 3 channels is used to map the i-th classification feature map to the three categories of background type, field type, and road type. A softmax operation is then performed to obtain the probability value for each category. For example, the output is [p1, p2, p3], where p1 is the probability value of the pixel being background type, p2 is the probability value of the pixel being road type, and p3 is the probability value of the pixel being field type.

[0157] For example, the fourth classification feature map is input into the fourth output layer 564 to obtain the fourth first classification result. The fourth first classification result is the probability of the type of each pixel point at the sampling scale of the fourth feature extraction layer.

[0158] The third classification feature map is input into the third output layer 563 to obtain the third first classification result. The third first classification result is the probability of the type of each pixel point at the sampling scale of the third feature extraction layer.

[0159] The second classification feature map is input into the second output layer 562 to obtain a second first classification result. The second first classification result is the probability of the type of each pixel point at the sampling scale of the second feature extraction layer.

[0160] The first classification feature map is input into the first output layer 561 to obtain the first first classification result. The first first classification result is the probability of the type of each pixel point at the sampling scale of the first feature extraction layer.

[0161] In the above technical solution, the classification feature maps obtained at different sampling scales are input into their respective corresponding output layers, so that a more accurate first classification result can be obtained.

[0162] The agricultural trajectory point recognition model further includes a post-processing layer 570 .

[0163] Furthermore, according to the M first classification results, the type of each trajectory point may specifically include:

[0164] Post-processing layer 570 is used to separate background pixels from non-background pixels to determine whether each pixel corresponding to a track point in the first classification result is a field or road type. The M first classification results are input into the post-processing layer to determine the type of each track point.

[0165] As a possible implementation method, in the post-processing layer, the type of each trajectory point is obtained according to the prior knowledge constraints of the background type and the non-background type and the probability constraint with the highest output probability value.

[0166] Specifically, since the classification result on the track can only be road or field, and the correct result for the background can only be background, the mask corresponding to the background type can be set to [1,0,0], and the mask corresponding to the non-background type can be set to [0,1,1].

[0167] For each pixel, multiply the probability of the three pixels with the highest probability value p1 by mask = [1,0,0] to obtain [p1,0,0], which represents the classification result of the background area. Since the probabilities of roads and fields are set to zero, the classification result of these areas can only be background.

[0168] Multiplying the probability values ​​of all pixels except for the above cases by mask = [0, 1, 1] yields [0, p2, p3], which represents the classification result on the trajectory line. Since the probability multiplier of the background is set to zero, the classification result on the trajectory line can only be either field or road.

[0169] In the above scheme, under these two constraints, the model will only classify the parts with tracks as fields or roads, and only recognize the background in the parts without tracks, without the common boundary blur and scattered point problems in semantic segmentation.

[0170] As a possible implementation method, the loss function of the agricultural trajectory point recognition model can be a Focal loss function, and the weight a of the road part in the loss function is increased.

[0171] For example, the weight a of the road portion in the vegetarian function can be 2.5-3.

[0172] In the above technical solution, due to the serious imbalance in the distribution of field operation trajectories and road driving trajectories in agricultural trajectory points (for example, after analyzing approximately 1,000 agricultural machinery operation trajectories and approximately 42,000 images, more than 80% of the trajectories are field types and less than 20% of the trajectories are road types), if conventional loss functions such as cross entropy loss function and sum of squares loss function are directly used, the model classification results will inevitably be more inclined to the field operation type. Therefore, by using the Focal loss function and increasing the weight of the road part in the loss function, not only can the accuracy of model recognition be improved, but the model is also easier to converge, thereby increasing the speed of model recognition.

[0173] FIG8 is a rendering of an agricultural trajectory recognition result provided by an embodiment of the present application.

[0174] Using the track point types obtained by the above recognition method, field-type track points are set to the first color, and road-type track points are set to the second color. The track point classification effect diagram shown in Figure 8 is drawn on the satellite map. The contrast between the first color and the second color is high.

[0175] The above describes in detail the agricultural trajectory point identification method. Below, a brief description of the agricultural trajectory point identification device will be provided in conjunction with Figures 9 and 10. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment. Therefore, any details not described in detail can be referred to the method embodiment above, and for the sake of brevity, they will not be repeated here.

[0176] FIG9 is a schematic diagram of an agricultural trajectory point recognition device provided by an embodiment of the present application. The device includes: an acquisition unit 9001 and a processing unit 9002.

[0177] The acquisition unit 9001 is used to acquire a first image, where the first image is a four-channel image, and the four channels include a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a trajectory channel corresponding to the first image.

[0178] The processing unit 9002 is configured to input the first image into an agricultural trajectory point recognition model, and output the type of each trajectory point corresponding to the first image, where the type of the trajectory point includes a field type or a road type.

[0179] It should be understood that the above description is merely an example. The agricultural trajectory point identification device is used to perform the methods or steps mentioned in the aforementioned method embodiments. Therefore, the device corresponds to the aforementioned method embodiments. For details, please refer to the description of the aforementioned method embodiments and will not be repeated here.

[0180] FIG10 is a schematic diagram of the hardware structure of an agricultural trajectory point identification device provided by an embodiment of the present application. The agricultural trajectory point identification device 1000 shown in FIG10 may include: a memory 1010, a processor 1020, and a communication interface 1030. The memory 1010, the processor 1020, and the communication interface 1030 are connected via an internal connection path. The memory 1010 is used to store instructions, and the processor 1020 is used to execute the instructions stored in the memory 1020 to control the input / output interface 1030 to receive / send at least some parameters of the second channel model. Optionally, the memory 1010 can be coupled to the processor 1020 via an interface or integrated with the processor 1020.

[0181] It should be noted that the communication interface 1030 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the communication device 1000 and other devices or a communication network. The communication interface 1030 may also include an input / output interface.

[0182] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 1020 or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1010, and the processor 1020 reads the information in the memory 1010 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0183] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0184] It should also be understood that in the embodiments of the present application, the memory may include read-only memory and random access memory, and provide instructions and data to the processor. A portion of the processor may also include non-volatile random access memory. For example, the processor may also store device type information.

[0185] It should be understood that the term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0186] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0187] An embodiment of the present application further provides an agricultural trajectory point identification system including a trajectory collection device 111 and a data processing device 120 , wherein the data processing device 120 may be the agricultural trajectory point identification device 9000 or the agricultural trajectory point identification device 1000 described above.

[0188] Optionally, the data processing device 120 may be located in a cloud server.

[0189] Optionally, if the data processing device 120 is a data processing device 120 located in a cloud server, the system further includes a device for receiving instructions from the data processing device 120 .

[0190] An embodiment of the present application also provides a device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the device performs the above-mentioned agricultural trajectory point recognition method.

[0191] An embodiment of the present application further provides a computer-readable medium storing a program code. When the computer program code is executed on a computer, the computer executes any one of the methods in FIG. 3 to FIG. 6 .

[0192] An embodiment of the present application also provides a chip, comprising: at least one processor and a memory, wherein the at least one processor is coupled to the memory and is configured to read and execute instructions in the memory to execute any one of the methods in Figures 3 to 6 above.

[0193] An embodiment of the present application also provides an agricultural vehicle, comprising: at least one processor and a memory, wherein the at least one processor is coupled to the memory and is configured to read and execute instructions in the memory to execute any one of the methods in Figures 3 to 6 above.

[0194] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0195] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0196] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0200] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0201] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for identifying agricultural trajectory points, characterized in that: include: Acquire a first image, where the first image is a four-channel image, and the four channels include a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a trajectory channel corresponding to the first image; The first image is input into an agricultural track point recognition model, and the type of each track point corresponding to the first image is output, where the type of the track point includes a field type or a road type.

2. The method according to claim 1, characterized in that The agricultural trajectory point recognition model includes M feature extraction layers and information interaction layers, the output of the i-th feature extraction layer is the input of the i+1-th feature extraction layer, M is a positive integer greater than or equal to 2, i is a positive integer, 1≤i≤M-1, the downsampling multiples of the M feature extraction layers are increased layer by layer, and the output of the M-th feature extraction layer is used as the input of the information interaction layer; The step of inputting the first image into the agricultural trajectory point recognition model and outputting the type of each trajectory point corresponding to the N first images comprises: Inputting the first image into the first feature extraction layer of the M feature extraction layers to obtain M local feature information layer by layer; Input the output of the Mth feature extraction layer to the information interaction layer, and output the global feature information of the N first images, where the global feature information includes the environmental features of the satellite images; The type of each trajectory point is obtained according to the M local feature information and the global feature information.

3. The method according to claim 2, characterized in that The agricultural trajectory point recognition model also includes M decoding layers, a part of the input of the i-th decoding layer is the output of the i+1-th decoding layer, another part of the input of the i-th decoding layer is the i-th local feature information corresponding to the i-th feature extraction layer, and the upsampling multiple of the i-th decoding layer is the same as the downsampling multiple of the i-th feature extraction layer; The obtaining the type of each trajectory point according to the M local feature information and the global feature information includes: Inputting the global feature information and the Mth local feature information into the Mth decoding layer to obtain the Mth classification feature map; Input the M-th classification feature map and the M-1th local feature information into the M-1th decoding layer, and obtain the other M-1 classification feature maps except the M-th classification feature map layer by layer, wherein the M classification feature maps include the M-th classification feature map and the other M-1 classification feature maps, and the detail information of the i+1th classification feature map is less than that of the i-th classification feature map; According to the M classification feature maps, the type of each trajectory point is obtained.

4. The method according to claim 3, characterized in that The agricultural trajectory point recognition model also includes M output layers; The type of each trajectory point obtained according to the M classification feature maps includes: Input the i-th classification feature map into the i-th output layer to obtain the i-th first classification result, wherein the i-th first classification result is the probability of the type of each pixel point under the downsampling scale of the i-th feature extraction layer, and the type of each pixel point includes background type, field type and road type; According to the M first classification results, the type of each trajectory point is obtained, and the M first output results include the i-th first classification result.

5. The method according to claim 4, characterized in that The agricultural trajectory point recognition model further includes a post-processing layer, which is used to separate background type and non-background type pixels to determine whether each pixel corresponding to the trajectory point in the first classification result is a field type or a road type; The M first classification results are input into a post-processing layer to obtain the type of each trajectory point.

6. The method according to any one of claims 1 to 5, characterized in that: The acquiring of the first image comprises: Get the trajectory points in the first time; A second image is determined according to the trajectory point and the satellite map corresponding to the trajectory point, wherein the first image is a partial image or the entire image of the second image.

7. The method according to claim 6, characterized in that The satellite map level corresponding to the first image is level 18.

8. An agricultural trajectory point recognition device, characterized in that: The identification device comprises an acquisition unit and a processing unit. The acquisition unit is used to acquire a first image, where the first image is a four-channel image, and the four channels include a red channel, a green channel, and a blue channel of a satellite map corresponding to the first image, and a trajectory channel corresponding to the first image; The processing unit is used to input the first image into an agricultural trajectory point recognition model, and output the type of each trajectory point corresponding to the first image, where the type of the trajectory point includes a field type or a road type.

9. The device according to claim 8, characterized in that The agricultural trajectory point recognition model includes M feature extraction layers and information interaction layers, the output of the i-th feature extraction layer is the input of the i+1-th feature extraction layer, M is a positive integer greater than or equal to 2, i is a positive integer, 1≤i≤M-1, the downsampling multiples of the M feature extraction layers are increased layer by layer, and the output of the M-th feature extraction layer is used as the input of the information interaction layer; The processing unit is specifically used for: Inputting the first image into the first feature extraction layer of the M feature extraction layers to obtain M local feature information layer by layer; Input the output of the Mth feature extraction layer to the information interaction layer, and output the global feature information of the N first images, where the global feature information includes the environmental features of the satellite images; The type of each trajectory point is obtained according to the M local feature information and the global feature information.

10. The device according to claim 9, characterized in that The agricultural trajectory point recognition model also includes M decoding layers, a part of the input of the i-th decoding layer is the output of the i+1-th decoding layer, another part of the input of the i-th decoding layer is the i-th local feature information corresponding to the i-th feature extraction layer, and the upsampling multiple of the i-th decoding layer is the same as the downsampling multiple of the i-th feature extraction layer; The processing unit is specifically used for: Inputting the global feature information and the Mth local feature information into the Mth decoding layer to obtain the Mth classification feature map; Input the M-th classification feature map and the M-1th local feature information into the M-1th decoding layer, and obtain the other M-1 classification feature maps except the M-th classification feature map layer by layer, wherein the M classification feature maps include the M-th classification feature map and the other M-1 classification feature maps, and the detail information of the i+1th classification feature map is less than that of the i-th classification feature map; According to the M classification feature maps, the type of each trajectory point is obtained.

11. The device according to claim 10, characterized in that The agricultural trajectory point recognition model also includes M output layers; The processing unit is specifically used for: Input the i-th classification feature map into the i-th output layer to obtain the i-th first classification result, wherein the i-th first classification result is the probability of the type of each pixel point under the downsampling scale of the i-th feature extraction layer, and the type of each pixel point includes background type, field type and road type; According to the M first classification results, the type of each trajectory point is obtained, and the M first output results include the i-th first classification result.

12. The device according to claim 11, characterized in that The agricultural trajectory point recognition model also includes a post-processing layer, which is used to separate background type and non-background type pixel points to determine whether each pixel point corresponding to the trajectory point in the first classification result is a field type or a road type; the processing unit is specifically used to: input the M first classification results into the post-processing layer to obtain the type of each trajectory point.

13. The device according to any one of claims 8 to 12, characterized in that The acquisition unit is specifically used for: Get the trajectory points in the first time; A second image is determined according to the trajectory point and the satellite map corresponding to the trajectory point, wherein the first image is a partial image or the entire image of the second image.

14. The device according to claim 13, characterized in that The satellite map level corresponding to the first image is level 18.

15. An agricultural trajectory point identification device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

17. An agricultural trajectory point recognition system, characterized in that: It comprises a trajectory collection device and an agricultural trajectory point recognition device, wherein the agricultural trajectory point recognition device is the agricultural trajectory point recognition device according to any one of claims 8 to 15.

Images