Method and apparatus for detecting land use, device, and storage medium

Abstract

A method and apparatus for detecting a land use, a device, and a storage medium. The method comprises: acquiring a current image to be detected of a land region to be detected (S110); an image block matrix of said current image and a preset detection encoding sequence being subjected to processing by means of encoding layers of an image embedding layer to obtain a first image feature matrix (S120); the first image feature matrix and the preset detection encoding sequence being subjected to processing by means of a detection layer network of the image embedding layer to obtain a second image feature matrix (S130); using phrase embedding layers to perform feature extraction on a pre-constructed label phrase comprising various land anomaly use labels so as to obtain a phrase feature matrix (S140); and on the basis of the second image feature matrix and the phrase feature matrix, performing alignment processing on the land anomaly use labels in the pre-constructed label phrase and said current image by means of a joint processing layer, and determining a detection result of said current image (S150).

Description

Land use detection method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 8, 2023, with application number 202311158378.X. The entire contents of this application are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of deep learning technology, for example, to a method, device, equipment and storage medium for detecting land use. Background Art

[0003] In the government project industry of monitoring illegal land use, in order to monitor the newly added illegal behaviors such as unauthorized changes in land use in the Natural Resources Surveying and Mapping Institute, the illegal land use change detection and recognition algorithms in related technologies, such as faster R-CNN, yolo, mask R-CNN, etc., simply calibrate the recognition targets without understanding the deep meaning of the specific recognition targets themselves; if only illegal behaviors such as unauthorized changes in land use need to be detected, traditional image detection and recognition algorithms need to add engineering methods on top of the model, and cannot expand and integrate this behavior into the illegal land use change model; the illegal land use change monitoring and recognition algorithm needs to use new pictures or video samples to re-label the data set and retrain expensively.

[0004] Summary of the Invention

[0005] The present application provides a land use detection method, apparatus, equipment and storage medium to enhance the flexible scalability of the detection model and conduct in-depth detection of land use to improve detection accuracy.

[0006] According to one aspect of the present application, a method for detecting land use is provided, the method comprising:

[0007] Obtaining a current image of the land area to be detected, and inputting the current image into a pre-trained detection network; wherein the pre-trained detection network includes an image embedding layer, a phrase embedding layer, and a joint processing layer; the image embedding layer includes an encoding layer and a detection layer network;

[0008] Inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer to perform feature fusion learning to obtain a first image feature matrix;

[0009] Inputting the first image feature matrix and the preset detection coding sequence into the detection layer network for secondary feature fusion learning to obtain a second image feature matrix;

[0010] Using the phrase embedding layer to extract features from pre-constructed label phrases to obtain a phrase feature matrix; wherein the pre-constructed label phrases include multiple abnormal land use labels;

[0011] According to the second image feature matrix and the phrase feature matrix, the abnormal land use label in the pre-constructed label phrase is aligned with the current image to be detected through the joint processing layer to determine the detection result of the current image to be detected.

[0012] According to another aspect of the present application, a device for detecting land use is provided, the device comprising:

[0013] a detection image input module configured to obtain a current image to be detected of the land area to be detected and input the current image to be detected into a pre-trained detection network; wherein the pre-trained detection network includes an image embedding layer, a phrase embedding layer, and a joint processing layer; the image embedding layer includes an encoding layer and a detection layer network;

[0014] A first image feature matrix acquisition module is configured to input the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer to perform a feature fusion learning to obtain a first image feature matrix;

[0015] a second image feature matrix acquisition module configured to input the first image feature matrix and the preset detection coding sequence into the detection layer network for secondary feature fusion learning to obtain a second image feature matrix;

[0016] a phrase feature matrix acquisition module configured to extract features from pre-constructed label phrases using the phrase embedding layer to obtain a phrase feature matrix; wherein the pre-constructed label phrases include multiple abnormal land use labels;

[0017] The detection result determination module is configured to align the abnormal land use label in the pre-constructed label phrase with the current image to be detected through the joint processing layer based on the second image feature matrix and the phrase feature matrix, and determine the detection result of the current image to be detected.

[0018] According to another aspect of the present application, an electronic device is provided, comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the land use detection method described in any embodiment of the present application.

[0022] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the land use detection method described in any embodiment of the present application when executed.

[0023] The technical solution of the embodiment of the present application is as follows: obtaining a current image to be detected of a land area to be detected, and inputting the current image to be detected into a pre-trained detection network including an image embedding layer, a phrase embedding layer and a joint processing layer; inputting the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer of the image embedding layer for a feature fusion learning to obtain a first image feature matrix; inputting the first image feature matrix and the preset detection coding sequence into the detection layer network of the image embedding layer for a second feature fusion learning to obtain a second image feature matrix; using the phrase embedding layer to extract features of a pre-constructed label phrase including multiple abnormal land use labels to obtain a phrase feature matrix; based on the second image feature matrix and the phrase feature matrix, aligning the abnormal land use labels in the pre-constructed label phrase with the current image to be detected through the joint processing layer to determine the detection result of the current image to be detected, thereby solving the problems in the related art that the detection model does not understand the deep meaning of the specific recognition target itself, cannot expand the behavior into the illegal land use change model, and needs to use new pictures or video samples to re-label the expensive data set and retrain, thereby improving the flexible scalability of the detection model and deeply detecting land use to improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The following is a brief introduction to the drawings required for the description of the embodiments. The drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] FIG1a is a flow chart of a method for detecting land use provided in Example 1 of the present application;

[0026] FIG1b is a schematic diagram of the structure of a pre-trained detection network provided in Example 1 of the present application;

[0027] FIG1c is a schematic diagram of the structure of a detection layer network provided in Example 1 of the present application;

[0028] FIG1d is an application example diagram of the land use detection method provided in Example 1 of the present application;

[0029] FIG2 is a schematic structural diagram of a land use detection device provided in Example 2 of the present application;

[0030] FIG3 is a schematic structural diagram of an electronic device for implementing the method for detecting land use according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The embodiments described are some embodiments related to the present application, but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Example 1

[0034] Figure 1a is a flow chart of a land use detection method provided in Example 1 of the present application. This embodiment is applicable to situations where land use is detected and verified. The method can be performed by a land use detection device, which can be implemented in the form of hardware and / or software. The land use detection device can be configured in an intelligent analysis algorithm platform. As shown in Figure 1a, the method includes:

[0035] S110. Obtain a current image to be detected of the land area to be detected, and input the current image to be detected into a pre-trained detection network; wherein the pre-trained detection network includes an image embedding layer, a phrase embedding layer, and a joint processing layer; the image embedding layer includes a coding layer and a detection layer network.

[0036] Among them, the pre-trained detection network can be obtained through the following steps:

[0037] Step 1: Dataset construction: Use more existing datasets (a collection of land area images).

[0038] Step 2: Sample Construction: Convert the existing labels on the land area image into phrases. This involves constructing a land area image and a phrase as a training sample. The construction method is "land area image + phrase." The phrase is constructed by connecting multiple labels with ### and adding "OBJDECT:" before the phrase. For example, if there are three abnormal land use labels, the phrase constructed might be "OBJDECT: ###Illegal farmland construction ###Mobile board house ###Simple shed." This setup allows samples to be constructed using the sample construction method in Step 2, regardless of whether the abnormal land use labels already exist or are newly added. This makes the detection network more flexible and allows for easy migration.

[0039] Step 3: Monitor image encoding: As shown in Figure 1b, the image embedding section first segments the land area image into patches. The tokens (feature encoding sequences) output by the feed-forward layer flow through the 6-layer Transformer layer. The output tokens are then processed three times through the DectionBlock Layer Network, as shown in Figure 1c. To ensure cyclic processing, the input and output of each layer of the DectionBlock Layer Network are processed through the subsequent feed-forward fully connected layer. The feed-forward fully connected layer is controlled to ensure the consistency of the input and output of each layer.

[0040] Six overlapping Transformer layers are used to extract deep image features. The processing logic of each layer is as follows: first, the feature map obtained according to the image patch flows through multi-head attention processing (multi-head means multiple groups of linear transformation layers. Here, a group of linear change layers is used, that is, three transformation tensors perform linear transformations on Wq, Wv and Wk respectively. These transformations do not change the size of the original tensor, so each transformation matrix is ​​a square matrix, which is convenient for GPU acceleration). This allows each patch of the image to interact globally and obtain global information of the image; then it is processed through the standard normalization layer and the residual network layer for better feature extraction and easy training convergence.

[0041] The input to this step is the patch embeddings of the original land area image and the learnable Dectection Block. The Dectection Block flows through the Feed-Forward layer, which outputs Dectection Tokens. The Transformer layer's attention mechanism self-learns to learn which patches belong to which Dectection Tokens.

[0042] By adding a learnable Dectection Token; calculating the loss value based on the predicted results and the actual results, by updating a set of linear change layers used here, that is, the three transformation tensors perform linear transformations on Wq, Wv and Wk respectively; that is, calculating the loss function value to control, backpropagating to the learnable Dectection Token to update the parameters to achieve the parameters of the detection feature module to which the patch belongs.

[0043] Step 4, Phrase Encoding: As shown in Figure 1b, the phrase embedding part is segmented to obtain multiple words; each word is treated as a token in the embedding; in addition, there are some additional prompt words prom tokens, such as "OBJDECT:", "###", etc., or special tokens in the language; such as [MASK], [NoObj], etc.; the phrase encoding in this step is encoded using BEAT, such as A=Bert encode (phrase).

[0044] The pre-training process for the detection network is as follows: After six Transformer layers, the network learns which dectection tokens belong to which patches, completing L0-level fusion. The dectection block's adapter then completes the dectection tokens, assigning image patch tokens to each dectection token and fusing them into L1-level dectection tokens with high-level semantic information (e.g., L1-level fused tokens with a dimension of 100×512).

[0045] The goal of training the fusion layer in the middle of the detection network is to further interact the image feature embeddings and phrase feature embeddings so that the final image-phrase joint feature space is better trained (that is, similar image-phrase joint feature embeddings in the joint feature space are distributed closer together, and dissimilar ones are distributed farther apart). After the image features and phrase features are trained, they are stronger and more cohesive.

[0046] Step 5. Loss definition: The loss function is defined as positioning loss LOSS location +Alignment loss LOSS align Composition, such as LOSS total =LOSS location +LOSS align , where the label of the alignment loss is the alignment of the phrase with each regin of the image; as shown in the following formulas 1, 2 and 3:

[0047] I=Transform encode (Image) Formula 1

[0048] A=Bert encode (Phrase) Formula 2

[0049] SCORE align =AI T Formula 3

[0050] Among them, Formula 1 indicates that the image encoder encodes the land area image to obtain M regin embeddings, I∈R M*D , D represents the dimension of embeddings, that is, the land area picture of each sample is taken as input, and the feature I of each picture is obtained after 6 layers of Transform encoding; similarly, multiple pictures are matrices; Formula 2 represents the phrase encoder to obtain N embeddings of the phrase after segmentation and encoding, and the number A∈R N*D Formula 3 shows that the image feature code and phrase feature code are multiplied to obtain the align score SCORE∈R N*M .

[0051] The current image to be detected can be acquired by a camera placed in the land area to be detected. In this embodiment, the pre-trained detection network obtained based on steps 1 to 5 above can be used to detect the current image to be detected.

[0052] S120 , inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer to perform feature fusion learning to obtain a first image feature matrix.

[0053] Optionally, before inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer for feature fusion learning, it can also include: slicing the current image to be detected to obtain multiple slice images; and obtaining the image block matrix based on the multiple slice images.

[0054] The sliced ​​image may be equivalent to the patch slice in step 3. The image block matrix may refer to the token (feature encoding sequence) output by the feed-forward layer after the patch slice in step 3.

[0055] As shown in the image embedding part on the left side of Figure 1b, the encoding layer is a repeated structure of the first preset number (for example, 6 layers); each encoding layer includes a first unit layer (Norm→Multi-Head-Attention→Norm) and a second unit layer (Norm→Feed-Forward→Norm); the structure of the first unit layer is the first standard norm layer (Norm), the multi-head attention layer (Multi-Head-Attention) and the second standard norm layer (Norm); the structure of the second unit layer is the third standard norm layer (Norm), the feedforward layer (Feed-Forward) and the fourth standard norm layer (Norm).

[0056] Accordingly, inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer for feature fusion learning to obtain a first image feature matrix may include:

[0057] The image block matrix and the preset detection coding sequence are used as the first connection item, and the first connection item is input into the first coding layer, and is processed in sequence by the first standard normalization layer, the multi-head attention layer, and the second standard normalization layer of the first unit layer in the first coding layer to obtain the second connection item; the first connection item and the second connection item are summed to obtain a first processing result; the first processing result is processed in sequence by the third standard normalization layer, the feedforward layer, and the fourth standard normalization layer of the second unit layer to obtain a third connection item; the first processing result and the third connection item are summed to obtain a second processing result; the second processing result is input into the next coding layer, and the operation of being processed in sequence by the first standard normalization layer, the multi-head attention layer, and the second standard normalization layer of the first unit layer in the first coding layer is executed until all coding layers are processed to obtain the first image feature matrix.

[0058] Exemplarily, the first connection item is the input of the transformer layer, that is, the patch matrix (M*D) of the current image to be detected + the learnable Dectection Token (Dectection Token matrix (N*D). The M*D patch matrix and the N*D Dectection Token matrix are subjected to the Norm→Multi-Head-Attention→Norm of the first unit layer to obtain the second connection item, and the first connection item and the second connection item are added to obtain the first processing result; then the first processing result is subjected to the Norm→Feed-Forward→Norm of the second unit layer to obtain the third connection item, and the first processing result and the third connection item are added to obtain the second processing result. The second processing result is input into the next coding layer, and the above operation is repeated until the processing of the 6 coding layers is completed to obtain the (M+N)*D matrix.

[0059] S130: Input the first image feature matrix and the preset detection coding sequence into the detection layer network for secondary feature fusion learning to obtain a second image feature matrix.

[0060] Optionally, the first image feature matrix and the preset detection coding sequence are input into the detection layer network for secondary feature fusion learning to obtain the second image feature matrix, including: using the transformation tensor of the linear transformation layer in the detection layer network to multiply the first image feature matrix and the preset detection coding sequence to obtain the transformed image feature matrix; adding the transformed image feature matrix and the preset detection coding sequence to obtain the second image feature matrix.

[0061] Referring to Figure 1c, Wv, Wk and Wq are three transformation tensors, and Wv, Wk and Wq are linearly transformed respectively; through this set of linear change layer distributions, the N*D Dectection Tocken matrix and the M*D patch matrix are matrix multiplied to obtain the N*M similarity matrix, and then the similarity matrix is ​​processed using the parameter Wo to obtain the N*M transformed image feature matrix; the preset detection coding sequence is added to the transformed image feature matrix, which represents the fusion and merging of the patch matrix into the Dectection Tocken matrix (the fusion and merging calculation strategy is to add the patch correlation matrix with the highest Dectection Tocken score); finally, the dectionBlock layer network output is the same size as the Dectection Tocken matrix, that is, N*D.

[0062] S140. Use a phrase embedding layer to extract features from pre-constructed label phrases to obtain a phrase feature matrix; wherein the pre-constructed label phrases include multiple abnormal land use labels.

[0063] Referring to the phrase embedding part on the right side of Figure 1b, the phrase embedding layer is a repeated structure of a second preset number (for example, 12 layers), and each phrase embedding layer includes a third unit layer and a fourth unit layer; the structure of the third unit layer is the same as that of the first unit layer; the structure of the fourth unit layer is the same as that of the second unit layer.

[0064] Correspondingly, using the phrase embedding layer to extract features of the pre-constructed label phrase to obtain a phrase feature matrix can include: taking the phrase matrix corresponding to the pre-constructed label phrase as the fourth connection item, and inputting the fourth connection item into the first phrase embedding layer, and sequentially processing it through the first standard normalization layer, the multi-head attention layer, and the second standard normalization layer of the third unit layer in the first phrase embedding layer to obtain the fifth connection item; summing the fourth connection item and the fifth connection item to obtain a third processing result; sequentially processing the third processing result through the third standard normalization layer, the feedforward layer, and the fourth standard normalization layer of the fourth unit layer to obtain the sixth connection item; summing the third processing result and the sixth connection item to obtain the fourth processing result; inputting the fourth processing result into the next phrase embedding layer, executing the operation of processing it through the first standard normalization layer, the multi-head attention layer, and the second standard normalization layer of the third unit layer in the first phrase embedding layer, until all phrase embedding layers are processed and the phrase feature matrix is ​​obtained.

[0065] For example, the input of the phrase embedding part on the right side of Figure 1b is a T*D phrase matrix. After 12 layers of Norm→Multi-Head-Attention→Norm, and Norm→Feed-Forward→Norm processing, the output is an N*D phrase feature matrix.

[0066] S150 , aligning the abnormal land use label in the pre-constructed label phrase with the current image to be detected through a joint processing layer according to the second image feature matrix and the phrase feature matrix, and determining a detection result for the current image to be detected.

[0067] Optionally, based on the second image feature matrix and the phrase feature matrix, the abnormal land use label in the pre-constructed label phrase is aligned with the current image to be detected through a joint processing layer to determine the detection result of the current image to be detected, which may include: transposing the second image feature matrix through the joint processing layer to obtain a transposed second image feature matrix; multiplying the transposed second image feature matrix and the phrase feature matrix to align the abnormal land use label in the pre-constructed label phrase with the current image to be detected to obtain an alignment score; and determining the detection result of the current image to be detected based on the alignment score.

[0068] Referring to Formula 3, the output of the image embedding layer I (i.e., the second image feature matrix) can be transposed to obtain I T ; Multiply it with A (i.e., phrase feature matrix) obtained by the phrase embedding layer to calculate the score, thereby determining the detection result of the current image to be detected.

[0069] Optionally, after outputting the detection result of the current image to be detected, it can also include: obtaining the target land abnormal use type from the detection result; generating early warning information based on the target land abnormal use type and displaying it to monitoring personnel for timely management.

[0070] Referring to Figure 1d, the camera monitors the video and images of the land area to be monitored, and pushes the image stream to the intelligent analysis center algorithm platform deployed with a pre-trained detection network. The platform analyzes and identifies the detection results of abnormal land use, and then pushes the detection results to the IoT perception and business system for early warning management and verification and disposal.

[0071] The technical solution of the embodiment of the present application is as follows: obtaining a current image to be detected of a land area to be detected, and inputting the current image to be detected into a pre-trained detection network including an image embedding layer, a phrase embedding layer and a joint processing layer; inputting the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer of the image embedding layer for a feature fusion learning to obtain a first image feature matrix; inputting the first image feature matrix and the preset detection coding sequence into the detection layer network of the image embedding layer for a second feature fusion learning to obtain a second image feature matrix; using the phrase embedding layer to extract features of a pre-constructed label phrase including multiple abnormal land use labels to obtain a phrase feature matrix; based on the second image feature matrix and the phrase feature matrix, aligning the abnormal land use labels in the pre-constructed label phrase with the current image to be detected through the joint processing layer to determine the detection result of the current image to be detected, thereby solving the problems in the related art that the detection model does not understand the deep meaning of the specific recognition target itself, cannot expand the behavior into the illegal land use change model, and needs to use new pictures or video samples to re-label the expensive data set and retrain, thereby improving the flexible scalability of the detection model and deeply detecting land use to improve detection accuracy.

[0072] Example 2

[0073] FIG2 is a schematic diagram of the structure of a land use detection device provided in Example 2 of the present application. As shown in FIG2 , the device includes: a detection image input module 210, a first image feature matrix acquisition module 220, a second image feature matrix acquisition module 230, a phrase feature matrix acquisition module 240, and a detection result determination module 250. Among them:

[0074] The detection image input module 210 is configured to obtain a current image to be detected of the land area to be detected and input the current image to be detected into a pre-trained detection network; wherein the pre-trained detection network includes an image embedding layer, a phrase embedding layer, and a joint processing layer; the image embedding layer includes an encoding layer and a detection layer network;

[0075] A first image feature matrix acquisition module 220 is configured to input the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer to perform a feature fusion learning to obtain a first image feature matrix;

[0076] A second image feature matrix acquisition module 230 is configured to input the first image feature matrix and the preset detection coding sequence into the detection layer network to perform secondary feature fusion learning to obtain a second image feature matrix;

[0077] a phrase feature matrix acquisition module 240 configured to extract features from pre-constructed label phrases using the phrase embedding layer to obtain a phrase feature matrix; wherein the pre-constructed label phrases include multiple abnormal land use labels;

[0078] The detection result determination module 250 is configured to align the abnormal land use label in the pre-constructed label phrase with the current image to be detected through the joint processing layer based on the second image feature matrix and the phrase feature matrix, and determine the detection result of the current image to be detected.

[0079] The technical solution of the embodiment of the present application is as follows: obtaining a current image to be detected of a land area to be detected, and inputting the current image to be detected into a pre-trained detection network including an image embedding layer, a phrase embedding layer and a joint processing layer; inputting the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer of the image embedding layer for a feature fusion learning to obtain a first image feature matrix; inputting the first image feature matrix and the preset detection coding sequence into the detection layer network of the image embedding layer for a second feature fusion learning to obtain a second image feature matrix; using the phrase embedding layer to extract features of a pre-constructed label phrase including multiple abnormal land use labels to obtain a phrase feature matrix; based on the second image feature matrix and the phrase feature matrix, aligning the abnormal land use labels in the pre-constructed label phrase with the current image to be detected through the joint processing layer to determine the detection result of the current image to be detected, thereby solving the problems in the related art that the detection model does not understand the deep meaning of the specific recognition target itself, cannot expand the behavior into the illegal land use change model, and needs to use new pictures or video samples to re-label the expensive data set and retrain, thereby improving the flexible scalability of the detection model and deeply detecting land use to improve detection accuracy.

[0080] Optionally, the land use detection device further includes an image block matrix acquisition module configured to, before inputting the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer for a feature fusion learning:

[0081] Slice the current image to be detected to obtain multiple slice images;

[0082] The image block matrix is ​​obtained according to the multiple tile images.

[0083] Optionally, the encoding layer is a first preset number of repeated structures; each encoding layer includes a first unit layer and a second unit layer; the structure of the first unit layer is sequentially a first standard normalization layer, a multi-head attention layer, and a second standard normalization layer; the structure of the second unit layer is sequentially a third standard normalization layer, a feedforward layer, and a fourth standard normalization layer;

[0084] Accordingly, the first image feature matrix acquisition module 220 can be set to:

[0085] Taking the image block matrix and the preset detection code sequence as a first connection item, inputting the first connection item into a first coding layer, and sequentially processing the first standard normalization layer, the multi-head attention layer, and the second standard normalization layer of the first unit layer in the first coding layer to obtain a second connection item;

[0086] performing summation processing on the first connection item and the second connection item to obtain a first processing result;

[0087] Processing the first processing result sequentially through the third standard specification layer, the feedforward layer, and the fourth standard specification layer of the second unit layer to obtain a third connection item;

[0088] performing summation processing on the first processing result and the third connection item to obtain a second processing result;

[0089] The second processing result is input into the next coding layer, and the operations of processing the first standard specification layer, the multi-head attention layer and the second standard specification layer of the first unit layer in the first coding layer are executed in sequence until all coding layers are processed to obtain the first image feature matrix.

[0090] Optionally, the second image feature matrix acquisition module 230 may be configured as follows:

[0091] Multiplying the transformation tensor of the linear transformation layer in the detection layer network with the first image feature matrix and the preset detection coding sequence to obtain a transformed image feature matrix;

[0092] The transformed image feature matrix is ​​added to the preset detection code sequence to obtain the second image feature matrix.

[0093] Optionally, the phrase embedding layer is a second preset number of repeated structures, each phrase embedding layer includes a third unit layer and a fourth unit layer; the structure of the third unit layer is the same as that of the first unit layer; the structure of the fourth unit layer is the same as that of the second unit layer;

[0094] Accordingly, the phrase feature matrix acquisition module 240 can be set to:

[0095] The phrase matrix corresponding to the pre-constructed label phrase is used as a fourth connection item, and the fourth connection item is input into the first phrase embedding layer, and is sequentially processed by the first standard normalization layer, the multi-head attention layer, and the second standard normalization layer of the third unit layer in the first phrase embedding layer to obtain a fifth connection item;

[0096] performing summation processing on the fourth connection item and the fifth connection item to obtain a third processing result;

[0097] Processing the third processing result sequentially through the third standard specification layer, the feedforward layer, and the fourth standard specification layer of the fourth unit layer to obtain a sixth connection item;

[0098] performing a summation process on the third processing result and the sixth connection item to obtain a fourth processing result;

[0099] The fourth processing result is input into the next phrase embedding layer, and the operations of processing by the first standard specification layer, the multi-head attention layer and the second standard specification layer of the third unit layer in the first phrase embedding layer are executed in sequence until all phrase embedding layers are processed to obtain the phrase feature matrix.

[0100] Optionally, the detection result determination module 250 may be configured to:

[0101] performing a transposition process on the second image feature matrix through the joint processing layer to obtain a transposed second image feature matrix;

[0102] multiplying the transposed second image feature matrix and the phrase feature matrix to align the abnormal land use label in the pre-constructed label phrase with the current image to be detected to obtain an alignment score;

[0103] A detection result for the current image to be detected is determined according to the alignment score.

[0104] Optionally, the land use detection device further includes a warning information display module, which is configured to, after outputting the detection result of the current image to be detected:

[0105] Obtaining the abnormal use type of the target land from the detection results;

[0106] Based on the type of abnormal use of the target land, early warning information is generated and displayed to monitoring personnel for timely management.

[0107] The land use detection device provided in the embodiments of the present application can execute the land use detection method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0108] The acquisition, storage, use, and processing of data in the technical solution of this application comply with relevant laws and regulations.

[0109] Example 3

[0110] FIG3 shows a block diagram of an electronic device 300 that can be used to implement an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples.

[0111] As shown in Figure 3, the electronic device 300 includes at least one processor 301 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 301. The memory stores a computer program that can be executed by the at least one processor. The processor 301 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 308 into the random access memory (RAM) 13. In RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The processor 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0112] Multiple components in the electronic device 300 are connected to the I / O interface 305, including an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a magnetic disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0113] Processor 301 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 301 include a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. Processor 301 executes the various methods and processes described above, such as the land use detection method.

[0114] In some embodiments, the land use detection method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by processor 301, one or more steps of the land use detection method described above can be performed. Alternatively, in other embodiments, processor 301 can be configured to execute the land use detection method in any other appropriate manner (e.g., via firmware).

[0115] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0119] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0120] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0121] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of this application can be achieved.

Claims

1. A method for detecting land use, comprising: Acquire a current image to be detected of the land area to be detected, and input the current image to be detected into a pre-trained detection network; wherein the pre-trained detection network includes an image embedding layer, a phrase embedding layer and a joint processing layer; the image embedding layer includes a coding layer and a detection layer network; Inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer to perform a feature fusion learning to obtain a first image feature matrix; Inputting the first image feature matrix and the preset detection coding sequence into the detection layer network for secondary feature fusion learning to obtain a second image feature matrix; The phrase embedding layer is used to extract features of the pre-constructed label phrases to obtain a phrase feature matrix; wherein the pre-constructed label phrases include a plurality of abnormal land use labels; According to the second image feature matrix and the phrase feature matrix, the abnormal land use label in the pre-constructed label phrase is aligned with the current image to be detected through the joint processing layer to determine the detection result of the current image to be detected.

2. According to the method of claim 1, before inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer for a feature fusion learning, the method further comprises: Performing slice processing on the current image to be detected to obtain a plurality of slice images; The image block matrix is ​​obtained according to the multiple tile images.

3. The method according to claim 1, wherein: The encoding layer is a first preset number of repeated structures; each encoding layer includes a first unit layer and a second unit layer; the structure of the first unit layer is a first standard specification layer, a multi-head attention layer, and a second standard specification layer in sequence; the structure of the second unit layer is a third standard specification layer, a feedforward layer, and a fourth standard specification layer in sequence; The step of inputting the image block matrix of the current image to be detected and the preset detection coding sequence into the coding layer for a feature fusion learning to obtain a first image feature matrix includes: The image block matrix and the preset detection coding sequence are used as a first connection item, and the first connection item is input into a first coding layer, and is processed in sequence by the first standard specification layer, the multi-head attention layer, and the second standard specification layer of the first unit layer in the first coding layer to obtain a second connection item; Performing a summation process on the first connection item and the second connection item to obtain a first processing result; Processing the first processing result sequentially through the third standard specification layer, the feedforward layer and the fourth standard specification layer of the second unit layer to obtain a third connection item; Performing a summation process on the first processing result and the third connection item to obtain a second processing result; The second processing result is input into the next coding layer, and the processing is performed sequentially through the first coding layer. The first standard specification layer, the multi-head attention layer and the second standard specification layer of a unit layer are processed until all coding layers are processed to obtain the first image feature matrix.

4. The method according to claim 1, wherein: The step of inputting the first image feature matrix and the preset detection coding sequence into the detection layer network for secondary feature fusion learning to obtain a second image feature matrix includes: Using the transformation tensor of the linear transformation layer in the detection layer network, multiplying it with the first image feature matrix and the preset detection coding sequence to obtain a transformed image feature matrix; The transformed image feature matrix is ​​added to the preset detection code sequence to obtain the second image feature matrix.

5. The method according to claim 3, wherein: The phrase embedding layer is a second preset number of repeated structures, each phrase embedding layer includes a third unit layer and a fourth unit layer; the structure of the third unit layer is the same as that of the first unit layer; The structure of the fourth unit layer is the same as that of the second unit layer; The phrase embedding layer is used to extract features from the pre-constructed tag phrases to obtain a phrase feature matrix, including: The phrase matrix corresponding to the pre-constructed label phrase is used as a fourth connection item, and the fourth connection item is input into the first phrase embedding layer, and is processed by the first standard specification layer, the multi-head attention layer, and the second standard specification layer of the third unit layer in the first phrase embedding layer in sequence to obtain a fifth connection item; performing a summation process on the fourth connection item and the fifth connection item to obtain a third processing result; Processing the third processing result sequentially through the third standard specification layer, the feedforward layer and the fourth standard specification layer of the fourth unit layer to obtain a sixth connection item; performing a summation process on the third processing result and the sixth connection item to obtain a fourth processing result; The fourth processing result is input into the next phrase embedding layer, and the operations of processing by the first standard specification layer, the multi-head attention layer and the second standard specification layer of the third unit layer in the first phrase embedding layer are executed in sequence until all phrase embedding layers are processed to obtain the phrase feature matrix.

6. The method according to claim 1, wherein: The step of aligning the abnormal land use label in the pre-constructed label phrase with the current image to be detected through the joint processing layer according to the second image feature matrix and the phrase feature matrix to determine the detection result of the current image to be detected includes: Through the joint processing layer, the second image feature matrix is ​​transposed to obtain a transposed second image feature matrix; The transposed second image feature matrix and the phrase feature matrix are multiplied to obtain the pre-structured Aligning the abnormal land use label in the label phrase with the current image to be detected to obtain an alignment score; A detection result of the current image to be detected is determined according to the alignment score.

7. The method according to claim 1, after outputting the detection result of the current image to be detected, the method further comprises: Obtaining the abnormal use type of the target land from the detection results; Based on the type of abnormal use of the target land, early warning information is generated and displayed to monitoring personnel for timely management.

8. A device for detecting land use, comprising: A detection image input module is configured to obtain a current image to be detected of a land area to be detected, and input the current image to be detected into a pre-trained detection network; wherein the pre-trained detection network includes an image embedding layer, a phrase embedding layer and a joint processing layer; the image embedding layer includes a coding layer and a detection layer network; A first image feature matrix acquisition module is configured to input the image block matrix of the current image to be detected and a preset detection coding sequence into the coding layer to perform a feature fusion learning to obtain a first image feature matrix; A second image feature matrix acquisition module, configured to input the first image feature matrix and the preset detection coding sequence into the detection layer network for secondary feature fusion learning to acquire a second image feature matrix; A phrase feature matrix acquisition module is configured to extract features of pre-constructed label phrases using the phrase embedding layer to obtain a phrase feature matrix; wherein the pre-constructed label phrases include a plurality of abnormal land use labels; The detection result determination module is configured to align the abnormal land use label in the pre-constructed label phrase with the current image to be detected through the joint processing layer according to the second image feature matrix and the phrase feature matrix, so as to determine the detection result of the current image to be detected.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the land use detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the method for detecting land use according to any one of claims 1 to 7 when executed.

Images