An image processing method and server applied to garden management

Abstract

The image processing method and the garden management server provided by the embodiments of the present application are applied to garden management, and are used for monitoring images of garden plants including multiple plant object clusters. The abnormal type in the current plant object cluster is analyzed by combining the cluster knowledge representation of the current plant object cluster, the target knowledge representation of the comparison target, and the second cluster description vector of the current plant object cluster. The second cluster description vector covers the information of the current plant object cluster and the information of the adjacent remaining plant object clusters. Therefore, the information of the adjacent other plant objects can be jointly analyzed when the abnormal type is analyzed, the feature information is utilized as a whole, and the accuracy of the abnormal type analysis can be increased.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image processing method and server for use in garden management. Background Technology

[0002] As an important component of urban residential areas, parks, and municipal greening, garden plants play a vital role in urban ecological construction. With most cities emphasizing the concept of "park city," the maintenance of urban gardens has become increasingly important. To diversify garden landscapes, the coverage area of ​​urban gardens has expanded, and the types of vegetation introduced have become more abundant. This has increased the challenges of garden management and maintenance, especially in identifying anomalies. For example, the categories of anomalies have become more diverse, including different insect pests, plant virus infections, human damage, and environmental degradation. Currently, with advancements in science and technology, drones can be used to capture images of garden plants and then perform image analysis to assist in anomaly management. However, traditional image analysis methods are mostly limited to general machine vision, with relatively weak generalization capabilities, unstable accuracy, and a lack of universality.

[0003] Therefore, a method is needed to accurately analyze garden anomalies. Summary of the Invention

[0004] The purpose of this invention is to provide an image processing method and server for use in garden management, so as to improve the above-mentioned problems.

[0005] The implementation method of this application embodiment is as follows:

[0006] In a first aspect, embodiments of this application provide an image processing method for garden management, applied to a garden management server, the method comprising:

[0007] Acquire cluster knowledge representations of multiple plant object clusters in a garden plant monitoring image, wherein the multiple plant object clusters are arranged according to the spatial distribution of plants, and each plant object cluster contains one or more plant objects;

[0008] Obtain a target knowledge representation of the comparison target, wherein the comparison target includes confirmation indication information of the comparison anomaly type of the target plant species;

[0009] By focusing feature information on the cluster knowledge representations of the multiple plant object clusters through the target knowledge representation, the first cluster description vector of each of the multiple plant object clusters is obtained;

[0010] The first cluster description vector of each of the multiple plant object clusters is integrated with the first cluster description vector of the adjacent plant object clusters to obtain the second cluster description vector of each of the multiple plant object clusters.

[0011] Anomaly type evaluation results for each of the plant object clusters are obtained by using the cluster knowledge representation of each of the plant object clusters, the target knowledge representation, and the second cluster description vector of each of the plant object clusters; the anomaly type evaluation results include the anomaly types in the plant object clusters.

[0012] As one implementation, the step of integrating the first cluster description vector of each of the plurality of plant object clusters with the first cluster description vector of adjacent plant object clusters to obtain the second cluster description vector of each of the plurality of plant object clusters includes:

[0013] For a first plant object cluster, the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue to the first plant object cluster is obtained by using the first cluster description vector of the first plant object cluster and the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue; wherein, the first plant object cluster is any one of the plurality of plant object clusters; the adjacent plant object cluster queue includes the plant object clusters that are adjacent to the first plant object cluster among the plurality of plant object clusters;

[0014] By using the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue to the first plant object cluster, the first cluster description vector of the first plant object cluster is integrated with the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue to obtain the second cluster description vector of the first plant object cluster.

[0015] As one implementation, the step of integrating the first cluster description vector of the first plant object cluster with the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue, based on the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue, to obtain the second cluster description vector of the first plant object cluster, includes:

[0016] The first cluster description vector of the first plant object cluster is combined with the first cluster description vector of the target plant object cluster to obtain the second cluster description vector of the first plant object cluster; wherein, the target plant object cluster is the plant object cluster with the largest focusing coefficient on the first plant object cluster in the adjacent plant object cluster queue.

[0017] As one implementation, obtaining the anomaly type evaluation results for each of the multiple plant object clusters through their respective cluster knowledge representations, the target knowledge representation, and their respective second cluster description vectors includes:

[0018] Plant species inference is performed using the second cluster description vector of each of the multiple plant object clusters to obtain plant species inference results for each of the multiple plant object clusters; the plant species inference results indicate whether the plant object cluster contains a plant object corresponding to the target plant species.

[0019] Anomaly type inference is performed using the plant species inference results of each of the multiple plant object clusters, the cluster knowledge representation of each of the multiple plant object clusters, and the target knowledge representation, to obtain the anomaly type evaluation results of each of the multiple plant object clusters.

[0020] As one implementation, the step of performing anomaly type inference through the plant species inference results of each of the multiple plant object clusters, the cluster knowledge representation of each of the multiple plant object clusters, and the target knowledge representation to obtain the anomaly type evaluation results of each of the multiple plant object clusters includes:

[0021] When the plant species reasoning result of the second plant object cluster represents a plant object in the second plant object cluster that corresponds to the target plant species, anomaly type reasoning is performed through the cluster knowledge representation in the second plant object cluster and the target knowledge representation to obtain the anomaly type evaluation result corresponding to the second plant object cluster; the second plant object cluster is any one of the multiple plant object clusters.

[0022] When the plant species reasoning result of the third plant object cluster indicates that there is no plant object corresponding to the target plant species in the third plant object cluster, the process of performing abnormal type reasoning through the cluster knowledge representation in the third plant object cluster and the target knowledge representation will no longer be executed.

[0023] As one implementation, the anomaly type evaluation result further includes the coverage range of the anomaly types in the plant object cluster within the plant object cluster;

[0024] The step of focusing feature information on the cluster knowledge representations of multiple plant object clusters through the target knowledge representation to obtain the first cluster description vector for each of the multiple plant object clusters includes:

[0025] By using the target knowledge representation and the cluster knowledge representation of each of the multiple plant object clusters, the focusing coefficient of each of the multiple plant object clusters toward the comparison target is obtained;

[0026] The cluster knowledge representation of each of the multiple plant object clusters is processed by the focusing coefficient of each of the multiple plant object clusters toward the comparison target, so as to obtain the first cluster description vector of each of the multiple plant object clusters;

[0027] The step of processing the cluster knowledge representation of each of the multiple plant object clusters by the focusing coefficient of each of the multiple plant object clusters on the comparison target to obtain the first cluster description vector of each of the multiple plant object clusters includes: determining the first cluster description vector of each of the multiple plant object clusters by multiplying the cluster knowledge representation of each of the multiple plant object clusters with the focusing coefficient of each of the multiple plant object clusters on the comparison target.

[0028] As one implementation, the cluster knowledge representation includes a knowledge representation for each plant object in the plant object cluster; before focusing feature information on the cluster knowledge representations of the multiple plant object clusters through the target knowledge representation to obtain the first cluster description vector for each of the multiple plant object clusters, the method further includes:

[0029] The knowledge representation of each plant object in the multiple plant object clusters is integrated with the knowledge representation of the same type of plant objects in the garden plant monitoring image to obtain the integrated cluster description vector of the same type of plant objects for each of the multiple plant object clusters.

[0030] The step of focusing feature information on the cluster knowledge representations of multiple plant object clusters through the target knowledge representation to obtain the first cluster description vector for each of the multiple plant object clusters includes:

[0031] By using the target knowledge representation, feature information is focused on the integrated cluster description vectors of the same type of plant objects in each of the multiple plant object clusters to obtain the first cluster description vector of each of the multiple plant object clusters.

[0032] The step of integrating the knowledge representation of each plant object in the plurality of plant object clusters with the knowledge representation of similar plant objects in the garden plant monitoring image for each plant object to obtain the integrated cluster description vector of similar plant objects for each of the plurality of plant object clusters includes:

[0033] By using the object encoding vector of the target plant object and the knowledge representation of the target plant object in the garden plant monitoring image, the focusing coefficient of the target plant object on the target plant object in the garden plant monitoring image is obtained; wherein, the target plant object is any plant object in the garden plant monitoring image;

[0034] By using the focusing coefficient of the target plant object on the same type of plant objects in the garden plant monitoring image, the knowledge representation of the target plant object is integrated with the knowledge representation of the target plant object on the same type of plant objects in the garden plant monitoring image to obtain the lower-level integrated knowledge representation of the target plant object; wherein, the lower-level integrated knowledge representation of the target plant object is the knowledge representation corresponding to the target plant object in the integrated cluster description vector of the same type of plant object of the plant object cluster corresponding to the target plant object.

[0035] In one implementation, the method is executed through a pre-set garden image anomaly analysis network, which is trained through the following steps:

[0036] The knowledge mining module in the garden image anomaly analysis network performs knowledge mining on multiple sample plant object clusters in the sample garden plant monitoring images to obtain the sample cluster knowledge representation of each of the multiple sample plant object clusters; wherein, the multiple sample plant object clusters are arranged according to the spatial distribution of plants, and each sample plant object cluster contains one or more plant objects;

[0037] The knowledge mining module performs knowledge mining on the comparison target to obtain the target knowledge representation of the comparison target; wherein, the comparison target includes confirmation indication information of the comparison anomaly type within the target plant species;

[0038] Through the contrast target focusing module in the garden image anomaly analysis network, and through the target knowledge representation, feature information focusing is performed on the example cluster knowledge representation of each of the multiple example plant object clusters to obtain the first example cluster description vector of each of the multiple example plant object clusters.

[0039] Through the first description vector integration module in the garden image anomaly analysis network, the first example cluster description vector of each of the multiple example plant object clusters is integrated with the first example cluster description vector of the adjacent example plant object clusters to obtain the second example cluster description vector of each of the multiple example plant object clusters.

[0040] The classification module in the garden image anomaly analysis network obtains anomaly type evaluation results for each of the multiple example plant object clusters through the example cluster knowledge representation, the target knowledge representation, and the second example cluster description vector of each of the multiple example plant object clusters; wherein, the anomaly type evaluation results include the anomaly types inferred from the example plant object clusters.

[0041] The network configuration variables of the garden image anomaly analysis network are adjusted based on the anomaly type evaluation results and anomaly type annotation information of each of the example plant object clusters; the anomaly type annotation information characterizes the true anomaly types in the example plant object clusters.

[0042] In one implementation, the classification module includes a plant species classification module and an anomaly type classification module, and the network configuration variables of the plant species classification module and the anomaly type classification module are the same.

[0043] The step involves using the classification module in the garden image anomaly analysis network to obtain anomaly type evaluation results for each of the multiple example plant object clusters through their respective example cluster knowledge representations, target knowledge representations, and second example cluster description vectors. This includes:

[0044] The plant species classification module performs plant species inference using the second example cluster description vector of each of the multiple example plant object clusters to obtain plant species inference results for each of the multiple example plant object clusters; the plant species inference results characterize whether the example plant object cluster contains a plant object corresponding to the target plant species.

[0045] The anomaly type classification module performs anomaly type inference based on the plant species inference results of each of the multiple example plant object clusters, the example cluster knowledge representation of each of the multiple example plant object clusters, and the target knowledge representation, and obtains the anomaly type evaluation results of each of the multiple example plant object clusters.

[0046] Before the step of focusing feature information on the example cluster knowledge representations of each of the multiple example plant object clusters through the contrast target focusing module in the garden image anomaly analysis network to obtain the first example cluster description vector for each of the multiple example plant object clusters, the method further includes:

[0047] Through the second description vector integration module in the garden image anomaly analysis network, the knowledge representation of each example plant object in the multiple example plant object clusters is integrated with the knowledge representation of the same type of plant object in the example garden plant monitoring image, so as to obtain the example same type of plant object integrated cluster description vector of each of the multiple example plant object clusters.

[0048] The step involves using the contrast target focusing module in the garden image anomaly analysis network to focus feature information on the example cluster knowledge representations of multiple example plant object clusters through the target knowledge representation, thereby obtaining the first example cluster description vector for each of the multiple example plant object clusters, including:

[0049] Through the comparison target focusing module and the target knowledge representation, feature information is focused on the integrated cluster description vectors of example plant objects of the same kind for each of the multiple example plant object clusters, so as to obtain the first example cluster description vector of each of the multiple example plant object clusters.

[0050] Secondly, this application also provides a garden management server, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program, it implements the aforementioned method.

[0051] This application includes at least the following beneficial effects:

[0052] The image processing method and garden management server provided in this application, for garden plant monitoring images including multiple plant object clusters, analyze the anomaly types in the current plant object cluster by combining the cluster knowledge representation of the current plant object cluster, the target knowledge representation of the comparison target, and the second cluster description vector of the current plant object cluster. The second cluster description vector covers the information of the current plant object cluster and the information of the other adjacent plant object clusters. Therefore, during anomaly type analysis, the information of other adjacent plant objects can be jointly analyzed to complete the utilization of feature information as a whole, which can increase the accuracy of anomaly type analysis.

[0053] Other features will be described in part in the following description. These features will be partially discovered by those skilled in the art upon examination of the following content and figures, or may be learned through production or application. The features of the present application can be implemented and obtained by practice or use of various aspects of the methods, tools, and combinations listed in the detailed examples described below. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein reference numerals in the various views of the drawings represent similar mechanisms.

[0056] Figure 1 These are schematic diagrams illustrating scenarios according to some embodiments of this application.

[0057] Figure 2 This is a schematic diagram illustrating the hardware and software composition of a garden management server according to some embodiments of this application.

[0058] Figure 3 This is a flowchart illustrating an image processing method applied to garden management according to some embodiments of this application.

[0059] Figure 4 This is a schematic diagram of the architecture of the image processing apparatus provided in the embodiments of this application. Detailed Implementation

[0060] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0061] In the detailed description below, numerous specific details are illustrated with examples to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that this application can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level without detail to avoid unnecessarily obscuring aspects of this application.

[0062] These and other characteristics, the functions disclosed in the current application, the methods of execution, the functions of related elements in the structure, the combination of components, and the economic efficiency of production, may become more apparent in consideration of the following description with reference to the accompanying drawings, all of which form part of this application. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this application. It should be understood that these drawings are not drawn to scale.

[0063] This application uses flowcharts to illustrate the execution process performed by a system according to embodiments of this application. It should be clearly understood that the execution processes in the flowcharts may not be executed sequentially. Instead, these execution processes may be executed in reverse order or simultaneously. Additionally, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0064] Figure 1 This is a schematic diagram of a scenario according to some embodiments of this application, including a garden management server 100 and a drone 300 that are connected to each other via a network 200. The drone 300 is used to take images of garden plants for monitoring and send them to the garden management server 100.

[0065] In some embodiments, please refer to Figure 2 This is a schematic diagram of the architecture of a garden management server 100, which includes an image processing device 110, a memory 120, a processor 130, and a communication unit 140. The memory 120, processor 130, and communication unit 140 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The image processing device 110 includes at least one software function module that can be stored in the memory 120 or embedded in the operating system (OS) of the garden management server 100 in the form of software or firmware. The processor 130 is used to execute executable modules stored in the memory 120, such as the software function modules and computer programs included in the image processing device 110.

[0066] The memory 120 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 120 stores programs, which are executed by the processor 130 upon receiving execution instructions. The communication unit 140 establishes a communication connection between the garden management server 100 and the drone 300 via a network, and transmits and receives data over the network.

[0067] The processor may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0068] Understandable. Figure 2 The structure shown is for illustrative purposes only; the garden management server 100 may also include components such as... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown. Figure 2 The components shown can be implemented using hardware, software, or a combination thereof.

[0069] Figure 3 This is a flowchart illustrating an image processing method applied to garden management according to some embodiments of this application. The method is applied to... Figure 1 The garden management server 100 in the system may specifically include the following steps S110 to S140. Based on the following steps, some optional embodiments will be described. These embodiments should be understood as examples and should not be construed as essential technical features for implementing this solution.

[0070] S110: Obtain the cluster knowledge representation of each of the multiple plant object clusters in the garden plant monitoring image, and obtain the target knowledge representation of the comparison target.

[0071] Garden plant monitoring images, such as images of urban gardens captured by unmanned aerial vehicles (UAVs), can contain multiple plant objects, such as multiple plants of different or the same species. In this embodiment, the garden plant monitoring image includes multiple plant object clusters arranged according to the spatial distribution of plants, with each cluster containing one or more plant objects. When a plant object cluster is acquired, the garden management server combines and stitches together one or more plant objects from the cluster. Knowledge representation mining is then performed on the combined and stitched image to obtain a cluster knowledge representation of the plant object cluster. This cluster knowledge representation can represent the semantic feature information of the plant object cluster, for example, it can be represented by vectors. In this embodiment, the comparison target includes confirmation indication information of the comparison anomaly type of the target plant species. Specifically, the comparison target includes an image of the comparison anomaly type and the confirmation indication information of the comparison anomaly type. The garden management server can perform knowledge representation mining on the comparison target to obtain the target knowledge representation of the comparison target. The confirmation indication information of the comparison anomaly type represents the confirmation status of the comparison anomaly type. For example, the contrast anomaly type can be leaf spot disease. The contrast anomaly type can include three contrast targets, such as leaf spot disease image (confirmed), leaf spot disease image (excluded), and leaf spot disease image (questionable). Among them, leaf spot disease image (confirmed) means that the image is leaf spot disease, leaf spot disease image (excluded) means that the image is not leaf spot disease, and leaf spot disease image (questionable) means that it is uncertain whether the image is leaf spot disease.

[0072] S120: Focus feature information on the cluster knowledge representations of multiple plant object clusters through target knowledge representation to obtain the first cluster description vector of each of the multiple plant object clusters.

[0073] When the target knowledge representation corresponding to the comparison target is obtained, the garden management server adopts a focusing strategy (such as Attention Mechanism) to focus the feature information of the cluster knowledge representation of each plant object cluster based on the target knowledge representation corresponding to the comparison target, so as to update the cluster knowledge representation of each plant object cluster and obtain the first cluster description vector of each plant object cluster.

[0074] S130: Integrate the first cluster description vector of each of the multiple plant object clusters with the first cluster description vector of the adjacent plant object clusters to obtain the second cluster description vector of each of the multiple plant object clusters.

[0075] When the garden management server obtains the first cluster description vectors corresponding to multiple plant object clusters, it integrates the first cluster description vectors corresponding to each plant object cluster with the first cluster description vectors of the other adjacent plant object clusters through a dynamic focusing strategy to obtain the second cluster description vectors of each of the two plant object clusters.

[0076] As one implementation method, the dynamic focusing strategy involves using the first cluster description vector corresponding to the current plant cluster as the search object, and integrating the first cluster description vector of the current plant cluster with the first cluster description vectors of the other adjacent plant clusters to achieve attention fusion. As the current plant cluster progresses, the number of adjacent plant clusters gradually decreases. In S120, the impact of the confirmation status of the anomaly type on the accuracy of anomaly type analysis in the plant cluster is taken into account. However, for a specific anomaly type, the situation in the current garden plant monitoring image may change. For example, in one plant cluster, the confirmation indication information for a possible anomaly type might be the first indication; when transitioning to other plant clusters, the indication for searching for possible anomaly types might become the second indication. For instance, in one plant cluster, if it is initially determined to be a piercing-sucking insect pest, then the confirmation indication information for this anomaly type is confirmed. However, in adjacent plant clusters, further analysis reveals that the confirmation indication information for this anomaly type is not confirmed, but excluded. In other words, for a plant object cluster, the information of the plant object cluster alone cannot determine whether its semantics correspond to the current contrast anomaly type; further analysis by combining information from adjacent plant objects is required. In this embodiment, for each plant object cluster, the first cluster description vector of the plant object cluster is integrated with the first cluster description vector of adjacent plant object clusters in the garden plant monitoring image to obtain a second cluster description vector. This second cluster description vector includes not only information about the current plant object cluster but also information about adjacent plant object clusters.

[0077] S140: Obtain the anomaly type evaluation results for each of the multiple plant object clusters by using their respective cluster knowledge representations, target knowledge representations, and second cluster description vectors. The anomaly type evaluation results include the anomaly types in the plant object clusters.

[0078] In this embodiment, based on the cluster knowledge representation and target knowledge representation of plant object clusters, the garden management server combines the second cluster description vector of the plant object cluster to infer the anomaly type in the current plant object cluster. There may be multiple comparative targets among the target plant species, such as insect pests, viruses, and natural disasters. For each comparative target, the garden management server performs anomaly type analysis on the garden plant monitoring images according to the same approach, completing the analysis of the anomaly types associated with the target plant species in the garden plant monitoring images.

[0079] Based on this, the method provided in this application provides that, for a garden plant monitoring image including multiple plant object clusters, the method analyzes the anomaly types in the current plant object cluster by combining the cluster knowledge representation of the current plant object cluster, the target knowledge representation of the comparison target, and the second cluster description vector of the current plant object cluster. The second cluster description vector covers the information of the current plant object cluster and the information of the other adjacent plant object clusters. Therefore, when analyzing anomaly types, the information of other adjacent plant objects can be jointly analyzed to make full use of feature information as a whole, which can increase the accuracy of anomaly type analysis.

[0080] As one implementation method, the method provided in this application embodiment can be executed by a pre-calibrated garden image anomaly analysis network. Specifically, the method in this application embodiment may include training the garden image anomaly analysis network and analyzing and processing garden plant monitoring images.

[0081] During the adjustment of the garden image anomaly analysis network, debugging samples (containing different example garden plant monitoring images, anomaly type annotation information corresponding to each example garden plant monitoring image, and comparison targets) are obtained from a local database or network to fine-tune the network, resulting in a calibrated garden image anomaly analysis network. In the garden plant monitoring image analysis and processing, this network is used to analyze and process the input garden plant monitoring images, obtaining anomaly type evaluation results for each plant cluster in the monitoring images. The training and application of the network can be performed on the same device, such as a garden management server.

[0082] The image processing method for garden management provided in this application embodiment includes the following steps in the training process of the garden image anomaly analysis network:

[0083] S101: The knowledge mining module in the garden image anomaly analysis network is used to mine knowledge of multiple sample plant object clusters in the sample garden plant monitoring image to obtain the sample cluster knowledge representation of each sample plant object cluster; wherein, the multiple sample plant object clusters are arranged according to the spatial distribution of plants, and each sample plant object cluster contains one or more plant objects.

[0084] S102: The knowledge mining module is used to mine the knowledge of the comparison target to obtain the target knowledge representation of the comparison target; wherein, the comparison target includes confirmation indication information of the comparison anomaly type within the target plant species.

[0085] S103: Through the contrast target focusing module in the garden image anomaly analysis network, feature information is focused on the example cluster knowledge representation of each of the multiple example plant object clusters through target knowledge representation, and the first example cluster description vector of each of the multiple example plant object clusters is obtained.

[0086] S104: Through the first description vector integration module in the garden image anomaly analysis network, the first example cluster description vectors of multiple example plant object clusters are integrated with the first example cluster description vectors of adjacent example plant object clusters to obtain the second example cluster description vectors of multiple example plant object clusters.

[0087] S105: Through the classification module in the garden image anomaly analysis network, the anomaly type evaluation results of multiple example plant object clusters are obtained through their respective example cluster knowledge representations, target knowledge representations, and second example cluster description vectors; wherein, the anomaly type evaluation results include the anomaly types inferred from the example plant object clusters.

[0088] The garden image anomaly analysis network can include a knowledge mining module, a comparison target focusing module, a first descriptive vector integration module, and a classification module. The garden management server uses each module of the garden image anomaly analysis network to analyze and process the comparison target and the target image (i.e., the garden plant monitoring image or example garden plant monitoring image) according to the methods described above, thereby obtaining the anomaly type of the target image.

[0089] S106: The network configuration variables of the garden image anomaly analysis network are adjusted by using the anomaly type evaluation results and anomaly type annotation information of multiple example plant object clusters; wherein, the anomaly type annotation information is used to characterize the true anomaly type in the example plant object clusters.

[0090] During training and debugging, the garden image anomaly analysis network infers the anomaly types in the example plant object clusters and obtains the anomaly type evaluation results. Then, the garden management server adjusts the network configuration variables of the garden image anomaly analysis network by combining the anomaly type annotation information and the anomaly type evaluation results, thereby optimizing the garden image anomaly analysis network. For example, iterates until preset conditions are met, such as network convergence.

[0091] Based on this, during the calibration process of the garden image anomaly analysis network, example garden plant monitoring images, anomaly type annotation information corresponding to the example garden plant monitoring images, and comparison targets are used to adjust the garden image anomaly analysis network. When the network is applied, for garden plant monitoring images containing multiple plant object clusters, the garden image anomaly analysis network combines the cluster knowledge representation of the current plant object cluster, the target knowledge representation of the comparison target, and the second cluster description vector of the current plant object cluster to analyze the anomaly type in the current plant object cluster. Because the second cluster description vector covers the information of the current plant object cluster and the information of the other adjacent plant object clusters, the information of other adjacent plant objects can be jointly analyzed during anomaly type analysis, so as to complete the utilization of feature information as a whole and increase the accuracy of anomaly type analysis.

[0092] As another embodiment, this application provides an image processing method for garden management, which may include the following steps:

[0093] S100: Obtain the cluster knowledge representation of each of the multiple plant object clusters in the garden plant monitoring image; obtain the target knowledge representation of the comparison target.

[0094] The garden management server uses the knowledge mining module in the garden image anomaly analysis network to perform knowledge mining on each plant cluster and the comparison target in the garden plant monitoring images, obtaining the cluster knowledge representation of each plant cluster and the target knowledge representation of the comparison target. As one implementation method, the knowledge mining module can be a feature mining network such as CNN, DNN, RNN, or LSTM; the specific implementation is not limited, for example, it can be a convolutional neural network.

[0095] S200: Integrate the knowledge representation of each plant object in multiple plant object clusters with the knowledge representation of the same type of plant objects in the garden plant monitoring image to obtain the integrated cluster description vector of the same type of plant objects for each of the multiple plant object clusters.

[0096] After obtaining the knowledge representation of each plant object in the garden plant monitoring image, the garden management server can acquire the focus coefficient (i.e., the attention invested) of similar plant objects in the garden plant monitoring image to the target plant object through the knowledge representation of any target plant object in the garden plant monitoring image. This integrates the knowledge representation of the target plant object with the knowledge representation of similar plant objects in the entire garden plant monitoring image, so as to update the knowledge representation of the target plant object and obtain the lower-level integrated knowledge representation of the target plant object. Furthermore, it obtains the integrated cluster description vector of similar plant objects in the plant object cluster. Based on the same idea, the garden management server updates the knowledge representation of target plant objects that contain similar plant objects in the entire garden plant monitoring image. Of course, if there are no similar plant objects in the garden plant monitoring image, the knowledge representation of the target plant object will not be updated.

[0097] As one implementation method, the garden management server can obtain the focusing coefficient of the target plant object on the target plant object in the garden plant monitoring image by using the object encoding vector of the target plant object and the knowledge representation of similar plant objects in the garden plant monitoring image. The target plant object is any plant object in the garden plant monitoring image. The garden management server integrates the knowledge representation of the target plant object with the knowledge representation of similar plant objects in the garden plant monitoring image using the focusing coefficient of the target plant object on the target plant object, obtaining a lower-level integrated knowledge representation of the target plant object, and then obtaining the integrated cluster description vector of similar plant objects for the plant object cluster. The lower-level integrated knowledge representation of the target plant object is the knowledge representation corresponding to the target plant object in the integrated cluster description vector of similar plant objects for the plant object cluster corresponding to the target plant object. Similar plant objects refer to plant objects of the same class as the target plant object in the garden plant monitoring image, excluding the target plant object. It can be understood that similar plant objects can be in the same plant object cluster and / or different plant object clusters as the target plant object.

[0098] As one implementation method, the garden management server integrates the knowledge representation of each plant object in multiple plant object clusters with the knowledge representation of similar plant objects in the garden plant monitoring image through the second description vector integration module in the garden image anomaly analysis network.

[0099] As one implementation method, the object encoding vector can be adjusted during network calibration. For example, a garden image anomaly analysis network may include a plant object encoding module, which is used to transform each plant object in the garden plant monitoring image into an object encoding vector, so as to be loaded into the knowledge mining module in the garden image anomaly analysis network. When the garden image anomaly analysis network is calibrated, the configuration variables of the plant object encoding module are also adjusted accordingly.

[0100] As one implementation method, for a target plant object Aa in a garden plant monitoring image, the object encoding vector Va is used as the focusing key for searching similar plant objects for the target plant object Aa. The focusing coefficient G between the target plant object Aa and the knowledge representation Kb of the b-th similar plant object can be obtained by the following formula:

[0101]

[0102] Where s is the dimension of the object encoding vector of Aa, Aa is the object encoding vector of the target plant object Aa, and Kb is the knowledge representation of the b-th plant object of the same type.

[0103] When calculating the focusing coefficient of a target plant object relative to similar plant objects in a garden plant monitoring image, the knowledge representation of the target plant object is integrated with the knowledge representations of similar plant objects in the same image to obtain a lower-level integrated knowledge representation of the target plant object. This lower-level integrated knowledge representation can be calculated by assigning different weights to the knowledge representations of the target plant object and its similar plant objects, and then summing them using a weighted average.

[0104] S300: By focusing on the feature information of the integrated cluster description vectors of the same type of plant objects in multiple plant object clusters through target knowledge representation, the first cluster description vector of each of the multiple plant object clusters is obtained.

[0105] As one implementation method, the garden management server can obtain the focus coefficients of each plant object cluster to the comparison target through the target knowledge representation and the cluster knowledge representations of each plant object cluster, and process the cluster knowledge representations of each plant object cluster through the focus coefficients of each plant object cluster to the comparison target to obtain the first cluster description vector of each plant object cluster.

[0106] As one implementation method, the garden management server determines the first cluster description vector of each of the multiple plant object clusters by multiplying the integrated cluster description vector of the same type of plant objects of each plant object cluster with the focusing coefficient of each of the multiple plant object clusters to the comparison target.

[0107] The focus coefficient of a plant object cluster to a comparison target represents the correlation between the plant object cluster and the comparison target; a higher focus coefficient indicates a higher correlation. The garden management server can obtain the integrated cluster description vector of similar plant objects within a plant object cluster. Using the target knowledge representation Kg of the comparison target as an embedding search, it obtains the focus coefficient between the target knowledge representation of the comparison target and the integrated cluster description vector of similar plant objects within a plant object cluster, thus obtaining the influence factor Wg. The influence factor Wg is the adjustment weight, which can be obtained using the following formula:

[0108] Wg = Kg × Ta

[0109] Where Kg is the target knowledge representation of the comparison target, and Ta is the integrated cluster description vector of the same type of plant objects (the a-th plant object cluster).

[0110] When obtaining the impact factor Wg, a standardization operation is performed on it, for example, based on a normalized exponential function to obtain the impact factor Sa. The impact factor Sa is then weighted and summed with the integrated cluster description vectors of similar plant objects for each plant object cluster. By comparing the focus coefficients between the target and the integrated cluster description vectors of similar plant objects in the plant object cluster, the first cluster description vector Ca corresponding to the plant object cluster is obtained, where:

[0111] Ca=∑SaTa

[0112] S400: The first cluster description vector of each of the multiple plant object clusters is integrated with the first cluster description vector of the adjacent plant object clusters to obtain the second cluster description vector of each of the multiple plant object clusters.

[0113] This application embodiment integrates the first cluster description vector of the current plant object cluster and the first cluster description vectors of the remaining adjacent plant object clusters using a focusing strategy to obtain the second cluster description vector of the current plant object cluster. As one implementation, for the first plant object cluster, the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue is obtained from the first cluster description vector of the first plant object cluster and the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue. Here, the first plant object cluster is any one of multiple plant object clusters, and the adjacent plant object cluster queue contains multiple plant object clusters adjacent to the first plant object cluster.

[0114] Next, the garden management server integrates the first cluster description vector of the first plant object cluster with the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue using the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue, to obtain the second cluster description vector of the first plant object cluster.

[0115] In one implementation, the garden management server combines the first cluster description vector of the first plant object cluster with the first cluster description vector of the target plant object cluster adjacent to the first plant object cluster to obtain the second cluster description vector of the first plant object cluster. The target plant object cluster is the plant object cluster with the highest focusing coefficient towards the first plant object cluster in the plant object cluster queue following the first plant object cluster.

[0116] S500: Obtain the anomaly type evaluation results for each of the multiple plant object clusters by using their respective cluster knowledge representations, target knowledge representations, and second cluster description vectors; the anomaly type evaluation results include the anomaly types in the plant object clusters.

[0117] In this system, the garden management server can process the cluster knowledge representation, target knowledge representation, and second cluster description vector of each plant object cluster using the classification module (e.g., a fully connected layer) in the garden image anomaly analysis network to obtain the anomaly type evaluation result for each plant object cluster. As one implementation, the plant species inference result for each plant object cluster can be obtained from the second cluster description vector. Then, the anomaly type evaluation result for each plant object cluster can be obtained from the plant species inference result, the cluster knowledge representation, and the target knowledge representation. The plant species inference result indicates whether the plant object cluster contains a plant object corresponding to the target plant species. As another implementation, the plant species inference result can be used to indicate whether the plant object cluster contains a plant object corresponding to the comparison target.

[0118] As one implementation method, when the plant species reasoning result of the second plant object cluster represents a plant object in the second plant object cluster that corresponds to the target plant species, anomaly type reasoning can be performed through the cluster knowledge representation in the second plant object cluster and the target knowledge representation to obtain the anomaly type evaluation result corresponding to the second plant object cluster. The second plant object cluster is any one of multiple plant object clusters.

[0119] Specifically, the classification module may include a plant species classification module and an anomaly type classification module. When obtaining the second cluster description vector of the current plant object cluster, the garden management server inputs it into the plant species classification module, and the plant species classification module processes and outputs the plant species inference results.

[0120] As one implementation method, when the plant species reasoning result of the second plant object cluster represents a plant object in the second plant object cluster that corresponds to the target plant species, anomaly type reasoning is performed through the cluster knowledge representation in the second plant object cluster and the target knowledge representation to obtain the anomaly type evaluation result corresponding to the second plant object cluster.

[0121] As one implementation method, the plant species classification module outputs a probability as the plant species inference result, representing the likelihood that the current plant object cluster contains a plant object corresponding to the target plant species. When the probability obtained by the plant species classification module exceeds a preset probability (adaptively selected based on actual conditions), then the current plant object cluster contains a plant object corresponding to the target plant species. Based on the cluster knowledge representation and target knowledge representation of the current plant object cluster, anomaly type inference is performed within the current plant object cluster. Conversely, when the probability obtained by the plant species classification module does not reach the preset probability, if the plant species inference result of the third plant object cluster indicates that the third plant object cluster does not contain a plant object corresponding to the target plant species, the step of performing anomaly type inference using the cluster knowledge representation and target knowledge representation of the third plant object cluster is no longer executed. Therefore, the current plant object cluster does not contain a plant object corresponding to the target plant species, and the process of performing anomaly type inference using the cluster knowledge representation and target knowledge representation of the current plant object cluster is no longer executed.

[0122] As one implementation, the anomaly type evaluation result includes the coverage of anomaly types in the plant object cluster.

[0123] As one implementation method, the garden management server, based on the cluster knowledge representation and target knowledge representation of the current plant object cluster, can perform anomaly type reasoning in the current plant object cluster by combining the cluster knowledge representation of the current cluster with the target knowledge representation of the comparison target based on the anomaly type classification module, and then determine the coverage range of the anomaly type in the current plant object cluster.

[0124] The plant species inference results obtained from the plant species classification module are used to determine the coverage of anomaly types in the current plant object cluster through the anomaly type classification module.

[0125] Based on this, the above-described method in this application, for a garden plant monitoring image including multiple plant object clusters, combines the cluster knowledge representation of the current plant object cluster, the target knowledge representation of the comparison target, and the second cluster description vector of the current plant object cluster to analyze the anomaly type in the current plant object cluster. Because the second cluster description vector covers the information of the current plant object cluster and the information of the other adjacent plant object clusters, the information of the other adjacent plant objects can be jointly analyzed during anomaly type analysis, and the feature information can be utilized as a whole, which can increase the accuracy of anomaly type analysis.

[0126] As one implementation, the garden image anomaly analysis network includes a knowledge mining module, a second descriptive vector integration module, a focusing module, a first descriptive vector integration module, a plant species classification module, and an anomaly type classification module. In this embodiment, it includes a comparison target sequence and garden plant monitoring images. The comparison target sequence includes one or more comparison targets, and the garden plant monitoring images include multiple plant object clusters. The garden management server loads one comparison target from the comparison target sequence and multiple plant object clusters from the garden plant monitoring images into the knowledge mining module. The comparison target receives a corresponding target knowledge representation, and the multiple plant object clusters receive their respective cluster knowledge representations. The garden management server loads the cluster knowledge representations of each of the multiple plant object clusters into the second descriptive vector integration module. The second descriptive vector integration module outputs integrated cluster description vectors for each of the multiple plant object clusters. The garden management server loads the target knowledge representation output by the knowledge mining module and the integrated cluster description vectors for each of the multiple plant object clusters output by the second descriptive vector integration module into the focusing module. The focusing module outputs the first cluster description vector for each of the multiple plant object clusters. The garden management server loads the first cluster description vectors of multiple plant object clusters into the first description vector integration module. Through a dynamic focusing strategy, the first description vector integration module outputs the first cluster description vectors of adjacent plant object clusters. Based on the integrated first cluster description vectors of multiple plant object clusters and adjacent plant object clusters, the second cluster description vectors of each plant object cluster are obtained. The garden management server then loads these second cluster description vectors into the plant species classification module. When the plant species inference result output by the plant species classification module for the current plant object cluster is greater than a preset value—in other words, when the plant species inference result indicates that the plant object cluster contains a plant object corresponding to the comparison target—the anomaly type classification module outputs the coverage range of anomaly types within the plant object cluster using the cluster knowledge representation of the plant object cluster and the target knowledge representation, and then processes another comparison target. When the plant species inference result indicates that the current plant object cluster does not contain a plant object corresponding to the comparison target, the anomaly type classification module no longer executes the process of outputting the coverage range of anomaly types within the plant object cluster using the cluster knowledge representation of the plant object cluster and the target knowledge representation. After classifying the anomalies of multiple plant object clusters according to the comparison target, the process proceeds to the next comparison target for iterative steps.

[0127] The training of a network for anomaly analysis of garden images may include the following steps:

[0128] S1000: The knowledge mining module in the garden image anomaly analysis network performs knowledge mining on multiple example plant object clusters in the example garden plant monitoring image to obtain the example cluster knowledge representation of each of the multiple example plant object clusters; the knowledge mining module performs knowledge mining on the comparison target to obtain the target knowledge representation of the comparison target.

[0129] The example garden plant monitoring image contains multiple example plant clusters arranged according to the spatial distribution of plants. Each example plant cluster contains one or more plant objects. When acquiring the example plant clusters, the knowledge mining module in the garden image anomaly analysis network performs knowledge mining on the multiple example plant clusters in the example garden plant monitoring image to obtain the example cluster knowledge representation for each of the multiple example plant clusters. The knowledge mining module then performs knowledge mining on the comparison target to obtain the target knowledge representation for the comparison target. The comparison target includes confirmation indication information of the comparison anomaly type within the target plant species.

[0130] S2000: The second description vector integration module in the garden image anomaly analysis network integrates the knowledge representation of each example plant object in multiple example plant object clusters with the knowledge representation of the same type of plant objects in the example garden plant monitoring image, to obtain the integrated cluster description vector of example plant objects for each of the multiple example plant object clusters.

[0131] After obtaining the knowledge representation of each example plant object in the example garden plant monitoring image, the second description vector integration module in the garden image anomaly analysis network is used. Through the knowledge representation of an example target plant object in the example garden plant monitoring image, the focusing coefficient of the example target plant object on the example target plant object in the example garden plant monitoring image is obtained. The knowledge representation of the example target plant object and the knowledge representation of the same type of plant object in the example garden plant monitoring image are integrated to update the knowledge representation of the example target plant object, and the lower-level integrated knowledge representation of the example target plant object is obtained. Then, the integrated cluster description vector of the same type of plant object of the example plant object cluster is obtained. The knowledge representation of the example target plant object containing the same type of plant object in the example garden plant monitoring image is updated in the same way. When an example target plant object does not have the same type of plant object in the example garden plant monitoring image, the knowledge representation of the example target plant object is not updated.

[0132] S3000: By comparing the target focusing module and using target knowledge representation, feature information is focused on the integrated cluster description vectors of example plant objects of the same type for each of the multiple example plant object clusters, so as to obtain the first example cluster description vector of each of the multiple example plant object clusters.

[0133] The comparison target contains confirmation indication information of the comparison anomaly type within the target plant species. When the target knowledge representation corresponding to the comparison target is obtained, the comparison target focusing module determines the focusing coefficient of each example plant object cluster to the target knowledge representation based on the focusing strategy. Based on the product of the integrated cluster description vector of the same type of plant objects of the example plant object cluster and the focusing coefficient of the example plant object cluster to the example comparison target, the integrated cluster description vector of the same type of plant objects corresponding to each plant object cluster is updated and adjusted to obtain the first cluster description vector of each example plant object cluster.

[0134] S4000: Through the first description vector integration module in the garden image anomaly analysis network, the first example cluster description vectors of multiple example plant object clusters are integrated with the first example cluster description vectors of adjacent example plant object clusters to obtain the second example cluster description vectors of multiple example plant object clusters.

[0135] When obtaining the first cluster description vectors corresponding to multiple example plant object clusters, based on the first description vector integration module in the garden image anomaly analysis network, and through a dynamic focusing strategy, the focusing coefficients of each example plant object cluster in the adjacent plant object cluster queues on the first example plant object cluster are obtained based on the first example cluster description vector of the first example plant object cluster and the first example cluster description vector of each example plant object cluster in the adjacent plant object cluster queues. The first example cluster description vector of the first example plant object cluster is combined with the first example cluster description vectors of the example target plant object clusters adjacent to the example plant object cluster, for example, by splicing them together to obtain the second example cluster description vector of the first example plant object cluster.

[0136] S5000: Through the plant species classification module, plant species reasoning is performed using the second example cluster description vector of each of the multiple example plant object clusters to obtain the plant species reasoning results of each of the multiple example plant object clusters. Anomaly type reasoning is performed using the plant species reasoning results of each of the multiple example plant object clusters, the example cluster knowledge representation of each of the multiple example plant object clusters, and the target knowledge representation of each of the multiple example plant object clusters to obtain the anomaly type evaluation results of each of the multiple example plant object clusters.

[0137] When obtaining the second example cluster description vector for each example plant object cluster, plant species inference is performed based on the plant species classification module using the second example cluster description vectors of multiple example plant object clusters. This yields plant species inference results for each example plant object cluster. These results indicate whether the example plant object cluster contains a plant object corresponding to the target plant species. The plant object corresponding to the target plant species is used to represent the anomaly type and corresponding confirmation indication information within the plant object cluster. When obtaining the plant species inference results for the example plant object clusters, the cluster knowledge representation and target knowledge representation of each example plant object cluster are combined. Based on the plant species classification module, anomaly type evaluation results for each example plant object cluster are obtained. These anomaly type evaluation results include the coverage of anomaly types within the example plant object clusters.

[0138] S6000: The network configuration variables of the garden image anomaly analysis network are adjusted by using the anomaly type evaluation results and anomaly type annotation information of multiple example plant object clusters. The anomaly type annotation information represents the true anomaly type in the example plant object clusters.

[0139] The garden management server can obtain a loss value based on the loss between the actual anomaly type in the example plant object cluster and the anomaly type inferred from the example plant object cluster. Specifically, the loss value can be calculated based on cross-entropy or log-likelihood. The network configuration variables in the garden image anomaly analysis network are adjusted by backpropagation based on the high loss.

[0140] The classification module includes a plant species classification module and a common type classification module, where the network configuration variables for the plant species classification module and the anomaly type classification module are identical. Based on the classification module in the garden image anomaly analysis network, the anomaly type evaluation results for multiple example plant object clusters are obtained through their respective example cluster knowledge representations, target knowledge representations, and second example cluster description vectors. Specifically, this can include: using the plant species classification module, plant species inference is performed using the second example cluster description vectors of each example plant object cluster to obtain plant species inference results for each example plant object cluster, where the plant species inference results indicate whether the example plant object cluster contains a plant object corresponding to the target plant species; using the anomaly type classification module, anomaly type inference is performed using the plant species inference results, example cluster knowledge representations, and target knowledge representations of each example plant object cluster to obtain anomaly type evaluation results for each example plant object cluster.

[0141] The plant species classification module and the anomaly type classification module each have their own knowledge representation mining and output structures. The knowledge representation mining structures in the plant species classification module and the anomaly type classification module have the same composition and the same parameters. Based on the contrast target focusing module in the garden image anomaly analysis network, before focusing feature information on the example cluster knowledge representations of multiple example plant object clusters through target knowledge representation to obtain the first example cluster description vector for each of the multiple example plant object clusters, the method may further include: integrating the knowledge representation of each example plant object in the multiple example plant object clusters with the knowledge representation of similar plant objects in the example garden plant monitoring image through the second description vector integration module in the garden image anomaly analysis network to obtain the integrated cluster description vector of the example similar plant objects for each of the multiple example plant object clusters.

[0142] Based on the contrast target focusing module in the garden image anomaly analysis network, feature information is focused on the example cluster knowledge representation of each of the multiple example plant object clusters through target knowledge representation, so as to obtain the first example cluster description vector of each of the multiple example plant object clusters. Specifically, it can include: through the contrast target focusing module, feature information is focused on the integrated cluster description vector of example similar plant objects of each of the multiple example plant object clusters through target knowledge representation, so as to obtain the first example cluster description vector of each of the multiple example plant object clusters.

[0143] Based on this, the method provided in this application embodiment is based on example garden plant monitoring images, anomaly type annotation information corresponding to example garden plant monitoring images, and a comparison target garden image anomaly analysis network. When the network is applied, for a garden plant monitoring image including multiple plant object clusters, the garden image anomaly analysis network combines the cluster knowledge representation of the current plant object cluster, the target knowledge representation of the comparison target, and the second cluster description vector of the current plant object cluster to analyze the anomaly type in the current plant object cluster. Because the second cluster description vector covers the information of the current plant object cluster and the information of the other adjacent plant object clusters, the information of the other adjacent plant objects can be jointly analyzed during anomaly type analysis, and the feature information can be utilized as a whole, which can increase the accuracy of anomaly type analysis.

[0144] Please refer to Figure 4 This is a functional module architecture diagram of the image processing device 110 provided in an embodiment of the present invention. The image processing device 110 can be used to execute an image processing method applied to garden management. The image processing device 110 includes:

[0145] The knowledge mining module 111 is used to obtain the cluster knowledge representation of each of the multiple plant object clusters in the garden plant monitoring image, wherein the multiple plant object clusters are arranged according to the spatial distribution of plants, and each plant object cluster contains one or more plant objects; and to obtain the target knowledge representation of the comparison target, wherein the comparison target contains the confirmation indication information of the comparison anomaly type of the target plant species.

[0146] The feature focusing module 112 is used to focus feature information on the cluster knowledge representation of each of the multiple plant object clusters through the target knowledge representation, so as to obtain the first cluster description vector of each of the multiple plant object clusters.

[0147] The vector integration module 113 is used to integrate the first cluster description vector of each of the multiple plant object clusters with the first cluster description vector of the adjacent plant object clusters to obtain the second cluster description vector of each of the multiple plant object clusters.

[0148] The anomaly determination module 114 is used to obtain the anomaly type evaluation results of multiple plant object clusters by using their respective cluster knowledge representations, target knowledge representations, and second cluster description vectors; the anomaly type evaluation results include the anomaly types in the plant object clusters.

[0149] Since the image processing method for garden management provided by the present invention has been described in detail in the above embodiments, and the principle of the image processing device 110 is the same as that method, the execution principle of each module of the image processing device 110 will not be described again here.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0151] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0152] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, an IoT data server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] It should be understood that for the technical terms for which no explanations are provided above, those skilled in the art can deduce their meanings without doubt based on the disclosed content. The content disclosed in the embodiments of this application is clear and complete to those skilled in the art. It should be understood that the process by which those skilled in the art deduce and analyze the unexplained technical terms based on the disclosed content is based on the content recorded in this application; therefore, the above content is not a judgment of the inventiveness of the overall solution.

[0155] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art can make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

[0156] It should also be understood that, in order to simplify the description disclosed in this application and thus aid in the understanding of at least one embodiment of the invention, multiple features may sometimes be grouped into a single embodiment, drawing, or description thereof in the foregoing description of the embodiments of this application. However, this method of disclosure does not imply that the subject matter of this application requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

Claims

1. An image processing method for garden management, characterized in that, The method, applied to a garden management server, includes: Acquire cluster knowledge representations of multiple plant object clusters in a garden plant monitoring image, wherein the multiple plant object clusters are arranged according to the spatial distribution of plants, and each plant object cluster contains one or more plant objects; Obtain a target knowledge representation of the comparison target, wherein the comparison target includes confirmation indication information of the comparison anomaly type of the target plant species; By focusing feature information on the cluster knowledge representations of the multiple plant object clusters through the target knowledge representation, the first cluster description vector of each of the multiple plant object clusters is obtained. The method involves integrating the first cluster description vectors of each of the multiple plant object clusters with the first cluster description vectors of adjacent plant object clusters to obtain the second cluster description vectors of each of the multiple plant object clusters. Specifically, for a first plant object cluster, the method obtains the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue relative to the first plant object cluster using the first cluster description vector of the first plant object cluster and the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue. The first plant object cluster is any one of the multiple plant object clusters. The adjacent plant object cluster queue includes plant object clusters adjacent to the first plant object cluster. The method integrates the first cluster description vector of the first plant object cluster with the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue using the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue to obtain the second cluster description vector of the first plant object cluster. Anomaly type evaluation results for each of the multiple plant object clusters are obtained by using their respective cluster knowledge representations, the target knowledge representation, and their respective second cluster description vectors. Specifically, this includes: performing plant species reasoning using the respective second cluster description vectors of the multiple plant object clusters to obtain plant species reasoning results for each of the multiple plant object clusters; the plant species reasoning results indicate whether the plant object cluster contains a plant object corresponding to the target plant species; and performing anomaly type reasoning using the respective plant species reasoning results, the respective cluster knowledge representations of the multiple plant object clusters, and the target knowledge representation to obtain anomaly type evaluation results for each of the multiple plant object clusters; the anomaly type evaluation results include the anomaly types within the plant object clusters.

2. The method according to claim 1, characterized in that, The step of integrating the first cluster description vector of the first plant object cluster with the first cluster description vector of each plant object cluster in the adjacent plant object cluster queue using the focusing coefficient of each plant object cluster in the adjacent plant object cluster queue to obtain the second cluster description vector of the first plant object cluster includes: The first cluster description vector of the first plant object cluster is combined with the first cluster description vector of the target plant object cluster to obtain the second cluster description vector of the first plant object cluster; wherein, the target plant object cluster is the plant object cluster with the largest focusing coefficient on the first plant object cluster in the adjacent plant object cluster queue.

3. The method according to claim 1, characterized in that, The step of performing anomaly type inference through the plant species inference results of each of the multiple plant object clusters, the cluster knowledge representation of each of the multiple plant object clusters, and the target knowledge representation, to obtain the anomaly type evaluation results of each of the multiple plant object clusters, includes: When the plant species reasoning result of the second plant object cluster represents a plant object in the second plant object cluster that corresponds to the target plant species, anomaly type reasoning is performed through the cluster knowledge representation in the second plant object cluster and the target knowledge representation to obtain the anomaly type evaluation result corresponding to the second plant object cluster; the second plant object cluster is any one of the multiple plant object clusters. When the plant species reasoning result of the third plant object cluster indicates that there is no plant object corresponding to the target plant species in the third plant object cluster, the process of performing abnormal type reasoning through the cluster knowledge representation in the third plant object cluster and the target knowledge representation will no longer be executed.

4. The method according to claim 1, characterized in that, The anomaly type evaluation result also includes the coverage range of the anomaly type in the plant object cluster in the plant object cluster; The step of focusing feature information on the cluster knowledge representations of multiple plant object clusters through the target knowledge representation to obtain the first cluster description vector for each of the multiple plant object clusters includes: By using the target knowledge representation and the cluster knowledge representation of each of the multiple plant object clusters, the focusing coefficient of each of the multiple plant object clusters toward the comparison target is obtained; The cluster knowledge representation of each of the multiple plant object clusters is processed by the focusing coefficient of each of the multiple plant object clusters toward the comparison target, so as to obtain the first cluster description vector of each of the multiple plant object clusters; The step of processing the cluster knowledge representation of each of the multiple plant object clusters by the focusing coefficient of each of the multiple plant object clusters on the comparison target to obtain the first cluster description vector of each of the multiple plant object clusters includes: determining the first cluster description vector of each of the multiple plant object clusters by multiplying the cluster knowledge representation of each of the multiple plant object clusters with the focusing coefficient of each of the multiple plant object clusters on the comparison target.

5. The method according to claim 1, characterized in that, The cluster knowledge representation includes the knowledge representation of each plant object in the plant object cluster; before focusing feature information on the cluster knowledge representations of the multiple plant object clusters through the target knowledge representation to obtain the first cluster description vector of each of the multiple plant object clusters, the method further includes: The knowledge representation of each plant object in the multiple plant object clusters is integrated with the knowledge representation of the same type of plant objects in the garden plant monitoring image to obtain the integrated cluster description vector of the same type of plant objects for each of the multiple plant object clusters. The step of focusing feature information on the cluster knowledge representations of multiple plant object clusters through the target knowledge representation to obtain the first cluster description vector for each of the multiple plant object clusters includes: By using the target knowledge representation, feature information is focused on the integrated cluster description vectors of the same type of plant objects in each of the multiple plant object clusters to obtain the first cluster description vector of each of the multiple plant object clusters. The step of integrating the knowledge representation of each plant object in the plurality of plant object clusters with the knowledge representation of similar plant objects in the garden plant monitoring image for each plant object to obtain the integrated cluster description vector of similar plant objects for each of the plurality of plant object clusters includes: By using the object encoding vector of the target plant object and the knowledge representation of the target plant object in the garden plant monitoring image, the focusing coefficient of the target plant object on the target plant object in the garden plant monitoring image is obtained; wherein, the target plant object is any plant object in the garden plant monitoring image; By using the focusing coefficient of the target plant object on the same type of plant objects in the garden plant monitoring image, the knowledge representation of the target plant object is integrated with the knowledge representation of the target plant object on the same type of plant objects in the garden plant monitoring image to obtain the lower-level integrated knowledge representation of the target plant object; wherein, the lower-level integrated knowledge representation of the target plant object is the knowledge representation corresponding to the target plant object in the integrated cluster description vector of the same type of plant object of the plant object cluster corresponding to the target plant object.

6. The method according to claim 1, characterized in that, The method is executed through a pre-set garden image anomaly analysis network, which is trained through the following steps: The knowledge mining module in the garden image anomaly analysis network performs knowledge mining on multiple sample plant object clusters in the sample garden plant monitoring images to obtain the sample cluster knowledge representation of each of the multiple sample plant object clusters; wherein, the multiple sample plant object clusters are arranged according to the spatial distribution of plants, and each sample plant object cluster contains one or more plant objects; The knowledge mining module performs knowledge mining on the comparison target to obtain the target knowledge representation of the comparison target; wherein, the comparison target includes confirmation indication information of the comparison anomaly type within the target plant species; Through the contrast target focusing module in the garden image anomaly analysis network, and through the target knowledge representation, feature information focusing is performed on the example cluster knowledge representation of each of the multiple example plant object clusters to obtain the first example cluster description vector of each of the multiple example plant object clusters. Through the first description vector integration module in the garden image anomaly analysis network, the first example cluster description vector of each of the multiple example plant object clusters is integrated with the first example cluster description vector of the adjacent example plant object clusters to obtain the second example cluster description vector of each of the multiple example plant object clusters. The classification module in the garden image anomaly analysis network obtains anomaly type evaluation results for each of the multiple example plant object clusters through the example cluster knowledge representation, the target knowledge representation, and the second example cluster description vector of each of the multiple example plant object clusters; wherein, the anomaly type evaluation results include the anomaly types inferred from the example plant object clusters. The network configuration variables of the garden image anomaly analysis network are adjusted based on the anomaly type evaluation results and anomaly type annotation information of each of the example plant object clusters; the anomaly type annotation information characterizes the true anomaly types in the example plant object clusters.

7. The method according to claim 6, characterized in that, The classification module includes a plant species classification module and an anomaly type classification module, and the network configuration variables of the plant species classification module and the anomaly type classification module are the same. The step involves using the classification module in the garden image anomaly analysis network to obtain anomaly type evaluation results for each of the multiple example plant object clusters through their respective example cluster knowledge representations, target knowledge representations, and second example cluster description vectors. This includes: The plant species classification module performs plant species inference using the second example cluster description vector of each of the multiple example plant object clusters to obtain plant species inference results for each of the multiple example plant object clusters; the plant species inference results characterize whether the example plant object cluster contains a plant object corresponding to the target plant species. The anomaly type classification module performs anomaly type inference based on the plant species inference results of each of the multiple example plant object clusters, the example cluster knowledge representation of each of the multiple example plant object clusters, and the target knowledge representation, and obtains the anomaly type evaluation results of each of the multiple example plant object clusters. Before the step of focusing feature information on the example cluster knowledge representations of each of the multiple example plant object clusters through the contrast target focusing module in the garden image anomaly analysis network to obtain the first example cluster description vector for each of the multiple example plant object clusters, the method further includes: Through the second description vector integration module in the garden image anomaly analysis network, the knowledge representation of each example plant object in the multiple example plant object clusters is integrated with the knowledge representation of the same type of plant object in the example garden plant monitoring image, so as to obtain the example same type of plant object integrated cluster description vector of each of the multiple example plant object clusters. The step involves using the contrast target focusing module in the garden image anomaly analysis network to focus feature information on the example cluster knowledge representations of multiple example plant object clusters through the target knowledge representation, thereby obtaining the first example cluster description vector for each of the multiple example plant object clusters, including: Through the comparison target focusing module and the target knowledge representation, feature information is focused on the integrated cluster description vectors of example plant objects of the same kind for each of the multiple example plant object clusters, so as to obtain the first example cluster description vector of each of the multiple example plant object clusters.

8. A garden management server, characterized in that, It includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, implements the method as described in any one of claims 1 to 7.

Images