A method and system for detecting melon powdery mildew lesion by combining text label self-prompting and soft supergraph reasoning

Abstract

This application discloses a method and system for detecting powdery mildew lesions in melons, combining text label self-prompting and soft hypergraph inference. The method, relating to the field of lesion detection, includes: based on an enhanced training set, inputting category labels into a frozen CLIP encoder to obtain text features; inputting sample images of powdery mildew-affected melon leaves into a YOLO backbone network to extract multi-scale original feature maps; performing the following operations on each scale of the original feature map: using a cross-modal interaction module to perform cross-modal attention interaction and fusion, cosine similarity calculation, and channel-level feature enhancement on the original feature map and text features to obtain an enhanced feature map; inputting the enhanced feature map into a soft hypergraph inference module in two paths, forming a soft hypergraph feature map through soft hyperedge generation and global information preservation; and inputting the soft hypergraph feature map into two parallel detection heads for prediction. This application improves the detection capability of powdery mildew lesions in melons.

Description

Technical Field

[0001] This application relates to the field of lesion detection technology, and in particular to a method and system for detecting powdery mildew lesions in melons by combining text label self-prompting and soft hypergraph inference. Background Technology

[0002] Melon is one of the important crops of the Cucurbitaceae family. It is mainly eaten fresh, but can also be made into dried melon, melon preserves, melon juice, melon sauce, and pickled products.

[0003] Powdery mildew of melons is a widespread global fungal disease characterized by a short incubation period, high transmissibility, and high incidence and secondary infection rates throughout the entire growth cycle, making it a key disease for production control. The disease primarily infects leaves, but can also damage petioles, stems, and even fruits. In the early stages, small white powdery mold spots appear on the upper or lower surfaces of leaves, damaging the photosynthetic organs, inhibiting normal carbohydrate synthesis, and causing a rapid decline in plant physiological functions. As the lesions rapidly expand and merge, the leaves gradually yellow, curl, and become brittle, eventually dying prematurely. This hinders fruit development, significantly reduces sugar content, and drastically diminishes quality and value. In years with widespread outbreaks, it can cause yield losses of 20%-30% in mild cases, and in severe cases, even total crop failure, resulting in economic losses for melon farmers.

[0004] Melon cultivation typically employs sheltered, rain-protected cultivation methods. Under these conditions, the greenhouse environment more readily reaches the threshold for powdery mildew reproduction and spread, creating conditions conducive to year-round occurrence of the disease. Early detection and lack of intervention can lead to sudden outbreaks, significantly increasing the difficulty of control. Due to the concealed and diverse nature of primary infection sources, as well as the heterogeneity of field microclimates, powdery mildew prevention largely relies on experience-based judgment and a passive response model of applying pesticides upon noticing symptoms. This easily leads to missing the optimal control window (early stage of disease), resulting in the overuse of chemical pesticides. This not only exacerbates the development of pesticide resistance in pathogens but also poses environmental pollution and risks to agricultural product safety.

[0005] In the early stages, powdery mildew manifests as small, powdery mold spots on the upper surface of leaves. Human observation of these spots is easily affected by subjective factors and environmental factors such as light. Furthermore, assessments of disease severity are primarily qualitative, based on symptom observation, which lacks accuracy. Disease spot detection using machine vision technology combined with image processing algorithms is currently a research hotspot, with the core aiming to diagnose powdery mildew and its severity. However, because early-stage powdery mildew lesions are small and have indistinct borders, they are easily confused with leaf texture, light, dew, dust, and shadows, leading to poor detection accuracy of related algorithms for melon powdery mildew. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for detecting powdery mildew spots in melons that combines text label self-prompting and soft hypergraph inference. This method fully utilizes the feature extraction capabilities of text label self-prompting and soft hypergraph inference to improve the detection capability of powdery mildew spots in melons.

[0007] To achieve the above objectives, this application provides the following solution.

[0008] Firstly, this application provides a method for detecting powdery mildew lesions in melons that combines text label self-prompting and soft hypergraph inference. The method includes: acquiring a sample dataset; the sample dataset includes images of different powdery mildew-infected melon leaves and corresponding text label information; the text label information includes at least manually labeled category labels of powdery mildew lesions on the melon leaf sample images; randomly dividing the sample dataset into a training set and a test set, and performing data augmentation on the training set; constructing a powdery mildew lesion detection model; the powdery mildew lesion detection model includes a text label self-prompting module and a soft hypergraph inference module; the text label self-prompting module includes at least a YOLO backbone network and a cross-modal interaction module; and inputting the category labels into a frozen C... The LIP encoder obtains text features. The sample image of a melon leaf with powdery mildew is input into the YOLO backbone network to extract multi-scale original feature maps. For each scale of the original feature map, the following operations are performed: The cross-modal interaction module performs cross-modal attention interaction and fusion, cosine similarity calculation, and channel-level feature enhancement on the original feature map and the text features to obtain an enhanced feature map; the enhanced feature map is input into the soft hypergraph inference module in two paths, forming a soft hypergraph feature map through soft hyperedge generation and global information preservation; the soft hypergraph feature map is input into two parallel detection heads for prediction; the trained powdery mildew lesion detection model is tested based on the test set; the image of the melon leaf to be detected is acquired, and the trained and tested powdery mildew lesion detection model is used to detect powdery mildew lesions on the melon leaf image.

[0009] Secondly, this application also provides a computer system, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for detecting powdery mildew spots in melons that combines text label self-prompting and soft hypergraph reasoning as described in the first aspect.

[0010] Based on the specific embodiments provided in this application, the following technical effects are disclosed.

[0011] This application addresses two aspects: cross-modal attention interaction and fusion, and soft hypergraph inference. First, it utilizes a frozen Contrastive Language-Image Pre-training (CLIP) encoder to obtain text features. These features are then fused with multi-scale original feature maps extracted by the YOLO backbone network. The aim is to improve the detection accuracy and feature discrimination ability for small, blurry lesions such as powdery mildew by leveraging textual semantic embeddings and visual features and compensating for insufficient visual information of small targets using textual priors. Secondly, this application also generates a feature map with a rich soft hypergraph structure through soft hyperedge generation and global information preservation. This feature map is fed back to two parallel detection heads to enhance their detection and representation capabilities, thereby achieving accurate detection of powdery mildew lesions in melons. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a method for detecting powdery mildew spots in melons that combines text label self-prompting and soft hypergraph inference in one embodiment of this application.

[0014] Figure 2 This is a schematic diagram of the powdery mildew lesion detection model structure in one embodiment of this application.

[0015] Figure 3 This is a flowchart illustrating the execution of a powdery mildew lesion detection model in one embodiment of this application.

[0016] Figure 4 This is a diagram of the internal structure of a computer system in another embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] The occurrence and spread of powdery mildew in melons are highly complex and unpredictable, posing a significant challenge to effective control. The main pathogens causing powdery mildew in melons are *Podosphaera xanthii* and *Golovinomyces cichoracearum*, typical obligate parasites. They can overwinter and oversummer as mycelium on perennial hosts or protected crops, or produce highly resistant cleistothecia that remain dormant in the soil with diseased plant debris, serving as the primary source of infection. The conidia produced, due to their small size, can be spread over long distances and large areas via air currents, allowing the disease to spread rapidly once it occurs, quickly engulfing entire fields and even regions. More importantly, its epidemic pattern is closely related to environmental conditions: warm, moderately humid microenvironments with poor ventilation and light penetration are highly conducive to spore germination and infection. However, unlike many diseases that thrive in high humidity, powdery mildew fungi produce conidia with extremely strong drought resistance, capable of germinating even in low relative humidity. This allows it to proliferate even during dry seasons or in greenhouses, defying the traditional belief that "diseases are always accompanied by rain." Furthermore, excessive nitrogen fertilizer, young plantlets, and overcrowding can significantly exacerbate the disease. The diversity and interaction of these factors make its dynamics difficult to grasp, ensuring that the prediction and early warning of powdery mildew in melons remains a significant challenge in both research and practice.

[0019] Object detection is a major research direction in machine vision applications and is also crucial for the detection and severity assessment of powdery mildew in melons. Many object detection algorithms exist, including those for small objects. However, the conditions under which melon leaf images are collected are inconsistent, and the lesions are small with unique disease characteristics. Researchers have proposed many object detection algorithms; however, while these algorithms have some generalization ability and are designed for public datasets or specific plant diseases, they may not achieve optimal results in detecting powdery mildew in melons.

[0020] The purpose of this application is to provide a method and system for detecting powdery mildew spots in melons that combines text label self-prompting and soft hypergraph inference. This method fully utilizes the feature extraction capabilities of text label self-prompting and soft hypergraph inference to improve the detection capability of powdery mildew spots in melons.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] In one exemplary embodiment, a method for detecting powdery mildew spots in melons is provided, combining text label self-prompting and soft hypergraph inference. Figure 1 As shown, the specific method for detecting powdery mildew spots in melons by combining text label self-prompting and soft hypergraph inference is as follows.

[0023] Step S1: Obtain the sample dataset.

[0024] In this embodiment, the sample dataset includes leaf sample images of different melons with powdery mildew and corresponding text label information. The leaf sample images of different melons with powdery mildew also need to be manually labeled. After the labeling is completed, a JSON format structured data file corresponding to the image is generated, which saves the outline coordinates of the diseased area, category labels and other text label information.

[0025] Step S2: Randomly divide the sample dataset into a training set and a test set, and perform data augmentation on the training set.

[0026] In this embodiment, for the training set, an image enhancement algorithm is used to amplify the powdery mildew lesion data of melons, that is, to perform the image processing process in the traditional YOLO model.

[0027] Step S3: Construct a powdery mildew lesion detection model.

[0028] In this embodiment, as Figure 2 As shown, the powdery mildew lesion detection model includes a text label self-prompt module and a soft hypergraph inference module. The text label self-prompt module comprises a YOLO backbone network and a cross-modal interaction module. The cross-modal interaction module further includes a cross-modal attention module (equipped with a max-pooling layer, connection layers, and a multi-head attention mechanism layer), a prompt score graph calculation module, and an SE attention module. The soft hypergraph inference module includes a two-branch soft hypergraph neural network, a Top-k module, a convergence propagation module, a contrast detection head, and a bounding box detection head.

[0029] As a preferred implementation, the YOLO backbone network adopts the YOLOV8x backbone network.

[0030] Step S4: Based on the enhanced training set, input the category label (also known as category information, powdery mildew) into the frozen CLIP encoder (using the ViT architecture and directly loading the pre-trained weights from OpenAI) to obtain text features. Input the sample images of powdery mildew leaves of melons into the YOLO backbone network to extract multi-scale original feature maps (i.e., C3, C4, and C5, corresponding to the visual representations of the shallow, medium, and deep layers, respectively). Perform the following operations on the original feature maps at each scale.

[0031] Step S41: Use the cross-modal interaction module to perform cross-modal attention interaction and fusion, cosine similarity calculation and channel-level feature enhancement on the original feature map and text features to obtain the enhanced feature map.

[0032] In this embodiment, step S41 is as follows.

[0033] The original feature map and text features are input into the cross-modal attention module; adaptive max pooling downsampling is performed on the original feature map (reducing spatial resolution to lower computational complexity) to obtain pooled image features (flattening the downsampled features into a series of spatial location nodes); the text features are projected into a query vector, and the pooled image features are projected into key vectors and value vectors respectively; a multi-head attention mechanism is used to calculate the similarity between the query vector and the key vector, and the attention weights are obtained after scaling and normalization; the value vectors are weighted and aggregated using the attention weights, and the enhanced text features fused with image information are output through residual connections; the original feature map and enhanced text features are input into the cue score map calculation module; the relationship between each spatial location in the original feature map and the enhanced text features is calculated. The cosine similarity between features is calculated, and the maximum similarity value is taken among all categories. After processing with an activation function, an attention mask is generated. The attention mask is then multiplied element-wise with the original feature map to obtain multimodal features guided by text semantics (Figure). The multimodal features are input into the SE attention module for channel-level feature enhancement. First, global average pooling is used to compress all spatial location information of each feature channel into a single value to obtain a global feature description vector. Then, two layers of fully connected neural networks are used for dimensionality reduction and dimensionality increase transformations. The importance weight coefficient of each channel is learned through an activation function. Finally, the learned weight coefficients are multiplied with the corresponding channels of the original multimodal features to enhance important channels and suppress redundant channels, outputting a channel-optimized enhanced feature map.

[0034] Step S42: Input the enhanced feature map into the soft hypergraph inference module in two ways, and form a soft hypergraph feature map through soft hyperedge generation and global information preservation.

[0035] In this embodiment, step S42 is as follows.

[0036] The enhanced feature map is duplicated into two paths. One enhanced feature map is flattened into node feature representations and input into a two-branch soft hypergraph neural network for high-order relation modeling. The other enhanced feature map is directly input into the aggregation and propagation module to retain global information. Within each soft hypergraph branch of the two-branch soft hypergraph neural network, the following operations are performed: First, global context information of the entire enhanced feature map is extracted through average pooling and max pooling operations, respectively. The two context vectors are concatenated and then passed through a multilayer perceptron network to generate a set of adaptive hyperedge prototype vectors. Then, the similarity between each node feature and all hyperedge prototypes is calculated. After parallel computation through a multi-head mechanism, the similarity is averaged and normalized to obtain the soft assignment probability matrix of the node to the hyperedge (this matrix represents the probability that each node belongs to each hyperedge).

[0037] To improve efficiency and generalization ability, the Top-k module's filtering strategy is used to retain only the nodes with the highest activation scores for each superedge, while the remaining connections are set to zero to form a sparse structure.

[0038] The aggregation propagation module performs a two-stage information propagation process. The aggregated features output from the two soft hypergraph branches are then reshaped, concatenated and fused with another enhanced feature map, and integrated along the channel dimension to obtain a soft hypergraph feature map containing rich high-order relational information. Specifically, the first stage is node-to-hyperedge aggregation, where the features of all nodes belonging to the same hyperedge are summed according to soft-assignment probabilities and then transformed nonlinearly to obtain the hyperedge feature representation. The second stage is hyperedge-to-node propagation, where all hyperedge features related to each node are aggregated back to that node according to soft-assignment probabilities, transformed, and then residually connected with the original node features to obtain the updated node features. The two independent soft hypergraph branches output aggregated features, which are reshaped back to their original two-dimensional spatial shape and then concatenated and fused with features from other processing paths. After channel dimension integration through one-dimensional convolution, a soft hypergraph feature map containing rich high-order relational information is formed.

[0039] Step S43: Input the soft hypermap feature map into two parallel detection heads for prediction.

[0040] In this embodiment, the two parallel detection heads refer to the contrastive learning head and the bounding box regression head. The contrastive learning head outputs the class confidence score after calculating the similarity between image features and text features, while the bounding box regression head outputs the bounding box coordinates (coordinates of the four points of the detection box). C4 and C5 use the exact same processing flow, only differing in feature map size and number of channels. The prediction results of the three scale layers are post-processed through coordinate unification, multi-scale fusion, and non-maximum suppression, ultimately outputting detection results containing bounding box coordinates, class labels, and confidence scores.

[0041] Step S5: Test the trained powdery mildew lesion detection model based on the test set.

[0042] In this embodiment, the testing process is the same as the specific process of step S4 above, except that it is no longer necessary to input CLIP category text features. Instead, the category text features cached after model training can be directly called to participate in inference.

[0043] Step S6: Obtain images of melon leaves to be detected, and use the powdery mildew lesion detection model after training and testing to detect powdery mildew lesions in the images of melon leaves to be detected.

[0044] like Figure 3As shown, in practical applications, the model is in the inference state. Only the image needs to be input, and there is no need to input CLIP category text features. Since the CLIP category text features have been cached during the training process, when the category text features are needed, the cached category text features after training can be directly called to participate in the inference, and the powdery mildew lesion detection results corresponding to the image can be output.

[0045] In addition, the following content is specifically executed in the text label self-suggestion module.

[0046] Text embeddings extracted using the CLIP encoder are used to guide feature refinement, followed by channel-level recalibration to emphasize salient features. Given textual cues describing powdery mildew features (e.g., labels and location information of white powdery spots on plant leaves), semantic embeddings are generated using the CLIP encoder. This indicates an input text prompt, and the CLIP encoder is parameterized as follows: Project it into a fixed-dimensional embedding space.

[0047] .

[0048] Among them, dimension (Applicable to CLIP ViT-B / 32), this guide embeds domain-specific knowledge of coding. This is used to guide visual feature processing. For the input feature map from the YOLO Backbone... (Number of channels C, height H, width W), flatten it into a token sequence. (sequence length) Then through a linear layer. and (dimension) )Will and Project onto the shared space and calculate the similarity matrix S.

[0049] .

[0050] Then, feature refinement is performed: .

[0051] in, This represents element-wise multiplication, generating the features after attention. To further enhance channel-level saliency, the Squeeze-and-Excitation module is applied to compress spatial information through average pooling: ( (for pixels in the image), and is excited through two fully connected layers (reduction ratio r=8): linear layer linear layer , For ReLU, It is sigmoid. The refined characteristic is... .

[0052] In the soft hypergraph inference module, SoftHGNNs are used to model the high-order semantic relationships between visual features, overcoming the limitations of traditional pairwise attention mechanisms. This module includes the definition, construction, and specific operations of soft hypergraphs.

[0053] (1) Definition of a hypergraph: A soft hypergraph extends the traditional hypergraph by allowing continuous and differentiable vertices to hyperedges, capturing detailed higher-order relationships. Formally, a soft hypergraph... The definition is as follows.

[0054] .

[0055] in, For vertex feature matrix (visual token, , ), To participate in the matrix, This represents the number of superedges (M=8). Each element... Represents vertices Participating in the super-border The degree of ambiguity supports flexible modeling of semantic ambiguity, such as partially occluded lesion areas.

[0056] (2) Construction of hypergraph: soft hyperedge construction is based on dynamic generation of participation matrix based on input features. For feature maps flattened Initialize a learnable global prototype The prototype is adjusted to fit each sample using context-aware offset. Global context is extracted through averaging and max pooling.

[0057] .

[0058] Connection as and by linear layers deal with.

[0059] .

[0060] The dynamic prototype is: .

[0061] Vertex feature preprojection: And divided into h=8 heads: Similar processing The similarity for each head is calculated as follows.

[0062] .

[0063] Cross-head average: .

[0064] The participation matrix is ​​as follows: .

[0065] (3) Specific operations of the hypergraph within the module: The soft hypergraph captures higher-order interactions through message passing, from... Start, application Convolution expands the channels ( ), and divided into three parts.

[0066] .

[0067] Among them, chunk This indicates a block operation, followed by two SoftHGNN branches for processing. For each branch, message passing aggregates vertex features into a hyperedge.

[0068] .

[0069] in, The intermediate dimension of the hypergraph output. , For GELU. Features are propagated back to the vertices.

[0070] .

[0071] in, And apply a dropout rate of 0.1. This generates the first and second branches respectively. and Subsequently, the Bottleneck block is processed. (Initially) ), iteration Second-rate.

[0072] .

[0073] Each Bottleneck is processed using a 3×3 convolution with residual connections. The feature list is updated as follows.

[0074] .

[0075] Finally, the features are connected along the channels and fused through a 1×1 convolution.

[0076] .

[0077] By leveraging soft hypergraphs to enhance feature representations, we can improve the detection performance of small-scale powdery mildew lesions by modeling complex semantic dependencies.

[0078] Finally, to verify the effectiveness of the above-mentioned method for detecting powdery mildew spots in melons by combining text label self-prompting and soft hypergraph inference, this embodiment also conducted comparative experiments using other methods. The specific experimental results are shown in Table 1 below.

[0079] Table 1 Comparison of Experimental Results for Different Detection Methods

[0080]

[0081] In another exemplary embodiment, a computer system is provided, which may be a server or a terminal, and its internal structure diagram may be as follows: Figure 4 As shown, the computer system includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned method for detecting powdery mildew spots in melons, combining text label self-prompting and soft hypergraph inference.

[0082] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer system to which the present application is applied. A specific computer system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0083] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0084] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0085] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0086] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0087] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting powdery mildew lesions in melons by combining text label self-prompting and soft hypergraph inference, characterized in that, The method for detecting powdery mildew lesions in melons by combining text label self-prompting and soft hypergraph inference includes: Obtain a sample dataset; the sample dataset includes leaf sample images of different melons with powdery mildew and corresponding text label information; the text label information includes at least manually annotated category labels of powdery mildew lesions on the leaf sample images of melons with powdery mildew; The sample dataset is randomly divided into a training set and a test set, and data augmentation is performed on the training set. A powdery mildew lesion detection model is constructed; the powdery mildew lesion detection model includes a text label self-prompting module and a soft hypergraph inference module; the text label self-prompting module includes at least a YOLO backbone network and a cross-modal interaction module. Based on the enhanced training set, the category labels are input into the frozen CLIP encoder to obtain text features. Images of melon powdery mildew leaf samples are input into the YOLO backbone network to extract multi-scale raw feature maps. For each scale of the raw feature map, the following operations are performed: cross-modal attention interaction and fusion, cosine similarity calculation, and channel-level feature enhancement are performed on the raw feature map and the text features using the cross-modal interaction module to obtain an enhanced feature map; the enhanced feature map is input into the soft hypergraph inference module in two paths, and a soft hypergraph feature map is formed through soft hyperedge generation and global information preservation; the soft hypergraph feature map is then input into two parallel detection heads for prediction. Based on the test set, the trained powdery mildew lesion detection model was tested. Acquire images of melon leaves to be detected, and use the powdery mildew lesion detection model after training and testing to detect powdery mildew lesions in the images of the melon leaves to be detected.

2. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference as described in claim 1, characterized in that, The cross-modal interaction module includes a cross-modal attention module, a cue score graph calculation module, and an SE attention module; the soft hypergraph inference module includes a bi-branch soft hypergraph neural network, a Top-k module, a convergence propagation module, a contrast detection head, and a bounding box detection head.

3. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference as described in claim 2, is characterized in that, The cross-modal interaction module is used to perform cross-modal attention interaction and fusion, cosine similarity calculation, and channel-level feature enhancement on the original feature map and the text features to obtain an enhanced feature map, specifically including: The original feature map and the text features are input into the cross-modal attention module; The original feature map is subjected to adaptive max pooling downsampling to obtain the pooled image features; The text features are projected and transformed into query vectors, and the pooled image features are projected into key vectors and value vectors respectively. A multi-head attention mechanism is used to calculate the similarity between the query vector and the key vector, and the attention weights are obtained after scaling and normalization. The value vector is weighted and aggregated using the attention weights, and the enhanced text features that fuse image information are output through residual connections. The original feature map and the enhanced text features are input into the prompt score map calculation module; Calculate the cosine similarity between each spatial location in the original feature map and the enhanced text feature, and take the maximum similarity value among all categories and generate an attention mask after processing with an activation function; The attention mask is multiplied element-wise with the original feature map to obtain multimodal features guided by text semantics; The multimodal features are input into the SE attention module for channel-level feature enhancement to obtain an enhanced feature map.

4. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference as described in claim 2, characterized in that, The enhanced feature map is input into the soft hypergraph inference module in two paths. Through soft hyperedge generation and global information preservation, a rich soft hypergraph feature map is formed, specifically including: The enhanced feature map is copied into two paths. One enhanced feature map is flattened into a node feature representation and input into the dual-branch soft hypergraph neural network for high-order relation modeling. The other enhanced feature map is directly input into the aggregation and propagation module to retain global information. Using the filtering strategy of the Top-k module, only the nodes with the highest activation scores for each superedge are retained, and the remaining connections are set to zero to form a sparse structure; The aggregation and propagation module is used to perform two-stage information propagation, and the aggregated features output by the two soft hypergraph branches are reshaped, spliced ​​and fused with another enhanced feature map, and integrated with the channel dimension to obtain a soft hypergraph feature map containing rich high-order relation information; the two-stage information propagation refers to the aggregation from node to hyperedge and the propagation from hyperedge to node.

5. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference according to claim 4, characterized in that, In each soft hypergraph branch of the dual-branch soft hypergraph neural network, the following operations are performed: Global context information of the entire enhanced feature map is extracted by average pooling and max pooling operations respectively. The two context vectors are concatenated and then passed through a multilayer perceptron network to generate a set of adaptive hyperedge prototype vectors. The similarity between each node feature and all hyperedge prototypes is calculated, and the average is taken after parallel computation through a multi-head mechanism. Then, the result is normalized to obtain the soft assignment probability matrix of the node to the hyperedge.

6. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference according to claim 2, characterized in that, The soft hypermap feature maps determined based on the original feature maps at different scales are input into the parallel contrast detection head and bounding box detection head respectively to obtain the prediction results of different scale layers. After the prediction results of all scale layers are processed by coordinate unification, multi-scale fusion and non-maximum suppression, the final output includes the detection results containing bounding box coordinates, class labels and confidence scores.

7. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference as described in claim 1, characterized in that, The powdery mildew lesion detection model uses the frozen CLIP encoder to encode the category labels to obtain text features during training; the powdery mildew lesion detection model directly calls the category text features cached after training during testing and actual application.

8. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference according to claim 1, characterized in that, The frozen CLIP encoder uses the ViT architecture and directly loads pre-trained weights from openAI.

9. The method for detecting powdery mildew spots in melons by combining text tag self-prompting and soft hypergraph inference as described in claim 1, characterized in that, The YOLO backbone network adopts the YOLOV8x backbone network; the multi-scale original feature maps correspond to the visual representations of the shallow, medium and deep layers, respectively.

10. A computer system, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method for detecting powdery mildew spots in melons combining text label self-prompting and soft hypergraph reasoning as described in any one of claims 1-9.

Images