Tropical fruit tree disease intelligent diagnosis method and system fused with multi-modal image

Abstract

The invention is suitable for the technical field of image recognition and intelligent diagnosis, and provides a tropical fruit tree disease intelligent diagnosis method and system fused with a multi-modal image, and the method comprises the steps: if the deviation index of a first response curve decreases with the increase of the coverage degree, and a second response curve is basically kept stable, judging that the disease of a tropical fruit tree is not damaged; if yes, it is judged that current film layer coverage has the optical diffusion improvement effect, and a correction coefficient is generated according to the deviation condition of the two response curves so as to correct the initial confidence coefficient parameter. According to the method, an optical abnormal region identification and multi-modal image fusion mechanism is introduced, a confidence coefficient adaptive correction model based on film optical characteristic difference is established, and the limitation of'coverage, namely interference 'in the prior art is broken through. By constructing a first response curve and a plurality of groups of second response curves and combining joint calculation of an edge identification stability index and a surface reflection consistency index, quantitative judgment of the optical diffusion improvement effect of the film layer and dynamic correction of confidence parameters are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition and intelligent diagnosis, in particular to a method and system for intelligent diagnosis of tropical fruit tree diseases by fusing multi-modal images. BACKGROUND

[0002] In the existing field of intelligent identification of fruit tree diseases and pests, automatic detection technology based on image recognition has gradually become an important means of orchard management and disease monitoring. Common technical solutions rely on the color, texture, shape and other characteristics of visible light images to achieve automatic identification of fruit surface lesions, insect damage or diseased areas through convolutional neural networks or fusion algorithms. However, in actual environments, factors such as natural reflection, humidity changes and uneven lighting on the surface of the fruit often cause local bright spots, blurring or overexposure of high light areas in the image, affecting the stability of feature extraction and model judgment. Therefore, existing recognition systems generally use confidence parameters to measure the reliability of the recognition results, and perform post-recognition when the confidence is below a threshold to reduce the risk of misjudgment.

[0003] However, for tropical fruits with thick wax layers and high curvature surfaces (such as mangoes), existing recognition algorithms often simply treat the fruit surface covering film layer or reflective area as an interference source, directly reducing the recognition confidence when such areas are detected, resulting in some normal areas being misjudged as abnormal. This processing method ignores the optical difference of the covering film layer and fails to distinguish the essential difference between the film layer with optical diffusion characteristics and the non-diffusion type film layer. In fact, some film layers with surface light diffusion characteristics (such as pesticide spray residual film, moisturizing agent film containing particles) form an optical diffusion coupling effect with the mango wax layer, which can improve the light distribution and enhance the edge features, thereby improving the recognition stability.

[0004] However, existing technologies lack research and quantitative modeling of this optical diffusion improvement effect, which cannot reflect its actual positive impact on the confidence parameter in the recognition model, resulting in a decrease in recognition accuracy of the system under complex lighting and film layer interference conditions. SUMMARY

[0005] The present application aims to provide a method and system for intelligent diagnosis of tropical fruit tree diseases by fusing multi-modal images, which aims to solve the problems raised in the background art.

[0006] The present application is implemented as follows: a method for intelligent diagnosis of tropical fruit tree diseases by fusing multi-modal images, the method comprising: When it is detected that the local optical abnormal area in the mango fruit image has a film layer covering with surface light diffusion characteristics, the initial confidence parameter of the optical abnormal area is obtained, and the recognition data set is called; Based on the identification dataset, a plurality of first samples with different film layer coverage degrees consistent with the film layer type, environmental conditions and surface curvature parameters of the optical abnormal area, and a plurality of second samples without surface light diffusion characteristics, but with consistent environmental conditions and surface curvature parameters but different film layer coverage degrees are extracted; A posteriori identification deviation index of each sample is calculated, which is determined by the combination of the edge identification stability index and the surface reflection consistency index; The deviation indexes of the first sample group and the second sample group are sorted according to the film layer coverage degree, and a first response curve and a second response curve are respectively generated; If the deviation index of the first response curve decreases with the increase of the coverage degree, and the second response curve remains basically stable, a correction coefficient is generated according to the deviation of the two response curves to correct the initial confidence parameter.

[0007] As a further limitation of the technical scheme of the embodiment of the application, the step of determining whether the film layer coverage and the film layer type exist in the optical abnormal area is performed by identifying the spectral characteristics and reflection distribution of the non-optical abnormal area around the optical abnormal area.

[0008] As a further limitation of the technical scheme of the embodiment of the application, the consistent environmental conditions specifically include that the differences between the first sample, the second sample and the current mango fruit image in terms of light intensity, incident angle and background brightness distribution are within a preset tolerance range.

[0009] As a further limitation of the technical scheme of the embodiment of the application, the film layer with surface light diffusion characteristics refers to a film layer with a fine particle structure or a milky distribution form on the surface of the mango fruit, which has a heterogeneous reflection interface, including pesticide spraying residual film and moisturizing agent film containing suspended particles; the film layer without surface light diffusion characteristics refers to a film layer with uniform and smooth surface morphology, which is mainly attached to the surface of the fruit in the form of liquid or smooth film, including rainwater thin layer, water vapor condensation film and transparent smooth adhesive film.

[0010] As a further limitation of the technical scheme of the embodiment of the application, the acquisition of the edge identification stability index specifically includes: based on the identification dataset, the edge identification result of each first sample or second sample in the posteriori identification stage is obtained, and the difference degree of the edge region and its adjacent region in the pixel gradient direction, brightness distribution and morphological continuity is analyzed according to the edge identification result, and the difference degrees are comprehensively taken as the edge identification stability index.

[0011] As a further limitation of the technical scheme of the embodiment of the present application, the obtaining of the surface reflection consistency index specifically comprises: obtaining surface reflection information of each first sample or second sample in the post-recognition stage based on the recognition data set, comparing and analyzing the reflection intensity distribution, color saturation and brightness gradient of the film layer covered area and the uncovered area in the first sample or the second sample, calculating the reflection difference degree, and comprehensively taking the reflection difference degrees as the surface reflection consistency index.

[0012] As a further limitation of the technical scheme of the embodiment of the present application, if the offset index of the first response curve decreases with the increase of the coverage degree, and the second response curve basically remains stable, the step of generating a correction coefficient to correct the initial confidence parameter according to the deviation of the two response curves comprises: If the offset index of the first response curve decreases with the increase of the coverage degree, and the second response curve basically remains stable, it is determined that the current film layer coverage has an optical diffusion improvement effect; determining the offset index corresponding to the film layer coverage degree of the optical abnormal area in the currently detected mango fruit image in the first response curve, and obtaining the offset index corresponding to the same coverage degree in the second response curve; calculating the deviation degree of the offset index corresponding to the second response curve relative to the offset index corresponding to the first response curve, and generating a correction coefficient by combining the deviation degree with a preset correction intensity coefficient; Based on the correction coefficient, the initial confidence parameter is corrected to reflect the positive influence of the film layer optical diffusion improvement effect on the recognition reliability.

[0013] A tropical fruit tree disease intelligent diagnosis system fusing multi-modal images, the system comprises: An image detection module is configured to obtain an initial confidence parameter of a local optical abnormal area in a mango fruit image when it is detected that the local optical abnormal area has a film layer coverage with surface reflection light diffusion characteristics, and to call a recognition data set; A sample extraction module is configured to extract, based on the recognition data set, a plurality of first samples having the same film layer type, environmental condition and surface curvature parameter but different film layer coverage degrees, and a plurality of second samples not having the surface reflection light diffusion characteristics but having the same environmental condition and surface curvature parameter but different film layer coverage degrees; A feature calculation module is configured to calculate a post-recognition offset index of each sample, the offset index being determined by a combination of an edge recognition stability index and a surface reflection consistency index; A curve generation module is configured to sort the offset indexes of the first sample group and the second sample group according to the film layer coverage degree, and to generate a first response curve and a second response curve, respectively; The confidence correction module is configured to generate a correction coefficient according to the deviation of the two response curves, to correct the initial confidence parameter, if the offset index of the first response curve decreases with the increase of the coverage degree, and the second response curve remains basically stable.

[0014] As a further limitation of the technical scheme of the embodiment of the present application, the image detection module is further configured to identify the spectral features and reflection distribution of the non-optical abnormal area around the optical abnormal area, to determine whether the optical abnormal area exists film layer coverage and the film layer type.

[0015] As a further limitation of the technical scheme of the embodiment of the present application, the consistent environmental conditions specifically include that the differences of the first sample and the second sample and the current mango fruit image in the light intensity, the incident angle and the background brightness distribution are within the preset tolerance range.

[0016] Compared with the prior art, the present application has the following beneficial effects: The present application introduces optical abnormal area recognition and multi-modal image fusion mechanism, establishes a confidence adaptive correction model based on the difference of film layer optical characteristics, breaks through the limitation of "coverage is interference" in the prior art. By constructing the first response curve and multiple second response curves, and combining the joint calculation of the edge recognition stability index and the surface reflection consistency index, the quantitative determination of the film layer optical diffusion improvement effect and the dynamic correction of the confidence parameter are realized. The present application can distinguish the systematic difference in recognition performance between the film layer with surface reflection diffusion characteristics and the non-diffusion type film layer, so as to maintain high recognition stability and accuracy in complex light or high curvature fruit surface environment. The method significantly improves the robustness and adaptive ability of the tropical fruit tree disease diagnosis system under natural light interference conditions, and has wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The flowchart of the method provided by the embodiment of the present application is shown; Figure 2 The flowchart of correcting the initial confidence parameter in the method provided by the embodiment of the present application is shown; Figure 3 The application architecture diagram of the system provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0019] Figure 1 The flowchart of the method provided by the embodiment of the present application is shown.

[0020] Specifically, the application discloses a method for intelligent diagnosis of diseases of tropical fruit trees by fusing multi-modal images, and the method specifically comprises the following steps. In step S100, when it is detected that the local optical abnormal area in the mango fruit image is covered by a film layer with surface reflection diffusion characteristics, the initial confidence parameter of the optical abnormal area is obtained, and a recognition data set is called.

[0021] The step of determining whether the optical abnormal area is covered by a film layer and the type of the film layer is performed by identifying the spectral characteristics and reflection distribution of the non-optical abnormal area around the optical abnormal area.

[0022] The film layer with surface reflection diffusion characteristics refers to a film layer with a fine particle structure or a milky distribution form on the surface of the mango fruit, which has a heterogeneous reflection interface, including pesticide residual film and moisturizing agent film containing suspended particles; the film layer without surface reflection diffusion characteristics refers to a film layer with a uniform and smooth surface, which is mainly attached to the surface of the fruit in a liquid or smooth film state, including a rainwater thin layer, a water vapor condensation film and a transparent smooth adhesive film.

[0023] In the embodiment of the application, during the image monitoring of diseases and pests, especially for mango fruits, due to the relatively thick natural wax layer on the surface and the large spherical curvature, when the external light is strong, the incident angle is large or the environmental humidity is high, a local optical overexposure or reflection blur area is easily formed in the image. The occurrence of such an area is usually related to factors such as light intensity, imaging angle, surface humidity and environmental scattered light, and mainly shows brightness saturation, loss of texture details or edge blur, which often occurs at positions with high curvature or concentrated reflection of the fruit.

[0024] The local optical abnormal area refers to an area in which the image details are damaged due to the reflection light intensity exceeding the dynamic range of the imaging system. In order to improve the recognition accuracy, the system generates an initial confidence parameter for each local area, which represents the reliability of the disease recognition result; when the confidence is higher than a preset threshold, the system directly outputs the recognition result, and when the confidence is lower than the threshold, the system triggers the posterior recognition. The initial confidence parameter is calculated by the recognition model before correction, which comprehensively considers local texture clarity, brightness gradient, color saturation and contrast, and is used to reflect the preliminary reliability of the recognition result of the area.

[0025] It is found through research that the existing recognition algorithm has certain deviation when processing mango fruits. When detecting that there is a covering on the surface of the fruit, the system usually considers the covering as optical interference, thereby automatically reducing the confidence parameter of the region, without distinguishing the optical property difference of the covering. In fact, for a film layer with surface light diffusion characteristics (such as pesticide spraying residual film or moisturizing agent film containing suspended particles), an optical diffuse coupling effect is formed between the film layer and the thick wax layer on the surface of the mango, which causes the reflected light to be scattered in multiple directions on the surface, the light intensity distribution is more uniform, the brightness balance of the local light reflection area is improved, and the edge texture is clearer, thereby visually improving the stable perception ability of the recognition model to the fruit surface disease spot or insect damage texture. The diffuse reflection effect produced by the film layer and the wax layer in cooperation has a positive recognition improvement effect under certain conditions, but since the existing algorithm only uses the presence of a covering as a negative criterion, the system confidence evaluation fails to reflect this actual improvement effect.

[0026] The recognition data set is used to provide contrast samples and statistical basis for confidence correction, which can be derived from artificially labeled orchard samples, automatically collected images of monitoring equipment, or historical samples in the agricultural image database. The recognition data set should include multi-modal image data of mango fruits collected under different light conditions, different film layer coverage levels, and different surface curvature conditions, and is accompanied by film layer type information, film layer spectral reflection characteristic parameters, disease area annotation results, and local optical anomaly area distribution annotation of each sample. Preferably, the data set contains visible light, near-infrared, and polarized reflection images to realize the differentiation and quantitative analysis of the optical diffusion characteristics and reflection consistency of the film layer, thereby providing physical and statistical double support for subsequent confidence correction.

[0027] Further, the intelligent diagnosis method for tropical fruit tree diseases based on fusion of multi-modal images further comprises the following steps: Step S200, based on the recognition data set, a plurality of first samples with the same film layer type, environmental conditions and surface curvature parameters but different film layer coverage levels, and a plurality of second samples without surface light diffusion characteristics, but with the same environmental conditions and surface curvature parameters but different film layer coverage levels are extracted.

[0028] The consistent environmental conditions specifically include that the differences in light intensity, incident angle and background brightness distribution between the first sample and the second sample and the current mango fruit image are within a preset tolerance range.

[0029] In the embodiment of the present application, in order to correct the confidence of the optical abnormal area recognition result, the system needs to extract two types of comparable sample groups based on the recognition data set. By establishing strict screening conditions, it is ensured that the comparison results can accurately reflect the influence of different film layer optical properties on the recognition performance. Specifically, the first sample is a sample with the same film layer type, environmental condition and surface curvature parameter as the optical abnormal area but different film layer coverage, which is used to reflect the recognition performance of the film layer with surface light diffusion characteristics under different coverage; the second sample is a sample without surface light diffusion characteristics, with the same environmental condition and surface curvature parameter but different film layer coverage, which is used as a reference comparison group.

[0030] By keeping the environmental conditions consistent, the influence of factors such as light intensity, incident angle, background brightness distribution, etc. on image recognition performance can be effectively excluded, so that the change of the offset index is only related to the optical behavior of the film layer. In addition, consistent environmental conditions can further include imaging system parameters (such as exposure time, sensor sensitivity) and surrounding scattered light intensity, etc. These factors will affect the reflection characteristics of light on the fruit surface.

[0031] During the data set construction process, it is of great significance to ensure sufficient sample quantity. Only on the basis of a large enough sample, can the correlation trend between film layer optical properties and recognition offset be accurately identified through statistical rules, avoiding curve deviation or unstable results caused by insufficient sample quantity.

[0032] It is worth noting that the selection of the second sample can be divided into multiple different groups, each group corresponding to a film layer type without surface light diffusion characteristics, such as a thin layer of rainwater, a water vapor condensation film or a transparent adhesive film. The purpose of this grouping method is to enable the system to establish corresponding control curves for different non-diffusion type film layers, avoiding offset misjudgment caused by differences in film layer material or morphology.

[0033] The core difference between the first sample and the second sample lies in the optical reflection characteristics of the film layer. The film layer corresponding to the first sample has light diffusion characteristics, and its interaction with the thick wax layer of the mango fruit surface may produce optical diffusion effect, thereby improving the recognition clarity; while the film layer corresponding to the second sample is smooth and concentrated in reflection, without such diffusion characteristics, so it will not produce the same positive optical improvement effect in the recognition process.

[0034] Further, the intelligent diagnosis method for tropical fruit tree diseases by fusing multi-modal images further comprises the following steps: Step S300, calculating the posterior recognition offset index of each sample, which is determined by the combination of the edge recognition stability index and the surface reflection consistency index.

[0035] The acquisition of the edge recognition stability index specifically includes: obtaining the edge recognition results of each first or second sample in the posterior recognition stage based on the recognition dataset, analyzing the degree of difference between the edge region and its neighboring regions in terms of pixel gradient direction, brightness distribution and morphological continuity based on the edge recognition results, and combining these degree of difference as the edge recognition stability index.

[0036] The acquisition of the surface reflection consistency index specifically includes: obtaining the surface reflection information of each first sample or second sample in the posterior recognition stage based on the recognition dataset; comparing and analyzing the reflection intensity distribution, color saturation, and brightness gradient of the film-covered and uncovered areas in the first sample or second sample; calculating their reflection difference; and combining these reflection differences as the surface reflection consistency index.

[0037] In this embodiment of the invention, step S300 is one of the core steps of the entire diagnostic method. By calculating the posterior identification offset index of each sample, a multi-dimensional evaluation and correction of the identification results of optically abnormal areas is achieved. This offset index is composed of an edge identification stability index and a surface reflection consistency index, both of which are closely related to the physical manifestation of "potential optical diffusion effects." Since the generation of optical diffusion effects in mango fruit surface image recognition depends on complex reflection conditions and film optical behavior, its formation process has strong randomness and multi-factor coupling characteristics. Therefore, directly judging with a single index often fails to reflect the actual identification offset. This invention indirectly reflects the existence and degree of optical diffusion improvement effect by integrating two representative posterior factors, thereby significantly improving the model's fault tolerance and judgment accuracy against complex optical interference.

[0038] The edge recognition stability index measures the robustness of a recognition system when processing sample edge features. It is obtained by acquiring the edge recognition results of each first or second sample in the posterior recognition stage based on the recognition dataset. The index is then calculated by averaging the differences between the edge region and its neighboring regions in pixel gradient direction, brightness distribution, and morphological coherence, resulting in a weighted index. This index reflects the system's noise sensitivity and feature consistency during edge region recognition and is a commonly used analytical method in existing image recognition technologies for evaluating the robustness of target edges.

[0039] The surface reflectance uniformity index is used to evaluate the uniformity of optical reflectance characteristics in a film-covered area. It is obtained by acquiring surface reflectance information for each first or second sample in the posterior identification stage based on the identification dataset, comparing the reflectance intensity distribution, color saturation, and brightness gradient between the film-covered and uncovered areas, calculating the reflectance difference, and using the magnitude of the overall difference to characterize reflectance uniformity. This index reflects the degree of difference in light distribution across different areas of the surface and is a quantitative method used in existing image reflectance analysis techniques to describe reflectance uniformity.

[0040] In this invention, the posterior recognition offset index, constructed by combining the edge recognition stability index and the surface reflectance consistency index, not only reflects the comprehensive influence of the film's optical properties on the image recognition process but also identifies potential optical diffusion improvement effects. When optical diffusion exists, the edge recognition stability of the sample is enhanced, and the reflectance consistency is improved, leading to an overall decrease in the offset index. Conversely, when there is no diffusion effect or the covering is a non-diffuse film, the offset index remains stable or fluctuates insignificantly. This trend provides a reliable quantitative basis for subsequent response curve analysis and confidence level correction.

[0041] Furthermore, it should be noted that the thickness of the film layer has an effective range in optical performance. When the film layer is too thin, its scattering effect on light is insufficient, and it cannot form obvious reflection modulation; when the film layer is too thick, the light penetration ability is reduced, and specular reflection or local occlusion is prone to occur, resulting in unstable recognition results. Usually, the film layer thickness formed on the surface of mango fruit under the actual planting and harvesting environment is mostly within the equilibrium range of optical reflection, that is, within this range, the thickness change will not significantly affect the reflection distribution characteristics. Therefore, the offset index change curve of the film layer without surface reflective diffusion characteristics under this condition usually shows a stable trend, while the film layer with reflective diffusion characteristics will show obvious response changes with the degree of coverage.

[0042] Furthermore, the intelligent diagnosis method for tropical fruit tree diseases by fusing multimodal images also includes the following steps: Step S400: Sort the offset indices of the first sample group and the second sample group according to the degree of film coverage, and generate the first response curve and the second response curve respectively.

[0043] In step S500, if the offset index of the first response curve decreases as the coverage increases, while the second response curve remains basically stable, then a correction coefficient is generated based on the deviation of the two response curves to correct the initial confidence parameter.

[0044] Specifically, Figure 2 A flowchart for correcting the initial confidence parameters is shown.

[0045] If the offset index of the first response curve decreases with increasing coverage, while the second response curve remains relatively stable, then a correction coefficient is generated based on the deviation between the two response curves to correct the initial confidence parameter. This process specifically includes the following steps: Step S501: If the offset index of the first response curve decreases with the increase of coverage, while the second response curve remains basically stable, then it is determined that the current film coverage has an optical diffusion improvement effect. Step S502: Determine the offset index corresponding to the film coverage of the optically abnormal region in the currently detected mango fruit image in the first response curve, and obtain the offset index corresponding to the same coverage in the second response curve. Step S503: Calculate the degree of deviation of the offset index corresponding to the second response curve relative to the offset index corresponding to the first response curve, and combine the degree of deviation with the preset correction intensity coefficient to generate a correction coefficient; Step S504: Based on the correction coefficient, the initial confidence parameter is improved and corrected to reflect the positive impact of the film optical diffusion improvement effect on the recognition reliability.

[0046] In this embodiment of the invention, step S400 aims to visualize and analyze the identification offset characteristics among different samples, so as to scientifically correct the confidence parameters subsequently. Specifically, after calculating the posterior identification offset index of each sample, the system first sorts all samples in order of increasing film coverage. Then, it plots the offset index change curves for the first sample group with surface reflective diffusion characteristics and the second sample group without these characteristics, thereby generating a first response curve and a second response curve. During implementation, multidimensional data regression or polynomial smoothing algorithms can be used to eliminate discrete fluctuations in single samples, allowing the curves to more accurately reflect the overall trend. In this way, the system can intuitively present the functional relationship between film coverage and identification offset, providing a quantitative basis for subsequent confidence correction.

[0047] The verification basis of step S501 echoes that of step S100 above, both being based on the identification results of the film's optical properties in the optically anomalous region. When the system detects a film with surface reflective diffusion characteristics on the surface of the optically anomalous region, it indicates that the region possesses the potential conditions to form an optical diffusion improvement effect. At this point, by statistically analyzing the changes in identification offset under different film coverage levels, it can be verified whether this potential effect actually exists.

[0048] If, in the response curves generated in step S400, the offset index of the first response curve decreases with increasing coverage, while the second response curve remains stable under multiple groups of different types of non-diffuse films, then this decreasing trend can be considered not to be caused by random factors such as ambient light, curvature changes, or sensing noise, but rather to the optical reflection-diffusion coupling effect between the film layer and the thick waxy layer on the surface of the mango fruit. This coupling effect can create optical diffusion within a certain film thickness range, making the local brightness distribution more uniform and edge recognition more stable, thus exhibiting the characteristic of a decreasing offset index. In other words, the decreasing trend of the first response curve represents enhanced recognition stability, while the stable performance of the second response curve provides a reference for comparison, verifying the authenticity and uniqueness of this improvement effect.

[0049] The multiple sets of second samples set in step S200—sample groups established for different types of films without surface reflective diffusion properties (such as rainwater thin layers, water vapor condensation films, or transparent adhesion films)—play a crucial verification role. By comparing the multiple second response curves generated from the multiple sets of second samples, if these curves remain stable and show no obvious downward trend, it indicates that regardless of the film type, as long as it lacks diffusion properties, the offset index identified by the system is not affected by the coverage degree. This cross-group consistency effectively eliminates other possible interference factors, allowing for a clearer confirmation that the decrease in the offset index observed in the first response curve is indeed due to the improved optical diffusion effect brought about by the film with surface reflective diffusion properties, rather than accidental environmental differences or data shifts.

[0050] In steps S502 to S503, the system finds the corresponding offset index in the first response curve based on the actual coverage of the film layer in the optically anomalous region, and simultaneously extracts the offset index under the same coverage level from the corresponding second response curve. The degree of deviation is obtained by calculating the difference or proportional difference between the two. For example, when the offset index of the second response curve is 20% higher than that of the first response curve, the degree of deviation can be defined as 0.2. The system weights this degree of deviation with a preset correction intensity coefficient (e.g., 0.5) to obtain a correction coefficient (0.1 in this example), and uses this to improve and correct the initial confidence parameter, thereby reflecting the positive contribution of the optical diffusion improvement effect in the recognition result.

[0051] Through the above mechanism, this invention establishes an adaptive confidence correction system based on the difference in response to sample data. Compared with the simple judgment mode of "coverage equals interference" in the prior art, this invention can identify and utilize the image quality improvement effect brought about by the film layer with optical diffusion characteristics to achieve dynamic optimization of the confidence parameter. As a result, the system can maintain the stability and accuracy of recognition when facing mango fruit images under complex optical conditions, significantly reducing the rate of repeated detection or false recognition caused by misjudgment.

[0052] Overall, this invention effectively solves the core research problem raised in step S100—namely, the inability of existing technologies to distinguish the optical properties of covering films, thus directly reducing confidence levels in the identification system. By establishing a comparison model between the first and second response curves and introducing a shift index calculation and confidence level correction mechanism, this invention achieves refined modeling and data-driven correction of the film's optical properties. This enables the system not only to identify interfering coverage but also to determine whether it possesses a positive optical improvement effect, fundamentally improving the identification system's ability to process images of mango fruit diseases and pests.

[0053] In terms of application prospects, this invention has strong practicality and scalability. This method can be widely applied to intelligent disease identification systems for tropical fruit trees, especially suitable for fruits with reflective surfaces, such as mangoes, bananas, and longans. It can effectively improve image recognition stability under complex environments such as strong light, after rain, and humidity. Furthermore, this technology system can be used in conjunction with multimodal image acquisition equipment (including visible light, infrared, and polarization imaging) to further expand its application in scenarios such as automatic agricultural monitoring, grading and detection, and orchard robot visual navigation, providing a scalable and adaptive recognition enhancement solution for intelligent agricultural image analysis.

[0054] Furthermore, Figure 3 An application architecture diagram of the system provided in an embodiment of the present invention is shown.

[0055] In another preferred embodiment of the present invention, a tropical fruit tree disease intelligent diagnosis system integrating multimodal imagery includes: The image detection module 100 is used to obtain the initial confidence parameters of the optically abnormal region and call the recognition dataset when a film layer with surface reflective diffusion characteristics is detected covering a local optically abnormal region in a mango fruit image.

[0056] The image detection module 100 is also configured to determine whether there is a film covering in the optically anomalous region and the type of film covering by identifying the spectral characteristics and reflectance distribution of the non-optically anomalous region surrounding the optically anomalous region.

[0057] Furthermore, the intelligent diagnostic system for tropical fruit tree diseases that integrates multimodal imagery also includes: The sample extraction module 200 is used to extract, based on the identification dataset, several first samples that are consistent with the film type, environmental conditions and surface curvature parameters of the optically anomalous region but have different film coverage, and several second samples that do not have surface reflective diffusion characteristics and are consistent with the environmental conditions and surface curvature parameters but have different film coverage.

[0058] The consistent environmental conditions specifically include: the differences between the first sample and the second sample and the current mango fruit image in terms of light intensity, incident angle and background brightness distribution are all within a preset tolerance range.

[0059] Furthermore, the intelligent diagnostic system for tropical fruit tree diseases that integrates multimodal imagery also includes: The feature calculation module 300 is used to calculate the posterior identification offset index of each sample. The offset index is determined by a combination of the edge recognition stability index and the surface reflection consistency index.

[0060] Furthermore, the intelligent diagnostic system for tropical fruit tree diseases that integrates multimodal imagery also includes: The curve generation module 400 is used to sort the offset indices of the first sample group and the second sample group according to the film coverage, and generate the first response curve and the second response curve respectively.

[0061] Furthermore, the intelligent diagnostic system for tropical fruit tree diseases that integrates multimodal imagery also includes: The confidence correction module 500 is used to generate a correction coefficient based on the deviation of the two response curves if the offset index of the first response curve decreases with the increase of coverage, while the second response curve remains basically stable, so as to correct the initial confidence parameter.

[0062] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent diagnosis of tropical fruit tree diseases by integrating multimodal imagery, characterized in that, The method includes: When a film layer with surface reflective diffusion characteristics is detected covering a local optically anomalous area in a mango fruit image, the initial confidence parameters of the optically anomalous area are obtained, and the recognition dataset is called. Based on the identification dataset, several first samples with the same film type, environmental conditions and surface curvature parameters as the optically anomalous region but different film coverage were extracted, as well as several second samples with the same environmental conditions and surface curvature parameters but different film coverage, which do not have surface reflective diffusion characteristics. Calculate the posterior identification offset index for each sample, wherein the offset index is determined by a combination of the edge identification stability index and the surface reflection consistency index; The offset indices of the first sample group and the second sample group are sorted according to the degree of membrane coverage, and the first response curve and the second response curve are generated respectively. If the deviation index of the first response curve decreases as the coverage increases, while the second response curve remains basically stable, then a correction coefficient is generated based on the deviation between the two response curves to correct the initial confidence parameter.

2. The intelligent diagnostic method for tropical fruit tree diseases based on multimodal image fusion according to claim 1, characterized in that, The step of determining whether an optically anomalous region is covered by a film and the type of film is performed by identifying the spectral characteristics and reflectance distribution of the non-optically anomalous region surrounding the optically anomalous region.

3. The intelligent diagnostic method for tropical fruit tree diseases based on multimodal image fusion according to claim 1, characterized in that, The consistent environmental conditions specifically include: the differences between the first sample and the second sample and the current mango fruit image in terms of light intensity, incident angle and background brightness distribution are all within a preset tolerance range.

4. The intelligent diagnostic method for tropical fruit tree diseases based on multimodal image fusion according to claim 1, characterized in that, The film layer with surface reflective diffusion properties refers to a film layer that forms a fine particulate structure or milky distribution on the surface of mango fruit and has a heterogeneous reflective interface, including pesticide spraying residue film and moisturizing agent film containing suspended particles; the film layer without surface reflective diffusion properties refers to a film layer with a uniform and flat surface that is mainly attached to the surface of the fruit in a liquid or smooth film form, including rainwater thin layer, water vapor condensation film and transparent smooth attachment film.

5. The intelligent diagnostic method for tropical fruit tree diseases based on multimodal image fusion according to claim 1, characterized in that, The acquisition of the edge recognition stability index specifically includes: obtaining the edge recognition results of each first or second sample in the posterior recognition stage based on the recognition dataset, analyzing the degree of difference between the edge region and its neighboring regions in terms of pixel gradient direction, brightness distribution and morphological continuity based on the edge recognition results, and combining these degree of difference as the edge recognition stability index.

6. The intelligent diagnostic method for tropical fruit tree diseases based on multimodal image fusion according to claim 5, characterized in that, The acquisition of the surface reflection consistency index specifically includes: obtaining the surface reflection information of each first sample or second sample in the posterior recognition stage based on the recognition dataset; comparing and analyzing the reflection intensity distribution, color saturation, and brightness gradient of the film-covered and uncovered areas in the first sample or second sample; calculating their reflection difference; and combining these reflection differences as the surface reflection consistency index.

7. The intelligent diagnostic method for tropical fruit tree diseases based on multimodal image fusion according to claim 6, characterized in that, If the offset exponent of the first response curve decreases with increasing coverage, while the second response curve remains relatively stable, the steps to generate correction coefficients based on the deviation between the two response curves to correct the initial confidence parameters include: If the offset index of the first response curve decreases with increasing coverage, while the second response curve remains basically stable, then the current film coverage is determined to have an optical diffusion improvement effect. In the first response curve, determine the offset index corresponding to the film coverage of the optically anomalous region in the currently detected mango fruit image, and obtain the offset index corresponding to the same coverage in the second response curve. Calculate the degree of deviation of the offset index corresponding to the second response curve relative to the offset index corresponding to the first response curve, and combine the degree of deviation with a preset correction intensity coefficient to generate a correction coefficient; The initial confidence parameter is improved and corrected based on the correction coefficient to reflect the positive impact of the film's optical diffusion improvement effect on recognition reliability.

8. A smart diagnostic system for tropical fruit tree diseases integrating multimodal imagery, characterized in that, The system includes: The image detection module is used to obtain the initial confidence parameters of the optically anomalous region when a film layer with surface reflective diffusion characteristics is detected in a local optically anomalous region of a mango fruit image, and to call the recognition dataset. The sample extraction module is used to extract, based on the identification dataset, several first samples that are consistent with the film type, environmental conditions and surface curvature parameters of the optically anomalous region but have different film coverage, and several second samples that do not have surface reflective diffusion characteristics and are consistent with the environmental conditions and surface curvature parameters but have different film coverage. The feature calculation module is used to calculate the posterior identification offset index of each sample, which is determined by a combination of the edge recognition stability index and the surface reflection consistency index. The curve generation module is used to sort the offset indices of the first sample group and the second sample group according to the film coverage, and generate the first response curve and the second response curve respectively. The confidence correction module is used to generate correction coefficients based on the deviation of the two response curves if the offset index of the first response curve decreases with the increase of coverage, while the second response curve remains basically stable, so as to correct the initial confidence parameters.

9. The intelligent diagnostic system for tropical fruit tree diseases based on multimodal imagery as described in claim 8, characterized in that, The image detection module is also configured to determine whether there is a film covering in the optically anomalous region and the type of film covering by identifying the spectral characteristics and reflectance distribution of the non-optically anomalous region surrounding the optically anomalous region.

10. The intelligent diagnostic system for tropical fruit tree diseases based on multimodal image fusion according to claim 9, characterized in that, The consistent environmental conditions specifically include: the differences between the first sample and the second sample and the current mango fruit image in terms of light intensity, incident angle and background brightness distribution are all within a preset tolerance range.

Images