Early detection method of internal lesion of mango based on thermal imaging and texture analysis

Abstract

The application belongs to the technical field of data processing and machine vision, and provides a mango internal lesion early detection method based on thermal imaging and texture analysis, specifically comprising the following steps: first, collecting mango thermal infrared images from a conveying belt and performing pretreatment to obtain target images; inputting the target images into a pre-trained infrared texture mapping model to obtain a mapping vector; then dynamically determining a current mango lesion detection result through the mapping vector; and finally determining whether the mango has an internal lesion risk according to the lesion detection result. The synergy degree or mathematical commonality degree of each texture index is quantified to determine whether the mango has a lesion risk, the judgment timing of the mango with a lesion risk is greatly advanced, the mango with a lesion risk is prevented from participating in logistics or cold chain transportation, the risk of the mango in the same batch being infected by a lesion is reduced, especially in the process with a long logistics cycle, the logistics efficiency can be effectively improved and the cost loss of ineffective transportation can be reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing and machine vision, and particularly relates to a mango internal lesion early detection method based on thermal imaging and texture analysis. BACKGROUND

[0002] Mangoes are prone to internal lesions caused by pathogenic bacteria during transportation and storage. Common lesion characteristics include black spots, softening or rotting, etc. Such lesions often need to be identified by machine vision technology in the later stage and cannot be identified before the visual features change. The mango monitoring methods popular on the market at present are based on high-resolution image acquisition and color feature analysis, and identify lesions by image feedback of color changes, gloss changes and lesion spread on the surface of mangoes. It is known that internal diseases of mangoes are hidden cost problems with high loss rate but not easy to detect in advance. The proportion of occurrence is usually not high, but once internal lesions occur during transportation, it will affect the rotting of the whole box of fruits, causing huge loss of unit value. Especially in the fruit and vegetable export logistics phenomenon in tropical regions, the case of whole batch return due to the inability to sort internal disease fruits is common. The improper temperature control within ten days after mango picking into the logistics process is the opportunity for pathogens to penetrate from the wound or fruit stem, causing internal diseases such as black spots and mold and spreading. Early detection by manual vision is difficult. Therefore, the mango internal lesion early detection method based on thermal imaging and texture analysis is proposed, which reflects the micro changes of the internal organization of mangoes in the early stage of lesion through the mapping model of the changes of thermal conductivity characteristics and the changes of tissue texture parameters. This method not only makes up for the defects of traditional visual detection that cannot identify internal lesions, but also can quickly screen at the early stage of fruit picking or storage, reducing transportation and storage costs. SUMMARY

[0003] The purpose of the present application is to provide a mango internal lesion early detection method based on thermal imaging and texture analysis to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a mango internal lesion early detection method based on thermal imaging and texture analysis is provided, which comprises the following steps:

[0005] S100, collecting a mango thermal infrared image from a conveyor belt and preprocessing to obtain a target image;

[0006] S200, inputting the target image into a pre-trained infrared texture mapping model to obtain a mapping vector;

[0007] S300, dynamically determining the lesion detection result of the current mango through the mapping vector.

[0008] S400, judging whether the mango has internal lesion risk according to the lesion detection result.

[0009] Further, in step S100, the method for collecting the mango thermal infrared image from the conveying belt and pre-processing to obtain the target image is: the mango to be packaged is placed in a room temperature constant for 2-4 hours, then conveyed by the conveying belt for packaging, a thermal imager is arranged on the conveying belt to collect the thermal infrared image of the conveying belt in real time; the target image is obtained by recognizing the mango in the thermal infrared image through a target detection algorithm or a thermal threshold segmentation algorithm.

[0010] The thermal imager is an infrared thermal imager, and the thermal sensitivity requirement is NETD≤50mK.

[0011] The reason for the room temperature constant for 2-4 hours is that the mango needs to enter the spontaneous heating distribution stable period after being picked, and too early thermal image collection will reduce the mapping vector accuracy of the further infrared-texture model.

[0012] The reason for the thermal sensitivity requirement of NETD≤50mK is that the thermal imager needs to use a recognition instrument with strong thermal sensitivity, because the detection of internal lesions of the mango needs to be based on subtle temperature difference.

[0013] Further, in step S200, the method for inputting the target image into the pre-trained infrared texture mapping model to obtain the mapping vector is: the infrared texture mapping model includes a plurality of stage labels, each stage label represents the distance of the mango lesion unit time, and the target image obtains a mapping vector from each stage label; the mapping vector is a sequence composed of hardness, viscosity, cohesion and elasticity as indexes, and each index in any mapping vector is accompanied by a confidence value; any target image is input into the pre-trained infrared texture mapping model to obtain the mapping vector of each stage label.

[0014] The value range of any index in the mapping vector is (0, 100), and the larger the value represents the stronger the lesion result directionality; the value range of the confidence value is (0, 1).

[0015] The confidence value accompanied by each index in the mapping vector belongs to the mapping confidence of the index obtained by the calculation of the target image from the infrared texture mapping model of a certain stage label, because different stage labels provide different texture possible states at different time points, the model output value trained by machine learning is not directly pointing to the real state of the hardness or elasticity index, but to the modelized index of the occurrence of this state, and the reliability of the modelized index needs to be further described by the mapping confidence to modify the phenomenon of overestimation or underestimation of the index result.

[0016] The pre-trained infrared texture mapping model is used for data mapping of infrared information and lesion texture information.

[0017] Further, in step S300, the method for dynamically determining the lesion detection result of the current mango by the mapping vector is:

[0018] Calculate the power average of each confidence of any mapping vector, filter out the stage labels whose power average is greater than or equal to the power average level of each mapping vector, and form a set denoted as a mapping analysis set;

[0019] The purpose of the power average is to preferentially average the large values of each confidence, and the power average level refers to the average value of each stage label corresponding to the mapping vector.

[0020] Read the normal vector and abnormal vector in the training data set of the infrared texture mapping model;

[0021] The normal vector is the average of the mapping vectors of the stage labels of all healthy and suitable mangoes in the training data set, and the abnormal vector is the average of the mapping vectors of the stage labels of all mangoes injected with bacteria liquid in the training data set.

[0022] For the mapping analysis set, calculate the weighted Euclidean distance between the mapping vector and the normal vector and the abnormal vector with the power average as the weight, and denote them as normal distance and abnormal distance respectively; define the joint reverse difference: the difference between the normal distance of a stage label and the normal distance of its previous stage label is D.Lx, and the difference between the abnormal distance is D.Bx, then the difference between D.Lx and D.Bx is the joint reverse difference; when the joint reverse difference of a stage label and its previous three stage labels are all positive values, mark it as a pre-warning label, otherwise mark it as a safe label;

[0023] Wherein, D.Lx is positive, indicating that the current stage label deviates from the healthy state more than the previous stage label, and D.Bx is less than 0, indicating that the current stage label is closer to the lesion state than the previous stage label. The joint reverse difference simultaneously captures the superposition effect of the two trends, so the joint reverse difference greater than 0 reflects the enhanced lesion risk of the current stage label, and less than 0 reflects the recovery of the healthy state. The judgment based on the joint reverse difference utilizes the continuity of the sequence, avoiding the noise interference of a single data point or mapping vector.

[0024] The pre-warning label represents the compliance trend of the continuous increase of the lesion risk of the mango, i.e. the deviation of the mango state from the healthy state derived from the increase of the consistency of the model judgment, which points to internal lesion identification, and the safe label does not constitute the internal lesion identification.

[0025] The ratio of the eutopic distance and the heterotopic distance is the lesion risk amount, the average value Ph.Bx of the lesion risk amount of the mapping analysis phase label is calculated, and the average value Ph.Lx and the standard deviation St.Lx of the lesion risk of each safety label are calculated; the risk benchmark value is ((Ph.Bx-Ph.Lx) / St.Lx)×100%×Rsfk; wherein Rsfk is the early warning label proportion;

[0026] The risk benchmark value represents the significant degree of the current state deviating from the healthy state, and the larger the value is, the higher the lesion risk is. The early warning label proportion represents the percentage of the early warning label in all phase labels, which reflects the relative weight of the risk, effectively avoiding the risk of over-warning when there are many safety labels or missing the judgment opportunity when the early warning label dominates.

[0027] If the risk benchmark value is greater than 130%-160% and there is an index value greater than 70 in the mapping vector corresponding to all early warning labels, the lesion detection result is true.

[0028] Since the format of this risk benchmark value adopts a data level index combined with a Z-score-like class and a dynamic label offset compensation term, and the statistical distribution assumption is approximately Gaussian distribution, the setting of the risk benchmark value is a statistically reasonable significant threshold selection. The larger the value is, the lower the judgment sensitivity is, that is, it is easier to trigger the judgment of the internal lesion mango. In the early sorting process of mangoes, if the logistics cycle is long or the cost of mangoes is high, the inflow of lesion individuals should be reduced as much as possible, and the acceptable level of false positives of the detection result is higher, so the threshold of the risk benchmark value is lowered.

[0029] The presence of an index value greater than 70 in the mapping vector corresponding to the early warning label is used as a supplementary judgment to prevent misjudgment, and the additional judgment of strong abnormal expression of the index itself is introduced to strengthen the significance of the diagnosis signal. Because the false positive trigger probability is also increased when the risk benchmark value threshold is low, the risk benchmark value threshold is a macro-trend-based judgment, and the supplementary judgment is an instance of abnormal state judgment. The combination of the two improves the output stability of the judgment result.

[0030] Preferably, in step S300, the method for dynamically judging the lesion detection result of the current mango through the mapping vector is: reading the eutopic vector and the heterotopic vector from the training data set of the infrared texture mapping model;

[0031] The eutopic vector is the mapping vector mean value of the phase label of all healthy and suitable picking mangoes in the training data set, and the heterotopic vector is the mapping vector mean value of the phase label of all mangoes injected with bacteria liquid in the training data set;

[0032] The index number is defined as k1, and the health tendency degree is HLC=mean{DW.Ck1 ×(1-|DW k1 -H.HPL k1 | / 100)};The lesion tendency is PLC=mean{DW.C k1 ×(1-|DW k1 -P.HPL k1 | / 100)};where k1 is the index number, DW.C k1 DW represents the confidence level of the k1-th index value in the current mapping vector. k1 H.HPL is the k1-th index value in the current mapping vector. k1 and P.HPL k1 This represents the k1th index value among the benign and heterogeneous vectors;

[0033] HLC is used to quantify the similarity between the current stage label and the benign vector. The larger the HLC value, the closer the current stage label is to the healthy state. PLC is used to quantify the similarity between the current stage label and the abnormal vector. The larger the PLC value, the closer the current stage label is to the diseased state. Both HLC and PLC are calculated by weighted average to ensure that the indicators with high confidence have a greater impact on the results.

[0034] If a stage label satisfies the condition that HLC is greater than PLC and the average confidence level of all mapping vectors is greater than 0.6-0.8, then the stage label is marked as a healthy label; otherwise, it is marked as a fluctuating label.

[0035] Calculate the average vector of all health label mapping vectors as the mean vector, and calculate the Manhattan distance between the current stage label mapping vector and the mean vector. Or Euclidean distance Let Mdp be the variable, and NFF be the number of fluctuation tags. Each fluctuation tag constitutes a set As.bd. The ratio of the range of each PLC in set As.bd to NFF is denoted as the lesion interval Lw.PLC.

[0036] The calculation of lesion interval aims to quantitatively characterize the degree of health deviation of a sample in the thermal and mass transfer and textural property mapping space. The internal tissue of a healthy mango is relatively uniform and stable in terms of thermal conductivity and textural properties. When a mango is infected by pathogens, it will cause localized minor abnormal fluctuations in thermal infrared images and textural features. The lesion interval quantifies these minor abnormalities through mapping vectors, that is, it calculates the difference between "normal tissue and diseased tissue" in a high-dimensional feature space, and can directly quantify the degree of disease through feature differences. If the lesion interval is high, it indicates that the disease tendency of different stage labels is large; if the lesion interval is low, it indicates that the disease process is relatively stable. The lesion interval improves the sensitivity detection rate of internal lesions and avoids the omission of early lesions without obvious appearance differences by traditional methods, thereby achieving early perception and accurate capture of the evolution process of minor lesions.

[0037] The absolute value of the difference between any element in set As.bd and the PLC of its next element is the interlesion degree abs.PLC of the fluctuation label corresponding to that element;

[0038] Intersubjectivity of lesion is used to process the absolute value of lesion deviation, further eliminating directional fluctuations and focusing on the intensity of the abnormality itself. This makes the identification of lesions insensitive to the direction of deviation, only concerned with the magnitude of the abnormality. If the intersubjectivity of lesion is large, it indicates significant lesion changes, which may correspond to pathogen invasion of key tissues or failure of host defense mechanisms. If the intersubjectivity of lesion is small and stable, it indicates that the lesion may be in a slow progression phase. It can reduce the interference of natural fluctuations in different directions caused by changes in transpiration and water distribution in the natural state of mangoes, effectively eliminating disturbances, eliminating directional noise, retaining only the strong abnormal signals unique to lesions, purifying the pathological signals, and improving the ability to identify the true lesion features in complex backgrounds, thereby improving the robustness and stability of lesion identification.

[0039] Calculate the critical evaluation value Crev for each fluctuation tag based on the set As.bd:

[0040] ;

[0041] Where k2 is the cumulative variable, abs.PLC k2 and ZLw.HLC k2 Let M and HLC represent the disease predisposition and health propensity of the k2th fluctuation label, respectively; M.HLC is the average health propensity of all health labels; and exp() is an exponential function with the natural constant e as the base.

[0042] The critical assessment measure is a dynamic threshold for lesion identification. By introducing dynamic parameters such as Manhattan distance, interlesion degree, and intersub-lesion degree, the discrimination standard is upgraded from absolute value comparison to relative trend analysis. It also has adaptive weight allocation, which can adaptively construct a lesion discrimination threshold with dynamic response capability. This balances the detection accuracy and generalization ability under different batches and environmental conditions. Therefore, the introduction of the critical assessment measure effectively avoids the decrease in detection rate or increase in false positive rate caused by fixed threshold methods. Healthy mango samples will naturally fluctuate within a certain range in the thermotexture mapping space. This fluctuation comes from microscopic mechanisms such as normal physiological metabolic dynamics and microscale tissue differences. If lesions occur, the characteristic fluctuation amplitude and distribution pattern will show a systematic shift. The critical assessment measure can dynamically adjust the discrimination standard to ensure the consistency and reliability of lesion identification.

[0043] When the extreme difference of the critical evaluation quantity of all fluctuation labels is greater than or equal to the average value of all fluctuation labels, and the number of fluctuation labels is more than half of the number of healthy labels, then the lesion detection result is true, otherwise the lesion detection result is false.

[0044] Beneficial effects: Since the lesion detection result is calculated by the analysis vector set composed of the four-dimensional texture indexes of each stage after the target image is respectively mapped to several stage labels, the synergistic degree or mathematical commonality degree of each texture index can be effectively quantified to determine whether the mango has a risk of lesion, and the prediction value under each stage label is not the current real texture state but the matching state based on the assumption, so the consistency of the explanation of the synergistic degree of the mapping vector between different stages can reflect whether the mango is reasonably in the continuous lesion process from the analysis vector set; the judgment opportunity of the risk of lesion of the mango is greatly advanced.

[0045] Further, in step S400, the method for determining whether the mango has an internal lesion risk according to the lesion detection result is: the thermal infrared image of any mango is collected several times during the transmission process of the conveyor belt and is constructed into a target image, the lesion detection result is obtained from each target image, and if more than half of the lesion detection results are true, it is determined that the mango has an internal lesion risk.

[0046] The method for collecting the thermal infrared image of the mango several times during the transmission process of the conveyor belt is: taking any tangent circle on the space cylinder with the conveyor belt as the axis, the upper half of the tangent circle is evenly divided by several radial rays with the center of the circle as the origin, and the reverse direction of each radial ray is taken as the shooting direction of the infrared thermal imager, so that the thermal infrared images in different shooting directions are obtained. Alternatively, when there is only one thermal infrared image, the thermal infrared images are obtained at different positions of the mango moving with the conveyor belt.

[0047] There are two reasons for the need for multiple lesion detection result comparisons here, the first is that the infrared thermal imager collects the response of the mango surface at different positions to the thermal radiation intensity, and the surface micro-thermal distribution image has differences under different transmission angles, which directly affects the model feature extraction and further affects the lesion detection result, the second reason is that the basis of the multi-task neural network model is the CNN neural network model, the CNN model has local spatial sensitivity to the input image, that is, the model has higher feature weight for the edge region or center region of the input image, and if the thermal features of the lesion fall within the key receptive field area during the collection process, it will not trigger the effective judgment of the model, and a single thermal infrared image data is easy to miss the opportunity to judge the lesion in the detection process.

[0048] Preferably, wherein all undefined variables in the present application can be threshold values set by artificial if not defined.

[0049] The application further provides a mango internal lesion early detection system based on thermal imaging and texture analysis, which comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the mango internal lesion early detection method based on thermal imaging and texture analysis when executing the computer program, and the mango internal lesion early detection system based on thermal imaging and texture analysis can run in a computing device such as a desktop computer, a notebook computer, a palm computer and a cloud data center, and the executable system can comprise, but is not limited to, a processor, a memory, a server cluster, and the processor executes the computer program to run in the following system units:

[0050] A target image acquisition unit is configured to acquire a mango thermal infrared image from a conveying belt and perform preprocessing to obtain a target image.

[0051] A target image mapping unit is configured to input the target image into a pre-trained infrared texture mapping model to obtain a mapping vector.

[0052] A lesion detection unit is configured to dynamically determine a lesion detection result of the current mango based on the mapping vector.

[0053] An internal lesion judgment unit is configured to determine whether the mango has an internal lesion risk based on the lesion detection result.

[0054] The mango internal lesion early detection method based on thermal imaging and texture analysis provided by the application can effectively quantify the synergistic degree or mathematical common degree of each texture index to determine whether the mango has a lesion risk, the prediction value under each stage label is not the current real texture state but a matching state based on an assumption, and therefore the consistent performance of the synergistic degree explanation of the mapping vectors between different stages from the analysis vector set can reflect whether the mango is reasonably in a continuous lesion process. The judgment time of the mango lesion risk is greatly advanced, the mango with a lesion risk is prevented from participating in logistics or cold chain transportation, the risk of lesion infection of mangoes in the same batch is reduced, especially in a long logistics cycle process, the logistics efficiency can be effectively improved and the cost loss of ineffective transportation can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0055] The above and other features of the application will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which like elements refer to like elements throughout the description. As best shown in the drawings, the following description describes some embodiments of the application. Other embodiments of the application will be apparent to those of ordinary skill in the art from consideration of the description and accompanying drawings. In the drawings:

[0056] Figure 1 FIG. 1 shows a flowchart of the early detection method of internal lesions of mangoes based on thermal imaging and texture analysis;

[0057] Figure 2 FIG. 2 shows a structural diagram of the early detection system of internal lesions of mangoes based on thermal imaging and texture analysis. DETAILED DESCRIPTION

[0058] The concept, specific structure and technical effects of the present application will be described clearly and completely in combination with the embodiments and the drawings, so as to fully understand the purpose, scheme and effects of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0059] Embodiment 1:

[0060] As Figure 1 FIG. 1 shows a flowchart of the early detection method of internal lesions of mangoes based on thermal imaging and texture analysis, and the early detection method of internal lesions of mangoes based on thermal imaging and texture analysis according to the embodiments of the present application will be described below in combination with Figure 1 The method comprises the following steps:

[0061] S100, collecting a thermal infrared image of mangoes from a conveyor belt and pre-processing to obtain a target image;

[0062] S200, inputting the target image into a pre-trained infrared texture mapping model to obtain a mapping vector;

[0063] S300, dynamically determining the lesion detection result of the current mango by the mapping vector;

[0064] S400, determining whether the mango has an internal lesion risk according to the lesion detection result.

[0065] Further, in step S100, the method of collecting a thermal infrared image of mangoes from a conveyor belt and pre-processing to obtain a target image is as follows: the mangoes to be packaged are placed in a constant temperature room for 2 hours, then packaged by a conveyor belt, a thermal imager is arranged on the conveyor belt to collect thermal infrared images of the conveyor belt in real time; a target detection algorithm or a thermal threshold segmentation algorithm is used to identify the mangoes in the thermal infrared images to obtain a target image.

[0066] The thermal imager is an infrared thermal imager, and the thermal sensitivity requirement is NETD≤50mK.

[0067] Preferably, the infrared thermal imager used in the embodiments is a FLIR A655sc infrared thermal imager, which has a sensitivity as low as 30mK and can fully identify subtle temperature differences.

[0068] The target detection algorithm is any one of YOLOv5 operator, Faster R-CNN operator or RetinaNet operator; when the thermal threshold segmentation algorithm is used, the infrared measurement value obtained by the conveying belt is used as a threshold for image segmentation to obtain the target area corresponding to each mango. The area corresponding to any mango in the thermal infrared image is the target image.

[0069] The target detection algorithm arrangement has high cost, and if the computing power is insufficient, it is easy to cause the monitoring result to have hysteresis, which is not conducive to the continuity and efficiency of the production environment operation, and is suitable for production scenes with sufficient computing power layout. The thermal threshold segmentation algorithm has fast processing speed and small memory occupation, and is suitable for popularizing existing mango production scenes, but it will lose image integrity, and the shooting direction and the angle between the conveying belt are small, which is easy to cause the problem of overlapping identification of mango individuals.

[0070] Further, in step S200, the method of inputting the target image into the pre-trained infrared texture mapping model to obtain the mapping vector is: the infrared texture mapping model includes a plurality of stage labels, each stage label represents the distance of the mango lesion unit time, and the target image obtains a mapping vector from each stage label; The mapping vector is a sequence composed of hardness, viscosity, cohesion and elasticity as indicators, and each indicator in any mapping vector is accompanied by a confidence value; any target image is input into the pre-trained infrared texture mapping model to obtain the mapping vector of each stage label.

[0071] The value range of any indicator in the mapping vector is (0, 100), and the larger the value represents the stronger the lesion result directionality; the value range of the confidence value is (0, 1).

[0072] The confidence value attached to each indicator in the mapping vector belongs to the mapping confidence of the indicator obtained by the calculation of the target image from the infrared texture mapping model at a certain stage label, because different stage labels provide different texture possible states at different time points. The output result value of the model trained by machine learning is not directly pointing to the real state of indicators such as hardness or elasticity, but to the modelized indicator of this state, and the reliability of this modelized indicator needs to be further described by the mapping confidence to modify the phenomenon of overestimation or underestimation of the indicator result.

[0073] The pre-trained infrared texture mapping model is used to map the infrared information and the lesion texture information. As shown in Table 1, the mapping relationship between the infrared thermal imaging features and the texture indicators is a comparison data table.

[0074]

[0075] Table 1 Mapping relationship between infrared thermal imaging features and texture indicators

[0076] It can be seen that as the lesion degree deepens, the internal tissue of mango is destroyed, leading to the enhancement of heat diffusion ability, the significant increase of infrared temperature difference ΔT, the significant decrease of hardness and the increase of viscosity, which indicates the phenomenon of tissue softening and water content increasing, and the decrease of cohesion and elasticity, which indicates the weakening of tissue structure integrity.

[0077] Since infrared thermal imaging reflects the temperature distribution of the object surface, and the internal lesion process of mango will cause changes in tissue structure and composition, which in turn affect its heat conduction performance. Therefore, at the early stage of lesion, the characteristics such as cell wall rupture and intercellular fluid exudation will lead to the decrease of tissue density and the increase of water content, which will significantly improve the specific heat capacity and thermal conductivity of the lesion area, thereby showing the characteristics of temperature difference anomaly in the thermal imaging image. The changes in texture such as the decrease of hardness, the increase of viscosity and the weakening of cohesion are essentially caused by the destruction of the microstructure of the tissue and the redistribution of the fluid components. The linkage between this microstructure and thermal performance belongs to the natural law of biological tissue heat conduction. Under this law, combined with the high coupling characteristics of texture indicators and thermal conduction parameters, it can be predicted that the data mapping of infrared information and lesion texture information at different lesion stages has continuity and predictability.

[0078] The pre-trained infrared texture mapping model is a multi-task neural network model, and its construction process is as follows: healthy and suitable mangoes are taken as sampling objects, among which the ones injected with pathogen liquid are taken as lesion sampling objects, and the ones not injected with pathogen liquid are taken as control sampling objects; after the lesion sampling objects are injected with pathogen liquid (pythium or anthracnose), a stage label is constructed every 6 hours, and a thermal infrared image and texture information are collected from the sampling objects for each stage label, including hardness, viscosity, cohesion and elasticity.

[0079] The training data obtained under the same stage label is used to construct the multi-task neural network model corresponding to the stage label. For any texture information under the stage label, the average value of the sample set composed of control sampling objects is taken as the control value, and the difference between the value of any lesion sampling object and the control value is taken as the correlation degree. Here, the difference value is used to represent the lesion characteristic data decay of the detection item; the correlation degrees of all lesion sampling objects under the same stage label are standardized to obtain the index value of the lesion sampling object with a value range of (0, 100).

[0080] For any control sampling object, the difference between the value of any texture information and the control value is taken as the correlation degree, and the difference between the average value of the same sample is used to represent the similarity of the non-lesion characteristic data of the detection item. The correlation degrees of all control sampling objects under the same stage label are standardized to obtain the index value of the control sampling object with a value range of (0, 100).

[0081] For any sampling object, the obtained index values of each index are taken as label features of training data, the obtained thermal infrared image is taken as prediction features, and the obtained data set is taken as training data to input a MobileNet shared CNN feature extraction backbone to construct a multi-task neural network model obtained as an infrared texture mapping model. The newly obtained target image obtains the index values and confidence values of softness, viscosity, cohesion, and elasticity through the multi-task neural network model.

[0082] Among the softness and viscosity in the softness and viscosity of the texture information, the tissue density or hardness is monitored by an ultrasonic pulse echo instrument, the degree of absorption of sound waves by liquid substances in the fruit pulp is used to change the viscosity, and the fruit pulp viscosity is obtained by low-frequency ultrasonic absorption inversion; the cohesion in the texture information of softness and viscosity is obtained by EIT low-frequency electrical impedance tomography, and the tissue continuity, i.e., cohesion, is determined by the characteristic that the electrical impedance significantly decreases after the cell membrane is broken; the elasticity in the texture information of softness and viscosity is obtained by a fruit dynamic rebound detector, and its working process is DVA dynamic vibration response analysis, the frequency and damping are obtained by knocking the mango and collecting the microseismic response spectrum, and the elasticity information is obtained by inversion. However, the method for collecting texture information is not limited to the above method, and the purpose of collecting texture information is to collect data existing in the decay of mango disease for subsequent inversion of mango disease detection results.

[0083] Further, in step S300, the method for dynamically determining the disease detection result of the current mango by mapping vectors is: calculating the power average value of each confidence of any mapping vector, screening out the stage labels whose power average values are greater than or equal to the power average value level of each mapping vector, and forming a set denoted as a mapping analysis set;

[0084] Among them, the purpose of the power average value is to preferentially average the values of each confidence, and the power average value level refers to the average value of each segment label corresponding to the mapping vector.

[0085] In the training data set of the infrared texture mapping model, the normal vector and the abnormal vector are read;

[0086] Among them, the normal vector is the average of the stage labels of the mapping vectors of all healthy and suitable picking mangoes in the training data set, and the abnormal vector is the average of the stage labels of the mapping vectors of all mangoes injected with bacteria liquid in the training data set;

[0087] For the mapping analysis set, the weighted Euclidean distance between the mapping vector and the eutopic vector and the dystopic vector is calculated with the power average as the weight, and is recorded as the eutopic distance and the dystopic distance respectively; the joint reverse difference is defined: the difference between the eutopic distance of a stage label and the eutopic distance of the previous stage label is D.Lx, and the difference between the dystopic distance is D.Bx, and the difference between D.Lx and D.Bx is the joint reverse difference; when the joint reverse difference of a stage label and its previous three stage labels are all positive values, it is marked as a warning label, otherwise it is marked as a safe label;

[0088] The ratio of the eutopic distance and the dystopic distance is recorded as the lesion risk amount, the average value of the lesion risk amount of the stage label in the mapping analysis set is calculated Ph.Bx, and the average value of the lesion risk of each safe label is Ph.Lx and the standard deviation St.Lx; the risk reference value is ((Ph.Bx-Ph.Lx) / St.Lx) x 100% x Rsfk; wherein Rsfk is the warning label proportion;

[0089] The risk reference value represents the significant degree of the current state deviating from the healthy state, and the larger the value is, the higher the lesion risk is. The warning label proportion represents the percentage of the warning label in all stage labels, which reflects the relative weight of the risk, effectively avoiding the risk of over-warning when there are more safe labels or missing the judgment opportunity when the warning label dominates.

[0090] If the risk reference value is greater than 150% and there is an index value greater than 70 in the mapping vector corresponding to all warning labels, the lesion detection result is true.

[0091] The "previous" described above with respect to the label represents the direction closer to the virus liquid injection, for example, a stage label is 30H, and the previous three stage labels are 24H, 18H and 12H.

[0092] Further, in step S400, the method for judging whether the mango has internal lesion risk according to the lesion detection result is: the thermal infrared image of any mango is collected several times during the transmission process of the conveyor belt and is constructed into a target image, the lesion detection result is obtained from each target image respectively, and if more than half of the lesion detection results are true, it is judged that the mango has internal lesion risk.

[0093] The method for collecting the thermal infrared image of the mango several times during the transmission process of the conveyor belt is: on the space cylinder with the conveyor belt as the axis, any cross-sectional circle is cut, and the upper half of the cross-sectional circle is evenly divided by several center rays, and the reverse direction of each ray is taken as the shooting direction of the infrared thermal imager, so that the thermal infrared images in different shooting directions are obtained. Or, when there is only one thermal infrared image, the thermal infrared images are obtained at different positions of the mango moving with the conveyor belt.

[0094] Embodiment 2:

[0095] Example 2 uses the same detection method as Example 1, the difference is that in step S300, the method of dynamically determining the disease detection result of the current mango by mapping vector is: reading the normal vector and abnormal vector in the training data set of the infrared texture mapping model;

[0096] Among them, the normal vector is the mean of the stage label mapping vector of all healthy and suitable picking mangoes in the training data set, and the abnormal vector is the mean of the stage label mapping vector of all mangoes injected with bacteria liquid in the training data set;

[0097] Taking k1 as the index number, the health tendency degree is defined as HLC = mean{DW.C k1 ×(1-|DW k1 -H.HPL k1 | / 100)}; the disease tendency degree is PLC = mean{DW.C k1 ×(1-|DW k1 -P.HPL k1 | / 100)}; wherein k1 is the index number, DW.C k1 is the confidence of the k1th index value in the current mapping vector, DW k1 is the k1th index value in the current mapping vector, H.HPL k1 and P.HPL k1 are the k1th index values in the normal vector and abnormal vector;

[0098] When the stage label satisfies HLC greater than PLC and the average value of all confidence of the mapping vector is greater than 0.6, the stage label is marked as a healthy label, otherwise it is marked as a fluctuation label;

[0099] The average vector of all healthy label corresponding mapping vectors is calculated as the normal vector, the Manhattan distance between the mapping vector of the current stage label and the normal vector is calculated as Mdp, and the number of fluctuation labels is NFF; Each fluctuation label constitutes a set As.bd; The ratio of the range of each PLC in the set As.bd to NFF is recorded as the disease interval Lw.PLC;

[0100] The absolute value of the difference between any element in the set As.bd and the PLC of the next element is the disease sub-interval abs.PLC of the fluctuation label corresponding to the element; The disease sub-interval of the last element in the set As.bd is the average of all disease sub-intervals; Here, the next element refers to the next element in the set As.bd scanned in the increasing direction of the stage label;

[0101] According to the set As.bd, the critical evaluation quantity Crev of each fluctuation label is calculated:

[0102] ;

[0103] wherein k2 is an accumulation variable, abs.PLC k2 and ZLw.HLC k2 are the k2th fluctuation label's lesion inter-subjectity and health tendency, respectively, M.HLC is the average of all healthy labels' health tendency, exp() is the exponential function with base e;

[0104] When the range of the critical evaluation quantity of all fluctuation labels is greater than or equal to the average of all fluctuation labels, and the number of fluctuation labels is more than half of the number of healthy labels, then the lesion detection result is true, otherwise the lesion detection result is false.

[0105] To verify the universality and detection stability of the method of the present application, a uniform sample control experiment was designed. A total of 40 mango samples were selected, of which 20 were healthy samples not injected with pathogen solution, 10 were lesion early stage samples injected with anthracnose culture for 24 hours, and 10 were lesion middle stage samples cultured for 72 hours. After all samples were placed at room temperature for 2 hours, thermal infrared images were collected and input into the pre-trained infrared texture mapping model to obtain the mapping vector. Subsequently, the three detection methods corresponding to Example 1, Example 2 and Comparative Example were performed respectively. The execution process is as follows: all samples collect infrared images, input the pre-trained infrared texture mapping model to obtain the vector, each sample executes the three detection methods in turn, the actual health status of the mango is confirmed by the artificial label of the pathogen injection group and the healthy group, the number of samples judged as lesion true is represented by the number of positive judgments, the accuracy rate is represented by the proportion of true positive and true negative to the total number of samples, and the proportion of the number of early stage lesions judged as true to the number of early stage lesions is used as the early identification rate.

[0106]

[0107] Table 2: Sample judgment effect statistics under the detection methods of the examples and comparative examples

[0108] As shown in Table 2, Example 1 has high identification accuracy in the lesion middle stage, and is suitable for mature lesion judgment with clear trend; while Example 2 is more sensitive to the early stage of lesion, and the identification rate is much higher than other methods, which is suitable for early screening scene, but there is a small phenomenon of misjudgment of the healthy group; while the comparative example uses a simple threshold method, resulting in many misjudgments and serious omissions, which is only suitable for rough preliminary screening; therefore, the combination of Examples 1 and 2 can respectively exert their advantages in different stages of lesion development, and if Comparative Example 1 is used as preliminary screening and Example 2 is used for fine screening, it is expected to more accurately mark abnormal mangoes.

[0109] Table The example provided by the present application provides a mango internal lesion early detection system based on thermal imaging and texture analysis, such as Figure 2As shown is a structure diagram of a mango internal lesion early detection system based on thermal imaging and texture analysis of the application, the mango internal lesion early detection system based on thermal imaging and texture analysis of this embodiment comprises a processor, a memory and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the above mango internal lesion early detection method embodiment based on thermal imaging and texture analysis when executing the computer program.

[0110] The system comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to run in the units of the following system:

[0111] A target image acquisition unit is configured to acquire mango thermal infrared images from a conveyor belt and perform preprocessing to obtain target images.

[0112] A target image mapping unit is configured to input the target images into a pre-trained infrared texture mapping model to obtain a mapping vector.

[0113] A lesion detection unit is configured to dynamically determine a lesion detection result of the current mango based on the mapping vector.

[0114] An internal lesion judgment unit is configured to determine whether the mango has an internal lesion risk based on the lesion detection result.

[0115] The mango internal lesion early detection system based on thermal imaging and texture analysis can run in desktop computers, notebook computers, palmtop computers and cloud servers, etc. The mango internal lesion early detection system based on thermal imaging and texture analysis can run in a system which can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the example is only an example of the mango internal lesion early detection system based on thermal imaging and texture analysis, and does not constitute a limitation on the mango internal lesion early detection system based on thermal imaging and texture analysis, and can include more or fewer components, or combine certain components, or different components, for example, the mango internal lesion early detection system based on thermal imaging and texture analysis can also include input and output devices, network access devices, buses, etc.

[0116] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor. The processor is a control center of the mango internal lesion early detection system based on thermal imaging and texture analysis, and is connected with various parts of the mango internal lesion early detection system based on thermal imaging and texture analysis through various interfaces and lines.

[0117] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the mango internal lesion early detection system based on thermal imaging and texture analysis by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0118] Although the description of the present application has been quite detailed and particularly described with respect to several embodiments, it is not intended to be limited to any of these details or embodiments or any special embodiment, so as to effectively cover the intended scope of the present application. In addition, the present application is described above with respect to the embodiments that the inventors can foresee, and the purpose is to provide a useful description, and those non-essential modifications to the present application that have not yet been foreseen can still represent equivalent modifications to the present application.

Claims

1. A method for early detection of internal disorders in mango based on thermography and texture analysis, characterized in that, The method comprises the following steps: S100, acquiring a mango thermal infrared image from a conveyor belt and preprocessing to obtain a target image; S200, inputting the target image into a pre-trained infrared texture mapping model to obtain a mapping vector; S300, dynamically determining the lesion detection result of the current mango through the mapping vector; S400, determining whether the mango has an internal lesion risk according to the lesion detection result; In step S300, the method of dynamically determining the lesion detection result of the current mango through the mapping vector is: calculating the power average of each confidence of any mapping vector, screening out the stage label whose power average is greater than or equal to the power average level of each mapping vector, and the stage label provides the texture possible state at different time points and constitutes a set, which is denoted as a mapping analysis set; Read the normal vector and abnormal vector of the training data set of the infrared texture mapping model; calculate the weighted Euclidean distance between the mapping vector and the normal vector and the abnormal vector with the power average as the weight, and denote them as normal distance and abnormal distance respectively; define the joint reverse difference: the difference between the normal distance of the stage label and the previous stage label is D.Lx, and the difference between the abnormal distance is D.Bx, then the difference between D.Lx and D.Bx is the joint reverse difference; when the joint reverse difference of a stage label and its previous three stage labels are all positive values, mark it as a warning label, otherwise mark it as a safety label; Denote the ratio of the normal distance and the abnormal distance as the lesion risk, calculate the average value Ph.Bx of the lesion risk of the stage label in the mapping analysis set, and the average value Ph.Lx and the standard deviation St.Lx of the lesion risk of each safety label; then the risk reference value is ((Ph.Bx-Ph.Lx) / St.Lx)×100%×Rsfk; wherein Rsfk is the proportion of warning labels; When the risk reference value is greater than 130%-160% and there is an index value greater than 70 in all mapping vectors corresponding to the warning labels, the lesion detection result is true.

2. The method for early detection of internal disorder in mango based on thermography and texture analysis as claimed in claim 1, wherein, In step S100, the method of acquiring a mango thermal infrared image from a conveyor belt and preprocessing to obtain a target image is: placing the packaged mangoes in a room temperature constant for 2-4 hours, then conveying the packaged mangoes through a conveyor belt, arranging a thermal imager on the conveyor belt and collecting thermal infrared images of the conveyor belt in real time; identifying the mangoes in the thermal infrared images to obtain a target image through a target detection algorithm or a thermal threshold segmentation algorithm; The thermal imager is an infrared thermal imager, and the thermal sensitivity requirement is NETD≤50mK.

3. The method for early detection of internal disorder in mango based on thermography and texture analysis as claimed in claim 1, wherein, In step S200, the method of inputting the target image into a pre-trained infrared texture mapping model to obtain a mapping vector is: the infrared texture mapping model includes a plurality of stage labels, and the target image obtains a mapping vector from each stage label; the mapping vector is a sequence composed of hardness, viscosity, cohesion and elasticity as indexes, and each index in the mapping vector is attached with a confidence value; inputting the target image into the pre-trained infrared texture mapping model to obtain the mapping vector of each stage label.

4. The method for early detection of internal disorder in mango based on thermography and texture analysis as claimed in claim 1, wherein, In step S300, the method for dynamically determining the disease detection result of the current mango by mapping the vector is: reading the normal vector and abnormal vector in the training data set of the infrared texture mapping model; defining the health tendency as HLC=mean{DW.C k1 ×(1-|DW k1 -H.HPL k1 | / 100)}; the disease tendency is PLC=mean{DW.C k1 ×(1-|DW k1 -P.HPL k1 | / 100)}; When the stage label satisfies HLC>PLC and the average value of all confidence values of the mapping vector is greater than 0.6, the stage label is marked as a health label, otherwise it is marked as a fluctuation label; The average vector of all health label corresponding mapping vectors is calculated as the equilibrium vector, the Manhattan distance between the mapping vector of the current stage label and the equilibrium vector is recorded as Mdp, the number of fluctuation labels is NFF; each fluctuation label constitutes a set As.bd; the ratio of the range of each PLC in the set As.bd to NFF is recorded as the lesion interval Lw.PLC; The absolute value of the difference between any element and the next element in the set As.bd is the lesion sub-interval abs.PLC of the fluctuation label corresponding to the element; the lesion abnormal intensity of each fluctuation label is quantified according to the set As.bd to obtain a critical evaluation quantity, and when the range of the critical evaluation quantities of all fluctuation labels is greater than or equal to the average value of all fluctuation labels, and the number of fluctuation labels is more than half of the number of health labels, the lesion detection result is true.

5. The method for early detection of internal disorder in mango based on thermography and texture analysis as claimed in claim 1, wherein, In step S400, the method for judging whether the mango has the risk of internal lesion according to the lesion detection result is: any mango is collected to obtain a thermal infrared image and is constructed into a target image several times in the conveying process of the conveying belt, and the lesion detection result is obtained from each target image, and if more than half of the lesion detection results are true, it is judged that the mango has the risk of internal lesion.

6. A system for early detection of internal disorders in mango based on thermal imaging and texture analysis, characterized in that, The mango internal lesion early detection system based on thermal imaging and texture analysis includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the steps in the mango internal lesion early detection method based on thermal imaging and texture analysis in any one of claims 1-5, and the mango internal lesion early detection system based on thermal imaging and texture analysis runs in a desktop computer, a notebook computer, a palm computer, and a cloud data center computing device.

Images