Industrial robot image processing method based on image fusion

Abstract

The invention discloses an industrial robot image processing method based on image fusion, and relates to the technical field of intelligent aquaculture, and the method comprises the steps: collecting an original image in real time through deploying an image collection device integrating visible light, polarization and multispectral imaging, and extracting suspended matter density, water body light transmittance and illumination intensity change information; generating a first image set; suppressing suspension interference through image filtering and enhancement processing to obtain a first corrected image; extracting an aquatic product individual region, performing multi-source image fusion, identifying color deviation, texture interruption and reflection feature anomaly regions, and constructing a lesion candidate set; gray scale reconstruction, edge gradient and brightness normalization correction of a multispectral channel are executed based on illumination and reflection changes, and a high-quality fusion image is generated; and calculating a health anomaly probability coefficient of the target individual by using the depth recognition model, comparing the health anomaly probability coefficient with a threshold value, and recording a recognition result and collecting information if the threshold value is exceeded. The method can significantly improve the accuracy of aquatic individual lesion recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent aquaculture, in particular to an industrial robot image processing method based on image fusion. BACKGROUND

[0002] With the rapid development of modern aquaculture industry, industrial robots are increasingly widely used in underwater farming environments, especially in automated monitoring and health detection. The underwater farming environment is complex and variable, with a large amount of suspended particles in the water, unstable and significant fluctuations in light conditions, which brings great challenges to the image quality collected by industrial robots, directly affecting the accuracy and reliability of lesion identification.

[0003] Traditional aquaculture image processing techniques usually rely on single spectral imaging or single type image data, and mostly use independent image processing algorithms, lacking effective fusion and collaborative analysis means of multispectral and polarized images, resulting in insufficient ability to capture abnormal features on the surface of aquatic individuals. At the same time, environmental factors such as suspended matter density, water transparency and light changes have a great impact on image quality, and existing methods often lack effective suppression and correction means for these interference factors.

[0004] In addition, traditional image recognition relies on shallow feature extraction and single feature dimension, lacking comprehensive recognition ability of multi-dimensional spectral and texture, reflection features, and is difficult to meet the precise positioning needs of fish body surface color deviation, texture fracture and reflection abnormalities and other diverse lesion areas. More importantly, there is a lack of systematic methods for training deep learning models based on high-quality fused image sets, which cannot fully utilize the complementary advantages of multi-source data, thereby limiting the precision and generalization ability of image anomaly recognition. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides an industrial robot image processing method based on image fusion to solve the problems mentioned in the background art.

[0006] To achieve the above purpose, the present application is implemented by the following technical scheme: an industrial robot image processing method based on image fusion, comprising the following steps:

[0007] Step one, deploy an industrial robot image acquisition device for underwater farming environment operation, real-time acquisition of original visible light image, polarized image and multispectral image, and obtain suspended matter density information, water transparency and light intensity change value in the image, generate a first image set;

[0008] Step two, image filtering and contrast enhancement operation is performed on the first image set to suppress the interference of suspended particles on image quality, and the first corrected image result is obtained;

[0009] Step three, after completing the first image processing, the area information of aquatic individual in the image is extracted, and the fusion calculation of visible light image and multispectral image is performed on the area of aquatic individual, the color deviation area, texture interruption area and reflection feature discontinuous area of fish body surface are identified, the second image set is constructed, and the candidate set of lesion area is constructed based on the fusion features;

[0010] Step four, in the image fusion process, according to the changes of illumination condition, shooting angle and surface reflectivity, the multispectral channel reflection features of the fusion image are subjected to multiple fitting correction, including gray scale reconstruction correction, edge gradient difference correction and local brightness normalization correction, so as to improve the identification accuracy of lesion area and obtain high-quality third image set;

[0011] Step five, based on the third image set, the health abnormality probability coefficient PABN of the i-th target individual is obtained by using the trained deep image recognition model, and compared with the preset fourth threshold Q4, if higher than the fourth threshold Q4, the current image acquisition unit data and identification result are recorded, and the image processing log data is constructed.

[0012] Preferably, step one comprises:

[0013] S11, deploying a high-resolution visible light camera, a polarization imaging camera and a multi-channel multispectral imager at the front end of the industrial robot;

[0014] S12, real-time acquisition of original visible light images of aquaculture area; acquisition of reflected polarization image information in water body; acquisition of multispectral reflection feature images of target individuals;

[0015] S13, using the local contrast reduction, edge blur degree and scattering light pattern in the image, the suspended particle density is obtained through the image filter response function and polarization angle change algorithm; based on the reflectivity attenuation characteristics of the water body area under different wave bands, the water body transmittance is obtained; through the overall brightness mean value and local brightness non-uniformity analysis, the brightness value of each pixel is obtained, and a brightness matrix is constructed to reflect the illumination distribution characteristics of the image; a first image set is generated.

[0016] Preferably, step two comprises:

[0017] S21, by using edge preserving filter on the first image set, the noise points caused by suspended particles in water are removed, while the fish body edge and details are retained; using histogram equalization or local contrast enhancement method, the light and dark difference between fish body and background in the image is improved, so that the color and texture features are clearer; combined with polarization image information, the water surface reflection interference is reduced, and the color deviation is corrected to improve the image reality;

[0018] S22. By extracting the density of suspended matter per unit area, the light transmittance of water at the location of the i-th image, and the average variance of the local brightness distribution in the i-th image, and after dimensionless processing, the suspension interference coefficient XFX is calculated.

[0019] Preferably, step two also includes:

[0020] S23. By setting a first threshold Q1 and comparing the floating interference coefficient XFX with the first threshold Q1, the first evaluation result is obtained, including:

[0021] When the suspension interference coefficient XFX < the first threshold Q1, it means that the i-th image has no suspended matter interference under water and light conditions, and the first corrected image result is obtained.

[0022] When the suspension interference coefficient XFX ≥ the first threshold Q1, it indicates that the i-th image has suspended interference under water and lighting conditions, triggering the first warning command and generating the first strategy: automatically triggering the image re-sampling strategy, adjusting the shooting angle and light source compensation parameters of the image acquisition device, reducing the acquisition frequency by 20% and delaying the acquisition of the next frame image, recalculating until the suspension interference coefficient XFX < the first threshold Q1.

[0023] Preferably, step three includes:

[0024] S31. Extract the aquatic individual region from the first corrected image, perform pixel-level fusion of the visible light image and the multispectral image within the target region to enhance surface detail features, identify feature regions in the fused image including: color deviation region, texture interruption region and reflection discontinuity region, and summarize them into a second image set;

[0025] S32. Based on the color deviation region, texture interruption region, and reflection discontinuity region identified in the fused image, statistical analysis and normalization processing are performed on the features of each abnormal region; for the image sample of aquatic individuals in the i-th image acquisition node, the proportion of body surface color difference of fish in the aquatic individual region in the i-th image, the intensity of body surface texture interruption of fish in the aquatic individual region in the i-th image, and the proportion of multispectral reflectance abnormal region in the i-th image are obtained. After dimensionless processing, the lesion feature aggregation coefficient BJX is calculated and obtained.

[0026] Preferably, step three also includes:

[0027] S33. By setting a second threshold Q2, and comparing the lesion feature aggregation coefficient BJX with the second threshold Q2, the second evaluation results are obtained, including:

[0028] When the aggregation coefficient of lesion features BJX is less than the second threshold Q2, it indicates that the fish's body surface features are in a normal state and should be continuously monitored.

[0029] When the lesion feature aggregation coefficient BJX is greater than or equal to the second threshold Q2, it indicates that the fish body surface feature is in an abnormal state and has a potential risk of lesion, a second warning instruction is triggered, and a second strategy is generated: a candidate set of lesion areas is established; step four is entered, and an image multi-channel fitting correction operation is started, the image is marked as a "candidate lesion image", and depth recognition processing is preferentially performed.

[0030] Preferably, step four includes:

[0031] S41, after obtaining the candidate set of lesion areas, combining the light conditions, shooting angles and surface reflectivity changes, the multispectral fusion image in the candidate area is corrected, specifically including:

[0032] S411, gray reconstruction correction is performed, the gray offset of the lesion area is corrected, and the brightness is unified;

[0033] S412, edge gradient difference correction is performed, the continuity and clarity of the edge texture of the lesion area are enhanced;

[0034] S413, local brightness normalization correction is performed, the local brightness difference caused by uneven light is eliminated, and the image contrast is improved;

[0035] S42, by extracting the gray reconstruction residual value before and after correction in the i-th image, the average of the edge gradient difference before and after in the i-th image, and the local brightness normalization index in the i-th image, after non-dimensional processing, the spectral consistency fitting coefficient GYNX is calculated and obtained.

[0036] Preferably, step four further includes:

[0037] S43, by comparing and analyzing the spectral consistency fitting coefficient GYNX with the third threshold Q3 through a preset third threshold Q3, a third evaluation result is obtained, including:

[0038] When the spectral consistency fitting coefficient GYNX is less than or equal to the third threshold Q3, it indicates that the image fitting correction is qualified, a high-quality third image set is established, and step five is entered;

[0039] When the spectral consistency fitting coefficient GYNX is greater than the third threshold Q3, it indicates that the image fitting correction is unqualified, a third warning instruction is triggered, and a third strategy is generated: return to step three to perform image refusion operation on the candidate set of lesion areas, and record the reason for the failure of this round of correction; update the fusion parameters, and re-correct the current image in step four, and re-calculate until the spectral consistency fitting coefficient GYNX is less than or equal to the third threshold Q3.

[0040] Preferably, step five includes:

[0041] S51, by utilizing a convolutional neural network, an image recognition initial model is constructed, and the third image set obtained in step four and the lesion region candidate set constructed in step three are used as training data and labeled data to train and test the image recognition initial model, so as to obtain a trained model for image health anomaly recognition; the trained image recognition initial model is used as a deep image recognition model for lesion image recognition, and the convolutional feature output of the intermediate layer of the third image set is extracted as an image fusion feature vector for representing the individual body surface abnormal structure features, and then the deep image recognition model is continuously trained and performance optimized based on the feature vector, and the normalized score of the i th aquatic individual in the texture structure continuity feature, the normalized score of the i th aquatic individual in the edge gradient integrity feature and the normalized score of the i th aquatic individual in the local brightness consistency feature are obtained, which are used for subsequent image anomaly recognition and health state discrimination operations.

[0042] Preferably, step five further comprises:

[0043] S52, based on the third image set, the normalized score SW of the i th aquatic individual in the texture structure continuity feature, the normalized score SB of the i th aquatic individual in the edge gradient integrity feature and the normalized score SL of the i th aquatic individual in the local brightness consistency feature are obtained by the trained deep image recognition model. i i i After dimensionless processing, the health anomaly probability coefficient PABN of the i th target individual is calculated and obtained.

[0044] S53, by presetting a fourth threshold Q4, and comparing and analyzing the health anomaly probability coefficient PABN of the i th target individual with the fourth threshold Q4, a fourth evaluation result is obtained, which comprises:

[0045] When the health anomaly probability coefficient PABN of the i th target individual is less than or equal to the fourth threshold Q4, it indicates that the health state of the target individual is normal, and continuous monitoring is performed.

[0046] When the health anomaly probability coefficient PABN of the i th target individual is greater than the fourth threshold Q4, it indicates that the health state of the target individual is abnormal, a fourth warning instruction is triggered, and a fourth strategy is generated: recording the current image acquisition metadata and recognition result, and constructing image processing log data.

[0047] The present application provides an industrial robot image processing method based on image fusion. It has the following advantages:

[0048] ​​(1) The industrial robot image processing method based on image fusion can accurately identify and evaluate the degree of suspended interference in water by fusing visible light, polarized images and multispectral images, combining dynamic perception of suspended matter density, water transparency and illumination intensity, automatically triggering image resampling or enhancement strategy, realizing dynamic optimization of image definition, and significantly improving image acquisition quality in complex aquaculture environment.

[0049] (2) The industrial robot image processing method based on image fusion extracts three key lesion characteristics of color deviation, texture interruption and reflectance anomaly based on pixel-level fusion of visible light and multispectral images, constructs a "lesion feature aggregation coefficient" and an evaluation threshold mechanism, realizes accurate positioning and candidate screening of abnormal areas on the body surface, and effectively enhances the sensitivity and accuracy of early lesion identification.

[0050] (3) The industrial robot image processing method based on image fusion introduces multi-channel gray reconstruction, edge gradient compensation and local brightness normalization correction algorithms for candidate lesion areas, establishes a "spectral consistency fitting coefficient" evaluation system to ensure that the image fusion presents uniform illumination performance and clear structure, provides high-quality training input for deep recognition models, and improves the model generalization ability and recognition stability.

[0051] (4) The industrial robot image processing method based on image fusion uses a trained deep image recognition model, combines texture continuity, edge integrity and brightness consistency features in the image, constructs a health abnormality probability index and compares it with a set threshold, can automatically judge the health status of aquatic individuals and generate an identification log, realizes automatic, fine and data traceable management of lesion monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The industrial robot image processing method based on image fusion is shown in the figure. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0054] Embodiment 1

[0055] Please refer to Figure 1 The present application provides an industrial robot image processing method based on image fusion, comprising the following steps:

[0056] Step one, deploy an industrial robot image acquisition device for operation in an underwater aquaculture environment, real-time acquisition of original visible light images, polarization images and multispectral images, and obtain the suspended matter density information, water transparency and illumination intensity change value in the image, generate a first image set;

[0057] Step two, image filtering and contrast enhancement operation is performed on the first image set to suppress the interference of suspended particulate matter on the image imaging quality, and a first corrected image result is obtained;

[0058] Step three, after completing the first image processing, the region information of the aquatic individual in the image is extracted, and the fusion calculation of visible light image and multispectral image is performed on the region of the aquatic individual, the color deviation region, the texture interruption region and the reflection feature discontinuous region of the fish body surface are identified, a second image set is constructed, and a lesion region candidate set is constructed based on the fusion features;

[0059] Step four, in the image fusion process, according to the changes of illumination condition, shooting angle and surface reflectivity, the multispectral channel reflection features of the fusion image are corrected by multiple fitting, including gray reconstruction correction, edge gradient difference correction and local brightness normalization correction, in order to improve the identification accuracy of the lesion region, and obtain a high-quality third image set;

[0060] Step five, based on the third image set, through the trained deep image recognition model, the health abnormality probability coefficient PABN of the i-th target individual is compared with the preset fourth threshold Q4, if it is higher than the fourth threshold Q4, the current image acquisition unit data and the recognition result are recorded, and the image processing log data is constructed.

[0061] In this embodiment, through the image acquisition and processing method, the interference of the suspended particles in the water on the image quality can be effectively suppressed, and the image can be corrected in multiple dimensions under different illumination conditions and shooting angles, the clarity and stability of the lesion features in the multispectral image are improved; on this basis, the lesion region candidate set constructed by fusing visible light and multispectral information can accurately depict the early lesion features such as color deviation, texture interruption and reflection anomaly on the fish body surface, combined with the health abnormality probability output by the deep recognition model and the preset threshold Q4 judgment mechanism, the system has high lesion detection accuracy and real-time response ability, which significantly improves the intelligent recognition level and early warning ability of the lesion target in the aquaculture environment.

[0062] Embodiment 2

[0063] This embodiment is an explanation and description in embodiment 1, please refer to Figure 1 , specifically, step one includes:

[0064] S11, deploy a high-resolution visible light camera, a polarization imaging camera, and a multi-channel multispectral imager at the front end of the industrial robot;

[0065] S12, real-time acquisition of raw visible light images of the aquaculture area; used to obtain the basic visual information of fish color and texture; acquisition of reflected polarization image information in the water body; used to enhance object edges, identify surface reflectivity changes, and assist in reflection correction; acquisition of multispectral reflection characteristic images of target individuals; obtain color depth, tissue differences, spectral response characteristics, etc.

[0066] S13, using the local contrast reduction, edge blur degree, and scattering light pattern in the image, through the image filter response function and polarization angle change algorithm, the suspended particle density is obtained; based on the reflectivity attenuation characteristics of the water area under different wavebands, the water transparency is obtained; through the overall brightness mean value and local brightness inhomogeneity analysis, the per-pixel brightness value is obtained, and a brightness matrix is constructed to reflect the light distribution characteristics of the image; a first image set is generated.

[0067] In this embodiment, by simultaneously deploying a high-resolution visible light camera, a polarization imaging camera, and a multi-channel multispectral imager at the front end of the industrial robot, multi-dimensional and real-time acquisition of visual information and spectral characteristics in the aquaculture environment is realized; combined with the filter response and polarization angle change algorithm based on local contrast, edge blur, and scattering light pattern, the suspended particle density in the water body is accurately quantified, and the water transparency and light distribution characteristics are effectively reflected; this multi-source information fusion acquisition and precise physical quantity extraction greatly improves the quality of the subsequent image correction and the adaptability of the basic data of the disease recognition, thereby providing a solid data guarantee for high-precision aquaculture health monitoring.

[0068] Embodiment 3

[0069] This embodiment is an explanation and description in Embodiment 2, please refer to Figure 1 , specifically, step two includes:

[0070] S21, by using edge-preserving filtering on the first image set, remove the noise caused by suspended particles in the water, while retaining the edges and details of the fish; use histogram equalization or local contrast enhancement method to improve the light and dark difference between the fish and the background in the image, so that the color and texture features are clearer; combined with the polarization image information, reduce the water surface reflection interference, and correct the color deviation, improve the image realism;

[0071] S22, by extracting the suspended particle density per unit area in the i-th image, the water transparency at the i-th image position, and the local brightness distribution average variance in the i-th image, after non-dimensional processing, the suspended interference coefficient XFX is calculated and obtained, the formula is as follows:

[0072] XFX=w1*Dxfi +w2*(1-Tt i )+w3*σ(Lg i );

[0073] Dxf i indicates the density of suspended solids per unit area in the i-th image, Tt i indicates the water transparency at the i-th image position, σ(Lg i ) indicates the average variance of local brightness distribution in the i-th image, w1, w2 and w3 indicate weight coefficients, 0 < w1 < 1, 0 < w2 < 1 and 0 < w3 < 1, and w1 + w2 + w3 = 1;

[0074] The acquisition method of w1, w2 and w3 is as follows: based on the analysis results of a large amount of aquaculture water image data, the influence degree of key image interference factors on imaging quality is extracted by statistically analyzing the relationship between different illumination conditions, suspended solids interference levels and image correction effects. In the experimental stage, based on the training performance of the image denoising enhancement model, combined with the change trend of image signal-to-noise ratio improvement value and structural similarity index, the weight distribution of each interference factor is optimized; at the same time, referring to the design guide of underwater machine vision system, the typical water product breeding environment monitoring standard and the industry application literature of image enhancement algorithm, the weight coefficient range is constrained to ensure that the coefficient combination can not only reflect the relative strength of the actual interference influence, but also meet the convergence and stability requirements of the image processing algorithm;

[0075] The brightness component is extracted from the original image to obtain the brightness matrix Lg i (x, y); using the sliding window technology, the whole image is divided into small blocks, and the local mean and variance of the brightness of each window pixel are calculated;

[0076]

[0077] In the formula, N indicates the total number of local windows, Var indicates the brightness variance, indicates the k-th local window in the i-th image;

[0078]

[0079] In the formula, n indicates the number of pixels in the local area, Lg j indicates the brightness value of the j-th pixel, indicates the average brightness value of the local area.

[0080] In this embodiment, by using edge-preserving filtering and histogram equalization and other enhancement techniques on the first image set, the noise caused by suspended particles in the water is effectively removed, the fish body edge and details are kept clear, and the color and texture features are improved in recognizability; combining with the polarization image correction of water surface reflection and color deviation, the image realness and contrast are significantly improved; at the same time, the suspended interference coefficient is calculated based on the suspended matter density, water body transmittance and local brightness variance, combined with the scientific optimization of weight distribution, the quantitative evaluation and dynamic compensation of image interference are realized, the adaptability and stability of the image denoising and enhancement algorithm are enhanced, and high-quality input data is provided for subsequent accurate lesion identification.

[0081] Embodiment 4

[0082] This embodiment is an explanation and description in embodiment 3, please refer to Figure 1 , specifically, step two further comprises:

[0083] S23, by presetting a first threshold Q1, and comparing and analyzing the suspended interference coefficient XFX with the first threshold Q1, a first evaluation result is obtained, including:

[0084] When the suspended interference coefficient XFX is less than the first threshold Q1, it indicates that the ith image has no suspended matter interference under the water body and light conditions, and the image clarity meets the fusion recognition requirements, and a first corrected image result is obtained;

[0085] When the suspended interference coefficient XFX is greater than or equal to the first threshold Q1, it indicates that the ith image has suspended matter interference under the water body and light conditions, and the image clarity does not meet the fusion recognition requirements, triggering a first warning instruction, generating a first strategy: automatically triggering an image resampling strategy, adjusting the shooting angle of the image acquisition device and the light source compensation parameter, reducing the acquisition frequency by 20% and delaying the next frame image acquisition, and recalculating until the suspended interference coefficient XFX is less than the first threshold Q1.

[0086] The acquisition method of the first threshold Q1: through a large number of experimental determination on the relationship between the suspended particle density change in image acquisition under different water body environments and the visual quality parameters such as image contrast and clarity, a quantitative model of the influence degree of suspended matter interference on image imaging is established, and combined with the statistical law of key texture recovery rate and edge gradient change before and after image filtering, the threshold distribution interval under the typical interference state is extracted. Then, through comprehensive analysis of the robustness of multiple batches of data and the actual recognition experience of professionals in the field of aquatic image processing, a reasonable judgment threshold Q1 for dynamically distinguishing normal image quality and serious suspended interference state is determined.

[0087] In this embodiment, a dynamic real-time assessment of the degree of interference from suspended particles in the image is achieved by setting a comparison mechanism between the suspended interference coefficient and a first threshold Q1. When suspended interference is detected to exceed the threshold, the system automatically triggers an image resampling strategy, intelligently adjusts the shooting angle and light source compensation parameters of the acquisition device, and appropriately reduces the acquisition frequency and delay acquisition time, effectively reducing the impact of suspended particles on image quality and ensuring that the acquired image meets the clarity requirements for fusion recognition, thereby significantly improving the accuracy of subsequent lesion recognition and the system's environmental adaptability.

[0088] Example 5

[0089] This embodiment is an explanation based on Embodiment 4. Please refer to it. Figure 1 Specifically, step three includes:

[0090] S31. Extract the aquatic individual region from the first corrected image, perform pixel-level fusion of the visible light image and the multispectral image within the target region to enhance surface detail features, identify feature regions in the fused image including: color deviation region, texture interruption region and reflection discontinuity region, and summarize them into a second image set;

[0091] S32. Based on the color deviation regions, texture interruption regions, and reflection discontinuity regions identified in the fused image, statistical analysis and normalization processing are performed on the features of each abnormal region. For the image sample of aquatic individuals in the i-th image acquisition node, the proportion of body surface color difference of fish in the aquatic individual region in the i-th image, the intensity of body surface texture interruption of fish in the aquatic individual region in the i-th image, and the proportion of multispectral reflectance abnormal regions in the i-th image are obtained. After dimensionless processing, the lesion feature aggregation coefficient BJX is calculated, as follows:

[0092] BJX=a1*Cp i +a2*Td i +a3*Ry i ;

[0093] In the formula, Cp i Td represents the percentage of color difference on the body surface of fish in the i-th image. i Ry represents the intensity of surface texture interruption of a fish in the region of the i-th image. i Let a1, a2, and a3 represent the proportion of the multispectral reflectance abnormal region in the i-th image, and let a1, a2, and a3 represent the weighting coefficients, where 0 < a1 < 1, 0 < a2 < 1, and 0 < a3 < 1, and a1 + a2 + a3 = 1.

[0094] Cp iThe acquisition method is: based on the comparison between the visible light image and the standard template, the color deviation distance is calculated using the color space (such as Lab or HSV); a deviation threshold is set, the proportion of pixels exceeding the threshold in the entire body surface area is calculated to obtain the normalized value;

[0095] Td i The acquisition method is: local directional gradient (such as Sobel, Gabor) filtering is performed on the target area in the fused image; the spatial continuity index (such as structural similarity, directional consistency) of the texture structure is calculated; the texture interruption length or density is normalized to the intensity;

[0096] Ry i The acquisition method is: the spectral reflectance values of each channel of the target area are extracted; the spectral deviation of each pixel is calculated by comparing with the spectral curve of the healthy individual in the training set; the proportion of abnormal pixels is calculated by setting an abnormal criterion; and the normalized proportion is obtained.

[0097] The acquisition method of a1, a2 and a3 is: by collecting and labeling a large number of typical aquatic disease sample images, the distribution characteristics of color deviation, texture interruption and multispectral reflectance anomaly in the diseased area of fish body surface are statistically analyzed, and the distinguishing ability and stability of the three types of features in actual disease recognition are evaluated. Combined with the results of multiple rounds of training and verification of artificial labeling data and deep recognition model, the influence contribution rate of different feature dimensions on recognition accuracy and false positive rate is calculated, and the optimal fusion proportion of the three types of features is determined by referring to the recognition experience of aquatic pathology experts. Finally, under the premise of ensuring stable recognition effect and strong generalization ability, reasonable weighting coefficients a1, a2 and a3 are set, and a1+a2+a3=1 is satisfied to construct the disease feature aggregation coefficient BJX, so as to improve the accuracy and robustness of the disease area candidate judgment.

[0098] In this embodiment, by pixel-level fusion of visible light images and multispectral images, the detailed features of aquatic individuals are significantly enhanced, and the disease features such as color deviation, texture interruption and reflectance discontinuity are more prominent and easy to identify; combined with normalization processing and weight coefficients optimized based on a large number of labeled samples and expert experience, the disease feature aggregation coefficient BJX constructed can accurately reflect the comprehensive performance of multi-dimensional abnormal features, effectively improve the recognition accuracy and robustness of the disease area candidate set, reduce the false positive rate, and enhance the precision diagnosis ability of the system for aquatic diseases in complex environments.

[0099] Embodiment 6

[0100] This embodiment is an explanation and description in embodiment 5, please refer to Figure 1 , specifically, step three further comprises:

[0101] S33, by presetting a second threshold Q2, and comparing the lesion feature aggregation coefficient BJX with the second threshold Q2, a second evaluation result is obtained, including:

[0102] When the lesion feature aggregation coefficient BJX is less than the second threshold Q2, it indicates that the fish body surface feature is in a normal state, and continuous monitoring is performed.

[0103] When the lesion feature aggregation coefficient BJX is greater than or equal to the second threshold Q2, it indicates that the fish body surface feature is in an abnormal state, and there is a risk of potential lesions, a second warning instruction is triggered, and a second strategy is generated: a candidate set of lesion areas is established; step four is entered, and an image multi-channel fitting correction operation is started, the image is marked as a "candidate lesion image", and depth recognition processing is preferentially performed.

[0104] The second threshold Q2 is obtained in the following manner: based on a plurality of aquaculture lesion sample image data, the spatial distribution of a plurality of lesion features such as color deviation, texture interruption, and reflection anomaly in the image is extracted, the distribution density, integration degree, and multi-feature co-occurrence rate are statistically analyzed by region fusion and clustering analysis method, and a reference model of lesion feature aggregation coefficient is constructed. Further, expert scoring data of labeled lesion areas and non-lesion areas are introduced, and the strongest discriminant ability of the aggregation feature combination interval is selected. In combination with the optimal classification boundary in the training process of the depth recognition model, the lesion feature aggregation threshold Q2 that can be used to divide the suspected lesion area and the normal area is determined.

[0105] In this embodiment, by introducing the comparison and analysis of the lesion feature aggregation coefficient BJX and the preset second threshold Q2, the system can efficiently distinguish the abnormal state of the fish body surface, and timely distinguish the normal and potential lesion individuals; when BJX≥Q2, the second warning instruction is automatically triggered and the candidate set of lesion areas is established, the suspected lesion image is preferentially marked and processed, the sensitivity and accuracy of early lesion detection are significantly improved, the real-time dynamic monitoring and rapid response of the breeding environment are promoted, the risk of disease spread is reduced, and the breeding benefit is improved.

[0106] Embodiment 7

[0107] This embodiment is an explanation and description in embodiment 6, please refer to Figure 1 , specifically, step four includes:

[0108] S41, after obtaining the candidate set of lesion areas, the multispectral fusion image in the candidate area is corrected in combination with the light condition, the shooting angle, and the surface reflectivity change, specifically including:

[0109] S411, gray reconstruction correction is performed, the gray offset of the lesion area is corrected, and the brightness is unified.

[0110] S412, edge gradient difference correction is performed, the continuity and clarity of the lesion area edge texture are enhanced;

[0111] S413, local brightness normalization correction is performed, local brightness difference caused by uneven illumination is eliminated, and image contrast is improved;

[0112] S42, by extracting the gray scale reconstruction residual value before and after correction in the ith image, the edge gradient difference value before and after correction in the ith image, and the local brightness normalization index in the ith image, after non-dimensional processing, the spectral consistency fitting coefficient GYNX is calculated and obtained, and the formula is as follows:

[0113]

[0114] In the formula, ΔG i represents the gray scale reconstruction residual value before and after correction in the ith image, represents the edge gradient difference value before and after correction in the ith image, U i represents the local brightness normalization index in the ith image, s1, s2 and s3 represent the fitting weight coefficients, 0

[0115] The acquisition method of s1 is as follows: by collecting multispectral image samples under different water quality and different illumination conditions, and comparing and analyzing the reconstructed gray scale channel with the original image, the influence degree of reconstruction error on image fusion effect is counted. Further, by combining the recognition accuracy change of the image recognition model under different gray scale error levels, a gray scale error sensitivity index system is constructed. Finally, by referring to the convergence stability and error tolerance of the image reconstruction algorithm under the standard sample set, and combining the experience weight setting strategy, the reasonable weight coefficient s1 of the gray scale reconstruction error term is determined, so as to ensure the response ability of the model to the change of gray scale characteristics.

[0116] The acquisition method of s2 is as follows: according to a large number of target image samples with different edge structure complexity, the functional relationship between edge gradient change and lesion feature boundary positioning accuracy is counted. By analyzing the frequency distribution of the gradient difference error of the edge area in the fusion image, the influence intensity of the gradient difference error on the extraction of lesion contour and the recognition of edge sensitive area is quantified. At the same time, by combining the performance of the image enhancement algorithm in gradient preservation, the influence degree of gradient deviation on image stability is comprehensively evaluated, and by referring to the commonly used loss weight setting standard in the field of edge recognition, the weight coefficient s2 of the edge gradient deviation term is determined.

[0117] The acquisition method of s3 is as follows: a plurality of fish body images are collected under different time periods, illumination intensities and shooting angles, an illumination change response database is constructed, and the local brightness normalization parameter U iQuantitative relationship between image fusion consistency. By modeling the correlation between the uneven brightness area of the image and the recognition error, the influence degree of normalization failure on the overall recognition accuracy is extracted. Based on the evaluation results of the combination of image illumination control mechanism, shooting standardization constraint and the robustness of the model to local light fluctuation, the weighting coefficient s3 of the illumination normalization error term is finally determined, which is used to reflect the importance of environmental light change in the overall fitting performance evaluation.

[0118]

[0119] In the formula, M represents the total number of pixels participating in statistics, represents the reference standard gray value of the pth pixel in the ith image, represents the corrected gray value of the pth pixel in the ith image.

[0120]

[0121] In the formula, m represents the total number of edge pixels, represents the gradient amplitude of the qth edge pixel in the ith image before correction, represents the gradient amplitude of the qth edge pixel in the ith image after correction.

[0122]

[0123] In the formula, σ(Lg i ) represents the average variance of local brightness distribution in the ith image.

[0124] In this embodiment, the multispectral fusion image in the lesion area candidate set is corrected by multiple correction steps, including gray reconstruction correction, edge gradient difference correction and local brightness normalization correction. The effects of uneven illumination, shooting angle change and surface reflectivity difference are effectively eliminated, and the image quality and detail performance of the lesion area are significantly improved. The spectral consistency fitting coefficient GYNX is introduced to quantitatively evaluate the correction effect, realize the accurate control and dynamic optimization of image correction quality, greatly improve the accuracy and stability of subsequent lesion recognition, and enhance the adaptability and diagnostic reliability of the system in complex environment.

[0125] Embodiment 8

[0126] This embodiment is an explanation and description in embodiment 7. Please refer to Figure 1 , specifically, step four further comprises:

[0127] S43, by presetting a third threshold Q3, and comparing and analyzing the spectral consistency fitting coefficient GYNX with the third threshold Q3, a third evaluation result is obtained, including:

[0128] When the spectral consistency fitting coefficient GYNX is less than or equal to the third threshold Q3, it indicates that the image fitting correction is qualified, a high-quality third image set is established, and step five is entered;

[0129] When the spectral consistency fitting coefficient GYNX is greater than the third threshold Q3, it indicates that the image fitting correction is unqualified, a third early warning instruction is triggered, and a third strategy is generated: returning to step three to perform image re-fusion operation on the lesion candidate region set, and recording the reason for failure of this round of correction; updating the fusion parameters, and re-performing the correction processing of step four on the current image, and re-calculating until the spectral consistency fitting coefficient GYNX is less than or equal to the third threshold Q3.

[0130] The third threshold Q3 is obtained in the following manner: by collecting multiple batches of aquatic individual image data under different illumination conditions, shooting angles and target postures, performing fitting analysis on the multi-spectral reflection characteristics of each channel, calculating the gray consistency, edge retention and local brightness normalization accuracy, and establishing the correlation model between them and the quality of the fused image. By statistically analyzing the aggregation degree distribution of the spectral feature difference between the lesion area and the healthy area in the fused image, a spectral consistency index system suitable for image fusion quality evaluation is constructed. Finally, referring to the performance of the recognition model in each fitting accuracy interval, the minimum spectral consistency fitting threshold Q3 required for stable recognition performance is determined.

[0131] In this embodiment, by setting the dynamic comparison mechanism of the spectral consistency fitting coefficient GYNX and the preset third threshold Q3, the system can evaluate the correction quality of the fused image in real time, and ensure that only high-quality images meeting the qualified standard enter the subsequent processing. When the correction is unqualified, an early warning is automatically triggered and the re-fusion step is returned, the fusion parameters are intelligently adjusted, the correction effect is cyclically optimized, the stability and accuracy of the image quality are significantly improved, and the reliability and robustness of the overall lesion recognition are enhanced, effectively avoiding the risk of misjudgment or omission caused by image correction failure.

[0132] Embodiment 9

[0133] This embodiment is an explanation and description in embodiment 8. Please refer to Figure 1 , specifically, step five includes:

[0134] S51, by using a convolutional neural network, an image recognition initial model is constructed, and the third image set obtained in step four and the lesion region candidate set constructed in step three are used as training data and labeled data to train and test the image recognition initial model, so as to obtain a trained model for image health anomaly recognition; the trained image recognition initial model is used as a deep image recognition model for lesion image recognition, and the convolutional feature output of the intermediate layer of the third image set is extracted as an image fusion feature vector for representing the individual abnormal structure features, and then the deep image recognition model is continuously trained and performance optimized based on the feature vector, and the normalized score of the i th aquatic individual in the texture structure continuity feature, the normalized score of the i th aquatic individual in the edge gradient integrity feature and the normalized score of the i th aquatic individual in the local brightness consistency feature are obtained, which are used for subsequent image anomaly recognition and health state discrimination.

[0135] In this embodiment, the third image set and the lesion region candidate set are systematically trained by using a convolutional neural network, so as to realize efficient recognition and discrimination of the deep image recognition model for the health anomaly of the aquatic individual; the model not only has good generalization ability, but also can dynamically extract and optimize multi-dimensional feature scores such as texture structure continuity, edge gradient integrity and local brightness consistency, thereby significantly improving the accuracy and robustness of image anomaly detection, effectively supporting real-time health state evaluation, and enhancing the adaptability and practical value of the system in complex aquaculture environments.

[0136] Embodiment 10

[0137] This embodiment is an explanation and description in embodiment 9. Please refer to Figure 1 , specifically, step five further comprises:

[0138] S52, based on the third image set, the normalized score SW i of the i th aquatic individual in the texture structure continuity feature, the normalized score SB i of the i th aquatic individual in the edge gradient integrity feature and the normalized score SL i of the i th aquatic individual in the local brightness consistency feature are obtained by using the trained deep image recognition model.

[0139] PABN = d1*SW i +d2*SB i +d3*SL i ;

[0140] In the formula, d1, d2 and d3 represent weight coefficients, 0 < d1 < 1, 0 < d2 < 1 and 0 < d3 < 1, and d1+d2+d3=1.

[0141] The acquisition method of d1, d2 and d3: through the correlation analysis between the health state annotation results of a large number of aquatic individual image samples and the key fusion feature score data (including texture structure integrity, edge gradient continuity and local brightness consistency) extracted from the third image set, the influence degree of each type of feature on the actual recognition accuracy is counted, and the weight distribution feature is extracted. Combined with professional image recognition model training experience, grid search and cross-validation methods are used to optimize a set of weighting coefficients with stable performance in different environments; in addition, reference is made to relevant research literature in the field of image recognition, industry standard model training paradigm and deep learning feature weighting practical experience, which usually gives importance evaluation method and parameter setting suggestion for each type of feature in multi-feature fusion recognition task. The finally determined d1, d2 and d3 are used as the weighting coefficients of the fusion feature score model, which is used to effectively represent the constituent proportion of individual health abnormal probability, and improve the stability and generalization ability of the recognition model.

[0142] S53, by presetting a fourth threshold Q4, and comparing and analyzing the health abnormal probability coefficient PABN of the i-th target individual with the fourth threshold Q4, a fourth evaluation result is obtained, including:

[0143] When the health abnormal probability coefficient PABN of the i-th target individual is less than or equal to the fourth threshold Q4, it indicates that the health state of the target individual is normal, and the monitoring is continued.

[0144] When the health abnormal probability coefficient PABN of the i-th target individual is greater than the fourth threshold Q4, it indicates that the health state of the target individual is abnormal, a fourth warning instruction is triggered, and a fourth strategy is generated: recording the current image acquisition metadata and recognition result, and constructing image processing log data.

[0145] The acquisition method of the fourth threshold Q4: based on a large number of artificially annotated aquatic individual health state images, a deep image recognition model is trained by using a supervised learning method, and the health abnormal probability value corresponding to each target individual is output. Through the consistency analysis between the recognition result and the artificial judgment result, the ROC curve and the precision-recall rate curve are drawn, and the probability division point with the optimal model recognition performance is determined. At the same time, referring to the health risk warning value setting method commonly used in the aquaculture industry, combined with the recognition false alarm rate and the missed detection rate, the health abnormality judgment threshold Q4 which can be used to trigger abnormal record and log generation is formulated.

[0146] In this embodiment, by combining the continuity of texture structure, the integrity of edge gradient and the consistency of local brightness, the system can more accurately evaluate the health status of aquatic individuals by calculating the health abnormality probability PABN through weighted scoring. The weight optimization method based on large-scale samples improves the recognition stability and generalization ability of the model under different environments. Based on the dynamic judgment mechanism of the preset threshold Q4, timely warning and automatic recording of abnormal individuals are realized, which effectively guarantees the real-time monitoring and health management of the breeding environment, significantly reduces the risk of false positives and false negatives, and improves the overall breeding efficiency and economic benefits.

[0147] The size of the threshold is set for easy comparison. The size of the threshold depends on the amount of sample data and the base number set by the person skilled in the art for each group of sample data; as long as it does not affect the proportional relationship between the parameters and the quantized values.

[0148] The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula close to the true value. The coefficients in the formula are set by the person skilled in the art according to the actual situation. The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application, according to the technical scheme and the inventive concept of the present application, can be equivalent to replace or change, which should be covered within the protection scope of the present application.

Claims

1. An image processing method for industrial robots based on image fusion, characterized in that, The method comprises the following steps: Step one, deploy an industrial robot image acquisition device for operation in an underwater aquaculture environment, real-time acquisition of original visible light images, polarization images and multispectral images, and obtain suspended matter density information, water transparency and illumination intensity change value in the image, generate a first image set; Step two, image filtering and contrast enhancement operation is performed on the first image set to suppress the interference of suspended particulate matter on image imaging quality, and a first corrected image result is obtained; Step three, after completing the first image processing, the region information of aquatic individuals in the image is extracted, and fusion calculation of visible light images and multispectral images is performed on the region of aquatic individuals, the color deviation region, texture interruption region and reflection feature discontinuous region of fish body surface are identified, a second image set is constructed, and a lesion region candidate set is constructed based on the fusion features; Step four, in the image fusion process, according to the changes of illumination condition, shooting angle and surface reflectivity, the multispectral channel reflection features of the fusion image are corrected by multiple fitting, including gray scale reconstruction correction, edge gradient difference correction and local brightness normalization correction, so as to improve the lesion region recognition accuracy and obtain a high-quality third image set; Step five, based on the third image set, through the trained deep image recognition model, the health abnormality probability coefficient PABN of the i-th target individual is calculated, and compared with the preset fourth threshold Q4, if higher than the fourth threshold Q4, the current image acquisition unit data and recognition result are recorded, and an image processing log data is constructed.

2. The image fusion-based industrial robot image processing method according to claim 1, characterized in that, Step one includes: S11, deploy a high-resolution visible light camera, a polarization imaging camera and a multi-channel multispectral imager at the front end of the industrial robot; S12, real-time acquisition of original visible light images of the aquaculture area; acquisition of reflected polarization image information in the water body; acquisition of multispectral reflection feature images of target individuals; S13, by using the local contrast reduction, edge blur degree and scattering light mode in the image, the suspended particle density is obtained through the image filtering response function and the polarization angle change algorithm; based on the reflectivity attenuation characteristics of the water body region under different wave bands, the water body transparency is obtained; through the overall brightness mean value and the local brightness non-uniformity analysis, the per-pixel brightness value is obtained, and a brightness matrix is constructed to reflect the illumination distribution characteristics of the image; a first image set is generated.

3. The image fusion-based industrial robot image processing method according to claim 2, characterized in that, Step two includes: S21, by using edge preserving filter on the first image set, remove the noise points caused by suspended particles in water, while retaining the fish body edge and details; use histogram equalization or local contrast enhancement method to improve the light and dark difference between fish body and background in the image, so that the color and texture features are clearer; combined with the polarization image information, reduce the water surface reflection interference, and correct the color deviation, improve the image reality; S22, by extracting the unit area suspended matter density in the i-th image, the water body transparency at the i-th image position and the local brightness distribution average variance in the i-th image, after non-dimensional processing, the suspended interference coefficient XFX is calculated and obtained.

4. The image fusion-based industrial robot image processing method according to claim 3, characterized in that, Step two also includes: S23, by presetting a first threshold Q1, and comparing and analyzing the suspended interference coefficient XFX with the first threshold Q1, a first evaluation result is obtained, including: When the suspended interference coefficient XFX is less than the first threshold Q1, it indicates that the i-th image is not disturbed by suspended solids under the water body and light conditions, and a first corrected image result is obtained; When the suspended interference coefficient XFX is greater than or equal to the first threshold Q1, it indicates that the i-th image is disturbed by suspended solids under the water body and light conditions, a first warning instruction is triggered, and a first strategy is generated: automatically triggering an image resampling strategy, adjusting the shooting angle of the image acquisition device and the light source compensation parameter, reducing the acquisition frequency by 20% and delaying the acquisition of the next frame of image, and recalculating until the suspended interference coefficient XFX is less than the first threshold Q1.

5. The image fusion-based industrial robot image processing method according to claim 4, characterized in that, Step three includes: S31, extracting the aquatic individual region from the first corrected image, performing pixel-level fusion on the visible light image and the multispectral image in the target region, enhancing the surface detail features, identifying the feature regions in the fused image, including color deviation regions, texture interruption regions and reflection discontinuous regions, and summarizing them into a second image set; S32, based on the color deviation regions, texture interruption regions and reflection discontinuous regions identified in the fused image, statistical analysis and normalization processing are performed on the features of each abnormal region; for the image sample of the aquatic individual in the i-th image acquisition node, the proportion of the surface color difference of the fish in the i-th image, the surface texture interruption intensity of the fish in the i-th image, and the proportion of the multispectral reflectance abnormal region in the i-th image are obtained. After dimensionless processing, the disease feature aggregation coefficient BJX is calculated and obtained.

6. The image fusion-based industrial robot image processing method according to claim 5, characterized in that, Step three also includes: S33, by presetting a second threshold Q2, and comparing and analyzing the disease feature aggregation coefficient BJX with the second threshold Q2, a second evaluation result is obtained, including: When the disease feature aggregation coefficient BJX is less than the second threshold Q2, it indicates that the fish surface features are in a normal state, and continuous monitoring is performed; When the disease feature aggregation coefficient BJX is greater than or equal to the second threshold Q2, it indicates that the fish surface features are in an abnormal state, and there is a risk of potential disease, a second warning instruction is triggered, and a second strategy is generated: establishing a disease region candidate set; entering step four, starting image multi-channel fitting correction operation, marking the image as "candidate disease image", and preferentially performing depth recognition processing.

7. The image fusion-based industrial robot image processing method according to claim 6, characterized in that, Step four includes: S41, after obtaining the disease region candidate set, combining the light conditions, shooting angle and surface reflectance change, the multispectral fusion image in the candidate region is corrected, specifically including: S411, gray reconstruction correction, by correcting the gray offset of the disease region, the brightness is unified; S412, edge gradient difference correction, by enhancing the continuity and clarity of the edge texture of the disease region; S413, local brightness normalization correction, by eliminating the local brightness difference caused by uneven light, the image contrast is improved; S42, by extracting the gray reconstruction residual value before and after correction in the i-th image, the average value of the edge gradient difference before and after in the i-th image, and the local brightness normalization index in the i-th image, after non-dimensional processing, the spectral consistency fitting coefficient GYNX is calculated and obtained.

8. The image fusion-based industrial robot image processing method according to claim 7, characterized in that, Step four further comprises: S43, by presetting a third threshold Q3, and comparing and analyzing the spectral consistency fitting coefficient GYNX with the third threshold Q3, a third evaluation result is obtained, including: When the spectral consistency fitting coefficient GYNX is less than or equal to the third threshold Q3, it indicates that the image fitting correction is qualified, a high-quality third image set is established, and step five is entered; When the spectral consistency fitting coefficient GYNX is greater than the third threshold Q3, it indicates that the image fitting correction is unqualified, a third warning instruction is triggered, and a third strategy is generated: return to step three to perform image re-fusion operation on the lesion candidate region set, and record the reason for the failure of this round of correction; update the fusion parameters, and re-calculate the correction processing of step four for the current image until the spectral consistency fitting coefficient GYNX is less than or equal to the third threshold Q3.

9. The image fusion-based industrial robot image processing method according to claim 8, characterized in that, Step five comprises: S51, by using a convolutional neural network, an image recognition initial model is constructed, and the third image set obtained in step four and the lesion region candidate set constructed in step three are used as training data and labeled data to train and test the image recognition initial model, so as to obtain a trained model for image health anomaly recognition; the trained image recognition initial model is used as a deep image recognition model for lesion image recognition, and the convolution feature output of the intermediate layer of the third image set is extracted as an image fusion feature vector for representing individual body surface abnormal structure features, and then the deep image recognition model is continuously trained and performance optimized based on the feature vector, and the normalized score of the i-th aquatic individual in the texture structure continuity feature, the normalized score of the i-th aquatic individual in the edge gradient integrity feature, and the normalized score of the i-th aquatic individual in the local brightness consistency feature are obtained, which are used for subsequent image anomaly recognition and health state discrimination operation.

10. The image fusion-based industrial robot image processing method according to claim 9, characterized in that, Step five further comprises: S52、based on the third image set, through the trained deep image recognition model, obtain the normalized score SW of the i th aquatic individual in the texture structure continuity feature i , the normalized score SB of the i th aquatic individual in the edge gradient integrity feature i , and the normalized score SL of the i th aquatic individual in the local brightness consistency feature i After dimensionless treatment, the health abnormality probability coefficient PABN of the i th target individual is calculated. S53, by presetting a fourth threshold Q4, and comparing and analyzing the health anomaly probability coefficient PABN of the i-th target individual with the fourth threshold Q4, a fourth evaluation result is obtained, including: When the health anomaly probability coefficient PABN of the i-th target individual is less than or equal to the fourth threshold Q4, it indicates that the target individual is in a normal health state, and continuous monitoring is performed; When the health anomaly probability coefficient PABN of the i-th target individual is greater than the fourth threshold Q4, it indicates that the target individual is in an abnormal health state, a fourth warning instruction is triggered, and a fourth strategy is generated: record the current image acquisition metadata and recognition result, and construct image processing log data.

Images