Cherry defect detection system based on feature fusion attention mechanism

Abstract

The invention relates to the technical field of cherry defect detection, and particularly discloses a cherry defect detection system based on a feature fusion attention mechanism, which is used for solving the problem that the wormhole recognition capability at a sunken area on the surface of an existing cherry is still relatively poor. The method comprises the following steps: acquiring first feature data and second feature data of a cherry sample, determining a concave region of the cherry sample based on the first feature data and the second feature data, extracting defect features according to the concave region of the cherry sample, fusing the defect features of the cherry sample, and judging whether the cherry sample has wormholes or not; according to the method, multiple features are fused, the attention mechanism is introduced, the sunken areas and the wormholes in the surfaces of the cherries are accurately recognized, the problem that existing wormhole detection capacity is poor is effectively solved, and the precision, robustness and efficiency of cherry wormhole recognition are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cherry defect detection, and more specifically, to a cherry defect detection system based on a feature fusion attention mechanism. Background Art

[0002] During the growth process, cherries are easily affected by genetics, environmental pressure, improper use of plant growth regulators, and attacks by pests and diseases, and are easily deformed into stimulated growth cherries, twin cherries, etc. Accurate identification of cherry defects is of great significance for improving its overall quality assessment, reducing economic losses, and optimizing the automated picking system. Traditional detection and identification of cherry defects mainly relies on manual visual inspection, which is time-consuming and labor-intensive and not suitable for large-scale detection. Existing technologies detect and identify cherry defects through feature fusion attention mechanism. For example, existing literature ([1] Dai Dongnan, Ma Rui, Liu Qi, et al. Research on cherry defect detection and identification based on feature fusion attention mechanism [J]. Shandong Agricultural Science, 2024, 56(03): 154-162. DO I: 10.14083 / ji ssn.1001-4942.2024.03.021.) recognizes defective cherries based on feature fusion attention mechanism, which not only saves time and effort, but also can effectively reduce the secondary damage of cherry fruits caused by manual sorting. In addition, the lightweight network structure also improves its feasibility of deployment in mobile intelligent screening equipment. However, the ability to identify insect eyes in the concave area of ​​the cherry surface is still poor.

[0003] The difficulty in identifying insect eyes on the surface of cherries lies mainly in the complex light and shadow changes in the concave area. Due to the change in the curvature of the surface in the concave area, it is easy to cause the reflection angle of the light to deviate, resulting in uneven light in the collected image, forming shadows or bright spots. This phenomenon not only interferes with the texture contrast between the insect eyes and the normal surface, but also may cover up the edge features of the insect eyes. In addition, insect eyes are usually small in size, diverse in shape and have no obvious geometric features, which makes it difficult for traditional image processing methods (such as those based on edge detection or texture analysis) to accurately identify them. In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a cherry defect detection system based on a feature fusion attention mechanism, which is used to solve the problem that the insect eye recognition ability in the concave area of ​​the existing cherry surface is still poor, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A cherry defect detection system based on feature fusion attention mechanism comprises a dual-angle feature capture processing module, a sunken area recognition module and an insect eye discrimination module; the dual-angle feature capture processing module is used to obtain first feature data and second feature data of a cherry sample; the sunken area recognition module is used to determine the sunken area of ​​the cherry sample based on the first feature data and the second feature data; the insect eye discrimination module is used to extract defect features according to the sunken area of ​​the cherry sample, and judge whether the cherry sample has insect eyes according to the defect features; the insect eye discrimination module comprises a sunken area scanning unit, a defect feature analysis unit and a feature fusion detection unit; the sunken area scanning unit is used to obtain the final sunken area R of the cherry sample through a three-dimensional scanner f The defect feature analysis unit is used to obtain the defect features of the cherry sample according to the three-dimensional regional grid model; the defect features of the cherry sample include a first defect feature and a second defect feature; the feature fusion detection unit is used to fuse the defect features of the cherry sample and determine whether there are insect eyes in the cherry sample: by extracting the first defect feature and the second defect feature, the first defect feature and the second defect feature are normalized to obtain the normalized first defect feature ΔZ d ' and the second defect feature α p ′, the defect fusion feature value is calculated based on the feature weighting of the attention mechanism:

[0007]

[0008] ω 1 +ω 2 =1;

[0009] Where: F fu is the defect fusion eigenvalue, ΔZ d ′ is the first defect feature after normalization of the dth surface point, ΔZ d is the first defect feature of the dth surface point, D is the number of surface points, α p ′ is the normalized second defect feature of vertex p, α p is the second defect feature of vertex p, and P is the number of vertices.

[0010] As a further solution of the present invention, the dual-angle feature capture processing module includes an image acquisition unit, an image processing module and a feature data capture unit;

[0011] The image acquisition unit is used to collect a first-angle cherry image and a second-angle cherry image of a cherry sample through a high-definition camera; the dual-angle cherry image is obtained by obtaining the maximum diameter of the concave surface of the cherry sample, taking the midpoint of the maximum diameter as the vertical line as the central axis of the cherry, calculating the center distance between the left and right outermost sides of the cherry and the central axis respectively, taking the center of the left and right center distances as the base point, and obtaining the cherry image at the same height of the base point through a high-definition camera, the cherry image taken by the high-definition camera on the left is the first-angle cherry image, and the cherry image taken by the high-definition camera on the right is the second-angle cherry image;

[0012] The image processing module is used for performing histogram equalization and noise removal on the cherry image at the first angle and the cherry image at the second angle;

[0013] The feature data capture unit is used to extract the pixel values ​​of the first-angle cherry image and the second-angle cherry image output by the image processing module to obtain the first feature data and the second feature data of the cherry sample; the first feature data is the pixel value of the first-angle cherry image; the second feature data is the pixel value of the second-angle cherry image.

[0014] As a further solution of the present invention, the sunken area identification module includes a primary data acquisition unit, a sunken area calculation unit and a sunken area positioning unit;

[0015] The primary data acquisition unit is used to acquire the first characteristic data and the second characteristic data of the cherry sample;

[0016] The sunken area calculation unit is used to construct a sunken area pixel coefficient calculation model according to the first feature data and the second feature data, and preliminarily determine the sunken area of ​​each cherry sample;

[0017] The depressed area positioning unit determines the depressed area position based on the depressed area pixel coefficient of the cherry sample.

[0018] As a further solution of the present invention, the sunken area calculation unit is used to construct a sunken area pixel coefficient calculation model according to the first feature data and the second feature data, and preliminarily determine the sunken area of ​​each cherry sample: the first feature data and the second feature data of each angle of a single cherry sample are imported into the sunken area pixel coefficient calculation model to calculate the sunken area pixel coefficient. The calculation formula of the sunken area pixel coefficient calculation model is:

[0019]

[0020]

[0021] Where: β p is the pixel coefficient of the concave area, β ais the pixel coefficient of the first concave area, is the first angle cherry image at (x 1 ,y 1 ), is the mean of the first feature data in the cherry image at the first angle, is the largest first feature data in the cherry image at the first angle, is the minimum first feature data in the cherry image at the first angle, β b is the pixel coefficient of the second concave area, is the second angle cherry image at (x 2 ,y 2 ), is the mean of the second feature data in the cherry image at the second angle, is the maximum second feature data in the cherry image at the second angle, is the minimum second feature data in the cherry image at the second angle.

[0022] As a further solution of the present invention, the sunken area positioning unit determines the position of the sunken area based on the sunken area pixel coefficient of the cherry sample: the sunken area pixel coefficient of the single cherry sample and the first feature data and the second feature data are respectively extracted, and the sunken area pixel coefficient is compared with the first feature data; if the first feature data is greater than or equal to the sunken area pixel coefficient, the position corresponding to the first feature data is not a sunken area; if the sunken area pixel coefficient is less than the first feature data, the position corresponding to the first feature data is a sunken area; the coordinates for determining the position of the sunken area are obtained to generate a first sunken area coordinate set R 1 ={(x 1 ,y 1 )};

[0023] The pixel coefficient of the sunken area is compared with the second feature data. If the second feature data is greater than or equal to the pixel coefficient of the sunken area, the position corresponding to the second feature data is not a sunken area; if the pixel coefficient of the sunken area is less than the second feature data, the position corresponding to the second feature data is a sunken area; the coordinates of the sunken area position are obtained to generate a second sunken area coordinate set R 2 ={(x 2 ,y 2 )};

[0024] Get the first concave area coordinate set r 1 ={(x 1 ,y 1 )} and the second concave area coordinate set r 2 ={(x 2 ,y 2)}, based on the intersection of the first concave area coordinate set and the second concave area coordinate set, determine the final concave area R of the cherry sample f =R 1 ∩R 2 .

[0025] As a further solution of the present invention, the defect feature analysis unit is used to obtain defect features of the cherry sample according to the three-dimensional regional grid model analysis;

[0026] The first defect feature is calculated by fitting the three-dimensional regional grid model, the depth mean of the final concave area is calculated, and the difference between each surface point and the depth mean is obtained as the first defect feature, that is:

[0027] ΔZ d =Z d -Z surface ;

[0028] Where: ΔZ d is the first defect feature of the dth surface point, Z d is the depth value of the dth surface point, Z surface is the mean depth of the final concave area;

[0029] The second defect feature is calculated by calculating the normal vector of each mesh vertex according to the three-dimensional regional mesh model, and calculating the normal deviation angle according to the normal vector of each mesh vertex as the second defect feature, that is:

[0030]

[0031] Where: α p is the second defect characteristic of vertex p, is the normal vector of vertex p, is the normal vector of the neighborhood point of vertex p, arccos is the inverse cosine function of the angle between two normal vectors, and N is the number of neighborhood vertices.

[0032] The present invention discloses a cherry defect detection system based on a feature fusion attention mechanism, and has the following technical effects and advantages: the present invention obtains first feature data and second feature data of a cherry sample, determines a sunken area of ​​the cherry sample based on the first feature data and the second feature data, extracts defect features according to the sunken area of ​​the cherry sample, fuses the defect features of the cherry sample, and judges whether there are insect eyes in the cherry sample; the first feature data can effectively capture the depth information of the sunken area and accurately describe the degree of the sunken area, and the second feature data can capture the sharpness and change trend of the edge of the sunken area, and supplement the deficiency of the depth feature; the feature weights are dynamically allocated according to the detection scene, so that the system can flexibly adapt to the diverse characteristics of the cherry samples, reduce the misjudgment or missed detection that may be caused by a single feature, and improve the accuracy of recognition; the defect feature value is calculated by feature fusion and compared with a threshold value, the position of the insect eye is accurately marked, and the natural depression and the insect eye area can be effectively distinguished; the present invention accurately identifies the sunken area and the insect eye on the cherry surface by fusing multiple features and introducing an attention mechanism, effectively solves the problem of poor existing insect eye detection capability, and improves the accuracy, robustness and efficiency of cherry insect eye recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the configuration of the image acquisition unit provided by the present invention for collecting cherry images;

[0034] Figure 2 A structural schematic diagram of a cherry defect detection system based on a feature fusion attention mechanism provided by the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solution in the present invention. Obviously, the described technical solution is only a part of the present invention, not all of it. Based on the technical solution in the present invention, all other technical solutions obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] A cherry defect detection system based on feature fusion attention mechanism includes a dual-angle feature capture processing module, a sunken area recognition module, and an insect eye discrimination module; the dual-angle feature capture processing module is connected to the sunken area recognition module, and the sunken area recognition module is connected to the insect eye discrimination module;

[0037] The dual-angle feature capture processing module is used to obtain the first feature data and the second feature data of the cherry sample;

[0038] The sunken area recognition module is used to determine the sunken area of ​​the cherry sample based on the first feature data and the second feature data;

[0039] The insect eye identification module is used to extract defect features based on the sunken areas of cherry samples and determine whether there are insect eyes in the cherry samples based on the defect features.

[0040] It should be noted that the cherry samples are cherries with insect holes in several concave areas on the surface.

[0041] Specifically, the dual-angle feature capture processing module includes an image acquisition unit, an image processing module and a feature data capture unit; the image acquisition unit is connected to the image processing module, and the image processing module is connected to the feature data capture unit.

[0042] The image acquisition unit is used to collect a first-angle cherry image and a second-angle cherry image of a cherry sample through a high-definition camera; the dual-angle cherry image is obtained by obtaining the maximum diameter of the concave surface of the cherry sample, taking the midpoint of the maximum diameter as the vertical line as the central axis of the cherry, calculating the center distance between the left and right outermost sides of the cherry and the central axis respectively, taking the center of the left and right center distances as the base point, and obtaining the cherry image at the same height of the base point through a high-definition camera, the cherry image taken by the high-definition camera on the left is the first-angle cherry image, and the cherry image taken by the high-definition camera on the right is the second-angle cherry image;

[0043] like Figure 1 The image acquisition unit shown is a configuration diagram for collecting cherry images; it includes a high-definition camera 1, a filter 2, a cherry sample 3, a black matte background 4, a liquid light guide 5, a halogen light source 6, a monochromator 7 and a computer 8. The halogen light source 6 provides illumination, and the light is filtered by the monochromator 7 for a specific wavelength and then transmitted to the cherry sample through the liquid light guide 5. The sample image is collected by the high-definition camera 1 and optimized in conjunction with the filter 2. The final image is transmitted to the computer 8 for processing and analysis. The black matte background is used to reduce interference and make the characteristics of the cherry sample more prominent.

[0044] The image processing module is used for performing histogram equalization and noise removal on the cherry image at the first angle and the cherry image at the second angle;

[0045] The feature data capture unit is used to extract the pixel values ​​of the first-angle cherry image and the second-angle cherry image output by the image processing module to obtain the first feature data and the second feature data of the cherry sample; the first feature data is the pixel value of the first-angle cherry image; the second feature data is the pixel value of the second-angle cherry image.

[0046] Shooting cherry samples from two angles can cover different directions and surface features of the sunken area, reduce the information that may be missed by single-angle images, and accurately align the sunken area through the acquisition method based on the maximum diameter and the central axis, thereby improving the efficiency of capturing target features. The combination of dual-angle images can provide spatial information of surface concavity and convexity, which helps to more accurately identify the sunken features of cherry samples. The feature data of images at different angles can be used to analyze the changing characteristics of the sunken area under changes in viewing angle, thereby improving the robustness of the defect detection algorithm. The image contrast is improved to make the detailed features clearer, especially in the case of complex lighting conditions or reflective sample surfaces, to eliminate noise interference in the acquired images, enhance the image quality, and ensure that the extracted feature data is accurate and reliable. Feature data is extracted directly based on pixel values, avoiding complex intermediate calculations and transformations, ensuring the integrity and accuracy of the data, and providing pixel feature data in two directions, which is convenient for subsequent difference analysis or fusion algorithms to enhance the recognition ability of sunken areas.

[0047] The sunken area recognition module includes a primary data acquisition unit, a sunken area calculation unit and a sunken area positioning unit, the primary data acquisition unit is connected to the sunken area calculation unit, and the sunken area calculation unit is connected to the sunken area positioning unit;

[0048] The primary data acquisition unit is used to acquire the first characteristic data and the second characteristic data of the cherry sample;

[0049] The sunken area calculation unit is used to construct a sunken area pixel coefficient calculation model according to the first feature data and the second feature data, and preliminarily determine the sunken area of ​​each cherry sample;

[0050] The depressed area positioning unit determines the depressed area position based on the depressed area pixel coefficient of the cherry sample.

[0051] The sunken area calculation unit is used to construct a sunken area pixel coefficient calculation model according to the first feature data and the second feature data, and preliminarily determine the sunken area of ​​each cherry sample: the first feature data and the second feature data of each angle of a single cherry sample are imported into the sunken area pixel coefficient calculation model to calculate the sunken area pixel coefficient. The calculation formula of the sunken area pixel coefficient calculation model is:

[0052]

[0053] Where: β p is the pixel coefficient of the concave area, β a is the pixel coefficient of the first concave area, is the first angle cherry image at (x 1 ,y 1 ), is the mean of the first feature data in the cherry image at the first angle, is the largest first feature data in the cherry image at the first angle, is the minimum first feature data in the cherry image at the first angle, β b is the pixel coefficient of the second concave area, is the second angle cherry image at (x 2 ,y 2 ), is the mean of the second feature data in the cherry image at the second angle, is the maximum second feature data in the cherry image at the second angle, is the minimum second feature data in the cherry image at the second angle.

[0054] The pixel coefficient of the concave area is constructed by using the feature data of the first angle and the second angle to avoid the problem of missed detection caused by a single-angle image. The fusion of dual-angle data can provide more comprehensive sample surface information, especially the characteristics of complex concave areas; the feature data of different angles are normalized by calculating the mean, maximum and minimum values ​​to reduce the influence of interference factors such as background illumination or reflection; the overall concave degree of the sample surface is judged by calculating the average value of the pixel coefficient of the concave area to reduce the error caused by local outliers; the concave area positioning unit can accurately locate the position of the concave area by analyzing the pixel coefficient of the concave area, thereby improving the ability to accurately detect concave; the concave detection is divided into three modules: primary data acquisition, concave area calculation, and concave area positioning. Each module has a clear division of labor to reduce repeated calculations. Only the pixels of the concave area are calculated instead of complex processing of the entire image, which significantly improves the calculation efficiency. The mean and normalization calculation of the pixel data can smooth the noise in the image, and the noise has little effect on the overall pixel coefficient of the concave area.

[0055] Specifically, the sunken area positioning unit determines the position of the sunken area based on the sunken area pixel coefficient of the cherry sample: extract the sunken area pixel coefficient of the single cherry sample and the first feature data and the second feature data respectively, compare the sunken area pixel coefficient with the first feature data, if the first feature data is greater than or equal to the sunken area pixel coefficient, then the position corresponding to the first feature data is not a sunken area; if the sunken area pixel coefficient is less than the first feature data, then the position corresponding to the first feature data is a sunken area; obtain the coordinates of the sunken area position to generate a first sunken area coordinate set R 1 ={(x 1 ,y 1 )};

[0056] The pixel coefficient of the sunken area is compared with the second feature data. If the second feature data is greater than or equal to the pixel coefficient of the sunken area, the position corresponding to the second feature data is not a sunken area; if the pixel coefficient of the sunken area is less than the second feature data, the position corresponding to the second feature data is a sunken area; the coordinates of the sunken area position are obtained to generate a second sunken area coordinate set R 2 ={(x 2 ,y 2 )};

[0057] Get the first concave area coordinate set R 1 ={(x 1 ,y 1 )} and the second concave area coordinate set R 2 ={(x 2 ,y 2 )}, based on the intersection of the first concave area coordinate set and the second concave area coordinate set, determine the final concave area R of the cherry sample f =R 1 ∩R 2 .

[0058] The first feature data and the second feature data are respectively compared with the pixel coefficients of the concave area to generate two independent coordinate sets of the concave area. Through the intersection operation, the error that may be caused by the single-angle feature is eliminated, thereby improving the reliability and accuracy of the positioning result. By comparing the pixel coefficients of the concave area with the feature data, the pixel position of the concave area can be accurately identified to avoid incorrectly marking the non-concave area as a concave area. The single-angle feature data may be affected by local reflection, acquisition angle, etc. By combining the feature data of the two angles and taking the intersection, the deviation caused by the abnormality of the single-angle data can be effectively eliminated. By comparing the pixel value with the pixel coefficient of the concave area, the concave area can be quickly judged and the calculation result can be obtained. The amount of calculation is low, and it can process large quantities of samples efficiently. It is not limited to the current feature data definition, and can be expanded to other features (such as curvature, normal deviation, etc.). By adjusting the calculation method of the pixel coefficient of the concave area, it can adapt to more detection scenarios. At this stage, it combines the feature data of two angles, and in the future it can be expanded to multi-angle feature fusion (such as adding top or bottom viewing angles) to further improve the comprehensiveness of concave area detection. The independent comparison results of the first feature data and the second feature data can be analyzed separately to help judge the quality of the specific angle data and the potential error source. By eliminating inconsistent areas through intersection, false alarms caused by single-angle acquisition can be further reduced, and the credibility of the results can be improved.

[0059] Specifically, the insect eye discrimination module includes a recessed area scanning unit, a defect feature analysis unit and a feature fusion detection unit; the recessed area scanning unit is connected to the defect feature analysis unit, and the defect feature analysis unit is connected to the feature fusion detection unit.

[0060] The concave area scanning unit is used to obtain the final concave area R of the cherry sample through a three-dimensional scanner. f Three-dimensional regional grid model;

[0061] The defect feature analysis unit is used to obtain defect features of the cherry sample according to the three-dimensional regional grid model analysis; the defect features of the cherry sample include a first defect feature and a second defect feature;

[0062] The feature fusion detection unit is used to fuse the defect features of the cherry samples and determine whether there are insect eyes in the cherry samples.

[0063] Specifically, the defect feature analysis unit is used to obtain defect features of the cherry sample according to the three-dimensional regional grid model analysis;

[0064] The first defect feature is calculated by fitting the three-dimensional regional grid model, the depth mean of the final concave area is calculated, and the difference between each surface point and the depth mean is obtained as the first defect feature, that is:

[0065] ΔZ d =Z d -Z surface ;

[0066] Where: ΔZ d is the first defect feature of the dth surface point, Z d is the depth value of the dth surface point, Z surface is the mean depth of the final concave area;

[0067] The second defect feature is calculated by calculating the normal vector of each mesh vertex according to the three-dimensional regional mesh model, and calculating the normal deviation angle according to the normal vector of each mesh vertex as the second defect feature, that is:

[0068]

[0069] Where: α p is the second defect characteristic of vertex p, is the normal vector of vertex p, is the normal vector of the neighborhood point of vertex p, arccos is the inverse cosine function of the angle between two normal vectors, and N is the number of neighborhood vertices.

[0070] The three-dimensional regional mesh model of the final sunken area of ​​the cherry sample is obtained using a three-dimensional scanner, which can capture the complete three-dimensional morphology of the sunken area, including the depth, shape, edge characteristics, etc. of the sunken area, which is more accurate than a two-dimensional image. For wormholes with complex shapes or irregular sunken areas (such as sharp and protruding wormhole edges), the three-dimensional mesh model can fully reflect their characteristics and avoid missing information from single-angle or two-dimensional data. The mesh model provided by the three-dimensional scanning contains vertex coordinates and normal information, providing rich data for subsequent depth and normal feature calculations. By calculating the difference between each point and the mean depth of the area, the depth change characteristics inside the sunken area can be directly reflected, especially the wormhole area has a deeper depth and greater internal changes, which is easy to distinguish from natural depressions. Quantitative features can be provided for both shallow depressions (such as natural depressions on the surface) and deep depressions (such as wormholes), which is suitable for defect analysis of different degrees. The edges of wormholes are usually sharper and the normal deviation angle is larger. By calculating the difference between each point and the mean depth of the area, the depth change characteristics inside the sunken area can be directly reflected. In particular, the wormhole area has a deeper depth and greater internal changes, which is easy to distinguish from natural depressions. Quantitative features can be provided for both shallow depressions (such as natural depressions on the surface) and deep depressions (such as wormholes), which is suitable for defect analysis of different degrees. The edges of wormholes are usually sharper and the normal deviation angle is larger. The deviation angle between the normal of each vertex and the normal of the neighborhood can effectively identify the local mutation characteristics of the concave area; compared with the depth feature, the normal feature can better reflect the boundary shape of the concave area and make the outline of the worm eye area clearer. Even when the depth difference is not significant, the normal deviation angle is still sensitive to local shape changes (such as sharp worm eye areas), and can be used as a supplementary feature to improve the accuracy of discrimination; the first defect feature (depth) reflects the overall change of the concave area, and the second defect feature (normal) captures local sharp changes. The combination of the two can more comprehensively describe the characteristics of worm eyes. The depth feature is sensitive to the severity of the concave, while the normal feature is sensitive to the sharpness of the edge. The possibility of misjudgment of a single feature is reduced through fusion discrimination, and the missed detection rate and false detection rate are significantly reduced, making the worm eye discrimination result more reliable; the feature fusion detection unit comprehensively analyzes the first and second defect features, can intelligently determine whether there are worm eyes in the cherry sample, and realize automatic classification and output.

[0071] Specifically, the feature fusion detection unit is used to fuse the defect features of the cherry sample and determine whether the cherry sample has insect eyes: by extracting the first defect feature and the second defect feature, normalizing the first defect feature and the second defect feature, and obtaining the normalized first defect feature ΔZ d ' and the second defect feature α p ′, the normalized formula is:

[0072]

[0073] Where: ΔZ d ′ is the first defect feature after normalization of the dth surface point, ΔZ d is the first defect feature of the dth surface point, ΔZ min is the minimum value in the first defect feature, ΔZ max is the maximum value of the first defect feature, α p′ is the normalized second defect feature of vertex p, α p is the second defect characteristic of vertex p, α min is the minimum value of the second defect feature, α max is the maximum value in the second defect feature;

[0074] The defect fusion feature value is calculated based on the feature weighting of the attention mechanism:

[0075]

[0076] ω 1 +ω 2 =1;

[0077] Where: F fu is the defect fusion eigenvalue, ΔZ d ′ is the first defect feature after normalization of the dth surface point, ΔZ d is the first defect feature of the dth surface point, D is the number of surface points, α p ′ is the normalized second defect feature of vertex p, α p is the second defect characteristic of vertex p, and P is the number of vertices;

[0078] Obtain the defect fusion feature value, and compare the defect fusion feature value with the preset defect fusion feature threshold. If the defect fusion feature value is greater than or equal to the preset defect fusion feature threshold, mark the cherry sample as having insect holes; if the defect fusion feature value is less than the preset defect fusion feature threshold, there is no need to mark the cherry sample as having insect holes.

[0079] By introducing normalization processing, weighted calculation of attention mechanism and feature fusion feature value threshold determination, accurate detection and marking of insect eye defects are achieved; the dimensions and value ranges of the first defect feature (depth difference) and the second defect feature (normal deviation angle) may be different. Normalizing the two features to the same range (0 to 1) eliminates the influence of the feature dimensions and ranges, making subsequent fusion calculations more reasonable; different cherry samples may have different concave features (such as some samples have deeper concave and other samples have sharper edges). Through the weight adjustment mechanism, the system can adapt to the discrimination needs of different types of insect eyes. After the first defect feature and the second defect feature are weighted and fused according to the weight ratio, the defect fusion feature generated is The value can more comprehensively describe the comprehensive characteristics of the insect eye, which includes both depth information and surface morphology. The threshold judgment process has a small amount of calculation and is suitable for assembly line detection of large quantities of cherry samples, which improves detection efficiency. The defect fusion feature threshold can be flexibly adjusted according to different detection scenarios and sample characteristics. For example, the threshold can be lowered in more stringent scenarios to reduce missed detections, and the threshold can be increased in ordinary scenarios to reduce false detections. The first defect feature (depth information) describes the degree of depression of the insect eye, and the second defect feature (normal deviation angle) describes the edge sharpness. The combination of the two more comprehensively reflects the characteristics of the insect eye and significantly improves the detection accuracy. The weight adjustment mechanism enables the system to adapt to different samples and detection conditions, thereby improving the adaptability and reliability of the overall detection.

[0080] The embodiment of the present invention obtains first feature data and second feature data of a cherry sample, determines a sunken area of ​​the cherry sample based on the first feature data and the second feature data, extracts defect features according to the sunken area of ​​the cherry sample, fuses the defect features of the cherry sample, and determines whether there are insect holes in the cherry sample; the first feature data can effectively capture the depth information of the sunken area and accurately describe the degree of the sunken area, and the second feature data can capture the sharpness and change trend of the edge of the sunken area, and supplement the deficiency of the depth feature; the feature weights are dynamically allocated according to the detection scene, so that the system can flexibly adapt to the diverse characteristics of the cherry samples, reduce the misjudgment or missed detection that may be caused by a single feature, and improve the accuracy of recognition; the defect feature value is calculated by feature fusion and compared with the threshold, the position of the insect hole is accurately marked, and the natural depression and the insect hole area can be effectively distinguished; the present invention accurately identifies the sunken area and the insect hole on the cherry surface by fusing multiple features and introducing an attention mechanism, effectively solves the problem of poor existing insect eye detection ability, and improves the accuracy, robustness and efficiency of cherry insect eye recognition.

[0081] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0082] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cherry defect detection system based on feature fusion attention mechanism, comprising a dual-angle feature capture processing module, a concave area recognition module, and an insect eye discrimination module; characterized in that: The dual-angle feature capture processing module is used to obtain the first feature data and the second feature data of the cherry sample; The sunken area recognition module is used to determine the sunken area of ​​the cherry sample based on the first feature data and the second feature data; The insect eye identification module is used to extract defect features according to the sunken area of ​​the cherry sample, and determine whether the cherry sample has insect eyes according to the defect features; the insect eye identification module includes a sunken area scanning unit, a defect feature analysis unit and a feature fusion detection unit; the sunken area scanning unit is used to obtain the final sunken area R of the cherry sample through a three-dimensional scanner f Three-dimensional regional grid model; The defect feature analysis unit is used to obtain defect features of cherry samples according to the three-dimensional regional grid model analysis; The defect characteristics of the cherry sample include a first defect characteristic and a second defect characteristic; feature The fusion detection unit is used to fuse the defect features of the cherry sample and determine whether the cherry sample has insect eyes: by extracting the first defect feature and the second defect feature, normalizing the first defect feature and the second defect feature, and obtaining the normalized first defect feature ΔZ d ′ And the second defect feature α p ′ , the defect fusion feature value is calculated based on the feature weighting of the attention mechanism: ω1+ω2=1; Where: F fu is the defect fusion eigenvalue, ΔZ d ′ is the first defect feature after normalization of the dth surface point, ΔZ d is the first defect feature of the dth surface point, D is the number of surface points, α p ′ is the normalized second defect feature of vertex p, α p is the second defect feature of vertex p, and P is the number of vertices.

2. The cherry defect detection system based on feature fusion attention mechanism according to claim 1 is characterized in that: The dual-angle feature capture and processing module includes an image acquisition unit, an image processing module and a feature data capture unit; The image acquisition unit is used to collect a first-angle cherry image and a second-angle cherry image of a cherry sample through a high-definition camera; the dual-angle cherry image is obtained by obtaining the maximum diameter of the concave surface of the cherry sample, taking the midpoint of the maximum diameter as the vertical line as the central axis of the cherry, calculating the center distance between the left and right outermost sides of the cherry and the central axis respectively, taking the center of the left and right center distances as the base point, and obtaining the cherry image at the same height of the base point through a high-definition camera, the cherry image taken by the high-definition camera on the left is the first-angle cherry image, and the cherry image taken by the high-definition camera on the right is the second-angle cherry image; The image processing module is used for performing histogram equalization and noise removal on the cherry image at the first angle and the cherry image at the second angle; The feature data capturing unit is used to extract the pixel values ​​of the first angle cherry image and the second angle cherry image output by the image processing module to obtain the first feature data and the second feature data of the cherry sample; The first feature data is the pixel value of the cherry image at the first angle; The second feature data is the pixel value of the cherry image at the second angle.

3. The cherry defect detection system based on feature fusion attention mechanism according to claim 1 is characterized in that: The sunken area recognition module includes a primary data acquisition unit, a sunken area calculation unit and a sunken area positioning unit; The primary data acquisition unit is used to acquire the first characteristic data and the second characteristic data of the cherry sample; The sunken area calculation unit is used to construct a sunken area pixel coefficient calculation model according to the first feature data and the second feature data, and preliminarily determine the sunken area of ​​each cherry sample; The depressed area positioning unit determines the depressed area position based on the depressed area pixel coefficient of the cherry sample.

4. The cherry defect detection system based on feature fusion attention mechanism according to claim 3 is characterized in that: The sunken area calculation unit is used to construct a sunken area pixel coefficient calculation model according to the first feature data and the second feature data, and preliminarily determine the sunken area of ​​each cherry sample: the first feature data and the second feature data of each angle of a single cherry sample are imported into the sunken area pixel coefficient calculation model to calculate the sunken area pixel coefficient. The calculation formula of the sunken area pixel coefficient calculation model is: Where: β p is the pixel coefficient of the concave area, β a is the pixel coefficient of the first concave area, is the first feature data at (x1, y1) in the cherry image at the first angle, is the mean of the first feature data in the cherry image at the first angle, is the largest first feature data in the cherry image at the first angle, is the minimum first feature data in the cherry image at the first angle, β b is the pixel coefficient of the second concave area, is the second feature data at (x2, y2) in the cherry image at the second angle, is the mean of the second feature data in the cherry image at the second angle, is the maximum second feature data in the cherry image at the second angle, is the minimum second feature data in the cherry image at the second angle.

5. The cherry defect detection system based on feature fusion attention mechanism according to claim 3 is characterized in that: The sunken area positioning unit determines the position of the sunken area based on the sunken area pixel coefficient of the cherry sample: extract the sunken area pixel coefficient of the single cherry sample and the first feature data and the second feature data respectively, compare the sunken area pixel coefficient with the first feature data, if the first feature data is greater than or equal to the sunken area pixel coefficient, then the position corresponding to the first feature data is not a sunken area; if the sunken area pixel coefficient is less than the first feature data, then the position corresponding to the first feature data is a sunken area; obtain the coordinates for determining the position of the sunken area, and generate a first sunken area coordinate set R1={(x1,y1)}; Compare the pixel coefficient of the sunken area with the second feature data. If the second feature data is greater than or equal to the pixel coefficient of the sunken area, the position corresponding to the second feature data is not the sunken area; if the pixel coefficient of the sunken area is less than the second feature data, the position corresponding to the second feature data is the sunken area; obtain the coordinates of the sunken area position, and generate a second sunken area coordinate set R2 = {(x2, y2)}; Obtain the first concave area coordinate set R1 = {(x1, y1)} and the second concave area coordinate set R2 = {(x2, y2)}, and determine the final concave area R of the cherry sample based on the intersection of the first concave area coordinate set and the second concave area coordinate set. f =R1∩R2.

6. The cherry defect detection system based on feature fusion attention mechanism according to claim 1, characterized in that: The defect feature analysis unit is used to obtain defect features of cherry samples according to the three-dimensional regional grid model analysis; The first defect feature is calculated by fitting the three-dimensional regional grid model, the depth mean of the final concave area is calculated, and the difference between each surface point and the depth mean is obtained as the first defect feature, that is: ΔZ d =Z d -Z surface ; Where: ΔZ d is the first defect feature of the dth surface point, Z d is the depth value of the dth surface point, Z surface is the mean depth of the final concave area; The second defect feature is calculated by calculating the normal vector of each mesh vertex according to the three-dimensional regional mesh model, and calculating the normal deviation angle according to the normal vector of each mesh vertex as the second defect feature, that is: Where: α p is the second defect characteristic of vertex p, is the normal vector of vertex p, is the normal vector of the neighborhood point of vertex p, arccos is the inverse cosine function of the angle between two normal vectors, and N is the number of neighborhood vertices.

Images