Grain depot cleaning robot visual identification platform and method based on deep learning

Abstract

The invention discloses a grain depot cleaning robot visual identification platform and method based on deep learning, and relates to the technical field of robot visual identification, and the method comprises the following steps: carrying out the comprehensive image collection of different regions in a grain depot environment through a visual system on a grain depot cleaning robot; according to the method, an image is divided into a plurality of sub-regions, for each sub-region, key features reflecting existence of potential glass fragments are extracted from the collected image, the extracted features are analyzed, and it is ensured that a deep learning model can receive accurate input. According to the invention, accurate model input is ensured through image acquisition and feature extraction, glass fragments are preliminarily identified by using a deep learning model and hidden danger areas are predicted, secondary identification is carried out after self-adaptive sharpening edge enhancement, the glass fragments are accurately marked and a cleaning instruction is generated, missing detection and false detection are reduced, the cleaning efficiency and accuracy are improved, and the method is suitable for large-scale popularization and application. Equipment damage and food safety risks are reduced, and grain depot operation safety and benefits are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of robot vision recognition, and in particular to a grain storage cleaning robot vision recognition platform and method based on deep learning. Background Art

[0002] The visual recognition technology of grain depot cleaning robots based on deep learning refers to the use of deep learning algorithms, especially deep learning models such as convolutional neural networks (CNNs), to analyze and process image or video data in the grain depot environment, so as to identify and classify different objects, obstacles or environmental features. In the application of grain depot cleaning robots, this visual recognition technology can help the robot identify problems such as debris in the grain pile, missed grains, damaged equipment or irregular stacking methods, and then perform precise cleaning and maintenance operations. By continuously learning from a large number of annotated images, the deep learning model can automatically extract useful features from the environment, perform object detection, classification and positioning, and thus realize automated and efficient cleaning work in complex grain depot environments. This technology not only improves the accuracy and reliability of the robot in actual operation, but also reduces the need for manual intervention, improving work efficiency and safety.

[0003] Glass fragments are difficult to be identified by deep learning models, mainly because their transparent or translucent properties result in a lack of significant texture and color features visually, and they easily blend into the background environment (such as grain piles or the ground). In addition, glass fragments have various forms and may present irregular shapes and reflective properties. This high reflectivity will produce strong light spots or artifacts under different lighting conditions, further interfering with the model's feature extraction and classification capabilities. These characteristics make it difficult for the model to capture the stable characteristics of glass fragments during training, thereby reducing the accuracy and robustness of recognition. Glass fragments that are not cleaned in time may cause serious damage to grain processing equipment and even cause safety accidents, such as equipment edge wear, conveyor belt punctures, or grain quality contamination, bringing economic and safety risks that cannot be ignored. Therefore, how to efficiently identify glass fragments in a grain warehouse environment is an urgent problem that needs to be solved.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a grain warehouse cleaning robot visual recognition platform and method based on deep learning. Through comprehensive image acquisition and feature extraction, the accuracy of deep learning model input is ensured, and a pre-trained model is used to perform preliminary identification of glass fragments and hidden danger prediction. For hidden danger areas, adaptive edge sharpening technology is used to enhance edge features, and secondary identification is performed to improve detection accuracy. Finally, the glass fragment area is accurately marked and cleaning instructions are generated, which effectively reduces missed detections and false detections, improves cleaning efficiency and accuracy, reduces equipment damage and food safety risks, and improves the safety and economic benefits of grain warehouse operations, so as to solve the problems in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a visual recognition method of a grain storage cleaning robot based on deep learning, comprising the following steps:

[0007] The vision system on the grain storage cleaning robot is used to collect comprehensive images of different areas in the grain storage environment;

[0008] Divide the image into several sub-regions. For each sub-region, extract key features that reflect the presence of potential glass fragments from the acquired image and analyze the extracted features to ensure that the deep learning model can receive accurate input.

[0009] The analyzed features are input into the pre-trained deep learning model, and the glass fragments in the grain warehouse are identified through the deep learning model to predict the glass fragment hidden danger area and glass fragment area in the grain warehouse;

[0010] Mark the glass fragments potential areas that are initially identified and depict the edges of the marked areas to provide a clear target range for further detailed processing;

[0011] For the marked glass fragments potential risk areas, the edge sharpening intensity is adaptively adjusted based on the prediction results of the deep learning model to enhance the edge features of the glass fragments;

[0012] The sharpened edge features are re-input into the deep learning model for secondary recognition, further improving the detection accuracy of the deep learning model for glass fragments;

[0013] The finally identified glass fragment area and its boundary information are output, and a cleaning instruction is generated for the cleaning robot to execute.

[0014] Preferably, the specific steps of performing comprehensive image acquisition of different areas in the grain storage environment are as follows:

[0015] According to the layout and environmental characteristics of the grain warehouse, the area where images need to be collected is clearly identified, and a collection strategy is formulated, including image resolution, collection frequency, and light source configuration;

[0016] Under the complex lighting conditions in the grain warehouse, the multi-light source system provides uniform and stable light to avoid image quality affected by insufficient light or light reflection;

[0017] By installing multi-angle and multi-type cameras on the robot, all-round image acquisition of the grain storage environment is achieved to ensure that no details are missed;

[0018] In a grain warehouse environment, the acquisition parameters of the camera and light source are dynamically adjusted to ensure that high-quality images are continuously acquired.

[0019] Preferably, for each sub-region, key features reflecting the presence of potential glass fragments are extracted, wherein the extracted features include halos formed around objects in the image due to high reflectivity, reflection spectrum features in the image, and reflection spectrum features of ordinary grains. After acquisition, the halos formed around objects in the image due to high reflectivity are analyzed to generate a halo effect reference value, and the halo effect reference value is used to quantify the intensity and range of the halo area formed around objects in the image due to high reflectivity of glass fragments to highlight the significance of the reflective object, and the difference between the reflection spectrum features in the image and the reflection spectrum features of ordinary grains is analyzed to generate a reflection spectrum offset reference value, and the reflection spectrum offset reference value is used to quantify the degree of offset of the reflection spectrum features in the image relative to the reflection spectrum of ordinary grains to identify objects with abnormal spectral characteristics.

[0020] Preferably, after obtaining the halo effect reference value and the reflection spectrum offset reference value generated by analyzing the extracted key features reflecting the existence of potential glass fragments, the halo effect reference value and the reflection spectrum offset reference value are input into a pre-trained deep learning model to generate a glass fragment feature index, and the glass fragments in the grain warehouse are identified by the glass fragment feature index, so as to predict the glass fragment hidden danger areas and glass fragment areas in the grain warehouse.

[0021] Preferably, the specific steps of generating a halo effect reference value after analyzing the halo formed around the object in the image due to high reflection are as follows:

[0022] The halo area is detected in the image by the light intensity threshold segmentation method. The image pixel matrix is ​​I(x, y), which is used to represent the position of each pixel in the image, where x and y are the coordinates of the image, and the pixel intensity at the corresponding position is P(x, y), which represents the brightness of each pixel in the image. The light intensity threshold is defined. When P(x, y)>T h , the pixel is marked as part of the halo region, forming the halo pixel set Φ h ,In order to quantify the intensity of the halo area, the accumulated value of the light intensity of all halo pixels in each halo area is calculated, and the calculation expression is:

[0023] ,

[0024] Among them, T h is the light intensity threshold, which is preset to determine the boundary of the halo area, S h is the cumulative value of the halo area intensity, which is used to quantify the total light intensity in the halo area, φ(P, T h ) is a nonlinear enhancement function, φ(P, T h ) = P(x, y) 2 ln(1+|P(x,y)-T h |), used to amplify the significant part of the halo intensity;

[0025] After completing the halo area detection, the geometric features of the halo area are obtained through morphological processing, including the halo area area and the halo area perimeter. The composite range index is introduced to measure the comprehensive characteristics of the halo range:

[0026] ,

[0027] Among them, A h is the area of ​​the halo region, represented by the number of pixels in the region, C h is the perimeter of the halo area, represented by the number of boundary pixels, κ is the shape complexity factor, and the calculation expression of the shape complexity factor is:

[0028] ,

[0029] Used to describe the irregular shape of the halo area, R c is a composite range index used to characterize the range characteristics of the halo region;

[0030] Get the cumulative value S of the halo area intensity h and the composite range index R that characterizes the range characteristics of the halo area c After that, the comprehensive halo area intensity cumulative value S h and the composite range index R c Generate a halo effect reference value. The expression for generating the halo effect reference value is:

[0031] HEI τ =λ1·S h +λ2·ln(1+R c ),

[0032] In the formula, HEI τ is the reference value of the halo effect, which is used to comprehensively quantify the intensity and range characteristics of the halo area. λ1 and λ2 are dynamic weight factors that balance the contribution of intensity and range according to environmental conditions.

[0033] Preferably, the specific steps of generating the reflection spectrum shift reference value after analyzing the difference between the reflection spectrum characteristics in the image and the reflection spectrum characteristics of ordinary grains are as follows:

[0034] Extract the reflectance spectrum characteristics of the area to be detected and the standard reflectance spectrum characteristics of ordinary grains from the image, λ is the spectral wavelength, and the calculation expression of the spectral feature difference matrix is: ΔS(x, y, λ)=S1-S2|, where

[0035] ,

[0036] S detected (x, y, λ) is the actual reflection spectrum intensity of the area to be detected at the coordinate (x, y),

[0037] ,

[0038] S reference (x, y, λ) is the reference reflectance spectrum characteristic of ordinary grains. As a reference benchmark, ΔS(x, y, λ) represents the spectral difference at each spatial coordinate. min and λ max are the minimum and maximum values ​​of the spectral wavelength range, respectively, and w(λ) is the wavelength weight function, which is used to emphasize the importance of different types of bands;

[0039] After calculating the spectral feature difference matrix, in order to quantify the overall degree of spectral shift, the reflection spectrum shift reference value is generated. The reflection spectrum shift reference value is calculated by performing spatial weighted integration of the spectral differences of all coordinates and introducing a nonlinear amplification factor to highlight the contribution of high difference areas. The calculation expression is:

[0040] ,

[0041] In the formula, Θ shiff is the reference value of the reflectance spectrum shift, which quantifies the overall shift of the reflectance spectrum features in the image relative to the reflectance spectrum of ordinary grains. φ(x, y) is the spatial weight function, which is used to highlight the contribution of key areas. α is the nonlinear amplification factor, which controls the response to the high spectral shift area. When α>1, the impact of the high difference area will increase significantly. min and x max are the starting point and end point of the image integration in the lateral direction, respectively, indicating the range of the image area to be analyzed on the x-axis, and y min and max are the starting point and end point of the image integration in the longitudinal direction, respectively, indicating the range of the image area to be analyzed on the y-axis.

[0042] Preferably, the glass fragment characteristic index generated when predicting the glass fragments in the grain depot by the deep learning model is compared and analyzed with the preset reference threshold range of the glass fragment characteristic index, and the glass fragments in the grain depot are identified. The identification steps are as follows:

[0043] If the glass fragment characteristic index is less than the minimum value of the glass fragment characteristic index reference threshold range, it indicates that there are no glass fragments in the sub-area, and the sub-area is divided into a non-glass fragment hidden danger area;

[0044] If the glass fragment characteristic index is within the glass fragment characteristic index reference threshold range, it indicates that the sub-area has a hidden danger of glass fragments, and the sub-area is divided into a glass fragment hidden danger area;

[0045] If the glass fragment characteristic index is greater than the minimum value of the glass fragment characteristic index reference threshold range, it indicates that glass fragments exist in the sub-region, and the sub-region is divided into a glass fragment region.

[0046] Preferably, the edge of the marked area is depicted using image segmentation technology to generate an accurate edge contour, and the specific steps are as follows:

[0047] Mark the initially identified glass fragments hidden danger areas and extract candidate areas to provide a basis for edge delineation;

[0048] Use image segmentation technology to generate accurate edge contours of objects in the candidate area to ensure that the edges of the glass fragment area are clearly visible;

[0049] Optimize and visualize the generated edge contours to ensure that the contour information can accurately reflect the potential areas of glass fragments and provide support for subsequent fine processing;

[0050] Based on the generated preliminary edge contour, morphological operations are applied to optimize the edge to eliminate noise or breaks in the segmentation process. The optimized edge contour will be used as the precise boundary of the final marked area. On this basis, the optimized edge contour will be visualized to clearly mark the potential danger areas of glass fragments.

[0051] Preferably, for the marked glass fragment hidden danger area, the edge sharpening intensity is adaptively adjusted according to the prediction result of the deep learning model, and the specific steps are as follows:

[0052] According to the maximum and minimum values ​​of the glass fragment characteristic index and the reference threshold range of the glass fragment characteristic index, the adaptive edge sharpening adjustment factor is calculated to adjust the edge sharpening strength. The calculation expression of the edge sharpening adjustment factor is:

[0053] ,

[0054] Glass Shard is the glass fragment characteristic index, threshold min 、threshold maxare the minimum and maximum values ​​of the reference threshold range of the glass fragment characteristic index, defining the judgment boundary between the hidden danger and the actual fragment area. κ' is the edge sharpening adjustment factor, controlling the dynamic adjustment of the edge sharpening intensity. β is the nonlinear enhancement factor. When β>1, the area corresponding to the high glass fragment characteristic index is sharpened more strongly.

[0055] The calculated edge sharpening adjustment factor is used to update the current edge sharpening strength. The calculation expression of the actual edge sharpening strength after the update is:

[0056] σ new =σ current ·(1+α′·κ′)

[0057] , where σ current is the current edge sharpening strength, indicating the initial sharpening level, σ new is the updated edge sharpening strength, reflecting the enhancement effect for a specific sub-region, α' is the sharpening enhancement coefficient, which adjusts the weight of the edge sharpening adjustment factor on the change of sharpening strength;

[0058] Apply the updated edge sharpening filter to the marked glass shard hazard areas. The core operation of the sharpening filter is as follows:

[0059] ,

[0060] in, Represents the Laplacian operator of the original image, used to extract edge details, I original is the image of the original marked area, I sharpened For the sharpened image, the edge information is more prominent.

[0061] The grain storage cleaning robot visual recognition platform based on deep learning includes image acquisition module, feature extraction and analysis module, glass fragment recognition module, hidden danger area marking and edge delineation module, adaptive edge sharpening module, secondary recognition module and cleaning instruction generation module;

[0062] The image acquisition module uses the visual system on the grain storage cleaning robot to perform comprehensive image acquisition of different areas in the grain storage environment;

[0063] The feature extraction and analysis module divides the image into several sub-regions. For each sub-region, key features that reflect the presence of potential glass fragments are extracted from the collected image and analyzed to ensure that the deep learning model can receive accurate input;

[0064] The glass fragment recognition module inputs the analyzed features into the pre-trained deep learning model, identifies the glass fragments in the grain warehouse through the deep learning model, and predicts the glass fragment hidden danger area and glass fragment area in the grain warehouse;

[0065] The module for marking and delineating the edges of potential danger areas of glass fragments that have been initially identified marks the edges of the marked areas, providing a clear target range for further fine-tuning.

[0066] The adaptive edge sharpening module adaptively adjusts the edge sharpening intensity for the marked glass fragment hidden danger areas according to the prediction results of the deep learning model to enhance the edge features of the glass fragments;

[0067] The secondary recognition module re-inputs the sharpened edge features into the deep learning model for secondary recognition, further improving the detection accuracy of the deep learning model for glass fragments;

[0068] The cleaning instruction generation module outputs the finally identified glass fragment area and its boundary information, and generates cleaning instructions for the cleaning robot to execute.

[0069] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0070] The present invention ensures that the deep learning model receives accurate input through comprehensive image acquisition and meticulous feature extraction. Then, a pre-trained machine learning model is used to perform preliminary identification of glass fragments and prediction of potential danger areas. For the identified potential danger areas, adaptive edge sharpening technology is used to enhance the edge features of the glass fragments, and secondary identification is performed to further improve the detection accuracy of the model. Finally, the glass fragment area is accurately located and marked, and cleaning instructions are generated for the cleaning robot to execute. This not only reduces missed detections and false detections, ensures the efficiency and accuracy of grain warehouse cleaning, reduces the potential risks of glass fragments to grain processing equipment and food safety, and improves the safety and economic benefits of grain warehouse operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0072] Figure 1 This is a flow chart of the method for visual recognition of a grain storage cleaning robot based on deep learning of the present invention.

[0073] Figure 2This is a module schematic diagram of the grain storage cleaning robot visual recognition platform based on deep learning in the present invention. DETAILED DESCRIPTION

[0074] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0075] The present invention provides Figure 1 The grain storage cleaning robot visual recognition method based on deep learning shown in the figure includes the following steps:

[0076] The vision system on the grain storage cleaning robot is used to collect comprehensive images of different areas in the grain storage environment;

[0077] The specific steps for comprehensive image acquisition of different areas in the grain storage environment are as follows:

[0078] According to the layout and environmental characteristics of the grain warehouse, the area where images need to be collected is clearly identified, and a collection strategy is formulated, including image resolution, collection frequency, and light source configuration;

[0079] First, conduct a preliminary survey of the grain storage environment and mark key areas (such as grain piles, ground, and equipment surroundings) and potential risk points (such as corners and narrow spaces). Set appropriate acquisition parameters based on the different characteristics of these areas. For example, increase image resolution or adjust exposure time in areas with low light to ensure that clear images are captured. In addition, plan dynamic acquisition strategies based on the robot's moving path, including a combination of static shooting and continuous video acquisition, to improve coverage of complex areas.

[0080] Under the complex lighting conditions in the grain warehouse, the multi-light source system provides uniform and stable light to avoid image quality affected by insufficient light or light reflection;

[0081] Natural light and artificial light in the grain storage environment may lead to uneven lighting. To solve this problem, a variety of light sources (such as LED light sources, infrared light sources, and ultraviolet light sources) can be configured on the robot. These light sources can be switched or used in combination according to the lighting needs of different areas. For example, when collecting ground reflection areas, infrared light can effectively reduce the interference of high light reflections; while in dark areas, ultraviolet light can enhance the prominence of transparent objects. The angle and brightness of the light source can be dynamically adjusted through the algorithm to ensure the overall quality of the collected image is consistent.

[0082] By installing multi-angle and multi-type cameras on the robot, all-round image capture of the grain warehouse environment can be achieved to ensure that no details are missed.

[0083] Multiple cameras are installed on the robot, including a high-resolution visible light camera, a wide-angle camera, and a depth camera. The high-resolution camera is used to capture details, the wide-angle camera is used to cover a large area, and the depth camera can provide three-dimensional spatial information to help identify tiny features and potential obstacles in the grain pile. When multiple cameras work synchronously, the collected data can be synthesized into a complete grain storage environment image through image stitching technology, eliminating blind spots and improving the spatial continuity and integrity of the image.

[0084] In a grain warehouse environment, conditions such as lighting, temperature, and dust may change in real time. By dynamically adjusting the acquisition parameters of the camera and light source, we can ensure that high-quality images are continuously acquired.

[0085] The visual system dynamically adjusts the camera exposure, white balance, and brightness and angle of the light source by monitoring the ambient light and image quality parameters (such as brightness, contrast, and noise level) in real time. For example, when the robot enters an area with strong light, the camera exposure value is automatically reduced to avoid overexposure; in areas with more dust, the image denoising algorithm is combined to perform real-time image optimization. In addition, the visual system can use the feedback mechanism to optimize the acquisition parameters in real time according to the clarity and completeness of the acquired image to ensure the stability and robustness of image acquisition.

[0086] Divide the image into several sub-regions. For each sub-region, extract key features that reflect the presence of potential glass fragments from the acquired image and analyze the extracted features to ensure that the deep learning model can receive accurate input.

[0087] The analyzed features are input into the pre-trained deep learning model, and the glass fragments in the grain warehouse are identified through the deep learning model to predict the glass fragment hidden danger area and glass fragment area in the grain warehouse;

[0088] For each sub-region, key features reflecting the presence of potential glass fragments are extracted, wherein the extracted features include halos formed around objects in the image due to high reflectivity, reflection spectrum features in the image, and reflection spectrum features of ordinary grains. After acquisition, the halos formed around objects in the image due to high reflectivity are analyzed to generate a halo effect reference value. The halo effect reference value is used to quantify the intensity and range of the halo area formed around objects in the image due to high reflectivity of glass fragments, thereby highlighting the significance of reflective objects. The difference between the reflection spectrum features in the image and the reflection spectrum features of ordinary grains is analyzed to generate a reflection spectrum offset reference value. The reflection spectrum offset reference value is used to quantify the degree of offset of the reflection spectrum features in the image relative to the reflection spectrum of ordinary grains, thereby identifying objects with abnormal spectral characteristics.

[0089] After obtaining the halo effect reference value and reflection spectrum shift reference value generated by analyzing the extracted key features reflecting the existence of potential glass fragments, the halo effect reference value and the reflection spectrum shift reference value are input into a pre-trained deep learning model to generate a glass fragment feature index. The glass fragments in the grain warehouse are identified by the glass fragment feature index, and the glass fragment hidden danger areas and glass fragment areas in the grain warehouse are predicted.

[0090] The pre-trained deep learning model is a multi-task learning model that is specifically designed to comprehensively analyze the possibility and location hazards of glass fragments from the multi-dimensional features of the input. This model usually uses an improved convolutional neural network (CNN) architecture combined with a multi-branch network design to process different types of input data, including halo effect reference values ​​and reflection spectrum shift reference values. The first branch of the model is mainly used to analyze spatial features, especially halo effect reference values, to capture the edge distribution and halo diffusion characteristics of highly reflective areas in the image. The second branch focuses on spectral features, analyzes the shift trend of the reflection spectrum, and identifies spectral abnormal areas. Through multi-branch design, the model can simultaneously learn the correlation between spatial features and spectral features, thereby improving the comprehensiveness and accuracy of glass fragment identification.

[0091] In addition, the model uses an attention mechanism (such as SE module or CBAM module) to dynamically focus on key areas in the feature map and enhance the ability to identify potential glass fragments. During the training process, the model uses a large amount of labeled data for supervised learning, and improves the prediction accuracy of hidden danger areas through the joint optimization of the cross entropy loss function and the IoU loss function. At the same time, the model also has good generalization ability and can adapt to diverse scenarios such as lighting conditions and background complexity in the grain storage environment.

[0092] Finally, the model generates a comprehensive glass fragment feature index by learning the intrinsic relationship between the halo effect reference value and the reflection spectrum offset reference value, thereby accurately predicting the potential areas and actual areas of glass fragments. This model design greatly improves the robustness and reliability of detecting glass fragments in complex environments.

[0093] When the intensity and range of the halo area formed by high reflectivity around the object in the image are large, it indicates that there is a high possibility of hidden dangers of glass fragments in the area. Glass fragments are highly reflective and will produce a significant halo effect under strong light, especially in the edge area. Due to multiple reflections and scattering of light, the halo intensity will be significantly enhanced, and the range may also extend to the surrounding area. Compared with ordinary grain piles or other non-reflective objects, this characteristic is particularly obvious in the image. Therefore, analyzing the intensity and range of the halo area can help identify potential hidden dangers of glass fragments and serve as an important basis for judgment by deep learning models.

[0094] The specific steps for generating a halo effect reference value after analyzing the halo formed by high reflection around an object in an image are as follows:

[0095] The halo area is detected in the image by the light intensity threshold segmentation method. The image pixel matrix is ​​I(x, y), which is used to represent the position of each pixel in the image, where x and y are the coordinates of the image, and the pixel intensity at the corresponding position is P(x, y), which represents the brightness of each pixel in the image. The light intensity threshold is defined. When P(x, y)>T h , the pixel is marked as part of the halo region, forming the halo pixel set Φ h ,In order to quantify the intensity of the halo area, the accumulated value of the light intensity of all halo pixels in each halo area is calculated, and the calculation expression is:

[0096] ,

[0097] Among them, T h is the light intensity threshold, which is preset to determine the boundary of the halo area, S h is the accumulated value of the halo area intensity, which is used to quantify the total light intensity in the halo area, φ(P, T h ) is a nonlinear enhancement function, φ(P, T h ) = P(x, y) 2 ln(1+|P(x,y)-T h |), used to amplify the significant part of the halo intensity;

[0098] After completing the halo region detection, the geometric features of the halo region are obtained through morphological processing, including the halo region area and the halo region perimeter. The halo region area can be calculated by summing the number of pixels in the region, while the halo region perimeter is extracted through the boundary detection algorithm. Then, the composite range index is introduced to measure the comprehensive characteristics of the halo range:

[0099] ,

[0100] Among them, A h is the area of ​​the halo region, represented by the number of pixels in the region, C h is the perimeter of the halo area, represented by the number of boundary pixels, κ is the shape complexity factor, and the calculation expression of the shape complexity factor is:

[0101] ,

[0102] Used to describe the irregular shape of the halo area, R c It is a composite range index used to characterize the range characteristics of the halo area.

[0103] Get the cumulative value S of the halo area intensity hand the composite range index R that characterizes the range characteristics of the halo area c After that, the comprehensive halo area intensity cumulative value S h and the composite range index R c Generate a halo effect reference value. The expression for generating the halo effect reference value is:

[0104] HEI τ =λ1·S h +λ2·ln(1+R c )

[0105] , where HEI τ is the reference value of the halo effect, which is used to comprehensively quantify the intensity and range characteristics of the halo area. λ1 and λ2 are dynamic weight factors, which balance the contribution of intensity and range according to environmental conditions. They are not specifically limited here.

[0106] It can be seen from the halo effect reference value that the larger the performance value of the halo effect reference value generated after analyzing the halo formed by high reflectivity around the objects in the image, the more significant the intensity and range of the halo area formed by highly reflective objects (such as glass fragments) in the image, which means that the potential hidden danger of glass fragments in the grain depot is greater. This is because glass fragments are highly reflective and will produce obvious halo effects under the illumination of light sources, including strong light reflection and irregular light scattering. Through quantitative analysis of these halo characteristics, the halo effect reference value can effectively characterize the degree of reflective anomaly in the area, and thus be used to assess the possibility of hidden dangers of glass fragments. Conversely, if the halo effect reference value performance value is small, it means that the halo feature is not significant and the risk of glass fragments in the grain depot is low.

[0107] When there is a large difference between the reflective spectral features in the image and the reflective spectrum of ordinary grain, it indicates that there is a high risk of glass fragments in the area. This is because glass fragments are significantly different from grains in terms of reflective properties, and they usually show special spectral shifts under multi-spectral (such as infrared and ultraviolet) conditions. This spectral shift is caused by the transparency and high reflectivity of the glass fragments, which is in sharp contrast to the overall spectral reflectance of the grain pile. Therefore, large spectral differences can effectively indicate areas where glass fragments may exist, providing an important reference for subsequent further detection and cleaning.

[0108] The specific steps of generating the reference value of the reflectance spectrum offset after analyzing the difference between the reflectance spectrum characteristics in the image and the reflectance spectrum characteristics of ordinary grains are as follows:

[0109] The reflectance spectrum characteristics of the area to be detected and the standard reflectance spectrum characteristics of ordinary grains are extracted from the image (where x, y are the spatial coordinates of the image), λ is the spectral wavelength, and the calculation expression of the spectral feature difference matrix is: ΔS(x, y, λ) = |S1-S2|, where

[0110] ,

[0111] S detected (x, y, λ) is the actual reflection spectrum intensity of the area to be detected at the coordinate (x, y),

[0112] ,

[0113] S reference (x, y, λ) is the reference reflectance spectrum characteristic of ordinary grains. As a reference benchmark, ΔS(x, y, λ) represents the spectral difference at each spatial coordinate. min and λ max are the minimum and maximum values ​​of the spectral wavelength range, respectively, and w(λ) is the wavelength weight function, which is used to emphasize the importance of certain specific bands (such as infrared or ultraviolet);

[0114] This step captures the spatial distribution and shift of spectral features through the difference matrix, providing a basis for the generation of reflectance spectrum shift reference values.

[0115] After calculating the spectral feature difference matrix, in order to quantify the overall degree of spectral shift, the reflection spectrum shift reference value is generated. The reflection spectrum shift reference value is calculated by performing spatial weighted integration of the spectral differences of all coordinates and introducing a nonlinear amplification factor to highlight the contribution of high difference areas. The calculation expression is:

[0116] ,

[0117] In the formula, Θ shiff is the reference value of the reflectance spectrum shift, which quantifies the overall shift of the reflectance spectrum features in the image relative to the reflectance spectrum of ordinary grains. φ(x, y) is the spatial weight function, which is used to highlight the contribution of key areas (such as high reflectance or edge areas) and increase attention to key areas. α is the nonlinear amplification factor, which controls the response to high spectral shift areas. When α>1, the impact of high difference areas will increase significantly. min and x max They are the starting point and end point of the image integration in the horizontal direction, indicating the range of the image area to be analyzed on the x-axis, and y min and max They are the starting point and end point of the image integration in the longitudinal (vertical) direction, respectively, indicating the range of the image area to be analyzed on the y-axis;

[0118] The spatial weight function can be a function that is dynamically adjusted according to the importance of different areas in the image. Common ones include Gaussian weight function, distance decay function and adaptive attention weight function. The Gaussian weight function assigns weights by gradually decaying outward from the target area to highlight the features of the key area; the distance decay function assigns weights according to the distance from each pixel to the specified center point. The farther the distance, the lower the weight; the adaptive attention weight function uses the attention mechanism in deep learning to dynamically assign weights according to the feature significance of each area in the image, enhancing the influence of areas with obvious feature differences. These functions can be flexibly selected according to specific application needs to highlight the key features of the glass fragments hidden danger area;

[0119] This step generates a quantitative indicator, the reflection spectrum shift reference value, by integrating spatial and spectral information. This is used to intuitively measure the degree of spectral abnormality in the glass fragment area, thus providing a key basis for further identification of glass fragments.

[0120] It can be seen from the reflection spectrum shift reference value that the larger the performance value of the reflection spectrum shift reference value generated after analyzing the difference between the reflection spectrum characteristics in the image and the reflection spectrum characteristics of ordinary grains, the more significant the difference between the reflection spectrum characteristics in the image and the reflection spectrum characteristics of ordinary grains, which is usually caused by the special optical properties of glass fragments (such as high reflectivity or spectral shift in a specific band). Therefore, the larger the reflection spectrum shift reference value, the greater the potential hidden danger of glass fragments in the grain warehouse; conversely, if the reflection spectrum shift reference value is small, it means that the reflection spectrum characteristics are closer to ordinary grains, and the possibility of glass fragments is low. The reflection spectrum shift reference value provides a quantitative basis for further cleaning and risk assessment, and can effectively help identify high-risk areas.

[0121] The deep learning model is not specifically limited here, and can achieve the halo effect reference value HEI τ and the reflected spectrum offset reference value Θ shiff Comprehensive analysis to generate glass fragment characteristic index Glass Shard In order to realize the technical solution of the present invention, the present invention provides a specific implementation method; Glass fragment feature index Glass Shard The resulting calculation formula is:

[0122] ,

[0123] Where ψ1 and ψ2 are the halo effect reference values ​​HEI, τ and the reflected spectrum offset reference value Θ shiff The preset proportional coefficient, and ψ1 and ψ2 are both greater than 0.

[0124] It can be seen from the glass fragment characteristic index that the greater the performance value of the halo effect reference value generated after analyzing the halo formed around the object in the image due to high reflection, the greater the performance value of the reflection spectrum offset reference value generated after analyzing the difference between the reflection spectrum characteristics in the image and the reflection spectrum characteristics of ordinary grain. In other words, the greater the performance value of the glass fragment characteristic index generated when predicting the glass fragments in the grain depot through the deep learning model, the greater the potential hidden danger of glass fragments in the grain depot, and vice versa.

[0125] The glass fragment characteristic index generated when predicting the glass fragments in the grain depot through the deep learning model is compared and analyzed with the preset reference threshold range of the glass fragment characteristic index to identify the glass fragments in the grain depot. The identification steps are as follows:

[0126] If the glass fragment characteristic index is less than the minimum value of the glass fragment characteristic index reference threshold range, it indicates that there are no glass fragments in the sub-area, and the sub-area is divided into a non-glass fragment hidden danger area;

[0127] If the glass fragment characteristic index is within the glass fragment characteristic index reference threshold range, it indicates that the sub-area has a hidden danger of glass fragments, and the sub-area is divided into a glass fragment hidden danger area;

[0128] If the glass fragment characteristic index is greater than the minimum value of the glass fragment characteristic index reference threshold range, it indicates that glass fragments exist in the sub-region, and the sub-region is divided into a glass fragment region.

[0129] Mark the glass fragments potential areas that are initially identified and depict the edges of the marked areas to provide a clear target range for further detailed processing;

[0130] The edge delineation of the marked area can use image segmentation technology (such as Mask R-CNN) to generate accurate edge contours. This step not only highlights the potential glass fragment area, but also provides more accurate edge information for subsequent steps.

[0131] Image segmentation technology is used to depict the edge of the marked area and generate accurate edge contours. The specific steps are as follows:

[0132] Mark the initially identified glass fragments hidden danger areas and extract candidate areas to provide a basis for edge delineation;

[0133] First, the deep learning model detects the potential areas of glass fragments and generates preliminary rectangular or polygonal marker boxes in the image. These boxes mark the areas where glass fragments may exist. Subsequently, these marked areas are refined and the areas that may contain multiple objects are further cropped into single target candidate areas. In order to reduce errors, non-maximum suppression (NMS) technology can be used to remove overlapping markers and retain the areas with the highest confidence. After the candidate areas are extracted, they are passed as input to the next step for accurate edge contour delineation.

[0134] Use image segmentation technology to generate accurate edge contours of objects in the candidate area to ensure that the edges of the glass fragment area are clearly visible;

[0135] In the extracted candidate areas, deep learning-based image segmentation techniques (such as Mask R-CNN or U-Net) are applied to segment the glass fragment potential areas. These techniques can divide the target area into foreground (possible glass fragments) and background, thereby generating more accurate contour information. After the segmentation is completed, edge detection algorithms (such as Canny or Sobel) can be combined to further enhance the details of the edge contour. This step captures the complex boundary shape of glass fragments through multi-level feature fusion, making the edges clearly identifiable, and even the edges of some tiny fragments can be accurately identified.

[0136] Optimize and visualize the generated edge contours to ensure that the contour information can accurately reflect the potential areas of glass fragments and provide support for subsequent fine processing;

[0137] Based on the generated preliminary edge contours, morphological operations (such as dilation and erosion) are applied to optimize the edges to eliminate noise or breaks that may occur during the segmentation process. The optimized edge contours will be used as the precise boundaries of the final marked area. On this basis, the optimized edge contours will be visualized, such as using different colors or line types to display them in the image to clearly mark the potential areas of glass fragments. This visual marking will provide a clear target area for the cleaning robot, making subsequent processing and cleaning operations more efficient and accurate.

[0138] For the marked glass fragments potential risk areas, the edge sharpening intensity is adaptively adjusted based on the prediction results of the deep learning model to enhance the edge features of the glass fragments;

[0139] For the marked glass fragments potential risk areas, the edge sharpening intensity is adaptively adjusted according to the prediction results of the deep learning model. The specific steps are as follows:

[0140] According to the maximum and minimum values ​​of the glass fragment characteristic index and the reference threshold range of the glass fragment characteristic index, the adaptive edge sharpening adjustment factor is calculated to adjust the edge sharpening strength. The calculation expression of the edge sharpening adjustment factor is:

[0141] ,

[0142] Glass Shard is the glass fragment characteristic index, which quantifies the possibility of the existence of glass fragments, threshold min 、threshold max are the minimum and maximum values ​​of the reference threshold range of the glass fragment characteristic index, defining the judgment boundary between the hidden danger and the actual fragment area. κ' is the edge sharpening adjustment factor, controlling the dynamic adjustment of the edge sharpening intensity. β is the nonlinear enhancement factor. When β>1, the area corresponding to the high glass fragment characteristic index is sharpened more strongly.

[0143] This step provides a basis for adaptive adjustment of edge sharpening intensity by calculating the edge sharpening adjustment factor, ensuring that the sharpening intensity of different hidden danger areas matches their risk levels.

[0144] The calculated edge sharpening adjustment factor is used to update the current edge sharpening strength. The calculation expression of the actual edge sharpening strength after the update is:

[0145] σ new =σ current ·(1+α′·k′)

[0146] , where σ current is the current edge sharpening strength, indicating the initial sharpening level, σ new is the updated edge sharpening strength, reflecting the enhancement effect for a specific sub-region, α' is the sharpening enhancement coefficient, which adjusts the weight of the edge sharpening adjustment factor on the change of sharpening strength;

[0147] This step ensures that high-risk areas are sharpened more strongly, while low-risk areas are sharpened less, thus highlighting the edges of the glass fragments.

[0148] Apply the updated edge sharpening filter to the marked glass shard hazard areas. The core operation of the sharpening filter is as follows:

[0149] ,

[0150] in, Represents the Laplacian operator of the original image, used to extract edge details, I original is the image of the original marked area, I sharpenedFor the sharpened image, the edge information is more prominent.

[0151] For the marked glass fragment hidden danger areas, the edge sharpening intensity is adaptively adjusted according to the prediction results of the deep learning model. Its role is to dynamically enhance the edge features of the glass fragments, making the fragments more prominent and clear in the image. Because glass fragments are usually transparent or translucent, their edges may be blurred or difficult to detect under ordinary lighting and image processing. By adaptively adjusting the sharpening intensity, the edge features can be enhanced according to the risk level of the hidden danger area, improving the model's recognition accuracy of the fragment boundaries. This process not only helps to reduce missed detections and false detections, but also provides more accurate target positioning for subsequent cleaning operations.

[0152] The sharpened edge features are re-input into the deep learning model for secondary recognition, further improving the detection accuracy of the deep learning model for glass fragments;

[0153] The sharpened edge features are re-entered into the deep learning model for secondary recognition, which uses the enhanced edge information to verify and refine the initial recognition results, thereby improving the detection accuracy of the model. This iterative recognition method further reduces the possibility of missed detection and false detection by strengthening boundary features, especially when dealing with small or irregularly shaped glass fragments. At the same time, combined with the confidence score of the model, re-evaluating low-confidence areas helps to optimize the reliability and accuracy of the final recognition results, providing a more accurate decision-making basis for subsequent cleaning.

[0154] Output the finally identified glass fragment area and its boundary information, and generate cleaning instructions for the cleaning robot to execute;

[0155] The recognition results are output in the form of annotated images, clearly marking the location of the glass fragments and their edges. At the same time, the location information of these areas is converted into path planning instructions for the robot to perform the cleaning task, ensuring the cleaning process is efficient and accurate. To improve the cleaning efficiency, the system can prioritize multiple marked areas and give priority to cleaning areas with more dangerous fragments.

[0156] Through the above-mentioned deep learning-based grain depot cleaning robot visual recognition method, the problem of glass fragments being difficult to accurately identify in the grain depot environment has been successfully solved, and the detection accuracy and reliability have been significantly improved. This method ensures that the deep learning model receives accurate input through comprehensive image acquisition and meticulous feature extraction; then, the pre-trained machine learning model is used to perform preliminary identification of glass fragments and predict potential danger areas. For the identified potential danger areas, the adaptive edge sharpening technology is used to enhance the edge features of the glass fragments, and secondary identification is performed, which further improves the detection accuracy of the model. Finally, the glass fragment area is accurately located and marked, and the cleaning instructions executed by the cleaning robot are generated. This solution not only reduces missed detections and false detections, ensures the efficiency and accuracy of grain depot cleaning, but also reduces the potential risks of glass fragments to grain processing equipment and food safety, and improves the safety and economic benefits of grain depot operations.

[0157] The present invention provides Figure 2 The grain storage cleaning robot visual recognition platform based on deep learning shown in the figure includes an image acquisition module, a feature extraction and analysis module, a glass fragment recognition module, a hidden danger area marking and edge delineation module, an adaptive edge sharpening module, a secondary recognition module, and a cleaning instruction generation module;

[0158] The image acquisition module uses the visual system on the grain storage cleaning robot to perform comprehensive image acquisition of different areas in the grain storage environment;

[0159] The feature extraction and analysis module divides the image into several sub-regions. For each sub-region, key features that reflect the presence of potential glass fragments are extracted from the collected image and analyzed to ensure that the deep learning model can receive accurate input;

[0160] The glass fragment recognition module inputs the analyzed features into the pre-trained deep learning model, identifies the glass fragments in the grain warehouse through the deep learning model, and predicts the glass fragment hidden danger area and glass fragment area in the grain warehouse;

[0161] The module for marking and delineating the edges of potential danger areas of glass fragments that have been initially identified marks the edges of the marked areas, providing a clear target range for further fine-tuning.

[0162] The adaptive edge sharpening module adaptively adjusts the edge sharpening intensity for the marked glass fragment hidden danger areas according to the prediction results of the deep learning model to enhance the edge features of the glass fragments;

[0163] The secondary recognition module re-inputs the sharpened edge features into the deep learning model for secondary recognition, further improving the detection accuracy of the deep learning model for glass fragments;

[0164] A cleaning instruction generation module outputs the finally identified glass fragment area and its boundary information, and generates cleaning instructions for the cleaning robot to execute;

[0165] The grain storage cleaning robot visual recognition method based on deep learning provided in an embodiment of the present invention is realized by the above-mentioned grain storage cleaning robot visual recognition platform based on deep learning. The specific methods and processes of the grain storage cleaning robot visual recognition platform based on deep learning are detailed in the embodiment of the grain storage cleaning robot visual recognition method based on deep learning, which will not be repeated here.

[0166] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0167] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0168] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A grain storage cleaning robot visual recognition method based on deep learning, characterized in that: The following steps are involved: The vision system on the grain storage cleaning robot is used to collect comprehensive images of different areas in the grain storage environment; Divide the image into several sub-regions. For each sub-region, extract key features that reflect the presence of potential glass fragments from the acquired image and analyze the extracted features to ensure that the deep learning model can receive accurate input. The analyzed features are input into the pre-trained deep learning model, and the glass fragments in the grain warehouse are identified through the deep learning model to predict the glass fragment hidden danger area and glass fragment area in the grain warehouse; Mark the glass fragments potential areas that are initially identified and depict the edges of the marked areas to provide a clear target range for further detailed processing; For the marked glass fragments potential risk areas, the edge sharpening intensity is adaptively adjusted based on the prediction results of the deep learning model to enhance the edge features of the glass fragments; The sharpened edge features are re-input into the deep learning model for secondary recognition, further improving the detection accuracy of the deep learning model for glass fragments; The finally identified glass fragment area and its boundary information are output, and a cleaning instruction is generated for the cleaning robot to execute.

2. The grain storage cleaning robot visual recognition method based on deep learning according to claim 1 is characterized in that: The specific steps for comprehensive image acquisition of different areas in the grain storage environment are as follows: According to the layout and environmental characteristics of the grain warehouse, the area where images need to be collected is clearly identified, and a collection strategy is formulated, including image resolution, collection frequency, and light source configuration; Under the complex lighting conditions in the grain warehouse, the multi-light source system provides uniform and stable light to avoid image quality affected by insufficient light or light reflection; By installing multi-angle and multi-type cameras on the robot, all-round image acquisition of the grain storage environment is achieved to ensure that no details are missed; In a grain warehouse environment, the acquisition parameters of the camera and light source are dynamically adjusted to ensure that high-quality images are continuously acquired.

3. The grain storage cleaning robot visual recognition method based on deep learning according to claim 1 is characterized in that: For each sub-region, key features reflecting the presence of potential glass fragments are extracted, wherein the extracted features include halos formed around objects in the image due to high reflectivity, reflection spectrum features in the image, and reflection spectrum features of ordinary grains. After acquisition, the halos formed around objects in the image due to high reflectivity are analyzed to generate a halo effect reference value. The halo effect reference value is used to quantify the intensity and range of the halo area formed around objects in the image due to high reflectivity of glass fragments, thereby highlighting the significance of reflective objects. The difference between the reflection spectrum features in the image and the reflection spectrum features of ordinary grains is analyzed to generate a reflection spectrum offset reference value. The reflection spectrum offset reference value is used to quantify the degree of offset of the reflection spectrum features in the image relative to the reflection spectrum of ordinary grains, thereby identifying objects with abnormal spectral characteristics.

4. The grain storage cleaning robot visual recognition method based on deep learning according to claim 3 is characterized in that: After obtaining the halo effect reference value and reflection spectrum shift reference value generated by analyzing the extracted key features reflecting the existence of potential glass fragments, the halo effect reference value and the reflection spectrum shift reference value are input into a pre-trained deep learning model to generate a glass fragment feature index. The glass fragments in the grain warehouse are identified by the glass fragment feature index, and the glass fragment hidden danger areas and glass fragment areas in the grain warehouse are predicted.

5. The grain storage cleaning robot visual recognition method based on deep learning according to claim 3 is characterized in that: The specific steps for generating a halo effect reference value after analyzing the halo formed by high reflection around an object in an image are as follows: The halo area is detected in the image by the light intensity threshold segmentation method. The image pixel matrix is ​​I(x, y), which is used to represent the position of each pixel in the image, where x and y are the coordinates of the image, and the pixel intensity at the corresponding position is P(x, y), which represents the brightness of each pixel in the image. The light intensity threshold is defined. When P(x, y)>T h , the pixel is marked as part of the halo region, forming the halo pixel set Φ h ,In order to quantify the intensity of the halo area, the accumulated value of the light intensity of all halo pixels in each halo area is calculated, and the calculation expression is: , Among them, T h is the light intensity threshold, which is preset to determine the boundary of the halo area, S h is the accumulated value of the halo area intensity, which is used to quantify the total light intensity in the halo area, φ(P, T h ) is a nonlinear enhancement function, φ(P, T h ) = P(x, y) 2 ln(1+|P(x,y)-T h |), used to amplify the significant part of the halo intensity; After completing the halo area detection, the geometric features of the halo area are obtained through morphological processing, including the halo area area and the halo area perimeter. The composite range index is introduced to measure the comprehensive characteristics of the halo range: , Among them, A h is the area of ​​the halo region, represented by the number of pixels in the region, C h is the perimeter of the halo area, represented by the number of boundary pixels, κ is the shape complexity factor, and the calculation expression of the shape complexity factor is: , Used to describe the irregular shape of the halo area, R c is a composite range index used to characterize the range characteristics of the halo region; Get the cumulative value S of the halo area intensity h and the composite range index R that characterizes the range characteristics of the halo area c After that, the comprehensive halo area intensity cumulative value S h and the composite range index R c Generate a halo effect reference value. The expression for generating the halo effect reference value is: HEI τ =λ1·S h +λ2·ln(1+R c ), In the formula, HEI τ is the reference value of the halo effect, which is used to comprehensively quantify the intensity and range characteristics of the halo area. λ1 and λ2 are dynamic weight factors that balance the contribution of intensity and range according to environmental conditions.

6. The grain storage cleaning robot visual recognition method based on deep learning according to claim 5 is characterized in that: The specific steps of generating the reference value of the reflectance spectrum offset after analyzing the difference between the reflectance spectrum characteristics in the image and the reflectance spectrum characteristics of ordinary grains are as follows: Extract the reflectance spectrum characteristics of the area to be detected and the standard reflectance spectrum characteristics of ordinary grains from the image, λ is the spectral wavelength, and the calculation expression of the spectral feature difference matrix is: ΔS(x, y, λ) = |S1-S2|, where , S detected (x, y, λ) is the actual reflection spectrum intensity of the area to be detected at the coordinate (x, y), , S reference (x, y, λ) is the reference reflectance spectrum characteristic of ordinary grains. As a reference benchmark, ΔS(x, y, λ) represents the spectral difference at each spatial coordinate. min and λ max are the minimum and maximum values ​​of the spectral wavelength range, respectively, and w(λ) is the wavelength weight function, which is used to emphasize the importance of different types of bands; After calculating the spectral feature difference matrix, in order to quantify the overall degree of spectral shift, the reflection spectrum shift reference value is generated. The reflection spectrum shift reference value is calculated by performing spatial weighted integration on the spectral differences of all coordinates and introducing a nonlinear amplification factor to highlight the contribution of high difference areas. The calculation expression is: , In the formula, Θ shiff is the reference value of the reflectance spectrum shift, which quantifies the overall shift of the reflectance spectrum features in the image relative to the reflectance spectrum of ordinary grains. φ(x, y) is the spatial weight function, which is used to highlight the contribution of key areas. α is the nonlinear amplification coefficient, which controls the response to the high spectral shift area. When α>1, the impact of the high difference area will increase significantly. min and x max are the starting point and end point of the image integration in the lateral direction, respectively, indicating the range of the image area to be analyzed on the x-axis, and y min and max are the starting point and end point of the image integration in the longitudinal direction, respectively, indicating the range of the image area to be analyzed on the y-axis.

7. The grain storage cleaning robot visual recognition method based on deep learning according to claim 4 is characterized in that: The glass fragment characteristic index generated when predicting the glass fragments in the grain depot through the deep learning model is compared and analyzed with the preset reference threshold range of the glass fragment characteristic index to identify the glass fragments in the grain depot. The identification steps are as follows: If the glass fragment characteristic index is less than the minimum value of the glass fragment characteristic index reference threshold range, it indicates that there are no glass fragments in the sub-area, and the sub-area is divided into a non-glass fragment hidden danger area; If the glass fragment characteristic index is within the glass fragment characteristic index reference threshold range, it indicates that the sub-area has a hidden danger of glass fragments, and the sub-area is divided into a glass fragment hidden danger area; If the glass fragment characteristic index is greater than the minimum value of the glass fragment characteristic index reference threshold range, it indicates that glass fragments exist in the sub-region, and the sub-region is divided into a glass fragment region.

8. The grain storage cleaning robot visual recognition method based on deep learning according to claim 1 is characterized in that: Image segmentation technology is used to depict the edge of the marked area and generate accurate edge contours. The specific steps are as follows: Mark the initially identified glass fragments hidden danger areas and extract candidate areas to provide a basis for edge delineation; Use image segmentation technology to generate accurate edge contours of objects in the candidate area to ensure that the edges of the glass fragment area are clearly visible; Optimize and visualize the generated edge contours to ensure that the contour information can accurately reflect the potential areas of glass fragments and provide support for subsequent fine processing; Based on the generated preliminary edge contour, morphological operations are applied to optimize the edge to eliminate noise or breaks in the segmentation process. The optimized edge contour will be used as the precise boundary of the final marked area. On this basis, the optimized edge contour will be visualized to clearly mark the potential danger areas of glass fragments.

9. The grain storage cleaning robot visual recognition method based on deep learning according to claim 7 is characterized in that: For the marked glass fragments potential risk areas, the edge sharpening intensity is adaptively adjusted according to the prediction results of the deep learning model. The specific steps are as follows: According to the maximum and minimum values ​​of the glass fragment characteristic index and the reference threshold range of the glass fragment characteristic index, the adaptive edge sharpening adjustment factor is calculated to adjust the edge sharpening strength. The calculation expression of the edge sharpening adjustment factor is: , Glass Shard is the glass fragment characteristic index, threshold min 、threshold max are the minimum and maximum values ​​of the reference threshold range of the glass fragment characteristic index, respectively, defining the judgment boundary between the hidden danger and the actual fragment area. K′ is the edge sharpening adjustment factor, controlling the dynamic adjustment of the edge sharpening intensity. β is the nonlinear enhancement factor. When β>1, the area corresponding to the high glass fragment characteristic index is sharpened more strongly. The calculated edge sharpening adjustment factor is used to update the current edge sharpening strength. The calculation expression of the actual edge sharpening strength after the update is: s new =s current ·(1+α′·κ′), In the formula, σ current is the current edge sharpening strength, indicating the initial sharpening level, σ new is the updated edge sharpening strength, reflecting the enhancement effect for a specific sub-region, α′ is the sharpening enhancement coefficient, which adjusts the weight of the edge sharpening adjustment factor on the change of sharpening strength; Apply the updated edge sharpening filter to the marked glass shard hazard areas. The core operation of the sharpening filter is as follows: , in, Represents the Laplacian operator of the original image, used to extract edge details, I original is the image of the original marked area, I sharpened For the sharpened image, the edge information is more prominent.

10. A grain depot cleaning robot visual recognition platform based on deep learning, used to implement the grain depot cleaning robot visual recognition method based on deep learning as described in any one of claims 1 to 9, characterized in that: It includes image acquisition module, feature extraction and analysis module, glass fragment recognition module, hidden danger area marking and edge delineation module, adaptive edge sharpening module, secondary recognition module and cleaning instruction generation module; The image acquisition module uses the visual system on the grain storage cleaning robot to perform comprehensive image acquisition of different areas in the grain storage environment; The feature extraction and analysis module divides the image into several sub-regions. For each sub-region, key features that reflect the presence of potential glass fragments are extracted from the collected image and analyzed to ensure that the deep learning model can receive accurate input; The glass fragment recognition module inputs the analyzed features into the pre-trained deep learning model, identifies the glass fragments in the grain warehouse through the deep learning model, and predicts the glass fragment hidden danger area and glass fragment area in the grain warehouse; The module for marking and delineating the edges of potential danger areas of glass fragments that have been initially identified marks the edges of the marked areas, providing a clear target range for further fine-tuning. The adaptive edge sharpening module adaptively adjusts the edge sharpening intensity for the marked glass fragment hidden danger areas according to the prediction results of the deep learning model to enhance the edge features of the glass fragments; The secondary recognition module re-inputs the sharpened edge features into the deep learning model for secondary recognition, further improving the detection accuracy of the deep learning model for glass fragments; The cleaning instruction generation module outputs the finally identified glass fragment area and its boundary information, and generates cleaning instructions for the cleaning robot to execute.

Images