Cargo identification method and system for roadway stacker

Abstract

The invention relates to the technical field of goods identification of a laneway stacker, in particular to a goods identification method and system of the laneway stacker, and the method comprises the following steps: collecting a goods image; performing quality evaluation on the cargo image, and judging the degradation type of the cargo image; based on the degradation type, performing adaptive image enhancement processing on the cargo image to obtain an enhanced image; generating a feature descriptor having resistance to the degradation influence of the cargo image; inputting the enhanced image and the feature descriptor into a deep learning model for cargo identification to obtain an identification result and confidence; and performing identification process control according to the confidence coefficient. According to the method, the recognition accuracy and stability of the deep learning model can be improved, so that the problems of recognition failure and efficiency reduction caused by image quality reduction are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of roadway stacker cargo identification, and particularly relates to a roadway stacker cargo identification method and system. BACKGROUND

[0002] In modern automated warehouses, as the core handling equipment, the efficient and accurate operation of the roadway stacker directly affects the smooth operation of the entire logistics system. In order to achieve accurate cargo identification, the stacker is usually equipped with a vehicle-mounted cargo identification system, which relies on an industrial camera installed on the stacker to capture cargo images. The camera lens surface applies a standard optical coating to optimize light transmittance, reduce reflection and scattering, and ensure that the image sensor receives clear and accurate image information.

[0003] However, in the actual warehouse environment, the high-speed movement of the stacker will produce significant airflow disturbance, causing fine particulate matter such as dust, fibers, and oil mist to deposit on the optical coating surface of the camera lens. The accumulation of these particulate matters changes the optical properties of the lens, increases the scattering effect of light, and thus reduces the quality of the image. In addition, changes in environmental humidity can also cause the oil-dust mixed film on the lens surface to absorb moisture and solidify, further changing its scattering properties of light, leading to a decrease in image quality, which in turn affects the quality of identification. SUMMARY

[0004] The present application aims to address the above-mentioned deficiencies by providing a roadway stacker cargo identification method and system.

[0005] The present application adopts the following technical solutions:

[0006] A roadway stacker cargo identification method, the method comprising the following steps:

[0007] Capturing a cargo image;

[0008] Performing quality assessment on the cargo image to determine the degradation type of the cargo image;

[0009] Based on the degradation type, performing adaptive image enhancement processing on the cargo image to obtain an enhanced image;

[0010] Generating a feature descriptor that is resistant to the degradation of the cargo image;

[0011] Taking the enhanced image and the feature descriptor as input, feeding them into a deep learning model for cargo identification to obtain an identification result and a confidence level;

[0012] Controlling the identification process according to the confidence level.

[0013] By the technical solution, the application can solve the problem of image degradation of the roadway stacker in a complex environment, significantly improve the recognition accuracy and stability of the deep learning model through adaptive image enhancement and generation of a robust feature descriptor, and effectively solve the problem of recognition failure and efficiency reduction caused by image quality degradation in the prior art.

[0014] Further, the step of performing quality assessment on the cargo image to determine the degradation type of the cargo image includes:

[0015] Obtaining motion state information of the stacker;

[0016] Based on the motion state information, predicting short-time optical distortion of the image region caused by mechanical vibration;

[0017] Performing local motion blur detection and geometric distortion analysis on the cargo image to obtain a processing region, the processing region including a local motion blur region and a geometric distortion region;

[0018] Distinguishing the processing region and persistent image degradation;

[0019] In the image region after the processing region is removed, evaluating the brightness distribution, contrast, definition and noise level of the cargo image to obtain brightness distribution evaluation results, contrast evaluation results, definition evaluation results and noise level evaluation results;

[0020] According to the brightness distribution evaluation results, contrast evaluation results, definition evaluation results and noise level evaluation results, the degradation type of the cargo image is determined.

[0021] Further, the step of obtaining the enhanced image includes:

[0022] Obtaining the humidity change rate and local temperature;

[0023] According to the humidity change rate and local temperature, predicting the thin film refractive index and surface topography change trend;

[0024] According to the degradation type, thin film refractive index and surface topography change trend, pre-adjusting the image enhancement algorithm parameters to perform image enhancement processing on the cargo image to obtain an initial image;

[0025] Analyzing the local pixel value distribution, edge gradient and color saturation of the initial image;

[0026] According to the local pixel value distribution, edge gradient and color saturation, adaptively adjusting the image enhancement algorithm parameters to perform image enhancement processing on the initial image to obtain an enhanced image.

[0027] Further, the step of predicting the thin film refractive index and surface topography change trend includes:

[0028] Obtain the temperature and humidity gradient change information of the area where the stacker is located to determine the humidity change rate and local temperature;

[0029] Obtain the stress state of the oil and dust film on the surface of the camera lens;

[0030] According to the temperature and humidity gradient change information and the stress state of the oil and dust film on the surface of the camera lens, the physical model parameters of the film optical characteristics are corrected in real time;

[0031] According to the corrected film optical characteristic physical model parameters, update the film optical characteristic physical model to predict the change trend of the film refractive index and surface topography.

[0032] Further, the step of obtaining the temperature and humidity gradient change information of the area where the stacker is located includes:

[0033] A forward temperature and humidity sensing array is arranged in front of the camera lens in the direction of the stacker travel;

[0034] A backward temperature and humidity sensing array is arranged behind the camera lens in the direction of the stacker travel;

[0035] Real-time acquisition of temperature and humidity data of the forward temperature and humidity sensing array and the backward temperature and humidity sensing array;

[0036] By comparing the temperature and humidity data of the forward temperature and humidity sensing array and the backward temperature and humidity sensing array, the temperature and humidity gradient change information of the area where the stacker is located is calculated.

[0037] Further, the step of obtaining the stress state of the oil and dust film on the surface of the camera lens includes:

[0038] Integrate a micro piezoelectric film sensor array in multiple directions on the surface of the camera lens;

[0039] Real-time acquisition of piezoelectric signals of the sensor array in different directions;

[0040] High-frequency filtering of the piezoelectric signal;

[0041] Instantaneous stress amplitude and frequency analysis of the filtered piezoelectric signal to obtain the stress state of the oil and dust film on the surface of the camera lens.

[0042] Further, the step of high-frequency filtering of the piezoelectric signal includes:

[0043] Frequency spectrum analysis of the piezoelectric signal to obtain the frequency spectrum characteristics of the piezoelectric signal;

[0044] According to the frequency spectrum characteristics, identify the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise in the piezoelectric signal;

[0045] According to the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise, the cutoff frequency and the attenuation slope of the high-pass filter are dynamically adjusted;

[0046] According to the dynamically adjusted cutoff frequency and attenuation slope, the piezoelectric signal is high-frequency filtered.

[0047] Further, the step of generating a feature descriptor resistant to the degradation of the cargo image includes:

[0048] Extracting the brightness change rate and color difference rate of the local area of the cargo image in multiple color channels;

[0049] Extracting the periodic texture pattern and non-periodic texture pattern of the local area of the cargo image;

[0050] Fusing the brightness change rate, color difference rate, periodic texture pattern and non-periodic texture pattern to generate a feature descriptor.

[0051] Further, the step of controlling the recognition process according to the confidence level includes:

[0052] Obtaining batch information, cargo value level and cargo visual similarity level of the cargo to be processed;

[0053] According to the batch information, cargo value level and cargo visual similarity level of the cargo to be processed, the recognition confidence threshold is dynamically adjusted;

[0054] Comparing the confidence level with the dynamically adjusted recognition confidence threshold;

[0055] When the confidence level is lower than the dynamically adjusted recognition confidence threshold, according to the cargo value level and the operation priority, triggering image reprocessing or issuing an artificial intervention alarm.

[0056] The application also discloses a roadway stacker cargo recognition system, which is applied to a roadway stacker cargo recognition method, and the system comprises:

[0057] An image acquisition module is configured to acquire a cargo image;

[0058] A quality evaluation module is configured to evaluate the quality of the cargo image and determine the degradation type of the cargo image;

[0059] An image enhancement module is configured to perform adaptive image enhancement processing on the cargo image based on the degradation type to obtain an enhanced image;

[0060] A feature descriptor generation module is configured to generate a feature descriptor resistant to the degradation of the cargo image;

[0061] An identification module takes the enhanced image and the feature descriptor as input, feeds them into a deep learning model for cargo identification, and obtains an identification result and a confidence level;

[0062] A control module controls the identification process according to the confidence level.

[0063] By this technical solution, the present application can integrate each step in the above method into a system through modular design, forming a complete and efficient solution for cargo identification of a roadway stacker, thereby providing hardware and software support for practical application and realizing the landing of the method.

[0064] The present application can effectively solve the problem in the prior art that the accumulation and solidification of an oil dust film on the surface of a camera lens caused by the operation of a roadway stacker in a complex warehouse environment, thereby causing image quality degradation, reduced identification accuracy of a deep learning model, and impaired system operation efficiency. Specifically, the present application realizes the adaptive adjustment of an image enhancement algorithm by accurately evaluating the image degradation type and predicting the optical properties of the film in combination with environmental parameters, thereby significantly improving the clarity and signal-to-noise ratio of the image. At the same time, the generation of a feature descriptor resistant to the effects of degradation enhances the robustness of the deep learning model when processing low-quality images. In addition, the introduction of cargo batch, value, and visual similarity level dynamically adjusts the identification confidence threshold and triggers reprocessing or manual intervention according to actual conditions, thereby realizing intelligent control of the identification process. Therefore, the method of the present application can significantly improve the accuracy and reliability of cargo identification of a roadway stacker, reduce the frequency of manual intervention, thereby improving the operation efficiency and throughput of the entire automated warehouse, and overcoming the shortcomings of high identification failure rate and low operation efficiency in the prior art.

[0065] To enable a further understanding of the features and technical contents of the present application, please refer to the following detailed description and drawings of the present application. However, the drawings provided are only for reference and illustration, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 A method flowchart of a roadway stacker cargo identification method of the present application;

[0067] Figure 2 A structural schematic diagram of a roadway stacker cargo identification system of the present application. DETAILED DESCRIPTION

[0068] The following embodiments will be described in conjunction with the preferred embodiments with reference to the drawings but the advantages and effects of the present application can be understood by those skilled in the art from the disclosure herein. The present application can be implemented or applied in other different embodiments, and various modifications and changes can be made to the details herein based on different views and applications without departing from the spirit of the present application. In addition, the drawings of the present application are only simple schematic illustrations and are not drawn according to the actual size, and it is declared in advance. The following embodiments will further illustrate the related technical contents of the present application, but the disclosed contents are not used to limit the protection scope of the present application.

[0069] The present embodiment provides a rack stocker cargo identification method and system, which combines Figure 1 and Figure 2 as shown.

[0070] Referring to Figure 1 , a rack stocker cargo identification method, the method comprising the following steps:

[0071] collecting cargo images;

[0072] performing quality assessment on the cargo images to determine the degradation type of the cargo images;

[0073] based on the degradation type, performing adaptive image enhancement processing on the cargo images to obtain enhanced images;

[0074] generating a feature descriptor resistant to the degradation of the cargo images;

[0075] inputting the enhanced images and the feature descriptor as input into a deep learning model for cargo identification to obtain an identification result and a confidence level;

[0076] controlling the identification process according to the confidence level.

[0077] Specifically, the method proposed in the present application first acquires original image data through the step of "collecting cargo images". The "cargo image" here refers to the visual information containing the cargo to be identified captured by the industrial camera installed on the rack stocker. These images may be affected by factors such as uneven lighting, blurring, noise, etc.

[0078] Subsequently, "quality assessment of the cargo image and determination of the degradation type of the cargo image" is one of the core steps of the method. This step aims to quantify the quality of the image and identify the specific reasons for image degradation. For example, the degradation type can include motion blur, defocus blur, low contrast, high noise, brightness overexposure or underexposure, color distortion, etc. By accurately determining the degradation type, targeted guidance can be provided for subsequent image enhancement. For example, methods based on image statistical features can be used, such as calculating the average gradient, information entropy, peak signal-to-noise ratio, or structural similarity index of the image to evaluate the image quality. The specific degradation type can also be determined by analyzing the histogram distribution, edge sharpness, noise variance, etc. of the image.

[0079] After determining the degradation type, the step of "performing adaptive image enhancement processing on the cargo image based on the degradation type to obtain an enhanced image" will select and adjust appropriate image enhancement algorithms based on the evaluation results. For example, if the image is judged to be low contrast, histogram equalization, gamma correction or local contrast enhancement algorithms can be used; if the image has motion blur, deconvolution algorithms or deep learning-based deblurring models can be used; if the image has high noise, non-local mean filtering, wavelet denoising or deep learning denoising networks can be used. The "adaptive" here refers to the dynamic adjustment of algorithm parameters to ensure the best enhancement effect and avoid over-enhancement or insufficient enhancement.

[0080] At the same time, the method also includes the step of "generating a feature descriptor resistant to the degradation of the cargo image". Traditional feature descriptors (such as SIFT, SURF, ORB, etc.) may not be stable or representative when the image is severely degraded. Therefore, this step aims to generate a feature representation that is more robust to image degradation. This can be achieved by designing new feature extraction algorithms, or by combining multiple features (such as texture, color, shape, local binary pattern, etc.) for fusion. For example, multi-scale feature fusion, deep learning feature extraction (such as intermediate layer features of CNN), or combining image semantic information to construct a more resistant descriptor can be used.

[0081] Next, the "input the enhanced image and feature descriptor into a deep learning model for cargo recognition, and obtain the recognition result and confidence" step inputs the processed image and feature descriptor into a pre-trained deep learning model. This deep learning model can be a convolutional neural network (CNN), recurrent neural network, or Transformer, etc., which has been trained on a large number of high-quality cargo images and can learn the complex visual patterns of the cargo. The enhanced image provides clearer visual information, while the robust feature descriptor supplements key features that are insensitive to degradation, and the combination of the two can significantly improve the recognition accuracy and stability of the model. The recognition result usually includes the category, position, etc. of the cargo, while the confidence represents the degree of confidence of the model in the recognition result.

[0082] Finally, the "control the recognition process according to the confidence" step is the decision-making link of the method. Confidence is an important indicator of the reliability of the recognition result. When the confidence is high, the system can consider the recognition result accurate and continue to perform subsequent stacking operations; when the confidence is low, it indicates that the recognition result may be uncertain, at which point the system can control the process according to the pre-set strategy, such as triggering image reprocessing, requesting manual review, or adjusting the stacking machine's operation strategy, etc., to avoid potential risks caused by false recognition.

[0083] Compared with the prior art, the core innovation of the present application lies in its "perception-enhancement-robustness-control" closed-loop processing mechanism for image degradation. First, by "evaluating the quality of the cargo image and determining the degradation type of the cargo image", the method can accurately identify the root cause of image degradation, rather than simply performing general enhancement. For example, when the image is degraded due to motion blur, the method will not mistakenly apply a contrast enhancement algorithm, but will choose a targeted deblurring technique. Second, "based on the degradation type, perform adaptive image enhancement processing on the cargo image" ensures the accuracy and efficiency of the enhancement strategy, avoiding over-processing or under-processing. Third, "generate a feature descriptor that is resistant to the degradation of the cargo image" enhances the robustness of recognition from the feature level, even in cases where image enhancement is limited, providing reliable identification basis. Finally, "control the recognition process according to the confidence" builds an intelligent decision-making mechanism that can dynamically adjust the subsequent process according to the reliability of the recognition result, effectively reducing invalid retries and manual intervention, and significantly improving the efficiency and automation level of the stacking machine operation.

[0084] For example, in a traditional method, when the camera lens is contaminated by an oil dust film causing image blur, the deep learning model may not be able to identify the barcode on the goods, the confidence continues to be below the threshold, and the system will repeatedly try to collect and identify, and eventually may need manual intervention. However, the method of the present application first evaluates the image to determine the degradation type, such as blur and low contrast, and then performs adaptive enhancement while generating feature descriptors that are resistant to blur and low contrast. These optimized inputs will greatly improve the accuracy and confidence of the identification when fed into the deep learning model, thereby avoiding unnecessary retries and manual intervention, ensuring the continuity and efficiency of the stacker operation. This integrated solution significantly improves the goods recognition ability and overall operating efficiency of the tunnel stacker in complex industrial environments.

[0085] Specifically, the step of quality evaluation of the goods image and determining the degradation type of the goods image includes: obtaining motion state information of the stacker; predicting short-time optical distortion of the image area caused by mechanical vibration based on the motion state information; performing local motion blur detection and geometric distortion analysis on the goods image to obtain a processing area, the processing area including a local motion blur area and a geometric distortion area; distinguishing the processing area and persistent image degradation; evaluating the brightness distribution, contrast, sharpness and noise level of the goods image in the image area after the processing area is removed to obtain brightness distribution evaluation results, contrast evaluation results, sharpness evaluation results and noise level evaluation results; and determining the degradation type of the goods image according to the brightness distribution evaluation results, contrast evaluation results, sharpness evaluation results and noise level evaluation results.

[0086] Wherein, obtaining the motion state information of the stacker can be understood as obtaining the speed, acceleration, vibration frequency and other parameters of the stacker when performing goods identification. These information can be obtained in real time by sensors (such as inertial measurement unit IMU, encoder, etc.) inside the stacker, which aims to provide basic data for subsequent prediction of image distortion. Based on the motion state information, the short-time optical distortion of the image area caused by mechanical vibration is predicted, which means that the motion state information of the stacker is used in combination with a pre-set mechanical vibration model or through a machine learning model to predict the short-time optical distortion such as image blur and jitter caused by mechanical vibration that may occur at the moment of image acquisition. For example, when the stacker moves at high speed or suddenly brakes, its mechanical vibration may cause transient blur or geometric distortion in the camera image.

[0087] The local motion blur detection and geometric distortion analysis of the cargo image obtain a processing region, and the processing region includes a local motion blur region and a geometric distortion region. The processing region refers to image processing algorithm analysis of the collected cargo image, and the local blur region caused by motion or mechanical vibration and the geometric distortion region caused by camera posture change or lens distortion in the image are identified. The local motion blur detection can be performed by using a method based on Fourier transform, wavelet analysis or deep learning; the geometric distortion analysis can be performed by using a feature point matching method, a grid deformation analysis method or the like. The purpose is to accurately identify the specific region of the damaged image.

[0088] The distinction between the processing region and the persistent image degradation refers to distinguishing the local motion blur region and the geometric distortion region caused by short-time mechanical vibration (i.e., the processing region) from the persistent and global image degradation caused by environmental factors (such as dust, haze, uneven illumination, etc.) or device aging (such as lens contamination). For example, the distinction can be completed by analyzing the spatial and temporal characteristics, frequency characteristics or morphological features of the degradation. The purpose is to adopt different evaluation and enhancement strategies for different types of degradation.

[0089] The brightness distribution evaluation result, the contrast evaluation result, the definition evaluation result and the noise level evaluation result of the cargo image in the image region after the processing region is removed are obtained by evaluating the brightness distribution, the contrast, the definition and the noise level of the cargo image in the image region after the processing region is removed. The remaining region of the image is subjected to conventional image quality evaluation after the short-time local degradation region caused by mechanical vibration is removed. The brightness distribution evaluation can be performed by using a histogram analysis method, an average brightness calculation method or the like; the contrast evaluation can be performed by using an RMS contrast, a Weber contrast or a Michelson contrast; the definition evaluation can be performed by using an edge gradient, an MTF (modulation transfer function) or a blind sharpness evaluation algorithm; and the noise level evaluation can be performed by using a signal-to-noise ratio, a peak signal-to-noise ratio or a noise variance. The purpose is to comprehensively quantify the intrinsic quality attributes of the image.

[0090] The degradation type of the cargo image is determined according to the brightness distribution evaluation result, the contrast evaluation result, the definition evaluation result and the noise level evaluation result. The main degradation type of the current cargo image is determined by comprehensively analyzing the above evaluation results, using a preset rule, a decision tree or a classification model. For example, if the brightness is too low and the contrast is insufficient, it can be determined that the illumination is insufficient; if the definition is low and the noise level is high, it can be determined that the ISO noise is high under low illumination; and if there is local motion blur, it can be determined that the motion blur degradation is caused. The purpose is to provide an accurate degradation type basis for subsequent adaptive image enhancement.

[0091] The application further provides a step of obtaining an enhanced image, including:

[0092] The humidity change rate and the local temperature are obtained.

[0093] predicting the change trend of the thin film refractive index and surface topography according to the humidity change rate and local temperature;

[0094] pre-adjusting the image enhancement algorithm parameters according to the degradation type, the change trend of the thin film refractive index and surface topography, and performing image enhancement processing on the cargo image to obtain an initial image;

[0095] analyzing local pixel value distribution, edge gradient and color saturation of the initial image;

[0096] adjusting the image enhancement algorithm parameters according to the local pixel value distribution, edge gradient and color saturation, and performing image enhancement processing on the initial image to obtain an enhanced image.

[0097] Specifically, the humidity change rate and local temperature can be understood as monitoring the environmental data in real time through the temperature and humidity sensors deployed near the stacker or the camera. The humidity change rate refers to the change amount of humidity per unit time, and the local temperature refers to the temperature of the area near the camera lens. These environmental parameters are crucial for evaluating the optical properties of the thin film (such as oil dust film or water vapor condensation film) that may exist on the surface of the camera lens. Among them, predicting the change trend of the thin film refractive index and surface topography refers to predicting the optical refractive index of the thin film that may exist on the surface of the camera lens and the change trend of its surface micro-topography based on the obtained humidity change rate and local temperature, combined with the pre-established physical model or empirical model. For example, high humidity may cause water vapor condensation, changing the refractive index of the thin film, while temperature changes may affect the stress state and topography of the thin film.

[0098] In practical applications, pre-adjusting the image enhancement algorithm parameters is to set the initial parameters of the image enhancement algorithm. This pre-adjustment process not only considers the degradation types of the goods image (such as blur, low contrast, noise, etc.), but also combines the predicted thin film refractive index and surface topography trend. For example, if it is predicted that the thin film refractive index change causes the image to appear a certain color cast or halo, the parameters of the color correction or de-halo algorithm can be pre-adjusted to provide an initial enhancement effect closer to the ideal state. Further, analyzing the local pixel value distribution, edge gradient, and color saturation of the initial image is a detailed quality assessment of the initial image after preliminary enhancement processing. The local pixel value distribution can reflect the brightness uniformity and dynamic range of the image; the edge gradient can measure the sharpness and detail retention of the image; and the color saturation reflects the vividness and authenticity of the image color. These analyses help to identify possible deficiencies or over-enhanced areas of the initial enhancement processing. Thus, based on the local pixel value distribution, edge gradient, and color saturation, the image enhancement algorithm parameters are adaptively adjusted, which is a feedback of the initial image analysis results. Based on these analyses, the parameters of the image enhancement algorithm are further fine-tuned to make up for the deficiencies of the initial enhancement, for example, if the edge gradient of the initial image is found to be insufficient, the sharpening parameter can be enhanced; if the color saturation is too high or too low, the color balance parameter can be adjusted. This secondary adaptive adjustment aims to obtain the best enhanced image.

[0099] The scheme of the present application introduces the perception of environmental factors (humidity change rate and local temperature) and combines the prediction of the optical properties of the camera lens surface film (thin film refractive index and surface topography trend), so that the parameters of the image enhancement algorithm can be pre-adjusted more forward-looking and targeted. This pre-adjustment mechanism can effectively cope with complex image degradation caused by environmental changes, avoiding the limitations brought by relying only on a single degradation type. On this basis, by analyzing the local pixel value distribution, edge gradient, and color saturation of the initial enhanced image, the image enhancement algorithm parameters are further adaptively adjusted, forming a closed-loop optimization process. It is precisely due to this multi-level and fine-tuned parameter adjustment strategy that the image enhancement processing can more accurately compensate for various degradation effects, thereby obtaining higher quality enhanced images.

[0100] In some preferred embodiments, the system assumes that the aisle stacker is operating in a warehouse environment that is humid and temperature fluctuates. When the humidity change rate increases and the local temperature drops, the system acquires these environmental data through the forward and backward temperature and humidity sensing arrays deployed near the camera lens. Based on these data, combined with the pre-established thin film optical property physical model, the system predicts that a micron-level water vapor condensation film may form on the surface of the camera lens, its refractive index will change, and the surface topography may become uneven, causing slight scattering and color deviation of the image. At this time, the system first adjusts the parameters of the image enhancement algorithm (such as the Retinex-based enhancement algorithm) according to the judged image degradation type (for example, slight blur and low contrast), and combines the predicted thin film refractive index and surface topography change trend. For example, the weight of the defogging parameter is increased in advance, and the color balance parameter is fine-tuned to compensate for the predicted color deviation. After the first image enhancement processing, the initial image is obtained. Subsequently, the system analyzes the initial image and finds that the local pixel value distribution is still slightly uneven, the edge gradient is not sharp enough in some areas, and the color saturation is slightly deviated. Based on these analysis results, the system further adjusts the parameters of the image enhancement algorithm. For example, for uneven pixel value distribution, local histogram equalization can be used; for insufficient edge gradient, an adaptive sharpening filter can be applied; for color saturation deviation, fine color correction can be performed. Through this secondary adaptive adjustment, a high-quality enhanced image is finally obtained, which not only has high clarity and moderate contrast, but also has true color restoration, effectively eliminating the degradation effects of environmental factors and providing an ideal input for subsequent deep learning recognition.

[0101] The step of predicting the thin film refractive index and surface topography change trend includes:

[0102] Obtain the temperature and humidity gradient change information of the area where the stacker is located to determine the humidity change rate and the local temperature;

[0103] Obtain the stress state of the oil and dust film on the surface of the camera lens;

[0104] According to the temperature and humidity gradient change information and the stress state of the oil and dust film on the surface of the camera lens, the physical model parameters of the thin film optical property are corrected in real time;

[0105] Update the thin film optical property physical model according to the corrected thin film optical property physical model parameters to predict the thin film refractive index and surface topography change trend.

[0106] Specifically, the "temperature and humidity gradient change information" refers to the spatial distribution difference of temperature and humidity in the environment where the stacker is located and its trend of change over time. By obtaining such information, the dynamic changes of the micro-environment on the camera lens surface can be more accurately understood, so as to more accurately determine the humidity change rate and local temperature affecting the characteristics of the film. Among them, this information can be understood as the difference between the temperature and humidity data collected at different positions near the camera lens or at different time points, and its purpose is to capture the local non-uniformity of the environment.

[0107] The "stress state of the oil and dust film on the camera lens surface" refers to the internal or external mechanical force borne by the oil and dust film attached to the surface of the camera lens. These stresses can cause slight deformation of the physical structure of the film, thereby affecting its optical refractive index and surface topography. Obtaining the stress state helps to more comprehensively evaluate the real-time optical characteristics of the film. In practical applications, the stress state can be specifically the tensile, compressive or shear stress of the film, and its purpose is to quantify the degree of deformation of the film due to mechanical action.

[0108] The "real-time correction of the physical model parameters of the film optical characteristics" refers to dynamically adjusting the various parameters in the mathematical model used to describe the optical behavior of the film according to the newly obtained temperature and humidity gradient change information and film stress state. For example, coefficients related to temperature, humidity and stress can be adjusted to make the model more accurately reflect the film characteristics under current environmental conditions. Among them, this correction process can be realized by adaptive filtering, Kalman filtering or machine learning methods, and its purpose is to ensure that the model parameters can closely track the actual environmental changes.

[0109] The "updating of the physical model of the film optical characteristics" refers to substituting the parameters that have been corrected in real time into the physical model of the film optical characteristics, thereby obtaining a model that is more consistent with the current actual environmental conditions. Based on this updated model, the refractive index and surface topography change trend of the film can be more accurately predicted, providing a more reliable basis for subsequent image enhancement processing.

[0110] The scheme of the present application can more comprehensively and finely capture environmental factors and mechanical factors affecting the optical properties of the film by introducing the temperature and humidity gradient change information of the area where the stacker is located and the stress state of the oil dust film on the camera lens surface. The temperature and humidity gradient change information provides more detailed environmental information than a single local temperature and humidity, which helps to identify the impact of local microclimate on the film, such as differences in local condensation or evaporation rate. At the same time, the stress state of the film directly reflects the physical deformation that the film may undergo, which directly changes its optical properties, such as refractive index or surface roughness. By taking these multi-dimensional, real-time environmental and physical parameters as inputs, the physical model parameters of the film optical properties are corrected in real time, so that the model can dynamically adapt to complex working conditions. It is precisely because of the accurate correction of the model parameters and the updating of the model that the prediction results of the film refractive index and surface morphology change trend are more accurate, thereby providing more reliable and fine inputs for subsequent adaptive image enhancement processing, effectively overcoming the problem of insufficient accuracy caused by relying on simple temperature and humidity prediction.

[0111] In some preferred embodiments, assuming that the stacker moves at high speed in the warehouse, the camera lens surface may generate a local temperature and humidity gradient due to air flow disturbance, and at the same time, the vibration of the stacker itself may also cause the oil dust film on the lens surface to bear a small stress. At this time, the system first acquires and compares the temperature and humidity data in real time through the temperature and humidity sensing arrays arranged along the front and rear of the camera lens, and calculates the temperature and humidity gradient change information of the area where the stacker is located. At the same time, the micro piezoelectric film sensor array integrated on the surface of the camera lens will monitor and analyze the stress state of the film in real time. These information are input into a pre-established physical model of the optical properties of the film. The model will dynamically adjust its internal parameters according to these real-time environmental and physical inputs, for example, if a large humidity gradient is detected, the model will adjust the parameters related to water vapor condensation or evaporation accordingly; if the film stress changes, the optical parameters related to the deformation of the film will be adjusted. In this way, the physical model of the optical properties of the film is updated in real time, so as to more accurately predict the refractive index and surface morphology change trend of the film at the current time. For example, when it is predicted that there is a small water droplet condensation or an oil film local bulge on the surface of the film, the system can adjust the defogging or deblurring parameters in the image enhancement algorithm accordingly to ensure that high-quality enhanced images can be obtained even in complex environments.

[0112] The present application further proposes a step of obtaining the temperature and humidity gradient change information of the area where the stacker is located, comprising:

[0113] A forward temperature and humidity sensing array is arranged along the front of the camera lens in the direction of travel of the stacker;

[0114] A backward temperature and humidity sensing array is arranged along the rear of the camera lens in the direction of travel of the stacker;

[0115] collecting the temperature and humidity data of the forward temperature and humidity sensing array and the backward temperature and humidity sensing array in real time;

[0116] calculating the temperature and humidity gradient change information of the area where the stacker is located by comparing the temperature and humidity data of the forward temperature and humidity sensing array and the backward temperature and humidity sensing array.

[0117] Specifically, the forward temperature and humidity sensing array and the backward temperature and humidity sensing array can each be composed of multiple temperature and humidity sensors, which are linearly or matrixly arranged along the direction of travel of the stacker to achieve fine perception of the temperature and humidity field in front of and behind the camera lens. Among them, the forward temperature and humidity sensing array is configured to perceive the temperature and humidity conditions of the area where the stacker is about to enter, while the backward temperature and humidity sensing array is configured to perceive the temperature and humidity conditions of the area where the stacker has left or is currently located. Collecting temperature and humidity data in real time means continuously obtaining temperature and humidity readings of each sensing array at a preset sampling frequency to ensure that dynamic changes in the environment can be captured. By comparing the temperature and humidity data of the forward temperature and humidity sensing array and the backward temperature and humidity sensing array, the temperature and humidity spatial gradient of the microenvironment where the camera lens is located can be calculated, for example, by calculating the average temperature and humidity difference between the forward and backward arrays or more complex spatial interpolation algorithms.

[0118] The scheme of the present application can accurately capture the temperature and humidity spatial gradient of the microenvironment where the camera lens is located during the travel of the stacker by setting temperature and humidity sensing arrays in front of and behind the camera lens and collecting and comparing their data in real time. This differential perception mechanism overcomes the limitations of traditional single sensors that cannot accurately reflect the dynamic changes of the local environment. It is precisely because accurate temperature and humidity gradient information can be obtained that the subsequent determination of humidity change rate and local temperature is more accurate, thereby providing more reliable input data for predicting the change trend of the refractive index and surface topography of the thin film.

[0119] In some preferred embodiments, it is assumed that the aisle stacker is operating in a warehouse environment with dramatic changes in temperature and humidity. For example, when the stacker is driving from a normal temperature area to a refrigerated area, the forward temperature and humidity sensing array will first detect a significant drop in temperature and an increase in humidity. Then, the rear temperature and humidity sensing array will also detect similar changes, but with a certain time lag and numerical difference. By comparing the temperature and humidity data of the two arrays in real time, the system can calculate the precise temperature and humidity gradient of the environment around the camera lens, for example, it can calculate that the temperature drops 2 degrees Celsius per meter and the humidity rises 5% RH. These precise gradient information is then used to correct the thin film optical property physical model parameters, so that the prediction of the change trend of the thin film refractive index and surface topography is closer to the actual situation, thereby ensuring that the image enhancement algorithm can timely and accurately adjust the parameters to compensate for the image degradation caused by the change of the thin film, for example, by adjusting the intensity of the contrast enhancement or dehazing algorithm, so that the cargo image still maintains high definition and recognizability after entering the refrigerated area.

[0120] The present application further proposes a step of obtaining the stress state of the oil and dust film on the surface of the camera lens, comprising:

[0121] integrating a micro piezoelectric film sensor array in multiple directions on the surface of the camera lens;

[0122] real-time acquisition of piezoelectric signals of the sensor array in different directions;

[0123] high-frequency filtering of the piezoelectric signals;

[0124] instantaneous stress amplitude and frequency analysis of the filtered piezoelectric signals to obtain the stress state of the oil and dust film on the surface of the camera lens.

[0125] Specifically, integrating a micro piezoelectric film sensor array in multiple directions on the surface of the camera lens means arranging a plurality of small-sized and sensitive piezoelectric film sensors in an array on the surface of the camera lens, and ensuring that these sensors can cover the key areas of the lens surface and their sensing directions can cover the possible stress directions. The piezoelectric film sensor can convert the mechanical stress or strain acting on its surface into an electrical signal, thereby realizing non-contact or micro-disturbance measurement of the stress state of the film. Integration in multiple directions aims to capture the anisotropic stress distribution that may exist in the oil and dust film and provide more comprehensive stress information.

[0126] Among them, real-time acquisition of piezoelectric signals of the sensor array in different directions can be understood as continuously acquiring the electrical signals generated by each sensor in its respective direction at a preset sampling frequency through the data acquisition unit connected to the sensor array. This real-time acquisition ensures timely monitoring of the dynamic stress changes of the oil and dust film.

[0127] In practical applications, the piezoelectric signal is high-frequency filtered, and the purpose is to remove the low-frequency mechanical vibration noise, environmental electromagnetic interference and other non-stress-related low-frequency signal components that may exist in the collected piezoelectric signal, so as to highlight the high-frequency signal caused by the stress change of the oil dust film.

[0128] Further, the instantaneous stress amplitude and frequency analysis of the filtered piezoelectric signal refers to further signal processing of the pure piezoelectric signal after filtering, such as Fourier transform, wavelet analysis and other methods, to extract the instantaneous amplitude of the signal (reflecting the stress size) and the frequency characteristics (reflecting the dynamic characteristics of the stress change). Through these analyses, the stress state of the oil dust film on the surface of the camera lens in different directions can be accurately quantified.

[0129] The scheme of the present application can directly and real-time perceive the mechanical stress borne by the oil dust film by integrating a micro piezoelectric film sensor array on the surface of the camera lens. Due to the high sensitivity and fast response characteristics of the piezoelectric film sensor, it can convert the tiny stress change on the oil dust film into a measurable electrical signal. The integrated mode in multiple directions ensures that even if the stress distribution of the oil dust film has anisotropy, it can be fully captured. Subsequently, by high-frequency filtering the collected piezoelectric signal, the low-frequency mechanical vibration and other interference generated during the operation of the stacker are effectively eliminated, so that the subsequent stress analysis can focus on the real stress change caused by the characteristics of the oil dust film itself and the external environment (such as airflow impact, particle adhesion). Finally, by analyzing the instantaneous stress amplitude and frequency of the filtered signal, the dynamic stress state of the oil dust film on the surface of the camera lens can be accurately quantified, providing accurate and reliable input data for real-time correction of the physical model of the film optical characteristics. It is precisely because of this refined stress state acquisition method that the prediction of the change trend of the film refractive index and surface topography is more accurate, thereby providing a solid foundation for adaptive image enhancement processing.

[0130] In some preferred embodiments, assuming the aisle stacker is running at high speed in the warehouse, the camera lens surface is covered with a dynamic changing oil dust film due to the airflow and the attachment of tiny dust particles in the environment. In order to accurately obtain the stress state of this film, a 16-micro-piezoelectric film sensor array can be integrated uniformly on the camera lens surface, arranged in a 4x4 grid, and each sensor can sense stress in different directions. When the stacker is running, the oil dust film is impacted by airflow and deformed, and the sensor array will generate corresponding piezoelectric signals in real time. For example, when a certain area of the film is subjected to shear stress, the corresponding direction sensor will output a specific voltage signal. These signals are collected by a high-speed data acquisition card at a frequency of 1000 times per second. Subsequently, the collected raw piezoelectric signals are sent to a digital signal processor for high-frequency filtering processing, such as using a high-pass filter with a cutoff frequency of 500Hz to filter out low-frequency noise generated by the stacker mechanical vibration. The filtered signals are then subjected to frequency spectrum analysis through fast Fourier transform to extract their instantaneous amplitude and dominant frequency components. Through comprehensive analysis of these amplitude and frequency data, the instantaneous stress tensor of the camera lens surface oil dust film in each direction can be accurately calculated, and its stress state can be obtained. For example, it can be identified that the film bears 0.5MPa tensile stress in a certain direction and 0.2MPa shear stress in another direction, and these stresses fluctuate at a frequency of 100Hz. These accurate stress state data are then input into the physical model of the film's optical properties to real-time correct the model parameters, so as to more accurately predict the refractive index and surface topography changes of the film, and provide reliable basis for subsequent image enhancement processing.

[0131] The application further proposes that the step of high-frequency filtering processing the piezoelectric signal comprises:

[0132] The piezoelectric signal is subjected to frequency spectrum analysis to obtain the frequency spectrum characteristics of the piezoelectric signal;

[0133] According to the frequency spectrum characteristics, the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise of the piezoelectric signal are identified;

[0134] According to the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise, the cutoff frequency and attenuation slope of the high-pass filter are dynamically adjusted;

[0135] The piezoelectric signal is subjected to high-frequency filtering processing according to the dynamically adjusted cutoff frequency and attenuation slope.

[0136] Specifically, the frequency spectrum analysis of the piezoelectric signal refers to converting the real-time collected piezoelectric signal from the time domain to the frequency domain through Fourier transform or other time-frequency analysis methods to obtain its energy distribution and phase information at different frequencies, thereby obtaining the frequency spectrum characteristics of the piezoelectric signal. The frequency spectrum characteristics can reveal various frequency components and their strengths contained in the signal. Among them, according to the frequency spectrum characteristics, identifying the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise of the piezoelectric signal refers to distinguishing the high-frequency signal component caused by the stress change of the camera lens surface dust film and the low-frequency noise component caused by the mechanical vibration of the stacker itself, environmental noise and other factors by analyzing the above frequency spectrum characteristics. The dominant frequency range of the high-frequency stress component usually corresponds to the frequency of the rapid change of the film stress, while the frequency range of the low-frequency mechanical vibration noise corresponds to the frequency of the stacker structure vibration or environmental noise. In practical applications, according to the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise, dynamically adjusting the cutoff frequency and attenuation slope of the high-pass filter refers to setting the cutoff frequency of the high-pass filter slightly lower than the lower limit of the dominant frequency range of the high-frequency stress component to ensure effective retention of the stress signal. At the same time, the attenuation slope is adjusted to provide sufficient attenuation in the frequency range of the low-frequency mechanical vibration noise, thereby maximizing the suppression of noise while avoiding excessive distortion of useful signals. This dynamic adjustment ensures that the filter parameters match the actual characteristics of the signal. Therefore, according to the dynamically adjusted cutoff frequency and attenuation slope, the high-frequency filtering processing of the piezoelectric signal refers to applying the high-pass filter with dynamically adjusted parameters to the original piezoelectric signal to remove the low-frequency mechanical vibration noise, thereby obtaining a pure high-frequency stress signal for subsequent stress state analysis.

[0137] In some preferred embodiments, the specific process of high-frequency filtering of the piezoelectric signal can be as follows: first, the frequency spectrum of the collected piezoelectric signal is analyzed by fast Fourier transform to obtain its amplitude-frequency characteristic curve. For example, when the stacker is running at high speed or passing through an uneven track, there may be obvious low-frequency peaks (e.g. 5-50 Hz) in the piezoelectric signal, which usually corresponds to mechanical vibration noise; at the same time, high-frequency signals (e.g. 100-500 Hz) caused by thin film stress changes may also exist. According to these spectral characteristics, the system can identify the dominant frequency range of low-frequency noise and the dominant frequency range of high-frequency stress components. For example, if it is detected that low-frequency noise is mainly concentrated below 20 Hz, and stress signals are mainly above 150 Hz, the cutoff frequency of the high-pass filter can be dynamically adjusted to about 100 Hz, and a steep attenuation slope (e.g. 40 dB / decade) is set to ensure that noise below 20 Hz is effectively suppressed, while stress signals above 150 Hz are completely preserved. Through this dynamic adjustment, even when the running state or environmental conditions of the stacker change, the filtering effect can be optimized to accurately extract the thin film stress information.

[0138] In some embodiments of the present application, a feature descriptor resistant to the degradation of the cargo image is generated. Specifically, the step of generating the feature descriptor resistant to the degradation of the cargo image includes:

[0139] extracting the brightness change rate and color difference rate of the local region of the cargo image in multiple color channels;

[0140] extracting the periodic texture pattern and non-periodic texture pattern of the local region of the cargo image;

[0141] fusing the brightness change rate, color difference rate, periodic texture pattern and non-periodic texture pattern to generate the feature descriptor.

[0142] Specifically, the brightness change rate can be understood as the degree of change of pixel intensity in the local region of the image with the spatial position, which reflects the edge and texture information of the image. For example, it can be obtained by calculating the image gradient or Laplacian operator. The color difference rate refers to the relative difference of pixel values between different color channels, which helps to capture the color features of the image, for example, it can be obtained by calculating the difference or ratio of each channel in the RGB or HSV color space. The purpose is to quantify the visual information of the image from different dimensions.

[0143] Among them, the periodic texture pattern refers to the structure or pattern that repeatedly appears in the image, such as grid, stripe, etc., which can be extracted by Fourier transform, wavelet analysis or local binary pattern (LBP) method. The aperiodic texture pattern refers to the irregular or random distributed texture in the image, such as noise, fuzzy area, etc., which can be described by statistical features (such as gray level co-occurrence matrix GLCM) or local entropy method. The purpose is to comprehensively capture the structural information of the image, including regular and irregular visual elements.

[0144] In practical applications, the fusion of brightness change rate, color difference rate, periodic texture pattern and aperiodic texture pattern is specifically realized by splicing, weighted summation of these extracted feature vectors or dimension reduction and combination through machine learning methods (such as principal component analysis PCA, linear discriminant analysis LDA). For example, these features can be used as different components of a multi-dimensional vector to form a comprehensive feature descriptor. The purpose is to integrate multi-source information to form a feature vector that has comprehensive and robust representation ability for image content.

[0145] The scheme of the present application can effectively capture the edge, texture and color information of the image by extracting the brightness change rate and color difference rate of the local area of the cargo image in multiple color channels. These information can maintain relative stability when the image is degraded in brightness, contrast or color distortion. At the same time, by extracting the periodic texture pattern and aperiodic texture pattern of the local area of the cargo image, the feature descriptor can finely describe the structural information in the image, whether it is regular texture or irregular noise or blur, which can be effectively coded. It is because these features have inherent resistance to image degradation, for example, the brightness change rate is not sensitive to overall brightness shift, the color difference rate has certain robustness to white balance drift, and the texture pattern can resist a certain degree of blur and noise, which makes the generated feature descriptor still maintain its distinguishability and stability when facing various complex degradation types, thereby providing high-quality input for subsequent deep learning models.

[0146] By the technical solution, the generated feature descriptor can effectively resist various degradation effects that the cargo image may encounter in actual application, such as uneven illumination, blur, noise, color distortion, etc. This resistance is derived from multi-dimensional and multi-scale feature extraction and fusion of the image, so that the feature descriptor not only contains rich semantic information, but also has robustness to degradation disturbance. Compared with features that only rely on a single type or are insensitive to degradation, the present scheme significantly improves the feature descriptor's representation ability for cargo images in complex industrial environments by comprehensively utilizing brightness, color and texture information, thereby providing a more reliable basis for deep learning models to accurately identify goods and effectively reducing the recognition error rate and false positive rate caused by image quality problems.

[0147] The step of controlling the recognition process according to the confidence level comprises:

[0148] Obtaining batch information, cargo value level and cargo visual similarity level of the cargo to be processed;

[0149] According to the batch information, cargo value level and cargo visual similarity level of the cargo to be processed, dynamically adjusting the recognition confidence threshold;

[0150] Comparing the confidence level with the dynamically adjusted recognition confidence threshold;

[0151] When the confidence level is lower than the dynamically adjusted recognition confidence threshold, triggering image reprocessing or issuing an artificial intervention alarm according to the cargo value level and operation priority.

[0152] Among them, obtaining the batch information, cargo value level and cargo visual similarity level of the cargo to be processed aims to provide multi-dimensional context information for subsequent dynamic threshold adjustment. Batch information can include production batch, warehouse batch or specific task batch of the cargo, etc. These information may be associated with specific attributes or recognition difficulty of the cargo. The cargo value level is used to distinguish high-value, medium-value or low-value goods. High-value goods usually have higher requirements for recognition accuracy, and the cost of incorrect identification is higher. The cargo visual similarity level assesses the similarity of different goods in appearance, for example, some goods may be difficult to distinguish due to slight differences in color, shape or texture, so more careful identification judgment is needed.

[0153] Based on the acquired batch information, the cargo value level, and the cargo visual similarity level, the recognition confidence threshold will be dynamically adjusted. For example, for high-value or high-visual-similarity cargo, a higher confidence threshold can be set to ensure that only when the recognition result has extremely high reliability will it be accepted, thereby reducing the risk of misrecognition. Conversely, for low-value and visually distinct cargo, the confidence threshold can be appropriately lowered to improve recognition efficiency. This dynamic adjustment can be achieved through a pre-set rule base, a machine learning model based on historical data, or an expert system.

[0154] Subsequently, the confidence output by the deep learning model is compared with the dynamically adjusted recognition confidence threshold. If the confidence is lower than the dynamically adjusted threshold, it indicates that the current recognition result may have a high uncertainty or error risk.

[0155] In this case, the system will intelligently trigger subsequent processing according to the value level of the cargo and the pre-set operation priority. Specifically, for high-value cargo or scenarios with extremely high recognition accuracy requirements, even if the confidence is slightly lower than the threshold, an artificial intervention alarm can be issued in priority to ensure final confirmation by humans, avoiding potential significant losses. For low-value or scenarios with higher real-time requirements, the system may prefer to trigger image reprocessing, such as reacquiring images, applying different image enhancement strategies, or using a backup recognition model, to attempt to improve recognition accuracy without human intervention. This strategic selection helps optimize resource allocation and balance recognition efficiency and accuracy.

[0156] The scheme of the present application introduces the batch information of the cargo to be processed, the cargo value level, and the cargo visual similarity level, and dynamically adjusts the recognition confidence threshold based on these information, thereby solving the limitations of the traditional fixed threshold method in complex and variable application scenarios. It is precisely because of the fine and situational management of the recognition confidence threshold that the system can flexibly adjust the strictness of recognition according to the characteristics of different cargos and business needs. When the recognition confidence is insufficient, the system does not simply reject the recognition result, but intelligently selects to trigger image reprocessing or issue an artificial intervention alarm in combination with the cargo value level and the operation priority. This multi-level decision-making mechanism enables the system to take more reasonable and efficient measures when facing uncertainty, avoiding resource waste or risk accumulation caused by one-size-fits-all judgment standards.

[0157] In some preferred embodiments, assume that the rack stacker is handling a batch of high-value precision electronic components, which have subtle differences in appearance, resulting in a high level of visual similarity. At this time, the system first obtains the high-value attribute and high visual similarity level information of the batch. Based on this information, the recognition confidence threshold will be dynamically adjusted to a relatively high level, for example, from the default 0.8 to 0.95. When the deep learning model has a recognition confidence of 0.90 for a certain electronic component, since it is lower than the dynamically adjusted 0.95 threshold, the system will judge that the recognition result has potential risks. Considering the high value level and operation priority of the batch of goods (for example, avoiding any misidentification), the system will not simply trigger image reprocessing, but will immediately issue an artificial intervention alarm to prompt the operator to manually check the specific component to ensure the absolute accuracy of the identification, thereby effectively avoiding significant economic losses caused by misidentification.

[0158] As another specific implementation, assume that the rack stacker is handling a batch of low-value ordinary packaging boxes, which have obvious visual feature differences and a low level of visual similarity. In this case, after the system obtains the low-value attribute and low visual similarity level information, the recognition confidence threshold may be dynamically adjusted to a relatively low level, for example, from the default 0.8 to 0.75. If the recognition confidence of a certain packaging box is 0.70, which is lower than the dynamically adjusted 0.75 threshold, the system will judge that the recognition result is uncertain. At this time, considering that the value level of the goods is low and the operation priority is allowed (for example, a certain delay can be accepted for automated processing), the system will preferentially trigger image reprocessing, such as reacquiring the image of the packaging box or applying a stronger image enhancement algorithm, in order to improve the recognition confidence without human intervention, thereby maximizing the efficiency of automated processing while ensuring a certain accuracy.

[0159] Reference Figure 2 The specific embodiments of the present application also disclose a rack stacker goods recognition system applied to a rack stacker goods recognition method, and the system comprises:

[0160] An image acquisition module is configured to acquire a goods image.

[0161] A quality evaluation module is configured to evaluate the quality of the goods image and determine the degradation type of the goods image.

[0162] An image enhancement module is configured to perform adaptive image enhancement processing on the goods image based on the degradation type to obtain an enhanced image.

[0163] A feature descriptor generation module is configured to generate a feature descriptor that is resistant to the degradation of the goods image.

[0164] a recognition module that inputs the enhanced image and the feature descriptor into a deep learning model for cargo recognition to obtain a recognition result and a confidence level;

[0165] a control module that controls the recognition process according to the confidence level.

[0166] In an automated warehouse environment, the lens surface of the on-board camera of the roadway stacker is easily contaminated by an oil dust film during high-speed operation, and is solidified due to changes in environmental temperature and humidity, resulting in a serious decline in image quality. Such degraded images make it difficult for the deep learning model to accurately extract features during cargo recognition, and the recognition confidence level continues to decline, frequently triggering the retry mechanism, ultimately leading to reduced work efficiency and increased frequency of human intervention, seriously affecting the continuity and reliability of the automated warehouse.

[0167] To this end, the roadway stacker cargo recognition system proposed in the present application aims to implement each step of the above cargo recognition method through modular design, thereby effectively addressing the image degradation problem caused by camera lens contamination in complex warehouse environments. The system acquires original images through an image acquisition module, then judges the degradation type of the images through a quality assessment module, and performs adaptive processing to improve image quality through an image enhancement module. At the same time, the feature descriptor generation module extracts features that are resistant to degradation. Finally, the recognition module uses enhanced images and robust features for deep learning recognition, and the control module intelligently regulates the recognition process according to the recognition confidence level, thereby significantly improving the accuracy and efficiency of cargo recognition and the stability of system operation. Each module works together to form a complete, intelligent cargo recognition closed loop, effectively solving the recognition problems caused by image degradation in the prior art.

[0168] Specifically, the content of collecting cargo images has been described in the above embodiments and will not be repeated here. It should be emphasized that the image acquisition module can be understood as a hardware and software component responsible for acquiring visual information of the cargo to be identified. Specifically, this module can include one or more industrial cameras, image sensors, optical lenses, and corresponding image acquisition cards or embedded processing units. For example, the industrial camera can be configured to continuously capture cargo images at a preset frame rate and resolution during the travel of the stacker. The image sensor can be a CMOS or CCD sensor, and its photosensitive performance and dynamic range can be selected according to the lighting conditions of the warehouse environment. The optical lens can use a fixed-focus or zoom lens to adapt to different distances and sizes of cargo. The image acquisition card or embedded processing unit is responsible for converting the analog signals output by the sensor into digital image data and performing preliminary image preprocessing such as white balance and gamma correction, and then transmitting the digital image data to the subsequent processing modules.

[0169] In the above embodiments, the content of quality evaluation of the cargo image and determination of the degradation type of the cargo image has been described, which will not be repeated here. It should be emphasized that the quality evaluation module can be understood as a processing unit for analyzing the collected cargo image to determine its quality condition and specific degradation type. This module can be implemented by a dedicated image processing chip, a digital signal processor or a software algorithm on a general-purpose processor (CPU / GPU). For example, this module can be configured to calculate the brightness histogram, contrast, edge sharpness indicators (such as Tenengrad gradient, Laplacian operator response) and noise level (such as by local variance or Fourier transform analysis) of the image. By comparing these indicators with preset thresholds or models, this module can identify whether the image has degradation problems such as low contrast, blur, excessive noise, abnormal brightness, and further determine the specific degradation type, such as motion blur, defocus blur or scattering degradation caused by an oil dust film.

[0170] In practical applications, the content of adaptive image enhancement processing of the cargo image based on the degradation type to obtain an enhanced image has been described in the above embodiments, which will not be repeated here. It should be emphasized that the image enhancement module can be understood as a processing unit for targeted optimization of the cargo image according to the degradation type determined by the quality evaluation module. This module can also be implemented by a hardware accelerator or a software algorithm. For example, when the quality evaluation module indicates that the image has low contrast, the image enhancement module can be configured to perform histogram equalization, adaptive gamma correction or local contrast enhancement (such as CLAHE algorithm). When the image has blur, the module can be configured to perform deconvolution algorithm or wavelet transform-based deblurring processing. When the image has high noise, the module can be configured to perform non-local mean filtering, bilateral filtering or deep learning-based denoising network. The "adaptive" feature of this module is reflected in its ability to dynamically adjust the parameters of the enhancement algorithm according to the real-time degradation type and degree, to achieve the best enhancement effect and avoid over-enhancement or insufficient enhancement.

[0171] Further, the content of generating feature descriptors that are robust to the degradation of the cargo image has been described in the above embodiments and will not be repeated here. It is emphasized that the feature descriptor generation module can be understood as a processing unit that extracts key visual features that are robust to degradation from the enhanced image. This module can be implemented by dedicated feature extraction hardware or software algorithms running on general-purpose processors. For example, this module can be configured to employ improved SIFT, SURF or ORB algorithms that take into account local gradients, orientation information and multi-scale characteristics of the image when detecting feature points and generating descriptors, thereby resisting the influence of light changes and slight blurring to some extent. In addition, this module can also be configured to extract texture features (such as LBP, Gabor filter responses), color features (such as color histograms, color moments) or shape features (such as HOG descriptors) of the image. These features can be fused to form a multi-dimensional feature vector that is more robust to image degradation, providing reliable input for the subsequent recognition module.

[0172] As a preferred embodiment, the content of feeding the enhanced image and the feature descriptor as input into a deep learning model for cargo recognition to obtain the recognition result and the confidence has been described in the above embodiments and will not be repeated here. It is emphasized that the recognition module can be understood as a processing unit that receives the enhanced image and the feature descriptor and utilizes a pre-trained deep learning model for cargo classification and positioning. This module is usually composed of a high-performance computing unit (such as GPU, NPU or FPGA) and a deep learning inference framework running on it. For example, this module can be configured to load a convolutional neural network-based image classification model (such as ResNet, EfficientNet) or an object detection model (such as YOLO, FasterR-CNN). The enhanced image and the feature descriptor are input into the model, and the model outputs the class label, the bounding box coordinates and the corresponding recognition confidence of the cargo through the complex feature representation it has learned. The performance of this module directly determines the accuracy and real-time performance of cargo recognition.

[0173] Therefore, the content of the identification process control according to the confidence level has been described in the above embodiments, and will not be repeated here. It should be emphasized that the control module can be understood as a management unit that intelligently decides and schedules the entire cargo identification process according to the confidence level output by the identification module. The module can be implemented by a microcontroller, a programmable logic controller or software logic on an embedded system. For example, the module can be configured to compare the identification confidence level with a preset threshold value. When the confidence level is higher than the threshold value, the control module can issue an instruction to allow the stacker to continue to perform the handling operation; when the confidence level is lower than the threshold value, the control module can trigger the image acquisition module to reacquire images, instruct the image enhancement module to perform deeper enhancement processing, or issue an alarm to the stacker main control system, requesting manual intervention for review according to a preset strategy. Through intelligent process control, the module effectively avoids the error operation caused by the low confidence level identification result, and improves the reliability and automation level of the system.

[0174] The above disclosed content is only the preferred feasible embodiments of the present application, and does not limit the protection scope of the present application, so any equivalent technical changes made according to the content of the present application and the drawings are included in the protection scope of the present application, and furthermore, the elements can be updated as the technology develops.

Claims

1. A method for identifying goods on a stacker crane in an aisle, characterized in that, The method includes the following steps: Capture images of the cargo; Perform quality assessment on cargo images to determine the type of quality degradation. Based on the degradation type, adaptive image enhancement processing is performed on the cargo image to obtain an enhanced image; Generate feature descriptors that are resistant to the degrading effects on cargo images; The enhanced image and feature descriptors are used as inputs to feed into a deep learning model for cargo recognition, and the recognition results and confidence scores are obtained. The identification process is controlled based on confidence level.

2. The method for identifying goods in a stacker crane in a lane as described in claim 1, characterized in that, The steps for quality assessment of cargo images and determining the type of degradation include: Obtain the motion status information of the stacker crane; Based on motion state information, predict short-term optical distortion of image regions caused by mechanical vibration; Local motion blur detection and geometric distortion analysis are performed on the cargo image to obtain the processing area, which includes the local motion blur area and the geometric distortion area. Differentiate between processed areas and persistent image degradation; In the image region after removing the processed area, the brightness distribution, contrast, sharpness and noise level of the cargo image are evaluated to obtain the brightness distribution evaluation results, contrast evaluation results, sharpness evaluation results and noise level evaluation results. Based on the evaluation results of brightness distribution, contrast, sharpness, and noise level, the type of degradation of the cargo image is determined.

3. The method for identifying goods in a stacker crane in a lane as described in claim 1, characterized in that, The steps to obtain an enhanced image include: Acquire the rate of humidity change and local temperature; Based on the rate of humidity change and local temperature, predict the trend of thin film refractive index and surface morphology changes; Based on the degradation type, thin film refractive index, and surface morphology change trends, the image enhancement algorithm parameters are pre-adjusted, and the cargo image is enhanced to obtain the initial image. Analyze the local pixel value distribution, edge gradient, and color saturation of the initial image; Based on the local pixel value distribution, edge gradient, and color saturation, the image enhancement algorithm parameters are adaptively adjusted to perform image enhancement processing on the initial image, resulting in an enhanced image.

4. The method for identifying goods in a stacker crane in a lane as described in claim 3, characterized in that, The steps for predicting the trends in refractive index and surface morphology of thin films include: To obtain information on the temperature and humidity gradient changes in the area where the stacker crane is located, in order to determine the rate of humidity change and local temperature; Obtain the stress state of the oil film on the camera lens surface; Based on the temperature and humidity gradient change information and the stress state of the oil and dust film on the camera lens surface, the physical model parameters of the film optical properties are corrected in real time. The physical model of thin film optical properties is updated based on the modified parameters of the physical model of thin film optical properties to predict the trend of thin film refractive index and surface morphology changes.

5. The method for identifying goods in a stacker crane in a lane as described in claim 4, characterized in that, The steps for obtaining information on the temperature and humidity gradient changes in the area where the stacker crane is located include: A forward-facing temperature and humidity sensing array is set in front of the camera lens along the direction of travel of the stacker crane; A rearward temperature and humidity sensing array is set behind the camera lens along the direction of travel of the stacker crane; Real-time acquisition of temperature and humidity data from the forward and backward temperature and humidity sensing arrays. By comparing the temperature and humidity data from the forward and backward temperature and humidity sensing arrays, the temperature and humidity gradient change information of the area where the stacker crane is located is calculated.

6. The method for identifying goods in a stacker crane in a lane as described in claim 4, characterized in that, The steps for obtaining the stress state of the oil film on the camera lens surface include: A micro piezoelectric thin-film sensor array is integrated along multiple directions on the surface of a camera lens; Real-time acquisition of piezoelectric signals from the sensor array in different directions; High-frequency filtering is applied to the piezoelectric signal; Instantaneous stress amplitude and frequency analysis were performed on the filtered piezoelectric signal to obtain the stress state of the oil film on the camera lens surface.

7. The method for identifying goods in a stacker crane in a lane as described in claim 6, characterized in that, The steps for high-frequency filtering of piezoelectric signals include: Spectral analysis of the piezoelectric signal is performed to obtain its spectral characteristics. Based on the spectral characteristics, identify the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise in the piezoelectric signal; Based on the dominant frequency range of the high-frequency stress component and the frequency range of the low-frequency mechanical vibration noise, the cutoff frequency and attenuation slope of the high-pass filter are dynamically adjusted. The piezoelectric signal is subjected to high-frequency filtering based on the dynamically adjusted cutoff frequency and attenuation slope.

8. The method for identifying goods in a stacker crane in a lane as described in claim 1, characterized in that, The steps for generating feature descriptors that are resistant to the degrading effects on cargo images include: Extract the brightness variation rate and color difference rate of local areas in a cargo image across multiple color channels; Extract periodic and non-periodic texture patterns from local regions of cargo images; The feature descriptor is generated by fusing brightness change rate, color difference rate, periodic texture pattern and non-periodic texture pattern.

9. The method for identifying goods in a stacker crane in a lane as described in claim 1, characterized in that, The steps for process control based on confidence level include: Obtain batch information, value level, and visual similarity level of the goods to be processed; The identification confidence threshold is dynamically adjusted based on the batch information, value level, and visual similarity level of the goods to be processed. Compare the confidence level with the dynamically adjusted recognition confidence threshold; When the confidence level is lower than the dynamically adjusted recognition confidence threshold, image reprocessing or manual intervention alarm is triggered based on the value level of the goods and the operation priority.

10. A cargo identification system for a stacker crane in an aisle, applied to the cargo identification method for a stacker crane as described in claim 1, characterized in that, The system includes: The image acquisition module is used to acquire images of the goods. The quality assessment module is used to assess the quality of cargo images and determine the type of quality degradation in the cargo images. The image enhancement module performs adaptive image enhancement processing on the cargo image based on the degradation type to obtain an enhanced image; The feature descriptor generation module is used to generate feature descriptors that are resistant to the degrading effects on cargo images; The recognition module takes the enhanced image and feature descriptors as input and feeds them into the deep learning model to identify the goods, and obtains the recognition result and confidence level. The control module controls the identification process based on the confidence level.

Images