Target image detection method and system for articles placed on unmanned warehouse shelf

Abstract

A target image detection method for articles placed on a shelf of an unmanned warehouse is characterized by comprising the following steps: dynamically setting a confidence threshold value of a to-be-detected target and an overlapped candidate frame suppression threshold value based on non-maximum suppression; dynamically inhibiting the overlapped candidate frame by using the overlapped candidate frame inhibition threshold value, and dynamically extracting a to-be-detected target by using the confidence coefficient; and detecting and marking a to-be-detected target by adopting a preset model so as to extract and track the placed object from the unmanned warehouse shelf.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a target image detection method and system for unmanned warehouses. Background Art

[0002] For the target image feature extraction method of items placed on shelves in unmanned warehouses, the relatively close existing technologies include object detection and recognition methods based on computer vision and deep learning. These technologies are widely used in fields such as automated warehousing systems, robot vision navigation, and smart retail. The following are several existing technologies and their application status, as well as their respective defects:

[0003] Traditional computer vision methods rely on hand-designed feature extraction algorithms, such as edge detection, corner detection, color histogram, SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features), etc. These methods are used to extract geometric and texture features of objects from images, and then perform object recognition and positioning.

[0004] Edge detection is a technique for identifying areas of image with significant intensity changes, which usually correspond to the boundaries of objects. Common edge detection algorithms include Sobel, Prewitt, Roberts, and Canny edge detection. Sobel and Prewitt use convolution kernels to calculate the gradient of the image and detect edges in the horizontal and vertical directions. Roberts uses diagonal convolution kernels, which is suitable for detecting diagonal edges. Canny edge detection includes steps such as noise filtering, gradient calculation, non-maximum suppression, and dual threshold detection, providing better detection results.

[0005] In the early days of unmanned warehouse systems, traditional computer vision methods were widely used for object detection and recognition. These methods can provide relatively stable performance in specific scenarios and lighting conditions, especially in environments with clear boundaries and simple backgrounds. However, edge detection is sensitive to noise and requires preprocessing to reduce the impact of noise on the detection results. In the case of complex backgrounds and weak boundaries, edge detection is less effective and it is difficult to accurately identify object boundaries. In addition, traditional computer vision methods are not robust enough and are very sensitive to lighting changes, occlusions, and background clutter, which can easily lead to recognition failures. Manually designed feature extraction methods also have limitations and cannot fully capture the high-level semantic information of complex objects. Moreover, some traditional algorithms (such as SIFT and SURF) have high computational complexity, resulting in poor real-time performance and difficulty in meeting the needs of rapid response. Therefore, although traditional methods are still valuable in specific applications, their limitations have prompted people to continue to seek more advanced solutions, such as image recognition technology based on deep learning.

[0006] Corner detection is a method to identify points with significant angle changes in an image. These points are usually highly characteristic and easy to match in different images. Corner detection algorithms use the local grayscale value changes of an image to locate these feature points. Commonly used corner detection algorithms include Harris corner detection and Shi-Tomasi corner detection. Harris corner detection: Based on the second-order matrix of the image gradient, the corners in the image are determined by calculating the corner response value. The calculation of the corner response value is based on the local autocorrelation function. When the image grayscale changes greatly, the response value is higher. Shi-Tomasi corner detection: Improved Harris corner detection, using the eigenvalue decomposition method to calculate the response value of the corner point, selecting the point with the smallest eigenvalue as the corner point, reducing the influence of noise and improving detection accuracy.

[0007] Corner detection is widely used in scenarios that require precise positioning and feature matching, such as aligning images taken at different times or from different perspectives. It is often used in stereo vision in medical imaging, remote sensing images, and computer vision. It tracks the motion trajectory of objects in videos and determines the position and direction of movement of objects by detecting and matching corner points in each frame. It restores the structural information of three-dimensional scenes by matching corner points in multi-view images. In image stitching, image retrieval, and object recognition, the correspondence between images is achieved by detecting and matching corner points in images.

[0008] Although corner detection performs well in specific applications, it also has some defects. Corner detection algorithms are usually not robust to scale changes and need to be detected at multiple scales to cope with corner changes at different scales. They are greatly affected by noise and are prone to false detection and missed detection in high-noise environments. Preprocessing is required to reduce the impact of noise. Some complex corner detection algorithms have high computational complexity. When processing high-resolution images or real-time applications, the computing speed may become a bottleneck. In some cases, corner detection algorithms may produce uneven distribution of feature points, resulting in too dense feature points in some areas and sparse feature points in other areas, affecting feature matching and image registration.

[0009] In recent years, the development of deep learning, especially convolutional neural networks (CNN), has significantly improved the performance of image feature extraction. Common methods include Faster R-CNN, YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector). These methods use deep neural networks to automatically learn features in images and detect and identify objects.

[0010] Faster R-CNN is part of the R-CNN series, which further improves the performance of R-CNN and Fast R-CNN. Its core idea is to introduce a Region Proposal Network (RPN) to generate candidate regions, and use CNN to classify and regress bounding boxes for these candidate regions. It uses a convolutional neural network to extract the feature map of the input image, slides a small network on the feature map to generate candidate regions, outputs a series of rectangular bounding boxes and their corresponding scores, projects the candidate regions generated by RPN onto the feature map, and uses the RoI pooling layer to unify the region size, and uses the fully connected layer to classify (item category) and regress bounding boxes (position adjustment) for each candidate region.

[0011] Although Faster R-CNN has many advantages in accuracy and performance, it also has some defects and limitations, mainly including the following aspects: The region proposal network (RPN) in Faster R-CNN needs to slide a small network on the input image to generate candidate regions, which is a very time-consuming process. Especially when processing large-scale high-resolution images, the computational complexity will increase significantly, resulting in a slower inference speed of the model. Although some optimization measures such as non-maximum suppression (NMS) can be used to reduce the number of candidate boxes, the overall computational cost is still high. Training needs to be performed on a large number of data sets, while also dealing with complex network structures and loss functions. This makes the model training time longer, especially when dealing with large-scale data sets, it takes a considerable amount of time to converge and tune. Faster R-CNN consists of multiple parts, including a convolutional feature extraction network, an RPN network, and subsequent classification and regression networks. The complexity of these parts increases the difficulty of model design and implementation, as well as the complexity of parameter adjustment and optimization. For general developers and researchers, it is necessary to have a high level of expertise in deep learning and computer vision to effectively apply and adjust the model. Since Faster R-CNN uses a fixed-size candidate region, it has poor detection results for small objects. Small objects usually have less pixel information, and candidate region generation and precise bounding box regression are prone to inaccuracies in this case. Although this problem can be partially solved by increasing the scale of the candidate box, it will increase computational complexity and the risk of false detection. The design of Faster R-CNN assumes that the pose of the object is fixed. For objects with rotation or unconventional poses, the detection effect of the model may drop significantly. In this case, additional pose estimation or other feature extraction techniques are needed to improve the robustness of detection.

[0012] YOLO is an end-to-end target detection algorithm that treats object detection as a regression problem. It predicts the object bounding box and category probability within each grid cell by dividing the input image into SxS grid cells. YOLO divides the input image into a fixed number of grid cells (such as 7x7 or 19x19), and each grid cell predicts a fixed number of bounding boxes (usually 2 or 5). Each bounding box contains the location of the object (the center coordinates, width and height of the bounding box). Each bounding box also predicts the confidence score for each category, indicating the probability that the bounding box contains an object of that category. YOLO uses a single loss function to optimize the coordinate prediction error and category prediction error of the bounding box, so that the model can be trained end-to-end.

[0013] YOLO has been widely used in video surveillance, autonomous driving, and smart security due to its high-speed detection capability and real-time performance. Since it can complete the entire detection task in a single forward propagation process, it is particularly suitable for scenarios that require fast response and processing of large amounts of data.

[0014] Although YOLO has obvious advantages in speed and real-time performance, it also has some significant drawbacks. YOLO achieves end-to-end training by converting the object detection task into a regression problem, which may result in less accurate bounding box positioning than traditional two-stage methods (such as Faster R-CNN). In particular, for small objects or when the object boundaries are not clear, YOLO often cannot accurately locate the object. Since YOLO uses a fixed number and size of grid cells to predict objects when it is designed, the detection effect for small-sized objects is usually poor. Small objects may be covered by larger grid cells, resulting in low detection accuracy or complete miss detection. YOLO's loss function usually uses mean squared error (MSE) to measure the error between bounding box position and category probability. This loss function may not perform well when dealing with difficult samples (such as highly occluded, unclear objects). In these cases, the model may not correctly predict the existence or location of the object. Training requires a large number of annotated object detection datasets, and the size and type of objects in the dataset must be diverse. In some scenarios, obtaining and annotating large-scale datasets can be an expensive and time-consuming task. Predict multiple bounding boxes in each grid cell and make category predictions for each bounding box. In the case of class imbalance, i.e., the number of objects in some categories is small, the model may tend to predict the categories with a larger number, resulting in a decrease in the detection accuracy of the minority categories.

[0015] Single Shot Multi-Box Detector (SSD) is a popular real-time object detection algorithm designed to overcome some of the shortcomings of traditional two-stage detection methods and achieve a certain balance between speed and accuracy. SSD uses feature maps at different levels to detect objects of different sizes and scales. Shallower feature maps can detect larger objects, while deeper feature maps can detect smaller objects. A set of default boxes (anchorboxes) with different sizes and aspect ratios are defined at each position of each feature map to predict the position and size of the object. Convolution operations are used on each feature map to simultaneously predict the bounding box offset and object category probability of each default box. A comprehensive loss function is used to optimize the prediction of the bounding box position and the recognition of the object category, including position loss and category loss.

[0016] However, SSD uses predefined default boxes for object detection. The design of these boxes may not be fine enough, especially for very small or very large objects, the detection accuracy may be reduced. Although SSD uses multi-scale feature maps to process objects of different sizes, its detection effect is still not as good as some two-stage methods based on region proposal networks (RPNs) for very small objects. Small objects usually have less pixel information, and SSD may be inaccurate when predicting their location and category. The design of default boxes in SSD is usually based on experience and data set statistics. The selection of these default boxes may not be suitable for different data sets and application scenarios. In some special cases, it is necessary to redesign and adjust the parameters of the default boxes according to specific problems to improve the accuracy and robustness of detection. After generating a large number of bounding boxes, non-maximum suppression (NMS) is required to remove overlapping bounding boxes to obtain the final object detection results. The threshold selection of NMS has an important impact on the quality of the final detection results. Inappropriate threshold selection may lead to false detection or missed detection.

[0017] Hybrid methods combine the advantages of traditional computer vision and deep learning, such as extracting high-level features through deep learning, combined with traditional methods for fine positioning and identification of objects. This method attempts to find a balance between complexity and robustness. In deep learning, convolutional neural networks (CNNs) can automatically learn and extract high-level features in images, while traditional computer vision methods (such as edge detection, SIFT, and SURF) can perform fine-grained object positioning and identification. By combining the two, hybrid methods can maintain high detection accuracy in complex backgrounds while using the geometric and texture information provided by traditional methods for fine positioning.

[0018] In some specific applications, hybrid methods have shown good performance. For example, in complex environments, the precise positioning of objects can be achieved by first detecting the approximate location of the object using deep learning methods, and then fine-tuning and confirming it using traditional methods. This combination enables the system to maintain high accuracy while handling complex tasks and cope with different environmental changes. Hybrid methods have been verified in some practical applications, such as object detection in unmanned warehouses, object recognition in autonomous driving, and robot navigation.

[0019] Although hybrid methods have significant advantages, they also have some drawbacks. First, the system complexity increases, and it is necessary to master and optimize both traditional computer vision technology and deep learning technology at the same time, which increases the complexity of system design and maintenance. Designing an effective hybrid system requires not only a deep understanding of the two technologies, but also optimization in algorithms and hardware to ensure the overall performance of the system. Secondly, integration is difficult, and the combination of traditional methods and deep learning methods needs to overcome technical compatibility and optimization problems. For example, how to seamlessly switch and integrate different feature extraction methods at different computing stages, and how to find the best balance between performance and resource utilization are key issues that need to be addressed. In addition, the differences in processing speed and data format between different methods also increase the difficulty of integration, requiring developers to have rich experience and technical accumulation.

[0020] The current image feature extraction methods used for placing items on shelves in unmanned warehouses are mainly based on deep learning, and are gradually developing towards hybrid methods. Although deep learning methods have excellent performance in accuracy and robustness, their high data requirements and computing resource consumption remain major challenges. Future developments may focus on improving the training efficiency of the model, reducing dependence on labeled data, and enhancing the real-time and adaptability of the system. Summary of the invention

[0021] In order to solve the shortcomings of the prior art, the present invention provides a target image detection method for items placed on unmanned warehouse shelves, which realizes target detection through multi-level feature fusion, deep learning feature extraction, optimized non-maximum suppression (NMS) and adaptive parameter adjustment strategy.

[0022] The present invention adopts the following technical solution.

[0023] A first aspect of the present invention relates to a target image detection method for items placed on unmanned warehouse shelves, the method comprising the following steps: dynamically setting a confidence threshold of a target to be detected and an overlapping candidate box suppression threshold based on non-maximum suppression; dynamically suppressing overlapping candidate boxes using the overlapping candidate box suppression threshold, and dynamically extracting the target to be detected using the confidence threshold; and detecting and labeling the target to be detected using a preset model to extract and track the placed objects from the unmanned warehouse shelves.

[0024] Preferably, dynamically setting the confidence threshold of the target to be detected and the overlapping candidate box suppression threshold based on non-maximum suppression includes: the confidence threshold of the target to be detected is:

[0025] T(E)=T0·(1+γ1·f1(L)+γ2·f2(C))

[0026] Where E is the set of external factors,

[0027] T0 is the initial confidence threshold,

[0028] γ1 and γ2 are environmental adjustment coefficients,

[0029] f1(·) and f2(·) are the first and second environmental impact functions,

[0030] L and C are the light intensity and scene complexity respectively.

[0031] Preferably, dynamically setting the confidence threshold of the target to be detected and the overlapping candidate box suppression threshold based on non-maximum suppression includes: the overlapping candidate box suppression threshold based on non-maximum suppression is:

[0032] λ(E)=λ0·(1+δ1·f3(L)+δ2·f4(C))

[0033] Where λ0 is the initial overlapping candidate box suppression threshold,

[0034] δ1 and δ2 are the candidate frame adjustment coefficients,

[0035] f3(·) and f4(·) are the third and fourth environmental influence functions respectively.

[0036] Preferably, the first environmental impact function and the third environmental impact function are:

[0037] f 1，3 (L) = log(1 + L)

[0038] The second environmental impact function is:

[0039] f2(C)=C 2

[0040] And the fourth environmental impact function is:

[0041] Preferably, the overlapping candidate box suppression threshold is used to dynamically suppress the overlapping candidate box, and the confidence level is used to dynamically extract the target to be detected, including: training the preset model, adjusting the parameters of the preset model

[0042] γ1, γ2, δ1 and δ2.

[0043] Preferably, a preset model is used to detect and mark the target to be detected so as to extract and track the placed objects from the unmanned warehouse shelves, including: the preset model is a deep learning convolutional neural network model; the deep learning convolutional neural network model uses an adaptive detection parameter adjustment strategy to adjust the parameters of the preset model so that the preset model extracts the target to be detected in the image.

[0044] Preferably, a preset model is used to detect and mark the target to be detected, so as to extract and track the objects placed on the shelves of the unmanned warehouse, including: the preset model extracts the target to be detected by a non-maximum suppression algorithm; in the non-maximum suppression algorithm, a set of candidate bounding boxes {B1, B2, ..., B m}, the confidence of the current box S i In the overlapping area IoU (B i , B j ), updated to:

[0045]

[0046] In the formula, the original confidence of the current box is:

[0047]

[0048] σ is an adjustment parameter used to control the decay rate of IOU, and j is ranked first.

[0049] The second aspect of the present invention relates to a target image detection system for objects placed on unmanned warehouse shelves, which is implemented using a target image detection method for objects placed on unmanned warehouse shelves in the first aspect of the present invention; wherein the system includes a setting module, an extraction module, and a preset module; wherein the setting module is used to dynamically set a confidence threshold of the target to be detected and an overlapping candidate box suppression threshold based on non-maximum suppression; the extraction module is used to dynamically suppress overlapping candidate boxes using the overlapping candidate box suppression threshold, and dynamically extract the target to be detected using the confidence; the preset module is used to detect and label the target to be detected using a preset model, so as to extract and track the objects placed on the shelves of the unmanned warehouse.

[0050] A third aspect of the present invention relates to a terminal, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method in the first aspect of the present invention.

[0051] A fourth aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in the first aspect of the present invention.

[0052] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a target image detection method for items placed on shelves in unmanned warehouses, and realizes target detection through multi-level feature fusion, deep learning feature extraction, optimized non-maximum suppression (NMS) and adaptive parameter adjustment strategy. The method adopts an advanced detection system designed for complex and changeable environments. The system combines local details with global semantic information through multi-level feature fusion technology to ensure comprehensive perception of target objects, and uses deep learning models such as ResNet and EfficientNet to extract advanced semantic features, thereby improving recognition accuracy and processing efficiency. At the same time, DIFF-DS adopts the Soft(er)-NMS algorithm to dynamically adjust the confidence of overlapping detection frames, reduce false detections, and optimize detection results. In addition, the system has strong adaptive capabilities and can adjust detection parameters in real time according to changes in ambient lighting, personnel density and other conditions, ensuring that in complex environments, such as densely populated areas, chaotic inventory in warehouses and scenes with changing lighting, it can still maintain efficient and accurate detection performance. This system is widely used in security monitoring, warehouse management and other fields, providing a flexible and reliable solution.

[0053] This method combines multiple technologies such as multi-level feature fusion, feature learning based on deep learning, non-maximum suppression optimization and adaptive adjustment strategy. Through multi-level feature extraction and fusion strategy, local detail features and global semantic information are integrated to effectively improve the detection ability of complex backgrounds and small objects. Convolutional neural network (CNN) and its variant networks (such as ResNet and EfficientNet) in deep learning technology are used to automatically learn and extract high-level semantic features to enhance the recognition ability of objects of different scales and postures. In the post-processing stage of detection results, the improved non-maximum suppression (NMS) algorithm is used to effectively reduce the number of overlapping bounding boxes and improve the accuracy and robustness of detection results. At the same time, an adaptive detection parameter adjustment strategy is introduced to dynamically adjust the detection algorithm parameters according to specific application scenarios and object features to improve the adaptability and generalization ability of the algorithm. This method can efficiently and accurately detect and identify items placed on shelves in unmanned warehouse environments, significantly improving the intelligent management level of unmanned warehouses.

[0054] The present invention achieves substantial technical breakthroughs by integrating multi-level feature fusion, deep learning-based feature extraction, optimized non-maximum suppression (NMS), and adaptive parameter adjustment strategies. The synergy of these methods greatly improves the accuracy and efficiency of feature extraction, ensuring that the model can skillfully adapt to highly complex and dynamic environments. This overcomes the main limitations of the existing technology, such as low recognition accuracy and insufficient robustness. By comprehensively optimizing the object detection task, this method is significantly different from traditional methods and provides a novel solution that not only improves detection accuracy, but also ensures performance consistency under different conditions, making it a significant step forward in unmanned warehouses and other demanding application scenarios. This comprehensive framework is based on state-of-the-art deep learning techniques and enhanced by dynamic algorithm adjustment, achieving a tangible leap in accuracy, scalability, and flexibility of object detection systems. Compared with the existing technology, the proposed solution solves some key problems, such as the inability to balance detection speed and accuracy in different scenarios. The result is a more reliable and resilient detection model that can outperform existing solutions, especially in high-density, unpredictable environments.

[0055] Compared with the prior art, this invention proposes a groundbreaking solution to the complexity of unmanned warehouse environments. By combining an adaptive adjustment strategy with an optimized non-maximum suppression (NMS) algorithm, the system can dynamically adjust detection parameters in real time to cope with the challenges posed by various unpredictable warehouse conditions. This approach greatly improves the monitoring accuracy of goods and personnel, ensuring efficient and safe warehouse management even in highly complex situations such as densely populated areas, unpredictable personnel flows, disorganized inventory, and fluctuating lighting conditions at different times of the day.

[0056] This comprehensive solution not only intelligently adapts to different levels of congestion, inconsistent light sources and random arrangement of items, thereby improving the level of automation in warehousing, but also solves the complex difficulties of the real world where factors such as heavy traffic, changing light conditions and changing item density are intertwined. By effectively addressing these challenges, the present invention sets a new standard for the practical application of intelligent warehouse systems. It not only optimizes operational performance in the current environment, but also lays a solid foundation for future applications in the field of intelligent logistics, demonstrating excellent scalability, adaptability and resilience in complex and changing industrial environments.

[0057] Through multi-level feature fusion, the problem of insufficient feature extraction in traditional methods is effectively solved. The core lies in the fusion of local detail features and global semantic information, which makes it perform better in complex background and small object detection, thereby significantly improving the detection accuracy of items placed on shelves in unmanned warehouses. For example, in an actual unmanned warehouse, this technology can accurately identify and locate each item on the shelf, whether it is a large box or a small commodity, and can obtain accurate detection results, thereby significantly reducing missed detections and false detections. This not only helps to improve the management efficiency of goods, but also ensures the orderly and reliable operation of warehouses.

[0058] Specifically, this patented technology adopts a feature learning method based on deep learning, using convolutional neural networks (CNN) and their variant networks (such as ResNet, EfficientNet, etc.) to automatically extract high-level semantic features in images, significantly enhancing the recognition ability of objects of different scales and postures. This means that regardless of whether the items on the shelves are neatly arranged or tilted and messy, the system can automatically adjust the recognition strategy to ensure the accuracy and reliability of the detection results. For managers of unmanned warehouses, this greatly improves the efficiency of inventory management, reduces the need for manual intervention, and also improves the level of automation in warehouses. In addition, the technology can maintain a high level of detection performance under various lighting conditions and complex backgrounds, greatly improving the practicality and reliability of the system.

[0059] Through the improved non-maximum suppression (NMS) algorithm and adaptive adjustment strategy, the number of overlapping bounding boxes is effectively reduced, and the accuracy and robustness of the detection results are improved. In practical applications, the adaptive adjustment strategy dynamically adjusts the detection algorithm parameters according to the specific application scenarios and object characteristics, so that the system can flexibly respond to various complex situations. This not only improves the management efficiency of unmanned warehouses, but also reduces the loss or misplacement of items due to detection errors, thereby ensuring the accuracy and timeliness of inventory management. For users, this means higher work efficiency and fewer operational errors, making the operation of unmanned warehouses smoother and more reliable. In addition, the improved NMS algorithm retains more valid bounding boxes through soft suppression, which helps to accurately detect items in high-density shelf environments, thereby optimizing warehouse space utilization and improving overall operational efficiency.

[0060] In general, this patented technology has positive significance in the extraction of target image features of items placed on shelves in unmanned warehouses, and provides managers and users with an efficient, reliable and intelligent solution. Through multi-level feature fusion and deep learning feature learning, this patented technology significantly improves the detection accuracy of complex backgrounds and small objects. The multi-level feature fusion strategy comprehensively utilizes local detail features and global semantic information, enabling the system to work stably in various complex environments. The deep learning method based on convolutional neural networks (CNN) and its variant networks (such as ResNet, EfficientNet, etc.) can automatically extract high-level semantic features in images and enhance the recognition ability of objects of different scales and postures, thereby improving the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a multi-level feature fusion map;

[0062] Figure 2 for deep learning-based feature learning graphs;

[0063] Figure 3 It is the non-maximum suppression optimization graph;

[0064] Figure 4 Adjust the strategy diagram for adaptability. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0066] The present invention discloses a target image detection method for objects placed on shelves in unmanned warehouses. The method comprises the following steps: dynamically setting a confidence threshold of a target to be detected and an overlapping candidate frame suppression threshold based on non-maximum suppression; dynamically suppressing overlapping candidate frames using the overlapping candidate frame suppression threshold, and dynamically extracting the target to be detected using the confidence threshold; and detecting and labeling the target to be detected using a preset model to extract and track the objects placed on the shelves in the unmanned warehouse.

[0067] Figure 1 is a multi-level feature fusion graph. Figure 1As shown in the figure, multi-level feature fusion is a technology widely used in the field of image processing and computer vision. It aims to solve the problem of insufficient feature extraction in traditional methods through multi-level feature extraction and fusion strategies. This method effectively combines local detail features with global semantic information to ensure that information at each level in the image can be fully captured.

[0068] Specifically, local detail features can help identify small objects and complex textures in images, while global semantic information provides contextual understanding of the image as a whole. This combination makes the model more robust when dealing with complex backgrounds. For example, in video surveillance systems, multi-level feature fusion significantly improves the ability to detect abnormal behaviors and events. By combining local detail features and global semantic information, the system can more accurately identify and analyze the behavior of people in the surveillance footage. Multi-level feature fusion enhances target detection by combining feature maps at different levels. Assume that there are feature maps F1, F2, ..., F3 at different levels, and the dimensions of each feature map are C i ×H i ×W i , where C i is the number of channels, H i and W i Represent the height and width respectively. Feature fusion can be expressed as:

[0069]

[0070] Among them, α i It is the fusion weight coefficient, which is used to adjust the contribution of feature maps at different levels.

[0071] Upsample(F i ,s i ) represents the feature map F i By proportion i Upsampling is performed to ensure that the size of all feature maps is consistent. Through the feature pyramid network (FPN) or other fusion strategies, multi-level fusion can be performed from top to bottom, so that the final feature map F fusion Contains rich multi-scale information.

[0072] If we want to consider cross-layer fusion loss optimization, we can define the fusion loss function as:

[0073]

[0074] Among them, β i is the weight coefficient used to balance the fusion contribution of feature maps of different layers.

[0075] In busy public places, surveillance systems can use detailed features to identify abnormal activities (such as theft) and use global information to judge the security status of the overall environment, thereby improving the reliability and security of surveillance.

[0076] First, different levels of features are extracted from the image through a convolutional neural network (CNN). Low-level features focus on capturing local detail information (such as edges and textures), while high-level features focus on global semantic information (such as the category and location of objects). Using structures such as the feature pyramid network (FPN), features at different levels can be effectively fused. FPN gradually combines high-level semantic features into low-level detail features through a top-down path, ensuring that feature maps at each scale contain rich multi-level information. Finally, the fused features are input into the object detection and recognition model, allowing the model to more accurately detect and recognize objects in the image, improving the robustness and accuracy of detection.

[0077] In daily video surveillance applications, adaptive adjustment strategies enable the system to dynamically adjust detection parameters according to changes in the monitoring environment and time period. For example, at night or in low light conditions, the system can increase detection sensitivity to enhance the ability to identify suspicious activities.

[0078] By introducing multi-level feature fusion and adaptive adjustment strategies, the detection system can maintain high performance and stability in various complex and changeable real-world scenarios. This not only improves the detection accuracy and efficiency of various applications, but also reduces the need for manual intervention, promoting the widespread application and development of intelligent systems. In general, multi-level feature fusion and adaptive adjustment strategies are of great significance in research and daily life, providing flexible and reliable solutions for various intelligent detection systems.

[0079] Figure 2 is a feature learning graph based on deep learning. Figure 2 As shown in the figure, deep learning technology significantly improves the image recognition effect by automatically learning and extracting high-level semantic features in images. As an important model of deep learning, convolutional neural network (CNN) can extract different levels of image features layer by layer, from low-level edges and textures to high-level object shapes and semantic information, through the stacking of multiple convolutional layers, pooling layers and fully connected layers. This multi-level feature representation not only captures rich image details, but also enhances the model's recognition ability in complex scenes.

[0080] On this basis, variant networks such as ResNet and EfficientNet further optimized the effect of feature learning. ResNet effectively solved the common gradient vanishing problem in deep networks by introducing residual connections, making it possible to train deeper networks. This innovation enables the model to capture more complex and abstract image features, improving the expressiveness and accuracy of the network. EfficientNet uses a compound scaling method to significantly reduce the consumption of computing resources while maintaining model performance. Its design concept is based on the comprehensive adjustment of network width, depth and resolution, so that the model can maintain high efficiency in different application scenarios.

[0081] In deep learning models (such as convolutional neural networks, CNNs), feature extraction of image I is achieved through a series of convolution operations. Assume that the input image is I∈R H×W×C , through the convolution kernel The output feature map K can be expressed as:

[0082] F=σ(BN(K*I+b))

[0083] Among them, * represents the convolution operation, σ represents the activation function (such as ReLU), BN is the batch normalization operation, and b is the bias term. Through multi-layer convolution and pooling operations (Pooling), low-level edge features are gradually extracted to high-level semantic features. When using multi-layer convolution, the output feature map F of the lth layer (l) for:

[0084] F (l) =σ (l) (BN (l) (K (l) *F (l-1) +b (l) ))

[0085] By stacking layers, higher-level semantic features are extracted.

[0086] These advanced network structures enable deep learning models to perform well in object recognition tasks of different scales and postures, and are suitable for various complex application scenarios, including autonomous driving, medical image analysis, and security monitoring.

[0087] Image recognition technology has been greatly improved through feature learning based on deep learning. Advanced network structures such as ResNet and EfficientNet not only optimize the effect of feature learning, but also provide strong technical support for image processing, computer vision and related fields. Its application prospects are broad and will promote the development of smart homes, video analysis and other fields, helping to achieve smarter and more efficient technical solutions.

[0088] Figure 3 is the non-maximum suppression optimization graph. Figure 3 As shown in Figure 2, in the post-processing stage of target detection, the non-maximum suppression (NMS) algorithm is used to reduce overlapping bounding boxes, thereby improving the accuracy of the detection results (e.g. Figure 3 As shown in Figure 2.2, the traditional NMS algorithm suppresses bounding boxes with excessive overlap by setting a threshold, but this method often misses overlapping detection results belonging to different objects. To solve this problem, improved NMS algorithms such as Soft(er)-NMS have emerged, which retains valuable detection results while reducing false detections by weighted adjustment of the confidence of overlapping bounding boxes.

[0089] Soft(er)-NMS shows significant improvement in detection accuracy and robustness in practice. It dynamically adjusts the confidence level of bounding boxes according to their overlap level, so that some high-confidence overlapping detection results are retained instead of being suppressed by simple threshold limits. This innovative method is particularly suitable for multi-target detection in complex scenes, and improves the reliability and accuracy of the model in practical applications.

[0090] Specifically, Soft(er)-NMS evaluates the overlapping area between the current highest confidence bounding box and other bounding boxes in each iteration. If the overlapping area of ​​two bounding boxes exceeds the set threshold, the traditional NMS will directly suppress the box with lower confidence. Soft(er)-NMS proportionally reduces the confidence of the lower confidence box to retain more valuable detection results.

[0091] The NMS algorithm is used to remove redundant bounding boxes. Set a set of candidate bounding boxes {B1, B2, ..., B m}, each box has a confidence S i And the overlapping area IoU with other boxes (B i , B j ). The traditional NMS is defined as:

[0092]

[0093] The optimized Soft(er)-NMS adjusts the confidence instead of simply suppressing the bounding box:

[0094] S i =S i (1-IoU(B i , B j ))

[0095] To further optimize NMS, we introduce a Gaussian decay function to make the score update smoother:

[0096]

[0097] Among them, σ is a tuning parameter used to control the decay rate of IoU. This formula can prevent excessive suppression of bounding boxes with higher confidence in densely populated areas.

[0098] This method prevents bounding boxes of highly similar but different targets from being completely removed, retaining more valuable detection information.

[0099] In an autonomous driving system, vehicles need to detect and identify in real time under changing weather, lighting, and road complexity. Through adaptive adjustment strategies, the system can dynamically adjust the parameters of the detection algorithm to adapt to the current environment. For example, in low light or rainy days, the system can increase the detection threshold to reduce the impact of noise and ensure the accuracy of detection of vehicles, pedestrians, and obstacles.

[0100] In video surveillance systems, adaptive adjustment strategies can dynamically adjust detection parameters based on the location of the camera and the characteristics of the surveillance area to enhance the ability to identify abnormal behaviors and events. On busy city streets, the algorithm can set a higher threshold to reduce false alarms, while in quiet residential areas, the threshold is lowered to capture potential threats.

[0101] In smart home devices such as security systems and home robots, the system can dynamically adjust detection parameters based on changes in indoor and outdoor environments and user behavior patterns to optimize performance and user experience. Smart cameras can automatically adjust exposure and contrast based on light changes to improve video quality and detection accuracy; home robots can adjust navigation and object recognition parameters based on home layout and activities to improve the safety and efficiency of autonomous operations.

[0102] First, the input data needs to be analyzed to extract feature information, such as light intensity, noise level, and scene complexity. Based on these features, the key parameters of the detection algorithm are dynamically adjusted, such as increasing the threshold when the light is insufficient and adjusting the non-maximum suppression parameters to reduce the impact of noise. In high-density scenes, the scale parameters are adjusted to better identify overlapping objects. The system can continuously iterate and optimize parameters during the actual detection process, and real-time feedback is used to adjust the detection strategy to ensure that it adapts to environmental changes and complex scene requirements. Finally, the effectiveness of the adjusted detection parameters is verified through experiments to improve the accuracy and robustness of detection.

[0103] Figure 4 is an adaptive adjustment strategy diagram. Figure 4As shown in the figure, by introducing the adaptive adjustment strategy and the improved non-maximum suppression algorithm, the detection system can maintain high performance and stability in various complex and changeable practical scenarios. This not only improves the detection accuracy and efficiency of various applications, but also reduces the need for manual intervention, and promotes the widespread application and development of intelligent systems. Overall, the combination of adaptive adjustment strategy and Soft(er)-NMS is of great significance in both research and daily life, providing a flexible and reliable solution for intelligent detection systems and demonstrating the potential for technological innovation.

[0104] The adaptive detection parameter adjustment strategy is a method of dynamically optimizing algorithm parameters, aiming to make real-time adjustments for different application scenarios and object features, such as Figure 4 As shown. Traditional detection algorithms usually rely on fixed parameters to handle all detection tasks, which often cannot achieve optimal performance when facing a changing actual environment. The adaptive adjustment strategy analyzes the characteristics of the input data and adjusts key parameters such as threshold, scale, and non-maximum suppression parameters in real time, so that the model can more effectively meet the detection needs of different environments. The detection model can dynamically adjust parameters based on factors such as light intensity and environmental complexity to improve the accuracy and robustness of detection of goods, employees, and obstacles. For example, when the light intensity is low, the model may need to set a higher threshold to reduce the impact of noise; when the light intensity is good, the threshold can be lowered to capture more detailed information.

[0105] The core idea of ​​the adaptive parameter adjustment strategy is to dynamically modify the key parameters of the detection algorithm according to environmental conditions. In target detection, the detection threshold T determines the confidence threshold of the detection result. i When the value is lower than the threshold T, the target will be ignored. Usually, T is a fixed value, such as 0.5 or 0.7. However, different scenes may require different detection thresholds. For example, when the light is dim or the background is complex, the detected target often has low confidence, so the threshold should be appropriately lowered to maintain the sensitivity of the detection. For example, given the light intensity L and the scene complexity C, the detection threshold T and the NMS parameter λ can be adjusted according to the external factor E = (L, C):

[0106] T(E)=T0·(1+γ1·f1(L)+γ2·f2(C))

[0107] Where T0 is the initial threshold, usually 0.5; γ1 and γ2 are environmental adjustment coefficients, used to control the impact of illumination and scene complexity on the threshold; f1(L) is a function of the illumination intensity L, which can usually be set as a linear or nonlinear mapping. Assuming that the range of L is [0, 1], we can define f1(L) = log(1+L), which means that the stronger the light, the smaller the impact; f2(C) = C 2, indicating that the effect of scene complexity on threshold adjustment is nonlinear. The more complex the scene, the higher the detection sensitivity required, so the threshold is appropriately lowered. In this way, the model can dynamically adjust the confidence threshold of target detection according to different lighting and scene complexity, thereby improving the detection effect.

[0108] Non-maximum suppression (NMS) is used to remove candidate boxes with excessive overlap. It depends on an overlap threshold λ, which is usually set to 0.5 or 0.7. If the IoU (intersection over union) of two candidate boxes exceeds λ, the candidate box with lower confidence will be suppressed. However, the distribution and density of objects in the scene may affect the setting of λ. In scenes with sparse objects, larger overlap may mean false detection, while in scenes with dense objects, a higher IoU threshold can avoid over-suppression. The adaptive adjustment of the NMS parameter λ can be defined as:

[0109] λ(E)=λ0·(1+δ1·f3(L)+δ2·f4(C))

[0110] Where λ0 is the initially set NMS threshold, such as 0.5 or 0.7; δ1 and δ2 are adjustment coefficients used to control the impact of illumination and scene complexity on NMS parameters; f3(L) is a function of illumination intensity L, similar to the case of detection threshold. The stronger the illumination, the clearer the candidate box. It can be set to f3(L) = log(1 + L), indicating that the NMS threshold increases when the illumination is strong; f4(C) is a function of scene complexity C. It is set to f4(C) = C. The more complex the scene, the higher the threshold is required to avoid false suppression of dense targets.

[0111] The influence of the external environment E = (L, C) on adaptive detection is reflected in: Light intensity (Luminance, L): Light directly affects the clarity and confidence of the target. Lower light may lead to lower confidence in the detected target or even missed detection. Therefore, in scenes with low light, the detection threshold T and NMS parameter λ need to be lowered. Scene complexity (SceneComplexity, C): The number of objects in the scene, the degree of occlusion, etc. determine the difficulty of detection. In complex scenes, the targets may be more dense and the IoU between candidate boxes is higher, so the suppression threshold λ of NMS should be appropriately increased to avoid excessive suppression. The adaptive adjustment coefficients γ1, γ2, δ1, δ2 should also be learned through the model during the training process to find the optimal weight distribution.

[0112] In the NMS algorithm, after setting a set of candidate bounding boxes, the confidence of the current box is indeed affected by its overlapping area with other boxes (measured by the intersection over union (IoU)), and is updated accordingly (actually performing retention or suppression operations rather than directly updating the confidence value). This is the basic process of NMS, so we directly give the update method as follows:

[0113] Confidence sorting: For each category, NMS first sorts each bounding box according to its confidence (i.e., the probability that the predicted box contains an object).

[0114] Select the highest confidence box: Start with the bounding box with the highest confidence and treat it as the "largest" candidate currently under consideration.

[0115] Calculate the intersection over union (IoU): Calculate the intersection over union (IoU) of the current largest candidate and all other bounding boxes. IoU is the ratio of the intersection area to the union area of ​​two bounding boxes, which is used to measure the degree of overlap between bounding boxes.

[0116] Suppress overlapping boxes: If the IoU between a bounding box and the current maximum candidate is higher than a preset threshold (e.g. 0.5), they are considered to be the same object, so the bounding box is removed from the candidate list (i.e., the bounding box is suppressed). This step indirectly affects the final confidence selection by removing bounding boxes with high overlap.

[0117] Update the candidate list: After removing all suppressed bounding boxes, select the one with the highest confidence from the remaining bounding boxes as the new "maximum" candidate, and then repeat steps 3 to 5 until all bounding boxes have been processed.

[0118] The suppression process is as follows: a preset model is used to detect and label the target to be detected, so as to extract and track the objects placed on the shelves of the unmanned warehouse, including: the preset model extracts the target to be detected by a non-maximum suppression algorithm; in the non-maximum suppression algorithm, a set of candidate bounding boxes {B1, B2, ..., B m}, the confidence of the current box S i In the overlapping area IoU (B i , B j ), updated to:

[0119]

[0120] In the formula, the original confidence of the current box is:

[0121]

[0122] σ is an adjustment parameter used to control the decay rate of IOU, and j is ranked first.

[0123] In this process, the confidence of the current box is not updated directly, but the algorithm logic of NMS is used to decide which bounding boxes are retained (i.e., bounding boxes with high confidence and low overlap with other bounding boxes) and which are suppressed (i.e., bounding boxes with high overlap and considered redundant). Ultimately, the bounding boxes in the output list are those that are considered to be most likely to represent the true target after NMS processing. Their confidence is based on the original prediction and is not directly modified numerically, but has been optimized through the logical screening of NMS. Although NMS does not directly update the confidence value, it affects the final output bounding boxes and their confidence by removing redundant bounding boxes. Therefore, after NMS processing, the bounding boxes and their confidence in the output list can be regarded as optimized and screened results.

[0124] In video surveillance, adaptive adjustment strategies can optimize detection parameters to enhance the recognition of abnormal behaviors and events based on the location of the camera and the characteristics of the monitored area. On busy city streets, the detection algorithm can set a higher threshold to reduce the risk of false alarms; while in quiet residential areas, lower thresholds can help capture potential threats. In addition, when monitoring at night, the algorithm can adjust non-maximum suppression parameters to cope with low light conditions and improve the detection of suspicious activities.

[0125] In the industrial field, adaptive adjustment strategies also play an important role. Different production lines and products have different requirements for detection accuracy and speed. The detection system can adjust parameters in real time according to the production environment and product characteristics to ensure product quality and production efficiency. For example, on high-speed production lines, the system can reduce computational complexity to increase detection speed, while in precision manufacturing, it can improve detection accuracy to ensure product consistency.

[0126] The field of medical image processing also demonstrates the unique advantages of adaptive adjustment strategies. Different patients and medical image types have different requirements for detection parameters. For example, in lung CT scans, different scan intensities and noise levels require different thresholds and scale parameters to accurately identify lesion areas. Through adaptive adjustment strategies, medical image processing systems can dynamically adjust parameters according to the specific conditions of patients, thereby improving the accuracy and reliability of diagnosis.

[0127] In daily life, adaptive adjustment strategies are widely used. For example, security systems and home robots in smart home devices can dynamically adjust detection parameters according to changes in indoor and outdoor environments and user behavior patterns to optimize performance and user experience. Smart cameras can automatically adjust exposure and contrast parameters according to light changes to improve video quality and detection accuracy; home robots can adjust navigation and object recognition parameters according to home layout and activities to enhance the safety and efficiency of autonomous operations.

[0128] By introducing an adaptive detection parameter adjustment strategy, the detection algorithm maintains high performance and stability in complex and changing real-world scenarios. This strategy not only improves the detection accuracy and efficiency of various applications, reduces the need for manual intervention, but also promotes the widespread application and development of intelligent systems. Overall, the adaptive adjustment strategy is of great significance in research and daily life, providing flexible and reliable solutions for various intelligent detection systems, and demonstrating great potential for technological innovation.

[0129] The deep learning convolutional neural network (CNN) is responsible for extracting features at different levels of the image. The low-level extracts details such as edges and textures, while the high-level focuses on semantic information such as the location and category of the target object. In the feature pyramid network (FPN), by designing a suitable hierarchical connection structure, such as layer-by-layer feedback from high-level semantic features to low-level detail features, setting scale balance parameters for feature maps at different levels, and ensuring that the features output by the convolution layer can be effectively fused at different scales, and realizing top-down feature transfer and integration in the network architecture through the FPN structure. Multi-level feature fusion integrates these different levels of features. During the fusion process, it ensures that the size of the feature maps at each level matches, and uses bilinear interpolation and other methods to adjust the size of the feature maps to avoid information loss. This achieves an effective combination of local details and global semantic information, and enhances the detection model's perception of complex scenes.

[0130] The deep learning model generates multiple candidate boxes in the image, which contain the prediction results of the target object. The role of NMS is to remove redundant overlapping boxes from these candidate boxes and retain the boxes with the highest confidence. The more accurate the feature extraction of the deep learning model, the more effective the NMS result is, which can reduce false detection and missed detection. The suppression threshold of Soft(er)-NMS is adjusted according to the scene and application. A higher threshold can reduce the number of overlapping boxes, while a lower threshold will retain more overlapping boxes. Thus, the optimal threshold is determined. At the same time, it is ensured that after using the candidate boxes generated by CNN in the target detection stage, when executing the sorting and suppression steps of NMS, the confidence of the box can be dynamically adjusted instead of removing low-confidence boxes in a one-size-fits-all manner, and the candidate boxes generated during the deep learning feature extraction process cannot be connected with the NMS algorithm to transmit confidence information and location information.

[0131] The adaptive parameter adjustment strategy can dynamically adjust the relevant parameters of NMS according to the environment (such as light, density, etc.), so that the system can maintain a high detection accuracy in different environments. Adaptive parameter adjustment adjusts the threshold of NMS according to changes in the external environment. The external environment information is obtained in real time through sensors or environmental analysis modules, and the NMS parameters suitable for the current environment are dynamically calculated through algorithms. At the code level, an interface is designed for NMS to allow the adaptive adjustment module to dynamically update NMS thresholds, confidence and other parameters. Secondly, external information such as light sensors and environmental monitoring equipment is introduced to adjust the sensitivity of the detection algorithm. For example, in a low-light environment, the detection threshold of NMS is increased in real time through data transmitted by external sensors to reduce noise interference. Through adaptive adjustment, NMS can retain valuable candidate boxes in complex scenarios and improve the robustness and sensitivity of detection.

[0132] Multi-level feature fusion provides effective extraction and integration of information at different levels in the image, but the importance of features at different levels may be different in different environments. The adaptive parameter adjustment strategy can dynamically adjust the feature fusion strategy according to environmental changes, so that certain levels of features have higher weights in specific environments, thereby optimizing the detection results. To solve this problem, a dynamic weight mechanism is introduced to assign different weights to features at different levels under different environmental conditions. For example, in a complex background, the weight of low-level features may be increased to enhance detail capture; in the case of sufficient light and simple background, the role of high-level features is enhanced. A dynamic weighting mechanism is designed for multi-level feature fusion, allowing the adaptive adjustment module to adjust the fusion ratio of each layer of features in real time according to environmental changes, and to transmit information such as light intensity and noise level in real time through external sensors or environmental analysis modules to adjust the weight distribution of feature fusion.

[0133] While ensuring the accuracy of detection, the present invention adopts an improved non-maximum suppression (NMS) algorithm and an adaptive adjustment strategy, which effectively improves the real-time performance and processing efficiency of the algorithm. The improved NMS algorithm reduces unnecessary computational burden and optimizes the efficiency of the post-processing stage by weighted processing of overlapping bounding boxes. The adaptive adjustment strategy dynamically adjusts the detection parameters according to the specific application scenario and object characteristics, so that the system can quickly respond to detection requirements in different environments, thereby achieving efficient real-time detection.

[0134] Through effective feature extraction and algorithm optimization, this patented technology reduces the reliance on large-scale annotated data sets, reduces application costs and implementation difficulties. Multi-level feature fusion and deep learning methods can extract rich feature information from less data, improving the generalization ability and adaptability of the model. In this way, not only the cost of data collection and annotation is reduced, but also the system can maintain high detection performance in the case of data scarcity.

[0135] In an actual unmanned warehouse, the use of this patented technology can significantly improve the efficiency of detecting and identifying items on the shelves. For example, the system can accurately locate each item, whether it is a large box or a small commodity, and can obtain accurate detection results, reducing missed detections and false detections. At the same time, the system can maintain a high level of detection performance under various lighting conditions and complex backgrounds, greatly improving the management efficiency and automation level of the warehouse. For managers of unmanned warehouses, this means reduced manual intervention, optimized inventory management processes, and improved work efficiency. Users can also enjoy higher operational accuracy and fewer errors, making the operation of unmanned warehouses smoother and more reliable.

[0136] In general, this patented technology has significant positive significance in extracting target image features of items placed on shelves in unmanned warehouses. By improving detection accuracy and robustness, optimizing computing efficiency and real-time performance, and reducing data requirements and costs, this technology provides managers and users with an efficient, reliable, and intelligent solution. This not only improves the management level and operational efficiency of unmanned warehouses, but also promotes the development of warehouse automation and intelligence, making unmanned warehouses an important part of the future logistics industry.

[0137] This patented technology significantly improves the detection accuracy, efficiency and robustness in the detection of items on shelves in unmanned warehouses through multi-level feature fusion, deep learning feature extraction, improved non-maximum suppression (NMS) algorithm and adaptive adjustment strategy. These innovations not only solve the problem of insufficient feature extraction in traditional methods, but also optimize computing efficiency, reduce data requirements and application costs, and provide a comprehensive solution for intelligent warehousing systems. The following will introduce the specific implementation and application effects of these core technologies in detail. The technology adopts a multi-level feature extraction and fusion strategy, comprehensively utilizes local detail features and global semantic information, and effectively solves the problem of insufficient feature extraction in traditional methods. This fusion strategy enables the system to better handle complex backgrounds and small object detection, and improves the accuracy and reliability of detection. In the feature extraction process, low-level features mainly capture detailed information such as edges, corners and textures of the image, middle-level features focus on the shape and local structure of the object, and high-level features extract semantic information of the image, such as the category and overall layout of the object. By fusing these different levels of features, the system can understand the image content more comprehensively. This fusion strategy is particularly suitable for processing objects of different scales, allowing the system to take into account both details and overall information when facing objects of different sizes. For example, in unmanned warehouse applications, it can not only accurately detect large boxes, but also identify small items placed in corners, which improves the applicability of the system.

[0138] By utilizing convolutional neural networks (CNN) and their variants (such as ResNet, EfficientNet, etc.), the technology automatically learns and extracts high-level semantic features from images. This innovation enhances the system's ability to recognize objects of different scales and postures, making detection more accurate, especially in variable practical application scenarios. Advanced network structures such as ResNet and EfficientNet are used, which perform well in feature extraction, have deeper layers and higher feature expression capabilities. The learning and generalization capabilities of the network are further improved through residual connections and compound scaling strategies. During the training process, the diversity of data is increased through data enhancement techniques (such as rotation, scaling, translation, color transformation, etc.), so that the model can better adapt to various changes in actual scenes and improve the robustness and robustness of detection.

[0139] Improved NMS algorithms (such as Soft(er)-NMS) reduce the number of overlapping bounding boxes and improve the accuracy and robustness of detection results by weighting overlapping bounding boxes in the post-processing stage. Compared with the traditional NMS algorithm, this improved method can better retain valuable detection results, especially in high-density environments. By performing weighted processing in overlapping areas instead of simply suppressing overlapping bounding boxes, the scores of bounding boxes are gradually attenuated, thereby retaining more potentially valuable detection results. Such a strategy is particularly effective when dealing with densely distributed objects. In an unmanned warehouse, there may be a large number of items on the shelves. Traditional NMS may miss some objects, while Soft(er)-NMS can effectively distinguish these densely distributed objects, improve the comprehensiveness and accuracy of detection, and ensure that each item can be accurately identified and located. An adaptive detection parameter adjustment strategy is introduced to dynamically adjust the detection algorithm parameters according to specific application scenarios and object features. This strategy improves the adaptability and generalization ability of the algorithm, enabling the system to flexibly respond to different environments and conditions, and significantly improves the detection efficiency and accuracy in unmanned warehouse environments. The system dynamically adjusts detection parameters, such as detection threshold and IoU threshold, according to real-time environmental changes (such as changes in illumination and object density), to ensure that the best detection effect can be maintained under different environmental conditions. Advanced technologies such as reinforcement learning and Bayesian optimization are introduced to optimize parameter configuration and improve the system's adaptability through continuous learning and adjustment. For example, in a warehouse with poor lighting, the system can automatically adjust parameters to improve detection accuracy and robustness in low-light environments. In unmanned warehouses, there are many types of goods and various placement methods. The adaptive adjustment strategy enables the system to quickly adapt to different types of goods and placement methods, provide accurate detection results, reduce human intervention, and improve warehouse management efficiency and intelligence. Through multi-level feature fusion, feature learning based on deep learning, improved non-maximum suppression (NMS) algorithm and adaptive adjustment strategy, this patented technology significantly improves the detection accuracy, efficiency and robustness of items placed on shelves in unmanned warehouses, and provides strong technical support for intelligent warehousing systems. These core innovations not only improve detection performance, but also optimize computing efficiency, reduce data requirements and application costs, and provide a comprehensive solution for the intelligent management of unmanned warehouses.

[0140] A second aspect of the present invention relates to a target image detection system for objects placed on unmanned warehouse shelves, which is implemented using a target image detection method for objects placed on unmanned warehouse shelves; wherein the system includes a setting module, an extraction module, and a preset module; wherein the setting module is used to dynamically set a confidence threshold of the target to be detected and an overlapping candidate box suppression threshold based on non-maximum suppression; the extraction module is used to dynamically suppress overlapping candidate boxes using the overlapping candidate box suppression threshold, and dynamically extract the target to be detected using the confidence; the preset module is used to detect and label the target to be detected using a preset model, so as to extract and track the objects placed on the shelves in the unmanned warehouse.

[0141] A third aspect of the present invention relates to a terminal, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method described in the first aspect of the present invention.

[0142] A fourth aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in the first aspect of the present invention.

[0143] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0144] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0145] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0146] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A target image detection method for objects placed on shelves in unmanned warehouses, characterized in that: The method comprises the following steps: Dynamically set the confidence threshold of the target to be detected and the overlapping candidate box suppression threshold based on non-maximum suppression; Dynamically suppress overlapping candidate frames using the overlapping candidate frame suppression threshold, and dynamically extract the target to be detected using the confidence level; A preset model is used to detect and mark the target to be detected, so as to extract and track the placed objects from the unmanned warehouse shelves.

2. The target image detection method for objects placed on shelves in an unmanned warehouse according to claim 1 is characterized in that: The dynamically setting of the confidence threshold of the target to be detected and the overlapping candidate box suppression threshold based on non-maximum suppression includes: The confidence threshold of the target to be detected is: T(E)=T0·(1+γ1·f1(L)+γ2·f2(C)) Where E is the set of external factors, T0 is the initial confidence threshold, γ1 and γ2 are environmental adjustment coefficients, f1(·) and f2(·) are the first and second environmental impact functions, L and C are the light intensity and scene complexity respectively.

3. The target image detection method for objects placed on shelves in an unmanned warehouse according to claim 2 is characterized in that: The dynamically setting of the confidence threshold of the target to be detected and the overlapping candidate box suppression threshold based on non-maximum suppression includes: The overlapping candidate box suppression threshold based on non-maximum suppression is: λ(E)=λ0·(1+δ1·f3(L)+δ2·f4(C)) Where λ0 is the initial overlapping candidate box suppression threshold, δ1 and δ2 are the candidate frame adjustment coefficients, f3(·) and f4(·) are the third and fourth environmental influence functions respectively.

4. The target image detection method for objects placed on shelves in an unmanned warehouse according to claim 3 is characterized in that: The first environmental impact function and the third environmental impact function are: f1,3(L)=log(1+L) The second environmental impact function is: f2(C)=C 2 。 5. The target image detection method for objects placed on shelves in an unmanned warehouse according to claim 4 is characterized in that: The method of dynamically suppressing the overlapping candidate frames by using the overlapping candidate frame suppression threshold and dynamically extracting the target to be detected by using the confidence level includes: The preset model is trained, and the parameters γ1, γ2, δ1 and δ2 of the preset model are adjusted.

6. The target image detection method for objects placed on shelves in an unmanned warehouse according to claim 5, characterized in that: The method of using a preset model to detect and mark the target to be detected so as to extract and track the objects placed on the shelves of the unmanned warehouse includes: The preset model is a deep learning convolutional neural network model; The deep learning convolutional neural network model uses an adaptive detection parameter adjustment strategy to adjust the parameters of the preset model so that the preset model extracts the target to be detected in the image.

7. The target image detection method for objects placed on shelves in an unmanned warehouse according to claim 6, characterized in that: The method of using a preset model to detect and mark the target to be detected so as to extract and track the objects placed on the shelves of the unmanned warehouse includes: The preset model extracts the target to be detected by a non-maximum suppression algorithm; In the non-maximum suppression algorithm, a set of candidate bounding boxes {B1, B2, ..., B m }, the confidence of the current box S i In the overlapping area IoU (B i ,B j ), updated to: In the formula, the original confidence of the current box is: σ is an adjustment parameter used to control the decay rate of IOU, and j is ranked first.

8. A target image detection system for items placed on shelves in unmanned warehouses, characterized by: The method is implemented by using a target image detection method for objects placed on shelves in an unmanned warehouse as described in any one of claims 1 to 7; wherein: The system includes a setting module, an extraction module, and a preset module; wherein, The setting module is used to dynamically set the confidence threshold of the target to be detected and the overlapping candidate box suppression threshold based on non-maximum suppression; The extraction module is used to dynamically suppress the overlapping candidate boxes using the overlapping candidate box suppression threshold, and dynamically extract the target to be detected using the confidence level; The preset module is used to detect and mark the target to be detected using a preset model, so as to extract and track the placed objects from the unmanned warehouse shelves.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images