An intelligent inventory identification method and system based on semantic segmentation and symbol regression, and a horizontal refrigerator

Abstract

The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application discloses an inventory intelligent identification method and system based on semantic segmentation and symbol regression and a horizontal refrigerator, and relates to the technical field of inventory management. The application disclosesa high-precision, high-adaptability and low-cost intelligent identification method suitable for inventory management scenes in a narrow space such as a horizontal cabinet.

Description

Technical Field

[0001] This invention relates to the field of computer vision inventory recognition technology, and in particular to an intelligent inventory recognition method, system, and horizontal refrigerated display case based on semantic segmentation and symbolic regression. Background Technology

[0002] Traditional warehouse inventory inspection mainly relies on manual counting or fixed calibrated sensors (such as RFID), which suffers from low efficiency, high error rates, and poor adaptability. For the unique characteristics of narrow spaces such as horizontal refrigerated display cases, existing visual inspection technologies face two major technical bottlenecks: first, the large aspect ratio structure causes lens tangential distortion, leading to amplified errors in physical height measurement at edges; second, perspective projection distortion caused by oblique viewing angles. Multi-view 3D reconstruction solutions are difficult to implement due to high hardware costs and computational complexity. Therefore, there is an urgent need for an intelligent identification solution for horizontal refrigerated display case inventory that combines high accuracy, low cost, and adaptability. Summary of the Invention

[0003] To overcome the aforementioned problems in the existing technology, this invention proposes an intelligent inventory identification method, system, and horizontal freezer based on semantic segmentation and symbolic regression.

[0004] The technical solution adopted by this invention to solve its technical problem is: an intelligent inventory identification method based on semantic segmentation and symbolic regression, comprising the following steps: Step 1, construct the inventory percentage annotation dataset: collect side view images of narrow space, use image annotation tools to draw polyline annotations along the same height direction, and assign inventory percentage values ​​to each polyline annotation. Read the polyline annotations and sample the two-dimensional coordinates of the points on the polyline at fixed intervals, and assign inventory percentage to each coordinate. Step 2, construct a custom semantic segmentation dataset: collect narrow spatial side view images, use polygon annotations to delineate the product area, and uniformly annotate the background area. In the end, it only contains two categories: product area and background area. Generate the corresponding binary mask image from the annotated data. Step 3: Train a semantic segmentation model using the custom semantic segmentation dataset obtained in Step 2, and train a symbolic regression model using the inventory percentage labeled dataset obtained in Step 1. Step 4: After acquiring a narrow space side view image and preprocessing it, input it into the semantic segmentation model trained in Step 3. The semantic segmentation model predicts the input image and outputs a binary mask matrix, where 1 represents the product area and 0 represents the background area. Step 5: Automatically generate adaptive sampling lines at the top and bottom of the results obtained in Step 4, and obtain sampling points based on the sampling lines; Step 6: Input the sampling point coordinates obtained in Step 5 into the symbolic regression model, convert the image pixel positions into inventory percentages, and average all sampling line results to obtain the overall inventory quantity.

[0005] The above-mentioned intelligent inventory identification method based on semantic segmentation and symbolic regression, in step 3, the semantic segmentation model adopts the DeepLabV3+ model, uses ResNet-50 as the backbone network, and evaluates the model performance by the average intersection-union ratio; the symbolic regression model adopts the PySR framework, which includes basic arithmetic operators and extended function sets.

[0006] The aforementioned intelligent inventory identification method based on semantic segmentation and symbolic regression, specifically includes step 5 as follows: Step 5.1, Sampling Line Generation: For the narrow space sidewall region, adaptive sampling lines are automatically generated based on the top and bottom of the sidewall. Sampling points are taken at the top and bottom edges according to the same proportion, and then corresponding sampling points in the same direction are paired to obtain a set of lines; Step 5.2, Sampling point acquisition: If the set of lines obtained in step 5.1 are parallel and of equal length in reality, the point where all lines intersect with the product area and the point closest to the top is the highest position of the product area in the display, and finally a set of pixel coordinates of the edge of the product area on the side wall are obtained.

[0007] An intelligent inventory identification system based on semantic segmentation and symbolic regression includes a monocular camera, an image processing module, a semantic segmentation module, a symbolic regression module, and an inventory calculation module. The monocular camera is used to acquire side-view images of a narrow space. The image processing module is used to preprocess the images acquired by the monocular camera. The semantic segmentation module is used to perform pixel-level segmentation on the preprocessed images to identify product areas. The symbolic regression module is used to establish a nonlinear mapping relationship between image coordinates and inventory height. The inventory calculation module is used to calculate the final inventory quantity.

[0008] The aforementioned intelligent inventory recognition system based on semantic segmentation and symbolic regression uses a semantic segmentation module based on the DeepLabV3+ model and a symbolic regression module based on the PySR framework.

[0009] The aforementioned intelligent inventory recognition system based on semantic segmentation and symbolic regression includes image preprocessing in the image processing module, which includes ineffective region removal and distortion correction.

[0010] A horizontal freezer uses the aforementioned intelligent inventory identification method based on semantic segmentation and symbolic regression for inventory identification, or is equipped with the aforementioned intelligent inventory identification system based on semantic segmentation and symbolic regression.

[0011] The beneficial effects of this invention are: 1. High-precision inventory identification: By employing the DeepLabV3+ model, this invention achieves a 98.29% mIoU (mean Intersection over Union) in the semantic segmentation stage, significantly improving the ability to identify SKU boundaries. In the symbolic regression stage, this invention uses a nonlinear mapping function established through the PySR framework to accurately characterize the complex nonlinear relationship between inventory height and image coordinates. This high-precision mapping effectively compensates for lens distortion and perspective deformation, controlling edge region errors to within 3%, far superior to traditional vision methods.

[0012] 2. High Adaptability: For the special structure of narrow spaces such as horizontal freezers, this invention employs a dynamic sampling line generation strategy to adaptively compensate for geometric distortion at the sidewall edges caused by lens tangential distortion. This dynamic strategy can adjust the sampling method according to the actual scene, ensuring accurate measurement of inventory height under different viewing angles and distortion conditions.

[0013] 3. High cost-effectiveness Monocular camera deployment: Compared to traditional multi-sensor or high-cost RFID solutions, this invention requires only a monocular camera to achieve fully automated inventory counting, significantly reducing hardware costs and deployment complexity. The widespread availability and low cost of monocular cameras make this solution a promising candidate for widespread applications.

[0014] Low data requirements and rapid training: Symbolic regression methods have low requirements for data distribution assumptions and can maintain good generalization performance even with limited training samples or measurement noise. This allows the system to be trained and put into use in a short time, further reducing time and labor costs.

[0015] 4. Highly interpretable: Explicit analytical expression: The mapping function generated by symbolic regression has an explicit analytical form, which is easier to verify in engineering and analyze theoretically compared to "black box" models such as deep learning. This interpretability makes the system easier to understand and accept in practical applications, facilitating subsequent optimization and improvement.

[0016] Theoretical analysis supports this: the nonlinear mapping function established through symbolic regression can clearly reveal the physical relationship between image coordinates and inventory height, providing theoretical support and decision-making basis for warehouse management. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process of this invention; Figure 2 This is a schematic diagram illustrating the mIoU variation during the training of the semantic segmentation model of this invention; Figure 3 This is a schematic diagram of the actual data points of basket A and the fitting function of the symbolic regression model in an embodiment of the present invention; Figure 4 This is a schematic diagram of the actual data points of basket B and the fitting function of the symbolic regression model in an embodiment of the present invention; Figure 5 This is a schematic diagram of the actual data points of basket C and the fitting function of the symbolic regression model in an embodiment of the present invention; Figure 6 This is a schematic diagram of the actual data points of basket D and the fitting function of the symbolic regression model in an embodiment of the present invention; Figure 7 This is a schematic diagram of the actual data points of basket E and the fitting function of the symbolic regression model in an embodiment of the present invention; Figure 8 This is a schematic diagram of the actual data points of the F-basket and the fitting function of the symbolic regression model in an embodiment of the present invention; Figure 9 This is a schematic diagram of the dynamic sampling line generated in an embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown, this embodiment discloses an intelligent inventory recognition method based on semantic segmentation and symbolic regression, which aims to solve the problems of low efficiency and poor adaptability of traditional warehouse inventory detection. In particular, it provides a high-precision and cost-effective solution for the special characteristics of the wardrobe scenario, such as narrow space, lens distortion, and dense stacking of SKUs.

[0020] The inventory percentage is defined as follows: the total height of the side walls of the horizontal freezer is... The height of the product at the side wall obstruction point is Then the inventory percentage at that point is Given the inventory ratios at all points, the overall inventory ratio is: .

[0021] The recognition method in this embodiment consists of two core stages. The first stage is semantic segmentation based on the DeepLabV3+ model. This model, through its advanced ASPP module and encoder-decoder structure, significantly improves the ability to recognize small-scale SKU boundaries. In the encoder part, the model uses dilated convolutions to expand the receptive field and capture multi-scale contextual information; in the decoder part, it recovers spatial detail information through progressive upsampling and feature fusion, achieving accurate segmentation of object boundaries. Ultimately, the DeepLabV3+ model achieved 98.29% mIoU on the custom semantic segmentation dataset of this invention, significantly improving SKU segmentation accuracy.

[0022] The second stage is nonlinear coordinate mapping based on symbolic regression. This invention innovatively applies symbolic regression to image-physical coordinate transformation modeling, its core being the establishment of an implicit functional relationship between the two-dimensional coordinates (x, y) of the sidewall and the actual inventory height z. The optimal expression is searched in a function space containing {+, -, ×, ÷, sin, cos, exp, log} using the PySR framework. The unique advantages of this method are automatic compensation for lens distortion and perspective distortion, with edge region errors controlled within 3%; and the function expression has an explicit analytical form, facilitating engineering verification and theoretical analysis. It effectively characterizes the physical laws governing the change of inventory proportion with spatial location.

[0023] The identification method in this embodiment specifically includes: Step 1, Data Acquisition: To perform inventory calculations, two datasets need to be built: Inventory percentage annotation dataset: Using the Labelme image annotation tool, polyline annotations were drawn along the same height of the cabinet, and an inventory percentage value (0-100%) was assigned to each polyline annotation. The polyline annotations were read and the two-dimensional coordinates (x, y) of the points on the polyline were sampled at fixed intervals, and a z value (corresponding to the inventory percentage) was assigned to each coordinate point.

[0024] The specific method used in this embodiment is as follows: A checkerboard ruler is used to attach fixed-length black and white checkerboard squares to the side wall of each basket. Along the edges of the black and white squares, an image annotation tool is used to mark the height percentage contour lines of each basket side wall. Specifically, based on the number of squares on the checkerboard ruler, the height percentage of each square in the image is calculated, and then contour lines are drawn along the edges of the squares in the image. In this way, height percentage data is assigned to the image area of ​​each basket side wall, thus achieving a mapping from two-dimensional image coordinates to inventory percentage.

[0025] To enhance the richness and reliability of the data, multiple sets of height percentage contour lines were marked at different locations along the sidewalls within the maximum visible area of ​​each basket. These contour lines, distributed at different heights along the sidewalls, cover various areas from bottom to top, thus providing rich training data for the symbolic regression model.

[0026] This inventory percentage annotation dataset has the following characteristics and advantages: High-precision annotation: By using a checkerboard annotation ruler and height percentage contour lines, we achieved high-precision annotation of the storage height on the side walls of the storage cabinet baskets. The annotation data achieved pixel-level accuracy, providing high-quality data support for subsequent symbolic regression model training.

[0027] Data richness: The dataset contains height percentage data for six baskets and multiple height positions, covering inventory status at different locations and heights within the wardrobe. This rich data distribution allows the symbolic regression model to better learn the complex nonlinear relationship between inventory height and image coordinates, improving the model's generalization ability and prediction accuracy.

[0028] Practical Applicability: The data is collected from actual commercial warehouse storage scenarios, making it highly relevant and valuable for real-world applications. By collecting and labeling data in real-world scenarios, we ensure that the dataset accurately reflects the actual distribution of warehouse storage inventory, providing a reliable solution for warehouse inventory management.

[0029] Through the meticulously designed data collection, preprocessing, and annotation methods and tools described above, we constructed a high-quality inventory percentage-annotated dataset. This dataset not only provides rich data support for training the symbolic regression model but also lays a solid foundation for subsequent inventory calculation and analysis.

[0030] A custom semantic segmentation dataset is used: polygon annotations are employed to accurately delineate the product SKU regions, precisely outlining the contour of each SKU. This method ensures accurate SKU region segmentation, especially in cases of densely stacked SKUs and occlusion, where polygon annotations better adapt to complex shapes and boundaries. Simultaneously, the background region is uniformly annotated, ultimately including only two categories: SKU and background. This simplified category classification helps improve model training efficiency and segmentation accuracy while reducing computational complexity in practical applications. The labeled data is converted to PASCAL VOC format, and corresponding binary mask images are generated to meet the training requirements of the semantic segmentation model.

[0031] This custom semantic segmentation dataset has the following features and advantages: High-precision annotation: Employing a polygon annotation method, the outline of the SKU can be accurately delineated, ensuring accuracy even when the SKU shape is complex or occluded. This high-precision annotation method provides a high-quality data foundation for model training, helping to improve the model's accuracy in recognizing SKU regions.

[0032] Data consistency: By uniformly labeling the background regions, the classification of the dataset is simplified, improving data consistency and processability. This consistency not only helps with convergence during model training but also reduces the false positive rate in practical applications.

[0033] Compatible with mainstream frameworks: The labeled data is converted to PASCAL VOC format, making the dataset seamlessly compatible with various mainstream deep learning frameworks (such as TensorFlow, PyTorch, etc.). This compatibility greatly facilitates model training and deployment, improving the dataset's versatility and practicality.

[0034] Step 2, Model Training and Validation 2.1 Semantic Segmentation Model Training Model selection: The DeepLabV3+ model is adopted, which combines the ASPP module and encoder-decoder structure, and can effectively extract multi-scale contextual information and restore spatial details.

[0035] Training parameters: ResNet-50 was used as the backbone network, with a learning rate of 0.001 and a batch size of 8, and 5000 iterations were performed. Data augmentation was also applied using random flipping and brightness adjustment.

[0036] Performance evaluation: Model performance was evaluated using mIoU (mean intersection-union ratio). The final model achieved an mIoU of 98.29% (e.g., Figure 2 As shown in the figure, where the horizontal axis represents the number of iterations and the vertical axis represents the mIoU value, the accuracy of SKU segmentation has been significantly improved.

[0037] 2.2 Training of the Symbolic Regression Model Data preparation: Use the inventory percentage annotation dataset, which includes image coordinates (x, y) and the corresponding inventory percentage z.

[0038] Algorithm configuration: The PySR framework is used, and the configuration includes the four basic arithmetic operators (+, -, *, / ) and the extended function set (cos, sin, exp, sqrt, log, inv(x)=1 / x, pow2(x)=x², pow3(x)=x³). The maximum formula complexity is set to 40, and it is optimized by a genetic algorithm (400 generations of iteration).

[0039] The above configuration is used to fit mapping functions to the sidewalls of the six basketball hoops. The fitted functions may contain outliers. 3D visualization is used to detect these outliers, and if any are found, the function is refitted to obtain a normal function. The final result is the following six mapping functions and their corresponding 3D visualizations (denoted as A, B, C, D, E, and F).

[0040] The fitting function for the basketball hoop is shown in formula (1), and the visualized 3D image is as follows. Figure 3 As shown; the B-basket fitting function is shown in formula (2), and the visualized 3D image is shown. Figure 4 As shown; the C-basket fitting function is shown in formula (3), and the visualized 3D image is shown. Figure 5As shown; the D-basket fitting function is shown in formula (4), and the visualized 3D image is shown. Figure 6 As shown; the E-basket fitting function is shown in formula (5), and the visualized 3D image is shown. Figure 7 As shown; the F-basket fitting function is shown in formula (6), and the visualized 3D image is shown. Figure 8 As shown.

[0041] z= (1) (2) z= (3) Z= (4) Z= (5) Z= (6) Experimental results show that the nonlinear mapping function exhibits a nonlinear pattern in the depth direction of the storage compartment and a complex nonlinear change in the horizontal direction, effectively characterizing the physical law of the change of inventory ratio with spatial location, and providing a reliable mathematical expression for subsequent analysis.

[0042] Step 3, Inventory Calculation Process 3.1 SKU Region Segmentation Input image: The system input is a picture of a wardrobe.

[0043] Model prediction: The trained DeepLabV3+ model is used to predict the input image and output a binary mask matrix, where 1 represents SKU pixels and 0 represents the background.

[0044] 3.2 Dynamic sampling line generation and sampling Sampling line generation: For the side wall area of ​​the wardrobe, adaptive sampling lines are automatically generated based on the top and bottom of the side wall. Sampling points are taken at the top and bottom edges in the same proportion, and then corresponding sampling points in the same direction are paired to obtain a set of lines.

[0045] Sampling point acquisition: It is assumed that this set of connecting lines are parallel and of equal length in reality (e.g., Figure 9 As shown in the figure, the point closest to the top among all the points where the lines intersect with the SKU area is the highest position of the SKU in the display, and finally a set of pixel coordinates of the edge of the SKU area on the side wall are obtained.

[0046] 3.3 Inventory Ratio Mapping Application of the mapping function: Input the coordinates of the sampling points into the sign regression function to convert the pixel positions of the image into the actual inventory percentage.

[0047] Inventory Calculation: The overall inventory is obtained by averaging the results of all sampling lines.

[0048] The recognition method disclosed in this embodiment is mainly designed around a monocular vision system. It adaptively compensates for geometric distortion at the sidewall edges caused by lens tangential distortion through a dynamic sampling line generation strategy. The core components of the system include: a monocular camera for acquiring side-view images of the cabinet; an image processing module responsible for preprocessing the acquired images, including removing ineffective regions and correcting distortion; a semantic segmentation module based on the DeepLabV3+ model to perform pixel-level segmentation of the image and identify SKU regions; a symbolic regression module using the PySR framework to establish a non-linear mapping relationship between image coordinates and inventory height; and an inventory calculation module combining the results of semantic segmentation and symbolic regression to calculate the final inventory level.

[0049] The intelligent recognition system is specifically deployed in actual commercial horizontal freezer scenarios. A monocular camera is installed on the side of the commercial horizontal freezer to ensure that the camera can clearly capture the side view image of the freezer.

[0050] Software integration: Integrate the trained semantic segmentation model and symbolic regression model into the inventory management system to achieve automated inventory detection and calculation.

[0051] Real-time monitoring: The system can process newly acquired images in real time, quickly calculate inventory levels, and feed the results back to warehouse management personnel, achieving efficient inventory management.

[0052] Through the above specific implementation methods, the present invention can effectively solve the technical problems in the detection of inventory in wardrobes, and provide a high-precision, highly adaptable, and low-cost intelligent identification method that is suitable for inventory management scenarios in narrow spaces such as wardrobes.

[0053] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its scope and protection, and such modifications or equivalent substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. An intelligent inventory identification method based on semantic segmentation and symbolic regression, characterized in that, Includes the following steps: Step 1, construct the inventory percentage annotation dataset: collect side view images of narrow spaces, use image annotation tools to draw polyline annotations along the same height direction, and assign inventory percentages to each polyline annotation. Read the polyline annotations and sample the two-dimensional coordinates of the points on the polyline at fixed intervals, and assign inventory percentages to each coordinate. Step 2, construct a custom semantic segmentation dataset: collect narrow spatial side view images, use polygon annotations to delineate the product area, and uniformly annotate the background area. In the end, it only contains two categories: product area and background area. Generate the corresponding binary mask image from the annotated data. Step 3: Train a semantic segmentation model using the custom semantic segmentation dataset obtained in Step 2, and train a symbolic regression model using the inventory percentage labeled dataset obtained in Step 1. Step 4: After acquiring a narrow space side view image and preprocessing it, input it into the semantic segmentation model trained in Step 3. The semantic segmentation model predicts the input image and outputs a binary mask matrix, where 1 represents the product area and 0 represents the background area. Step 5: Automatically generate adaptive sampling lines for the top and bottom regions obtained in Step 4, and obtain sampling points based on the sampling lines; Step 6: Input the sampling point coordinates obtained in Step 5 into the symbolic regression model, convert the image pixel positions into inventory percentages, and average all sampling line results to obtain the overall inventory percentage. Step 5 specifically includes: Step 5.1, Sampling line generation: For the narrow space sidewall area, an adaptive sampling line is automatically generated based on the top and bottom of the sidewall. Sampling points are taken at the top and bottom edges in the same proportion. Then, the sampling points are connected one by one in the same direction to obtain a set of connecting lines. Step 5.2, Sampling point acquisition: If the set of lines obtained in step 5.1 are parallel and of equal length in reality, the point closest to the top among all the points where the lines intersect with the product area is the highest position of the product area in the display, and finally a set of pixel coordinates of the edge of the product area on the side wall are obtained.

2. The intelligent inventory identification method based on semantic segmentation and symbolic regression according to claim 1, characterized in that, In step 3, the semantic segmentation model adopts the DeepLabV3+ model, using ResNet-50 as the backbone network, and the model performance is evaluated by the average intersection-over-union ratio; the symbolic regression model adopts the PySR framework, which includes basic arithmetic operators and extended function sets.

3. An intelligent inventory identification system based on semantic segmentation and symbolic regression, characterized in that, The method for intelligent inventory identification based on semantic segmentation and symbolic regression as described in claim 1 includes a monocular camera, an image processing module, a semantic segmentation module, a symbolic regression module, and an inventory calculation module. The monocular camera is used to acquire side view images of a narrow space. The image processing module is used to preprocess the images acquired by the monocular camera. The semantic segmentation module is used to perform pixel-level segmentation on the preprocessed images to identify the product areas. The symbolic regression module is used to establish a nonlinear mapping relationship between image coordinates and inventory percentage. The inventory calculation module is used to calculate the final inventory percentage.

4. The intelligent inventory identification system based on semantic segmentation and symbolic regression according to claim 3, characterized in that, The semantic segmentation module is based on the DeepLabV3+ model, and the symbolic regression module is based on the PySR framework.

5. The intelligent inventory identification system based on semantic segmentation and symbolic regression according to claim 3, characterized in that, The image processing module includes image preprocessing, such as removal of ineffective regions and distortion correction.

6. A horizontal freezer, characterized in that, Inventory identification is performed using the intelligent inventory identification method based on semantic segmentation and symbolic regression as described in any one of claims 1-2, or an intelligent inventory identification system based on semantic segmentation and symbolic regression as described in any one of claims 3-5 is installed.

Images