Intelligent three-dimensional storage management system for glass bottles and carrying terminal

Abstract

This invention relates to the field of intelligent warehousing technology and discloses an intelligent automated storage and retrieval management system and handling terminal for glass bottles. The system includes a storage space monitoring module, a storage change analysis module, a handling decision control module, and an automated handling module. The storage space monitoring module periodically acquires images of storage shelves, forming a time-stamped image sequence. The storage change analysis module extracts multi-dimensional spatial distribution features of glass bottle stacks through a dedicated network, generating a dynamic feature stream, and outputs a quantitative deviation index after time-series analysis. The handling decision control module compares this index with a preset threshold; only when the index continuously exceeds the threshold for a set period does it generate a control command containing the target location and operation category. The automated handling module executes the command to complete the operation. This invention achieves deep perception of micro-changes in storage status and highly reliable decision triggering, improving management accuracy and system stability.

Description

Technical Field

[0001] This invention relates to the field of intelligent warehousing technology, specifically to an intelligent three-dimensional warehousing management system and handling terminal for glass bottles. Background Technology

[0002] In the field of automated storage and retrieval system (AS / RS) management of glass bottles, existing technologies mainly rely on fixed scanning or timed image acquisition for inventory status monitoring. These methods are typically based on simple image difference comparisons or pre-set rule judgments, resulting in a relatively limited perceptual dimension. The systems lack the ability to deeply analyze and quantitatively track the gradual changes in spatial attributes such as stacking posture and arrangement density of glass bottles caused by environmental or human factors during the storage process. Conventional technologies struggle to effectively extract key features characterizing stacking stability from visual information and are slow to respond to unstructured, microscopic state evolutions.

[0003] The shortcomings of existing solutions lie in the accuracy of state assessment and the reliability of decision triggering. Triggering mechanisms based on thresholds or single changes are highly susceptible to misjudgments due to transient environmental disturbances, leading to unnecessary equipment actions. The system cannot distinguish between incidental disturbances and genuine, continuous state deviations, resulting in a lack of robust decision-making basis for initiating automated material handling operations. This not only potentially impacts warehousing efficiency but also increases the costs of ineffective equipment operation and maintenance. A technology that deeply integrates time-series perception and intelligent decision-making is needed to accurately identify real changes in warehousing state and generate highly reliable operating instructions accordingly.

[0004] The purpose of this invention is to solve the above-mentioned problems and to find a management method that can deconstruct and quantify the spatial distribution characteristics of glass bottle stacking from time-series images and intelligently trigger handling operations based on continuous deviation analysis. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent three-dimensional warehouse management system and handling terminal for glass bottles, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides an intelligent automated storage and retrieval system for glass bottles, the system comprising: The warehouse space monitoring module periodically collects image information of the warehouse shelves, forming a sequence of glass bottle storage space images with time stamps; The storage change analysis module receives the sequence of glass bottle storage space images, extracts the multi-dimensional spatial distribution features of glass bottle stacks through the storage image feature deconstruction network, forms a dynamic feature stream of glass bottle storage, performs time-series correlation parsing operation on the dynamic feature stream of glass bottle storage, and calculates the deviation quantification index of the spatial distribution features of glass bottles between adjacent acquisition times. The handling decision control module receives the deviation measurement index, compares the deviation measurement index with the preset operation activation threshold, and generates a glass bottle handling control instruction containing the target location identifier and operation category when the deviation measurement index continuously exceeds the operation activation threshold for a preset duration. The automated handling module receives the glass bottle handling control command, performs positioning and navigation according to the target storage location identifier, and completes the picking, placing or transferring of glass bottles according to the operation category.

[0007] Preferably, the step of extracting the multi-dimensional spatial distribution features of glass bottle stacks through a warehouse image feature deconstruction network includes the following processing steps: Multi-level feature extraction processing is performed on single-frame images in the glass bottle storage space image sequence to obtain an initial glass bottle space contour feature map reflecting the glass bottle contour edge information and an initial glass bottle stacking density feature map reflecting the glass bottle stacking density information. Spatial geometric structure analysis is performed on the initial glass bottle spatial contour feature map to calculate the contour continuity index and contour symmetry index, and synthesize the glass bottle geometric structure description vector. Regional density gradient analysis is performed on the initial glass bottle stacking density feature map to identify the boundaries between high-density stacking regions and low-density stacking regions, and the density change rate between regions is calculated to form a glass bottle stacking density distribution vector. The geometric structure description vector of the glass bottle and the stacking density distribution vector of the glass bottle are fused by feature dimension to generate the dynamic feature flow of glass bottle storage that includes spatial structure and stacking state information.

[0008] Preferably, the storage change analysis module performs time-series correlation analysis on the dynamic feature flow of glass bottle storage, including the following steps: From the dynamic feature stream of glass bottle storage, extract historical dynamic feature segments of glass bottle storage belonging to the previous analysis period and real-time dynamic feature segments of glass bottle storage belonging to the current analysis period in chronological order. On multiple preset feature analysis dimensions, the changes in the feature values ​​of the historical glass bottle storage dynamic feature segments and the real-time glass bottle storage dynamic feature segments on the feature analysis dimensions are calculated respectively, so as to obtain the single-dimensional feature changes of glass bottle storage in multiple dimensions. A dynamic weighting coefficient is configured for each of the single-dimensional feature changes in the glass bottle storage. The dynamic weighting coefficient is adaptively adjusted according to the fluctuation stability of the corresponding feature dimension in the historical period. The dimension with smaller fluctuation is assigned a higher dynamic weighting coefficient. The changes in the single-dimensional features of glass bottle storage across all dimensions are multiplied by their respective dynamic weight coefficients, and then weighted and summed to finally output the deviation quantification index that characterizes the intensity of changes in the overall storage status.

[0009] Preferably, the process of adaptively adjusting the dynamic weight coefficients based on the fluctuation stability of the corresponding feature dimensions in historical periods specifically involves: The sequence of feature values ​​at multiple consecutive time points in each feature dimension of the historical glass bottle storage dynamic feature segment is statistically analyzed. Calculate the standard deviation of the feature value sequence, which is used as the historical fluctuation range value of the feature dimension; The preset baseline weight value is compared with the historical fluctuation amplitude value. If the historical fluctuation amplitude value is less than the preset stability threshold, the baseline weight value is increased by a fixed gain value to obtain the dynamic weight coefficient after the feature dimension is adjusted. If the historical fluctuation amplitude value is greater than or equal to the preset stability threshold, the benchmark weight value is reduced by a fixed attenuation value to obtain the dynamic weight coefficient after the feature dimension is adjusted.

[0010] Preferably, the workflow of the handling decision control module further includes: While generating the glass bottle handling control command, the handling decision control module accesses the pre-stored 3D model database of the storage rack to obtain the rack structure parameters and adjacent rack occupancy status corresponding to the target location identifier. The handling decision control module selects a target handling strategy from a plurality of preset candidate handling strategies based on the rack structure parameters and the occupancy status of adjacent storage locations. The candidate handling strategies include at least a direct pick-and-place strategy, a nearby temporary storage and then pick-and-place strategy, and a path replanning pick-and-place strategy. The identification code of the target handling strategy is added to the glass bottle handling control command to form an enhanced glass bottle handling control command, which is then sent to the automated handling module.

[0011] Preferably, the rule for selecting a target handling strategy from a plurality of preset candidate handling strategies is as follows: If the adjacent storage location occupancy status shows that there is sufficient direct operating space for the target storage location, then the direct pick-and-place strategy is selected. If the occupancy status of the adjacent storage location shows that the direct operating space of the target storage location is insufficient, but there is an available adjacent storage location that meets the preset safety distance requirements, then the adjacent temporary storage and retrieval strategy is selected. If the adjacent storage location occupancy status shows that the direct operating space of the target storage location is insufficient and there is no available adjacent storage location that meets the preset safety distance requirements, then a path replanning pick-and-place strategy is selected. The pick-and-place strategy instructs the automated handling module to approach the target storage location along a non-straight path.

[0012] Preferably, the automated handling module performs a positioning and navigation process based on the target cargo location identifier, including: The automated handling module has a built-in terminal positioning module that obtains the current position coordinates of the terminal in the warehouse space in real time. The automated handling module parses the target location identifier from the received glass bottle handling control command, and queries the corresponding target location spatial coordinates from the warehouse electronic map based on the target location identifier; The automated handling module calls the path planning engine, takes the current location coordinates of the terminal as the starting point, the target cargo space coordinates as the ending point, and takes into account the preset fixed obstacle area and the real-time updated dynamic obstacle information to calculate a collision-free terminal navigation path. The drive control system of the automated handling module controls the movement of its mobile chassis according to the terminal navigation path until it reaches the position corresponding to the spatial coordinates of the target cargo location.

[0013] Preferably, the automated handling module completes the picking, placing, or transferring of glass bottles according to the operation category, specifically including: When the operation category is "removal", the automated handling module controls the gripper at the end of its robotic arm to move to the preset glass bottle gripping preparation position, drives the gripper to perform a clamping action, clamps the target glass bottle, lifts it and transports it to the designated outlet conveyor line position. When the operation category is "storage", the automated handling module obtains the glass bottle to be stored from its carrying platform, controls the robotic arm to move the glass bottle to the precise storage position of the target location, releases the gripper, and places the glass bottle stably. When the operation category is "transfer", the automated handling module first performs a "retrieve" operation, moving the glass bottles from the original warehouse to its carrying platform. According to the new target warehouse location identifier in the instruction, it plans the path to the new warehouse location and moves it. Finally, it performs a "store" operation at the new warehouse location.

[0014] Preferably, the warehouse space monitoring module includes multiple high-resolution industrial cameras fixed at different viewing positions in the warehouse area, and an image sequence scheduler; Each of the high-resolution industrial cameras periodically captures images of the warehouse shelves within its assigned monitoring sub-area according to a unified clock signal, and sends the captured raw shelf images to the image sequence scheduler. The image sequence scheduler receives raw shelf images from all high-resolution industrial cameras. Based on the camera identifier and timestamp attached to each raw shelf image, it aligns and stitches all images in chronological order and spatial location to generate an image sequence of the glass bottle storage space that reflects the complete state of the entire storage area.

[0015] Preferably, the present invention also includes a glass bottle intelligent automated storage and handling terminal, the terminal including a memory, a processor and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the glass bottle intelligent automated storage and handling management system as described above.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The warehouse image feature deconstruction network can extract multi-dimensional spatial distribution features of glass bottle stacks from temporal image sequences and form a dynamic feature stream for temporal correlation analysis. It transforms visual information into a quantifiable data stream of stacking spatial attributes, enabling continuous measurement of deep state indicators such as packing density and posture stability. This allows the system to capture slow, unstructured state degradation trends that are difficult to identify using traditional methods, resulting in a deeper and more accurate perception dimension. The system's sensitivity to common environmental disturbances such as lighting and shadows is reduced, enhancing the robustness of state perception.

[0017] By setting a decision logic that requires a deviation metric to continuously exceed a preset activation threshold before triggering a handling command, an intelligent triggering mechanism with time window verification was constructed. This solution effectively distinguishes between accidental, momentary disturbances and real, continuous anomalies. Subsequent physical operations are only initiated after the system confirms a stable deviation, thus avoiding equipment malfunctions caused by noise or momentary disturbances. This enhances the necessity and effectiveness of each automated handling operation, reducing idle operation and ineffective cycles. The overall decision-making quality of the system is improved, the operation is more stable and reliable, and resource utilization is optimized. Attached Figure Description

[0018] Figure 1 This is a timing diagram of the intelligent three-dimensional warehouse management system for glass bottles described in this invention; Figure 2 A flowchart for extracting multi-dimensional spatial distribution features; Figure 3 The flowchart for time-series correlation analysis is shown below; Figure 4 A multi-dimensional evaluation and comparison chart of candidate routes for glass bottle storage and handling; Figure 5 Heatmaps showing the operation time consumption of different handling strategies during the automated handling execution phase. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 This invention provides an intelligent three-dimensional warehouse management system for glass bottles. The system includes: a warehouse space monitoring module that periodically collects image information of warehouse shelves, forming a sequence of glass bottle warehouse space images with time stamps; a warehouse change analysis module that receives the image sequence, extracts multi-dimensional spatial distribution features of glass bottle stacks through a built-in warehouse image feature deconstruction network, forming a dynamic feature stream of glass bottle storage, and performs time-series correlation analysis on the feature stream to calculate a deviation quantification index of the spatial distribution features of glass bottles between adjacent collection times; a handling decision control module that receives the deviation quantification index, compares it with a preset operation activation threshold, and when the deviation quantification index continuously exceeds the operation activation threshold for a preset duration, generates a glass bottle handling control instruction containing a target location identifier and operation category; and an automated handling module that receives the instruction, performs positioning and navigation based on the target location identifier, and completes the picking, placing, or transferring of glass bottles according to the operation category.

[0021] In one embodiment of the present invention, see [reference] Figure 2 In the process of extracting multi-dimensional spatial distribution features of glass bottle stacks using a warehouse image feature deconstruction network, multi-level feature extraction processing is performed on single-frame images in the glass bottle warehouse spatial image sequence to obtain initial glass bottle spatial contour feature maps reflecting glass bottle outline edge information and initial glass bottle stacking density feature maps reflecting glass bottle stacking density information. Spatial geometric structure analysis is performed on the initial glass bottle spatial contour feature maps, calculating contour continuity and contour symmetry indices to synthesize a glass bottle geometric structure description vector. Regional density gradient analysis is performed on the initial glass bottle stacking density feature maps to identify the boundaries between high-density and low-density stacking regions and calculate the density change rate between regions, forming a glass bottle stacking density distribution vector. The glass bottle geometric structure description vector and the glass bottle stacking density distribution vector are fused in terms of feature dimensions to generate a dynamic feature flow of glass bottle storage containing spatial structure and stacking state information.

[0022] In practical implementation, the storage image feature deconstruction network of the intelligent three-dimensional storage management system for glass bottles extracts multi-dimensional spatial distribution features of glass bottle stacks. The processing can be described using an example of a storage space image sequence containing regularly arranged glass bottle shelves. The resolution of a single frame image in the image sequence is 2048 pixels by 2048 pixels and encoded in RGB format. In practical implementation, multi-level feature extraction processing is performed on a single frame image in the glass bottle storage space image sequence. This processing is completed by a pre-trained deep convolutional neural network. The first to third convolutional operations of the network extract the edge and texture information of the image, outputting a feature map with a dimension of 512 pixels by 512 pixels by 64 channels as the initial glass bottle spatial contour feature map reflecting the glass bottle contour edge information. The fourth to fifth convolutional operations of the network fuse a wider range of contextual information, outputting a feature map with a dimension of 256 pixels by 256 pixels by 128 channels as the initial glass bottle stacking density feature map reflecting the glass bottle stacking density information.

[0023] In some embodiments, spatial geometric structure analysis is performed on the initial spatial contour feature map of the glass bottle. The analysis process applies edge connection and contour finding algorithms to the initial glass bottle spatial contour feature map to identify all closed or nearly closed glass bottle contours in the image. For each identified contour, the continuity of pixels along its perimeter is calculated, and the contour continuity index C is determined. c formula:

[0024] Where: N breaks L represents the number of discontinuous breakpoints detected during contour tracking. total The total length of the contour is represented by the contour continuity index C. Taking an example contour calculation result, when 15 breakpoints are detected and the total contour length is 1200 pixels, the contour continuity index C... c The calculated result is 0.9875. At the same time, the contour symmetry index is calculated by mirroring the contour around the central axis of its smallest bounding rectangle and calculating the overlap area ratio between the original contour and the mirrored contour. The overlap area ratio of an example contour of 0.92 is recorded as the contour symmetry index. The average contour continuity index and the average contour symmetry index of all the calculated contours are combined into a two-dimensional vector as the geometric structure description vector of the glass bottle.

[0025] It is understandable that regional density gradient analysis is performed on the initial glass bottle stack density feature map. This analysis involves a sliding window statistical method, with each window being 16 pixels by 16 pixels. The sum of the feature activation values ​​within each window is used as the local density value, generating a density distribution map. A gradient operator is then applied to this map to calculate the direction and magnitude of density change at each pixel location. Pixel chains with gradient magnitudes significantly higher than the average level are identified as the boundaries between high-density and low-density stacking regions. For each pair of adjacent high-density and low-density regions, the average density change between their centroids is calculated, and the density change rate R is determined. d The density change rate R is defined as the ratio of the difference in average density values ​​between two regions to the Euclidean distance between their centroids. For example, if the average density value of the high-density region is 85, the average density value of the low-density region is 20, and the centroid distance is 50 pixels, then the density change rate R... d The calculation result is 1.3. The number of all identified boundaries, the average gradient magnitude, and the calculated average density change rate are combined into a multidimensional vector, which serves as the density distribution vector of the glass bottle stack.

[0026] Optionally, the geometric description vector of the glass bottle and the stacking density distribution vector of the glass bottle are fused in terms of feature dimensions. The feature dimension fusion operation directly concatenates the two-dimensional geometric description vector of the glass bottle with the multi-dimensional stacking density distribution vector of the glass bottle. Assuming that the stacking density distribution vector of the glass bottle is a five-dimensional vector, a seven-dimensional fused feature vector is generated after concatenation. In some embodiments, the two vectors are normalized before concatenation so that the numerical range of each dimension in the vector is unified to the interval [0,1]. This fused feature vector is used as a feature sample in the dynamic feature stream of glass bottle storage of the current frame image. As the image sequence is input, the system generates a series of such fused feature vectors in chronological order, which together constitute the complete dynamic feature stream of glass bottle storage.

[0027] In one embodiment of the present invention, see [reference] Figure 3The warehouse change analysis module performs time-series correlation parsing on the dynamic feature stream of glass bottle storage. It extracts historical dynamic feature segments belonging to the previous analysis period and real-time dynamic feature segments belonging to the current analysis period from the dynamic feature stream in chronological order. For multiple preset feature analysis dimensions, it calculates the changes in eigenvalues ​​of the historical and real-time dynamic feature segments across these dimensions, obtaining single-dimensional feature changes for glass bottle storage across multiple dimensions. A dynamic weighting coefficient is assigned to each single-dimensional feature change, adaptively adjusting based on the stability of the corresponding feature dimension's fluctuations over historical periods; dimensions with smaller fluctuations are assigned higher dynamic weighting coefficients. All single-dimensional feature changes are multiplied by their respective dynamic weighting coefficients, weighted, and summed to output a deviation metric representing the intensity of overall warehouse status changes. The process of adaptively adjusting the dynamic weight coefficient based on the fluctuation stability of the corresponding feature dimension in the historical period is as follows: statistically analyze the feature value sequence of the historical glass bottle storage dynamic feature segment at multiple consecutive time points in each feature dimension, calculate the standard deviation of the feature value sequence as the historical fluctuation amplitude value of the feature dimension, compare the preset benchmark weight value with the historical fluctuation amplitude value, if the historical fluctuation amplitude value is less than the preset stability threshold, the benchmark weight value is increased by a fixed gain value to obtain the dynamic weight coefficient after feature dimension adjustment, if the historical fluctuation amplitude value is greater than or equal to the preset stability threshold, the benchmark weight value is decreased by a fixed decay value to obtain the dynamic weight coefficient after feature dimension adjustment.

[0028] In practice, the storage change analysis module of the intelligent three-dimensional storage management system for glass bottles performs time-series correlation analysis on the dynamic feature stream of glass bottle storage. Taking a system with a preset analysis period of 10 minutes as an example, the dynamic feature stream of glass bottle storage is continuously generated at a rate of one feature vector per second. In practice, historical dynamic feature segments of glass bottle storage belonging to the previous analysis period and real-time dynamic feature segments of glass bottle storage belonging to the current analysis period are extracted from the dynamic feature stream of glass bottle storage in chronological order. The historical dynamic feature segments of glass bottle storage are selected from 600 consecutive feature vectors before the start time of the analysis period, and the real-time dynamic feature segments of glass bottle storage are selected from 600 consecutive feature vectors before the current time. Each feature vector contains data from seven preset feature analysis dimensions.

[0029] In some embodiments, the changes in the eigenvalues ​​of historical and real-time dynamic feature segments of glass bottle storage are calculated across seven preset feature analysis dimensions. The calculation process involves taking the average of 600 eigenvalues ​​for each feature analysis dimension for the historical dynamic feature segment of glass bottle storage. And calculate the average of 600 feature values ​​of the real-time glass bottle storage dynamic feature segment in this dimension. The change in the single-dimensional characteristic ΔV of glass bottle storage is expressed by the formula. To perform the calculation, taking the result of a sample feature dimension as an example, the historical average value... It is 0.85, the real-time average. If the value is 0.72, then the calculated result of the single-dimensional characteristic change ΔV of glass bottle storage in this dimension is 0.13.

[0030] This can be understood as assigning dynamic weighting coefficients to the changes in each single-dimensional feature of glass bottle storage. These dynamic weighting coefficients are adaptively adjusted based on the stability of the corresponding feature dimension's fluctuations over historical periods. The system statistically analyzes the feature value sequence of each feature dimension across multiple consecutive time points for each historical dynamic feature segment of glass bottle storage. Specifically, it uses 600 feature values ​​from the historical segment as the feature value sequence and calculates the standard deviation of the feature value sequence as the historical fluctuation amplitude of the feature dimension. The preset baseline weight value W b Compared with historical fluctuation values Comparison, stability threshold T S The default value is 0.05, based on historical fluctuation values. Less than the preset stability threshold T S Then the benchmark weight value W b Increase by a fixed gain value △ inc The dynamic weight coefficients after feature dimension adjustment are obtained. If the historical fluctuation range value Greater than or equal to the preset stability threshold T S Then the benchmark weight value W b Reduce by a fixed attenuation value Δ dec The dynamic weight coefficients after feature dimension adjustment are obtained. Gain value △ inc Set to 0.1, attenuation value Δ dec Set to 0.05, baseline weight value W b Initially set to 1.0, taking a single feature dimension as an example, its historical fluctuation range value is... The value is 0.03, which is less than the stability threshold T. S If W = 0.05, then the dynamic weight coefficient W for this dimension is... a The calculation result is 1.1.

[0031] Optionally, the changes in the single-dimensional features of glass bottle storage across all dimensions are multiplied by their respective dynamic weight coefficients and then weighted and summed. The final output is a deviation metric that represents the intensity of changes in the overall storage status. Assuming the changes in the seven dimensions of glass bottle storage single-dimensional features are [0.13, 0.08, 0.22, 0.05, 0.11, 0.03, 0.17], and the corresponding dynamic weight coefficients are [1.1, 0.95, 0.95, 1.1, 1.1, 1.1, 0.95], the weighted summation operation is performed on each change and its weight coefficient. After multiplying and summing the numbers, the deviation metric D is calculated as follows: D = (0.13 × 1.1) + (0.08 × 0.95) + (0.22 × 0.95) + (0.05 × 1.1) + (0.11 × 1.1) + (0.03 × 1.1) + (0.17 × 0.95). In some embodiments, the value obtained after weighted summation is directly used as the deviation metric output. For example, the above calculation example yields a specific deviation metric value of 0.7805, which is transmitted to the transport decision control module for subsequent decision-making.

[0032] In one embodiment of the present invention, the workflow of the handling decision control module further includes, while generating the glass bottle handling control instruction, accessing a pre-stored 3D model database of warehouse racks to obtain the rack structure parameters corresponding to the target storage location identifier and the occupancy status of adjacent storage locations. Based on the rack structure parameters and the occupancy status of adjacent storage locations, the handling decision control module selects a target handling strategy from a set of preset candidate handling strategies. The candidate handling strategies include at least a direct pick-and-place strategy, a nearby temporary storage followed by pick-and-place strategy, and a path replanning pick-and-place strategy. The identifier code of the target handling strategy is added to the glass bottle handling control instruction to form an enhanced glass bottle handling control instruction, which is then sent to the automated handling module. The rule for selecting a target handling strategy from multiple preset candidate handling strategies is as follows: if the occupancy status of adjacent storage locations shows that there is sufficient direct operating space for the target storage location, the direct pick-and-place strategy is selected; if the occupancy status of adjacent storage locations shows that there is insufficient direct operating space for the target storage location but there is an available nearby storage location that meets the preset safety distance requirement, the adjacent storage followed by pick-and-place strategy is selected; if the occupancy status of adjacent storage locations shows that there is insufficient direct operating space for the target storage location and there is no available nearby storage location that meets the preset safety distance requirement, the path replanning pick-and-place strategy is selected. The pick-and-place strategy instructs the automated handling module to approach the target storage location along a non-linear path.

[0033] In practical implementation, the handling decision control module of the intelligent automated storage and retrieval system for glass bottles executes an enhanced decision-making process while generating glass bottle handling control instructions. The handling decision control module accesses a pre-stored 3D model database of storage racks. This database stores the 3D spatial coordinates, physical dimensions, and status attributes corresponding to each location identifier in key-value pairs. Assuming that the target location identifier in the received glass bottle handling control instruction is "A-05-12", the handling decision control module uses this identifier as the key to query the database and obtain the corresponding rack structure parameters and the occupancy status of adjacent locations. The rack structure parameters include the net height H of the rack layer where the target location is located, the effective depth D of the pallet, and the rated load W. The occupancy status of adjacent locations is returned in the form of an array, indicating whether the six logical adjacent locations above, below, left, right, front, and back relative to the target location "A-05-12" are currently occupied by Boolean values.

[0034] In some embodiments, the handling decision control module selects a target handling strategy from a set of preset candidate handling strategies based on the acquired rack structure parameters and the occupancy status of adjacent storage locations. The preset candidate handling strategy library includes three strategies: direct pick-and-place strategy, adjacent temporary storage followed by pick-and-place strategy, and path replanning pick-and-place strategy. The selection process is based on a set of predefined rule logic. The core of the rule logic is to evaluate the direct operability of the target storage location. Direct operability is quantified by a space sufficiency index S.

[0035] Among them: A e A represents the effective operating area of ​​the storage location calculated based on the racking structure parameters. m η represents the minimum working area required for the robotic arm of the automated handling module to perform pick-and-place operations. η is an influence factor calculated based on the occupancy status of adjacent storage locations. When neither adjacent storage location is occupied, η = 1.0; when some adjacent storage locations are occupied, the value of η decreases according to a preset rule. It can be understood that the rule for selecting a target handling strategy from multiple preset candidate handling strategies is: if the calculated space sufficiency index S is greater than the preset operation threshold T... op If the adjacent storage location occupancy status indicates sufficient direct operating space for the target storage location, the handling decision control module selects a direct pick-and-place strategy; if the space sufficiency index S is not greater than the preset operating threshold T... op However, there is at least one available storage location in the adjacent storage location occupancy status array, and the spatial distance d between the available storage location and the target storage location is less than the preset safety distance threshold D. s If there is a nearby available storage location that meets the preset safety distance requirement, the handling decision control module selects a nearby temporary storage and then pick-up / place strategy; if the space sufficiency index S is not greater than the preset operation threshold T op Furthermore, the spatial distance d between all adjacent storage locations that are occupied or vacant is not less than the safety distance threshold D.s If the target storage location is deemed to have insufficient direct operational space and no available adjacent storage location meeting the preset safety distance requirements, the handling decision control module will select a route replanning pick-and-place strategy. In one example, assuming the calculated space sufficiency index S for the target storage location "A-05-12" is 0.75, and the preset operational threshold T... op Since the inequality is 1.0, S ≤ T op Checking the occupancy status of adjacent storage locations revealed that storage location "A-05-11" to its left was vacant, and the calculated distance d was 1.2 meters. The preset safe distance threshold D... s The length is 2.0 meters, satisfying d < D. s Therefore, the handling decision control module selects a strategy of temporary storage in the vicinity followed by retrieval and placement.

[0036] Optionally, the handling decision control module adds the identifier code of the selected target handling strategy to the original glass bottle handling control instruction. The identifier codes of the direct pick-up and place strategy, the adjacent temporary storage and then pick-up and place strategy, and the path replanning pick-up and place strategy are preset as "DIRECT", "BUFFER", and "RECON", respectively. Taking the aforementioned example, the system combines the strategy identifier code "BUFFER" with the original target storage location identifier "A-05-12" and the operation category "retrieve" to form an enhanced glass bottle handling control instruction. In some embodiments, the enhanced instruction also includes parameters related to the selected strategy. For example, when the adjacent temporary storage and then pick-up and place strategy is selected, the instruction will include the specified free adjacent storage location identifier "A-05-11". This complete enhanced glass bottle handling control instruction is then sent to the automated handling module for execution.

[0037] In one embodiment of the present invention, the automated handling module performs a positioning and navigation process based on the target storage location identifier. This includes the automated handling module's built-in terminal positioning module acquiring its current terminal position coordinates within the storage space in real time. The automated handling module parses the target storage location identifier from the received glass bottle handling control command and queries the corresponding target storage location space coordinates from the warehouse electronic map based on the target storage location identifier. The automated handling module calls the path planning engine, using the current terminal position coordinates as the starting point and the target storage location space coordinates as the ending point, and considering preset fixed obstacle areas and real-time updated dynamic obstacle information, to calculate a collision-free terminal navigation path. The automated handling module's drive control system controls its mobile chassis to move according to the terminal navigation path until it reaches the position corresponding to the target storage location space coordinates. The automated handling module completes the glass bottle picking, placing, or transferring operations according to the operation category. Specifically, when the operation category is "retrieve," the automated handling module controls the gripper at the end of its robotic arm to move to a preset glass bottle picking preparation position, drives the gripper to perform a clamping action, clamps the target glass bottle, lifts it, and transports it to the designated exit conveyor line position. When the operation category is "storage," the automated handling module retrieves the glass bottles to be stored from its platform, controls the robotic arm to move the bottles to the precise storage location of the target storage location, and releases the gripper to place the bottles stably. When the operation category is "transfer," the automated handling module first performs a "retrieve" operation to move the bottles from their original storage location to its platform, plans the path to the new target storage location according to the instructions, and moves them accordingly. Finally, it performs a "storage" operation at the new storage location.

[0038] In practical implementation, the automated handling module of the intelligent automated storage and retrieval system for glass bottles executes positioning and navigation based on the target storage location identifier. The terminal positioning module built into the automated handling module obtains the terminal's current position coordinates in the storage space in real time by fusing LiDAR point cloud matching and ultra-wideband wireless positioning base station signals. The terminal's current position coordinates are represented in three-dimensional form, for example, (X_c, Y_c, Z_c) is (15.2, 8.7, 0.0), and the unit is meters. The automated handling module parses the target storage location identifier from the received glass bottle handling control command and queries the corresponding target storage location spatial coordinates from the warehouse electronic map based on the target storage location identifier. The warehouse electronic map is a database that stores the coordinate information of all shelves, aisles, and workstations. Assuming the target storage location identifier is "B-03-08", the retrieved target storage location spatial coordinates are (45.6, 22.3, 1.5).

[0039] It is understandable that the automated handling module calls the path planning engine, taking the current coordinates of the terminal as the starting point and the spatial coordinates of the target storage location as the ending point, and considering the preset fixed obstacle area and the real-time updated dynamic obstacle information, to calculate a collision-free terminal navigation path. The fixed obstacle area includes the boundary coordinates of the shelf base and load-bearing columns, while the dynamic obstacle information is provided by the mobile sensor network deployed in the warehouse, including the real-time position contours of other moving equipment or personnel. The path planning engine uses an improved A* algorithm for search, and the evaluation function for the path search is:

[0040] Where: C p W represents the total cost of the calculated candidate paths. i f represents the preset weight coefficient of the i-th cost factor. i This represents the specific value of the i-th cost factor. Cost factors include path length, number of turns, minimum distance to nearby dynamic obstacles, etc. See Table 1, which shows a comparison of some parameters of three candidate paths calculated in a path planning.

[0041] Table 1: Evaluation Table of Candidate Path Planning Solutions

[0042] In some embodiments, the path planning engine selects the total path cost C. p The candidate path with the lowest cost is selected as the output terminal navigation path. Based on the comparison in the table above, the total path cost C of candidate path 3 is... p The value is 41.8, which is the lowest value, and therefore it was selected as the terminal navigation path. The drive control system of the automated handling module controls the movement of its mobile chassis according to the terminal navigation path. The drive control system analyzes the terminal navigation path as a series of dense path points and controls the speed and direction of the hub motors through a proportional-integral-derivative controller until the target cargo space coordinates are reached.

[0043] In practice, the automated handling module performs glass bottle retrieval and transfer operations according to the operation category. When the operation category is "retrieval," the automated handling module controls the gripper at the end of its robotic arm to move to the preset glass bottle gripping position. The preset position is a three-dimensional coordinate calculated based on the spatial coordinates of the target storage location and the geometric model of the storage location. The gripper is driven to perform a clamping action, and after clamping the target glass bottle, the robotic arm lifts the glass bottle 20 cm along a vertical lifting trajectory, and then transports it to the designated exit conveyor position along a planned horizontal path. After placement, the gripper is released. When the operation category is "storage," the automated handling module retrieves the glass bottle to be stored from its carrying platform. After the photoelectric sensor on the carrying platform confirms the presence of the glass bottle, it controls the robotic arm to move the glass bottle to the precise storage position of the target storage location. The precise storage position is finally accurately positioned by a laser rangefinder sensor. After releasing the gripper, the robotic arm remains still for 0.5 seconds to ensure that the glass bottle is placed stably, and then withdraws. Optionally, when the operation category is "relocation", the automated handling module first performs a complete "retrieve" operation at the source location, moving the glass bottles from the original location to the fixed slot of its carrying platform. Based on the new target location identifier in the instruction, it plans the path to the new location and moves the bottles. Finally, it performs a complete "store" operation at the new location. In one relocation instruction, the automated handling module may receive two glass bottle handling control instructions in succession. The first instruction is an operation category of "retrieve" and the target location identifier is the source location. The second instruction is an operation category of "store" and the target location identifier is the destination location.

[0044] See Figure 4 In the evaluation phase of intelligent automated warehousing and handling path planning for glass bottles, this graph uses the candidate path number as the horizontal axis and simultaneously displays three core indicators for the three candidate paths: total path length (meters, blue bars), estimated time (seconds, orange bars), and total path cost (red line and nodes). Specifically, candidate path 1 has a total path length of 38.5 meters, an estimated time of 48 seconds, and a total path cost of 42.6; candidate path 2 has a total path length of 36.2 meters, an estimated time of 52 seconds, and a total path cost of 45.1; and candidate path 3 has a total path length of 39.1 meters, an estimated time of 46 seconds, and a total path cost of 41.8. The graph clearly presents the differences in performance of each path in terms of length and time through multi-indicator visualization comparison. At the same time, the red line highlights the changing trend of the total path cost. The total path cost of candidate path 3 (41.8) is the lowest of the three, which is consistent with the optimal path selection logic of the path planning engine based on total cost.

[0045] In one embodiment of the present invention, the warehouse space monitoring module includes multiple high-resolution industrial cameras fixed at different viewing positions within the warehouse area, and an image sequence scheduler. Each high-resolution industrial camera periodically captures images of the warehouse shelves within its assigned monitoring sub-area according to a unified clock signal, and sends the captured raw shelf images to the image sequence scheduler. The image sequence scheduler receives the raw shelf images from all the high-resolution industrial cameras, aligns and stitches all the images according to time sequence and spatial location based on the camera identifier and timestamp attached to each raw shelf image, and generates an image sequence of the glass bottle warehouse space reflecting the complete state of the entire warehouse area.

[0046] In practical implementation, the intelligent automated storage and retrieval system for glass bottles includes a storage space monitoring module comprising multiple high-resolution industrial cameras fixed at different angles within the storage area, and an image sequence scheduler. The storage area is a 20-meter-long, 15-meter-wide, and 10-meter-high automated warehouse. Four high-resolution industrial cameras are fixed to pillars at the four corners of the warehouse, 8 meters above the ground. Each high-resolution industrial camera has a sensor resolution of 4096 pixels by 3000 pixels, equipped with an 8mm fixed-focus lens, and its field of view covers a monitored sub-area. In practical implementation, each high-resolution industrial camera operates according to a unified time... The clock signal periodically captures images of the warehouse shelves within its monitored sub-area. A unified clock signal is distributed by a central time server via a network protocol to ensure that the system time deviation of all high-resolution industrial cameras is less than 10 milliseconds. The shooting cycle is preset to 30 seconds. When each cycle is triggered, the high-resolution industrial camera automatically focuses and exposes, captures a color original shelf image, and sends the captured original shelf image with camera identification "CAM_01", "CAM_02", "CAM_03", "CAM_04" and acquisition timestamp accurate to milliseconds to the image sequence scheduler.

[0047] In some embodiments, the image sequence scheduler receives raw shelf images from all high-resolution industrial cameras. The image sequence scheduler runs on a dedicated industrial server equipped with a high-speed network interface and a large-capacity cache. It aligns and stitches all images according to their temporal order and spatial location based on the camera identifier and timestamp attached to each raw shelf image. The alignment process first groups images arriving within the same acquisition period into the same processing batch based on their timestamps. For a given batch, for example, four images with a timestamp around "2023-10-27 14:30:00.030", the image sequence scheduler checks the deviation δ between the actual arrival timestamp of each image and the batch reference time. t The deviation calculation formula is:

[0048] Where: t arrivalt represents the actual arrival timestamp of a single original shelf image. batch This represents the baseline timestamp set for this batch. If the deviation of a certain image is δ t If the frame exceeds the preset 50-millisecond threshold, the image is marked as an asynchronous frame and a retransmission or interpolation compensation process is initiated.

[0049] It is understandable that after time alignment, the images need to be aligned and stitched together in spatial position. The image sequence scheduler internally stores a pre-calibrated storage space mapping table. The mapping table defines the transformation relationship between the pixel coordinates of each high-resolution industrial camera image and the global three-dimensional coordinates of the storage space. Using the perspective transformation matrix, four distortion-corrected original shelf images are projected onto the same unified global two-dimensional overhead view plane. The projected images have overlapping areas. The image sequence scheduler uses a feature point matching-based fusion algorithm to smoothly stitch the overlapping parts, eliminate stitching seams, and ensure brightness consistency. Finally, a composite image covering the complete state of the entire storage area is generated. This composite image is a glass bottle storage space image sequence frame at a certain acquisition time. Arranging the sequence frames generated in multiple consecutive periods according to the time identifier constitutes the glass bottle storage space image sequence used by the system.

[0050] See Figure 5 In the automated handling execution phase of the intelligent automated storage and retrieval system for glass bottles, a heatmap visually presents the time differences among three candidate handling strategies—direct strategy, temporary storage strategy, and replanning strategy—in the three operations of "retrieval," "storage," and "transfer." The color gradient in the graph corresponds to the operation time (15 to 45 seconds), with darker colors representing longer times: Under the direct strategy, "retrieval" and "storage" operations have the shortest times (15.2s and 14.8s respectively), while "transfer" takes 28.5s; under the temporary storage strategy, the times for all three operations increase ("retrieval" 22.5s, "storage" 21.8s, and "transfer" 40.2s); the replanning strategy has the longest operation time, with "transfer" reaching 48.7s. This time distribution is consistent with the logic of the system's handling strategy selection rules: the direct strategy relies on ample operating space for efficient operation; the temporary storage strategy increases time due to the addition of temporary storage in adjacent locations; and the replanning strategy further increases time due to non-linear path navigation and the complexity of the operation process.

[0051] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0052] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A smart three-dimensional warehouse management system for glass bottles, characterized in that, The system includes; The warehouse space monitoring module periodically collects image information of the warehouse shelves, forming a sequence of glass bottle storage space images with time stamps; The storage change analysis module receives the sequence of glass bottle storage space images, extracts the multi-dimensional spatial distribution features of glass bottle stacks through the storage image feature deconstruction network, forms a dynamic feature stream of glass bottle storage, performs time-series correlation parsing operation on the dynamic feature stream of glass bottle storage, and calculates the deviation quantification index of the spatial distribution features of glass bottles between adjacent acquisition times. The handling decision control module receives the deviation measurement index, compares the deviation measurement index with the preset operation activation threshold, and generates a glass bottle handling control instruction containing the target location identifier and operation category when the deviation measurement index continuously exceeds the operation activation threshold for a preset duration period. The automated handling module receives the glass bottle handling control command, performs positioning and navigation according to the target storage location identifier, and completes the picking, placing or transferring of glass bottles according to the operation category.

2. The intelligent three-dimensional warehouse management system for glass bottles according to claim 1, characterized in that, The process of extracting the multi-dimensional spatial distribution features of glass bottle stacks using a warehouse image feature deconstruction network includes the following steps: Multi-level feature extraction processing is performed on single-frame images in the glass bottle storage space image sequence to obtain an initial glass bottle space contour feature map reflecting the glass bottle contour edge information and an initial glass bottle stacking density feature map reflecting the glass bottle stacking density information. Spatial geometric structure analysis is performed on the initial glass bottle spatial contour feature map to calculate the contour continuity index and contour symmetry index, and synthesize the glass bottle geometric structure description vector. Regional density gradient analysis is performed on the initial glass bottle stacking density feature map to identify the boundaries between high-density stacking regions and low-density stacking regions, and the density change rate between regions is calculated to form a glass bottle stacking density distribution vector. The geometric structure description vector of the glass bottle and the stacking density distribution vector of the glass bottle are fused by feature dimension to generate the dynamic feature flow of glass bottle storage that includes spatial structure and stacking state information.

3. The intelligent three-dimensional warehouse management system for glass bottles according to claim 2, characterized in that, The storage change analysis module performs time-series correlation analysis on the dynamic feature stream of the glass bottle storage, including the following steps: From the dynamic feature stream of glass bottle storage, extract historical dynamic feature segments of glass bottle storage belonging to the previous analysis period and real-time dynamic feature segments of glass bottle storage belonging to the current analysis period in chronological order. On multiple preset feature analysis dimensions, the changes in the feature values ​​of the historical glass bottle storage dynamic feature segments and the real-time glass bottle storage dynamic feature segments on the feature analysis dimensions are calculated respectively, so as to obtain the single-dimensional feature changes of glass bottle storage in multiple dimensions. A dynamic weighting coefficient is configured for each of the single-dimensional feature changes in the glass bottle storage. The dynamic weighting coefficient is adaptively adjusted according to the fluctuation stability of the corresponding feature dimension in the historical period. The dimension with smaller fluctuation is assigned a higher dynamic weighting coefficient. The changes in the single-dimensional features of glass bottle storage across all dimensions are multiplied by their respective dynamic weight coefficients, and then weighted and summed to finally output the deviation quantification index that characterizes the intensity of changes in the overall storage status.

4. The intelligent three-dimensional warehouse management system for glass bottles according to claim 3, characterized in that, The process of adaptively adjusting the dynamic weight coefficients based on the stability of the corresponding feature dimensions' fluctuations in historical periods is as follows: The sequence of feature values ​​at multiple consecutive time points in each feature dimension of the historical glass bottle storage dynamic feature segment is statistically analyzed. Calculate the standard deviation of the feature value sequence, which is used as the historical fluctuation range value of the feature dimension; The preset baseline weight value is compared with the historical fluctuation amplitude value. If the historical fluctuation amplitude value is less than the preset stability threshold, the baseline weight value is increased by a fixed gain value to obtain the dynamic weight coefficient after the feature dimension is adjusted. If the historical fluctuation amplitude value is greater than or equal to the preset stability threshold, the benchmark weight value is reduced by a fixed attenuation value to obtain the dynamic weight coefficient after the feature dimension is adjusted.

5. The intelligent three-dimensional warehouse management system for glass bottles according to claim 1, characterized in that, The workflow of the handling decision control module also includes: While generating the glass bottle handling control command, the handling decision control module accesses the pre-stored 3D model database of the storage rack to obtain the rack structure parameters and adjacent rack occupancy status corresponding to the target location identifier. The handling decision control module selects a target handling strategy from a plurality of preset candidate handling strategies based on the rack structure parameters and the occupancy status of adjacent storage locations. The candidate handling strategies include at least a direct pick-and-place strategy, a nearby temporary storage and then pick-and-place strategy, and a path replanning pick-and-place strategy. The identification code of the target handling strategy is added to the glass bottle handling control command to form an enhanced glass bottle handling control command, which is then sent to the automated handling module.

6. The intelligent three-dimensional warehouse management system for glass bottles according to claim 5, characterized in that, The rule for selecting a target handling strategy from a plurality of preset candidate handling strategies is as follows: If the adjacent storage location occupancy status shows that there is sufficient direct operating space for the target storage location, then the direct pick-and-place strategy is selected. If the occupancy status of the adjacent storage location shows that the direct operating space of the target storage location is insufficient, but there is an available adjacent storage location that meets the preset safety distance requirements, then the adjacent temporary storage and retrieval strategy is selected. If the adjacent storage location occupancy status shows that the direct operating space of the target storage location is insufficient and there is no available adjacent storage location that meets the preset safety distance requirements, then a path replanning pick-and-place strategy is selected. The pick-and-place strategy instructs the automated handling module to approach the target storage location along a non-straight path.

7. The intelligent three-dimensional warehouse management system for glass bottles according to claim 1, characterized in that, The automated handling module performs a positioning and navigation process based on the target cargo location identifier, including: The automated handling module has a built-in terminal positioning module that obtains the current position coordinates of the terminal in the warehouse space in real time. The automated handling module parses the target location identifier from the received glass bottle handling control command, and queries the corresponding target location spatial coordinates from the warehouse electronic map based on the target location identifier; The automated handling module calls the path planning engine, takes the current location coordinates of the terminal as the starting point, the target cargo space coordinates as the ending point, and takes into account the preset fixed obstacle area and the real-time updated dynamic obstacle information to calculate a collision-free terminal navigation path. The drive control system of the automated handling module controls the movement of its mobile chassis according to the terminal navigation path until it reaches the position corresponding to the spatial coordinates of the target cargo location.

8. The intelligent three-dimensional warehouse management system for glass bottles according to claim 7, characterized in that, The automated handling module completes the picking, placing, or transferring of glass bottles according to the operation category, specifically including: When the operation category is "removal", the automated handling module controls the gripper at the end of its robotic arm to move to the preset glass bottle gripping preparation position, drives the gripper to perform a clamping action, clamps the target glass bottle, lifts it and transports it to the designated outlet conveyor line position. When the operation category is "storage", the automated handling module obtains the glass bottle to be stored from its carrying platform, controls the robotic arm to move the glass bottle to the precise storage position of the target location, releases the gripper, and places the glass bottle stably. When the operation category is "transfer", the automated handling module first performs a "retrieve" operation, moving the glass bottles from the original warehouse to its carrying platform. According to the new target warehouse location identifier in the instruction, it plans the path to the new warehouse location and moves it. Finally, it performs a "store" operation at the new warehouse location.

9. The intelligent three-dimensional warehouse management system for glass bottles according to claim 1, characterized in that, The warehouse space monitoring module includes multiple high-resolution industrial cameras fixed at different viewing positions in the warehouse area, as well as an image sequence scheduler. Each of the high-resolution industrial cameras periodically captures images of the warehouse shelves within its assigned monitoring sub-area according to a unified clock signal, and sends the captured raw shelf images to the image sequence scheduler. The image sequence scheduler receives raw shelf images from all high-resolution industrial cameras. Based on the camera identifier and timestamp attached to each raw shelf image, it aligns and stitches all images in chronological order and spatial location to generate an image sequence of the glass bottle storage space that reflects the complete state of the entire storage area.

10. A smart three-dimensional storage and handling terminal for glass bottles, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent three-dimensional storage management system for glass bottles according to any one of claims 1-9.

Images