Warehouse automatic checking system integrated with visual identification

Abstract

The invention relates to the technical field of warehouse checking, in particular to an automatic warehouse checking system integrating visual identification, which comprises a checking matching unit, a feature extraction unit, a goods shelf classification unit, a goods identification unit and a goods shelf distribution unit, and further comprises a goods shelf coding unit, the goods shelf coding unit is used for coding a plurality of goods shelves of a target warehouse to obtain a goods shelf coding set corresponding to the target warehouse, the goods shelf coding set comprises the number of goods shelf rows, the number of goods shelf columns and the number of goods shelf layers, and the goods shelf coding set corresponding to the target warehouse is transmitted to the inventory matching unit. Through a high-definition camera and an automatic image processing technology, the system can quickly and accurately complete image acquisition, feature extraction and classification of the goods shelf, and the automatic process greatly reduces the time and labor cost required by manual checking.

Description

Technical Field

[0001] The present invention relates to the technical field of warehouse inventory counting, and in particular to an automated warehouse inventory counting system integrating visual recognition. Background Art

[0002] Visual recognition refers to the process of automatically identifying and classifying objects, scenes or features in images or videos through computer technology and algorithms; a warehouse is a facility used to store goods, materials or goods. It is often used in the commercial and logistics industries to manage inventory between production and sales.

[0003] Traditional systems usually rely on manual inventory counting, which is time-consuming and inefficient, and can easily lead to delays in inventory management. Moreover, due to reliance on manual data input, errors are prone to occur, resulting in inaccurate inventory information, which affects decision-making. Traditional systems are often unable to update inventory status in real time, and managers are slow to grasp inventory conditions, making it difficult to make timely replenishment and inventory adjustments. In addition, due to the lack of effective visualization tools, it is difficult for managers to intuitively understand the use of shelves, which may lead to waste of space and irrational allocation of resources. There is usually a lack of intelligent decision-making support, and resource allocation relies on experience-based judgment, which can easily lead to waste of resources and inefficiency. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a warehouse automated inventory system integrating visual recognition.

[0005] The technical solution adopted to solve the above technical problems is: an integrated visual recognition warehouse automated inventory system, including an inventory matching unit, a feature extraction unit, a shelf classification unit, a cargo identification unit and a shelf allocation unit, and also includes:

[0006] A shelf coding unit, which is used to code multiple shelves in a target warehouse to obtain a shelf coding set corresponding to the target warehouse, wherein the shelf coding set includes the number of shelf rows, the number of shelf columns and the number of shelf layers, and transmits the shelf coding set corresponding to the target warehouse to an inventory matching unit;

[0007] An image acquisition unit, the image acquisition unit is used to acquire images of a plurality of shelves of the target warehouse through a high-definition camera at a fixed position to obtain a shelf image set corresponding to the target warehouse, and transmit the shelf image set corresponding to the target warehouse to an image preprocessing unit and a goods identification unit;

[0008] An image preprocessing unit, which receives the shelf image set corresponding to the target warehouse transmitted by the image acquisition unit, performs image preprocessing on the shelf image set corresponding to the target warehouse to obtain a standard shelf image set corresponding to the target warehouse, and transmits the standard shelf image set corresponding to the target warehouse to the image segmentation unit;

[0009] An image segmentation unit receives the standard shelf image set corresponding to the target warehouse transmitted by the image preprocessing unit, performs shelf image segmentation on the standard shelf image set to obtain a target shelf image set corresponding to the standard shelf image set, and transmits the target shelf image set corresponding to the standard shelf image set to a feature extraction unit.

[0010] Preferably, the feature extraction unit receives the target shelf image set corresponding to the standard shelf image set transmitted by the image segmentation unit, and performs feature extraction on each target shelf image in the target shelf image set corresponding to the standard shelf image set to obtain an image feature set corresponding to the target shelf image set, and transmits the image feature set corresponding to the target shelf image set to the shelf classification unit.

[0011] Preferably, the shelf classification unit receives the image feature set corresponding to the target shelf image set transmitted by the feature extraction unit, and classifies the image feature set corresponding to the target shelf image set based on a trained shelf classification model to obtain the shelf type corresponding to the image feature set, wherein the shelf type includes shelf types with goods and shelf types without goods, and the shelf type corresponding to the image feature set is transmitted to the goods identification unit.

[0012] Preferably, the cargo identification unit receives the shelf image set corresponding to the target warehouse transmitted by the image acquisition unit and the shelf type corresponding to the image feature set transmitted by the shelf classification unit, obtains a shelf image with goods as the shelf type corresponding to the image feature set based on the shelf image set corresponding to the target warehouse, performs image recognition on the shelf image with goods as the shelf type based on a trained image recognition model to obtain cargo information corresponding to the shelf image with goods as the shelf type, updates the inventory table pre-stored in the database based on the cargo information corresponding to the shelf image with goods as the shelf type and the shelf image with no goods as the shelf type to obtain a dynamic inventory table corresponding to the target warehouse, and transmits the dynamic inventory table corresponding to the target warehouse to the inventory matching unit.

[0013] Preferably, the inventory matching unit receives the shelf coding set corresponding to the target warehouse transmitted by the shelf coding unit and the dynamic inventory table corresponding to the target warehouse transmitted by the goods identification unit, and matches the shelf coding set corresponding to the target warehouse based on the dynamic inventory table corresponding to the target warehouse to obtain the shelf corresponding to the dynamic inventory table, performs lighting display on the shelf corresponding to the dynamic inventory table that has no goods stored, and transmits the information of the shelf corresponding to the dynamic inventory table that has no goods stored to the shelf allocation unit.

[0014] Preferably, the shelf allocation unit (9) receives the shelf information without goods corresponding to the dynamic inventory table transmitted by the inventory matching unit (2), and constructs an objective function of shortest allocation time based on the shelf information without goods corresponding to the dynamic inventory table, constructs constraint conditions based on the remaining free storage spaces on the shelf, and solves the objective function of shortest allocation time through a linear programming algorithm based on the constraint conditions to obtain an allocation plan corresponding to the shelf information without goods corresponding to the dynamic inventory table.

[0015] Preferably, performing shelf image segmentation on the standard shelf image set to obtain a target shelf image set corresponding to the standard shelf image set includes:

[0016] The minimum and maximum grayscale values ​​of the standard shelf image are obtained, and a first segmentation threshold is calculated based on the minimum and maximum grayscale values. The calculation formula of the first segmentation threshold is as follows:

[0017]

[0018] Among them, T0 represents the first segmentation threshold, Z min Represents the minimum gray value, Z max Indicates the maximum gray value;

[0019] Segmenting the standard shelf image into a first target shelf image and a first background image based on the first segmentation threshold;

[0020] The average grayscale value of the first target shelf image and the average grayscale value of the first background image are obtained, and a second segmentation threshold is calculated based on the average grayscale value of the first target shelf image and the average grayscale value of the first background image. The second segmentation threshold calculation formula is as follows:

[0021]

[0022] Among them, T K represents the second segmentation threshold, H1 represents the average gray value of the first target standard shelf image, and H2 represents the average gray value of the first background standard shelf image;

[0023] The first segmentation threshold is compared with the second segmentation threshold. If the first segmentation threshold is equal to the second segmentation threshold, the standard shelf image is segmented into a second target standard shelf image based on the second segmentation threshold. The second target standard shelf image is the final target standard shelf image. Otherwise, the second segmentation threshold is assigned to the first segmentation threshold, and the above operation is repeated.

[0024] Preferably, image preprocessing is performed on the shelf image set corresponding to the target warehouse to obtain a standard shelf image set corresponding to the target warehouse, including:

[0025] Binarizing the shelf image to obtain a shelf binary image;

[0026] Traversing each pixel point in the binary image of the shelf, when traversing to the current pixel point, judging whether the current pixel point is an isolated point by using a preset isolated point judgment rule;

[0027] If the current pixel is an isolated point, the current pixel is deleted; otherwise, the next pixel of the current pixel is traversed until each pixel in the shelf binary image is traversed to obtain a standard shelf image;

[0028] The above operation is repeated until all shelf images in the shelf image set are traversed to obtain a standard shelf image set corresponding to the target warehouse.

[0029] Preferably, the isolated point judgment rule includes a first isolated point judgment formula, a second isolated point judgment formula, a third isolated point judgment formula and a fourth isolated point judgment formula, and the first isolated point judgment formula is as follows:

[0030] B(P0)=1;

[0031] Wherein, P0 represents the current pixel point, and B(P0) represents the pixel value of the current pixel point;

[0032] The second isolated point judgment formula is as follows:

[0033]

[0034] Where n represents the serial number of the pixel adjacent to the current pixel, N4 represents the first digital sequence, and N4 = {1, 2, 3, 4}, a 2n-1 Indicates whether the pixel value of the adjacent pixel numbered 2n-1 is 0, and

[0035] The third isolated point judgment formula is as follows:

[0036]

[0037] Wherein, N8 represents a second digital sequence, and N8={1,2,...,8};

[0038] The fourth isolated point determination formula is as follows:

[0039]

[0040] Among them, C n Indicates whether the pixel value of the adjacent pixel with sequence number n is 1, and

[0041] Preferably, feature extraction is performed on each target shelf image in the target shelf image set corresponding to the standard shelf image set to obtain an image feature set corresponding to the target shelf image set, including:

[0042] Extracting the pixel area of ​​the target shelf image to obtain a first image feature corresponding to the target shelf image;

[0043] Calculating the joint distribution of the grayscale values ​​of pixels in the target shelf image, and extracting the texture feature of the target shelf image based on the joint distribution of the grayscale values ​​of pixels in the target shelf image to obtain a second image feature corresponding to the target shelf image;

[0044] Calculating the image gradient of the target shelf image to obtain edge features of the target shelf image in the horizontal and vertical directions, and using the edge features of the target shelf image in the horizontal and vertical directions as the third image features of the target shelf image;

[0045] Traverse each pixel of the target shelf image, check whether the pixel value of each pixel is 1 and the mark of the pixel is not in the mark array, wherein the mark array is used to record the mark of the pixel whose pixel value is 1 of the traversed pixel, if the pixel currently traversed is in the mark array, traverse the neighborhood within a preset range of the pixel currently traversed, and record the mark of the pixel whose pixel value is 0 in the neighborhood within the preset range in the connected array, repeat the above operation until each pixel in the target shelf image is traversed to obtain multiple connected arrays, compare the number of elements in the multiple connected arrays with a preset threshold to obtain connected arrays greater than the preset threshold, count the number of connected arrays greater than the preset threshold, compare the number of connected arrays greater than the preset threshold with the preset number threshold to determine whether the target shelf image has connectivity, and use whether the target shelf image has connectivity as the fourth image feature of the target shelf image;

[0046] The first image feature, the second image feature, the third image feature and the fourth image feature corresponding to the target shelf image are combined to obtain an image feature set corresponding to the target shelf image set.

[0047] Preferably, the shelf classification model adopts an improved support vector machine, and optimizes the parameters of the support vector machine based on a particle swarm optimization algorithm to obtain an improved support vector machine, wherein the parameters of the support vector machine include a penalty coefficient and a kernel function, wherein the parameters of the support vector machine are optimized based on a particle swarm optimization algorithm to obtain an improved support vector machine, including:

[0048] Initializing the parameters of the particle swarm optimization algorithm, wherein the parameters of the particle swarm optimization algorithm include the population size N, the problem dimension D, the maximum number of iterations T, the upper bound ub of the search space, and the lower bound lb of the search space;

[0049] Generate an initial particle population in the search space, calculate the fitness value of each individual particle in the particle population, and sort the fitness value of each individual particle in the particle population to update the best fitness value bF and the worst fitness value wF under the current number of iterations;

[0050] The position of each individual particle in the particle population is disturbed, and the fitness value of the individual particle is recalculated. The disturbance formula is as follows:

[0051]

[0052] Among them, X(t+1) represents the position of the individual particle at the t+1th iteration, X b (t) represents the position of the best individual at the tth iteration, X A (t) and X B (t) represents two individuals randomly selected from the particle population at the tth iteration, vb represents a random number in [-a, a], a represents a dynamically updated parameter, and vc represents a parameter that decreases linearly from 1 to 0 as the number of iterations increases, r represents a random number in [0,1], p represents a dynamic threshold, and p = tanh(S(i)-DF), S(i) represents the fitness value of the individual particle, DF represents the currently obtained optimal fitness value of the individual particle, W represents the dynamic weight of the individual particle, and

[0053] Repeat the above steps until the maximum number of iterations is reached to output the optimal individual particle position, which is the optimal combination of the parameters of the support vector machine. The support vector machine is improved based on the optimal combination of the parameters of the support vector machine to obtain an improved support vector machine.

[0054] Preferably, the image recognition model adopts Fast R-CNN.

[0055] Preferably, the objective function for the shortest allocation time is as follows:

[0056]

[0057] Among them, T ST represents the objective function of the shortest distribution time, that is, the total time for the last delivery device to complete the distribution task, T o It represents the time to complete the task of delivering the o-th delivery device, T p and T r It represents the total time for the delivery equipment to complete the picking and distribution tasks, n t Indicates the total number of delivered devices;

[0058] The constraints are as follows:

[0059]

[0060] in, Indicates shelf Z i Is it placed in V l Location, Indicates shelf Z i Whether it is delivered by R o allocation, m represents the number of free shelves, lg represents the total number of available positions on the free shelves, V l Indicates the placement position of the lth shelf in the storage area, It means that when the o-th delivery equipment performs the i-th assigned task, the picking work area P k Waiting time in queue, Indicates that the oth delivery device performs the ith assigned task and leaves the picking work area P k Entrance time, It indicates the time when the o-th delivery equipment arrives at the entrance of the replenishment workspace after executing the i-th assigned task.

[0061] The beneficial effects of the present invention are as follows: (1) The present invention uses high-definition cameras and automated image processing technology to enable the system to quickly and accurately complete image acquisition, feature extraction and classification of shelves. This automated process greatly reduces the time and labor costs required for manual inventory counting. In addition, by using advanced image recognition and classification algorithms, the system can identify the status of goods and shelves with high accuracy, reduce human errors, ensure the accuracy of inventory data, make dynamic inventory tables more reliable, and timely update inventory information based on the results of real-time image recognition, so that warehouse managers can instantly grasp the inventory status and optimize replenishment and inventory management; (2) The present invention analyzes and displays lights on shelves without goods, so that managers can intuitively understand which shelves need to reallocate goods. This visual feedback helps improve the space utilization and management efficiency of the warehouse, and uses a linear programming algorithm to intelligently allocate shelf resources through the shelf allocation unit, thereby achieving an optimal balance between allocation efficiency and shelf stability. This intelligent decision support system can reduce resource waste and improve overall operational efficiency. In addition, through the high degree of integration between the various units, it ensures the smooth and real-time nature of information flow, making the overall system operation more coordinated, which helps to further improve the automation level of warehouse management. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of the system architecture of an overall system in an embodiment of the present invention.

[0063] Figure numerals: 1. Shelf coding unit; 2. Inventory matching unit; 3. Image acquisition unit; 4. Image preprocessing unit; 5. Image segmentation unit; 6. Feature extraction unit; 7. Shelf classification unit; 8. Goods identification unit; 9. Shelf allocation unit. DETAILED DESCRIPTION

[0064] Embodiment 1, as Figure 1 As shown, the present invention proposes an integrated visual recognition warehouse automated inventory system, comprising an inventory matching unit 2, a feature extraction unit 6, a shelf classification unit 7, a goods identification unit 8 and a shelf allocation unit 9, characterized in that:

[0065] The shelf coding unit 1 is used to code multiple shelves in the target warehouse to obtain a shelf coding set corresponding to the target warehouse, wherein the shelf coding set includes the number of shelf rows, the number of shelf columns and the number of shelf layers, and transmits the shelf coding set corresponding to the target warehouse to the inventory matching unit 2;

[0066] The image acquisition unit 3 is used to acquire images of multiple shelves in the target warehouse through a high-definition camera at a fixed position to obtain a shelf image set corresponding to the target warehouse, and transmit the shelf image set corresponding to the target warehouse to the image preprocessing unit 4 and the goods identification unit 8;

[0067] The image preprocessing unit 4 receives the shelf image set corresponding to the target warehouse transmitted by the image acquisition unit 3, performs image preprocessing on the shelf image set corresponding to the target warehouse to obtain a standard shelf image set corresponding to the target warehouse, and transmits the standard shelf image set corresponding to the target warehouse to the image segmentation unit 5;

[0068] The image segmentation unit 5 receives the standard shelf image set corresponding to the target warehouse transmitted by the image preprocessing unit 4, and performs shelf image segmentation on the standard shelf image set to obtain a target shelf image set corresponding to the standard shelf image set, and transmits the target shelf image set corresponding to the standard shelf image set to the feature extraction unit 6.

[0069] In an optional embodiment, the feature extraction unit 6 receives the target shelf image set corresponding to the standard shelf image set transmitted by the image segmentation unit 5, and performs feature extraction on each target shelf image in the target shelf image set corresponding to the standard shelf image set to obtain an image feature set corresponding to the target shelf image set, and transmits the image feature set corresponding to the target shelf image set to the shelf classification unit 7.

[0070] In an optional embodiment, the shelf classification unit 7 receives the image feature set corresponding to the target shelf image set transmitted by the feature extraction unit 6, and classifies the image feature set corresponding to the target shelf image set based on a trained shelf classification model to obtain the shelf type corresponding to the image feature set, wherein the shelf type includes shelf types with goods and shelf types without goods, and transmits the shelf type corresponding to the image feature set to the goods identification unit 8.

[0071] In an optional embodiment, the cargo identification unit 8 receives the shelf image set corresponding to the target warehouse transmitted by the image acquisition unit 3 and the shelf type corresponding to the image feature set transmitted by the shelf classification unit 7, obtains a shelf image with a shelf type corresponding to the image feature set having goods based on the shelf image set corresponding to the target warehouse, performs image recognition on the shelf image with a shelf type having goods based on a trained image recognition model to obtain cargo information corresponding to the shelf image with a shelf type having goods, updates the inventory table pre-stored in the database based on the cargo information corresponding to the shelf image with a shelf type having goods and the shelf image with a shelf type having no goods to obtain a dynamic inventory table corresponding to the target warehouse, and transmits the dynamic inventory table corresponding to the target warehouse to the inventory matching unit 2.

[0072] In an optional embodiment, the inventory matching unit 2 receives the shelf coding set corresponding to the target warehouse transmitted by the shelf coding unit 1 and the dynamic inventory table corresponding to the target warehouse transmitted by the goods identification unit 8, and matches the shelf coding set corresponding to the target warehouse based on the dynamic inventory table corresponding to the target warehouse to obtain the shelf corresponding to the dynamic inventory table, displays lights on the shelf corresponding to the dynamic inventory table that has no goods, and transmits the shelf information corresponding to the dynamic inventory table that has no goods to the shelf allocation unit 9.

[0073] In an optional embodiment, the shelf allocation unit 9 receives the shelf information without goods corresponding to the dynamic inventory table transmitted by the inventory matching unit 2, and constructs an objective function of shortest allocation time based on the shelf information without goods corresponding to the dynamic inventory table, constructs constraints based on the remaining free storage spaces on the shelves, and solves the objective function of shortest allocation time through a linear programming algorithm based on the constraints to obtain an allocation plan corresponding to the shelf information without goods corresponding to the dynamic inventory table.

[0074] It should be noted that the linear programming algorithm is a mathematical method used to solve linear optimization problems. Its purpose is to maximize or minimize a linear objective function under a given set of linear constraints.

[0075] Embodiment 2, a warehouse automated inventory system integrating visual recognition proposed by the present invention, compared with embodiment 1, this embodiment further includes: performing shelf image segmentation on the standard shelf image set to obtain a target shelf image set corresponding to the standard shelf image set, including:

[0076] Get the minimum and maximum grayscale values ​​of the standard shelf image, and calculate the first segmentation threshold based on the minimum and maximum grayscale values. The calculation formula of the first segmentation threshold is as follows:

[0077]

[0078] Among them, T0 represents the first segmentation threshold, Z min Represents the minimum gray value, Z max Indicates the maximum gray value;

[0079] Segmenting the standard shelf image into a first target shelf image and a first background image based on a first segmentation threshold;

[0080] The average grayscale value of the first target shelf image and the average grayscale value of the first background image are obtained, and the second segmentation threshold is calculated based on the average grayscale value of the first target shelf image and the average grayscale value of the first background image. The second segmentation threshold calculation formula is as follows:

[0081]

[0082] Among them, T K represents the second segmentation threshold, H1 represents the average gray value of the first target standard shelf image, and H2 represents the average gray value of the first background standard shelf image;

[0083] The first segmentation threshold is compared with the second segmentation threshold. If the first segmentation threshold is equal to the second segmentation threshold, the standard shelf image is segmented into a second target standard shelf image based on the second segmentation threshold. The second target standard shelf image is the final target standard shelf image. Otherwise, the second segmentation threshold is assigned to the first segmentation threshold, and the above operation is repeated.

[0084] In an optional embodiment, image preprocessing is performed on the shelf image set corresponding to the target warehouse to obtain a standard shelf image set corresponding to the target warehouse, including:

[0085] Binarize the shelf image to obtain a shelf binary image;

[0086] Traverse each pixel point in the shelf binary image, and when traversing to the current pixel point, determine whether the current pixel point is an isolated point by using the preset isolated point judgment rule;

[0087] If the current pixel is an isolated point, the current pixel is deleted; otherwise, the next pixel of the current pixel is traversed until each pixel in the shelf binary image is traversed to obtain a standard shelf image.

[0088] Repeat the above operation until all shelf images in the shelf image set are traversed to obtain the standard shelf image set corresponding to the target warehouse.

[0089] In an optional embodiment, the isolated point judgment rule includes a first isolated point judgment formula, a second isolated point judgment formula, a third isolated point judgment formula and a fourth isolated point judgment formula, and the first isolated point judgment formula is as follows:

[0090] B(P0)=1;

[0091] Wherein, P0 represents the current pixel point, and B(P0) represents the pixel value of the current pixel point;

[0092] The second isolated point judgment formula is as follows:

[0093]

[0094] Where n represents the serial number of the pixel adjacent to the current pixel, N4 represents the first digital sequence, and N4 = {1, 2, 3, 4}, a 2n-1 Indicates whether the pixel value of the adjacent pixel numbered 2n-1 is 0, and

[0095] The third isolated point judgment formula is as follows:

[0096]

[0097] Wherein, N8 represents a second digital sequence, and N8={1,2,...,8};

[0098] The fourth isolated point judgment formula is as follows:

[0099]

[0100] Among them, C n Indicates whether the pixel value of the adjacent pixel with sequence number n is 1, and

[0101] In an optional embodiment, feature extraction is performed on each target shelf image in the target shelf image set corresponding to the standard shelf image set to obtain an image feature set corresponding to the target shelf image set, including:

[0102] Extracting the pixel area of ​​the target shelf image to obtain a first image feature corresponding to the target shelf image;

[0103] Calculating the joint distribution of the grayscale values ​​of pixels in the target shelf image, and extracting the texture features of the target shelf image based on the joint distribution of the grayscale values ​​of pixels in the target shelf image, so as to obtain a second image feature corresponding to the target shelf image;

[0104] Calculating the image gradient of the target shelf image to obtain edge features in the horizontal and vertical directions of the target shelf image, and using the edge features in the horizontal and vertical directions of the target shelf image as the third image features of the target shelf image;

[0105] Traverse each pixel of the target shelf image, check whether the pixel value of each pixel is 1 and the mark of the pixel is not in the mark array, wherein the mark array is used to record the mark of the pixel whose pixel value is 1 of the traversed pixel, if the currently traversed pixel is in the mark array, traverse the neighborhood within a preset range of the currently traversed pixel, and record the mark of the pixel whose pixel value is 0 in the neighborhood within the preset range in the connected array, repeat the above operation until each pixel in the target shelf image is traversed to obtain multiple connected arrays, compare the number of elements in the multiple connected arrays with a preset threshold to obtain a connected array greater than the preset threshold, count the number of connected arrays greater than the preset threshold, compare the number of connected arrays greater than the preset threshold with the preset number threshold to determine whether the target shelf image has connectivity, and use whether the target shelf image has connectivity as the fourth image feature of the target shelf image;

[0106] The first image feature, the second image feature, the third image feature and the fourth image feature corresponding to the target shelf image are combined to obtain an image feature set corresponding to the target shelf image set.

[0107] It should be noted that pixel area refers to the number of pixels contained in a specific area in the image, such as the target shelf. In a binary image, the target is usually represented by an area with a pixel value of 1; the joint distribution of pixel gray values ​​refers to the probability distribution between the gray values ​​of two or more variables, which are pixels in this case, indicating the possibility of their simultaneous occurrence. It is usually used to analyze the relationship between different pixel gray values ​​in an image.

[0108] In an optional embodiment, the shelf classification model adopts an improved support vector machine, and optimizes the parameters of the support vector machine based on a particle swarm optimization algorithm to obtain an improved support vector machine, wherein the parameters of the support vector machine include a penalty coefficient and a kernel function, wherein the parameters of the support vector machine are optimized based on a particle swarm optimization algorithm to obtain an improved support vector machine, including:

[0109] Initialize the parameters of the particle swarm optimization algorithm, where the parameters of the particle swarm optimization algorithm include the population size N, the problem dimension D, the maximum number of iterations T, the upper bound ub of the search space, and the lower bound lb of the search space;

[0110] Generate an initial particle population in the search space, calculate the fitness value of each individual particle in the particle population, and sort the fitness value of each individual particle in the particle population to update the best fitness value bF and the worst fitness value wF under the current number of iterations;

[0111] The position of each individual particle in the particle population is disturbed, and the fitness value of the individual particle is recalculated. The disturbance formula is as follows:

[0112]

[0113] Among them, X(t+1) represents the position of the individual particle at the t+1th iteration, X b (t) represents the position of the best individual at the tth iteration, X A (t) and X B (t) represents two individuals randomly selected from the particle population at the tth iteration, vb represents a random number in [-a, a], a represents a dynamically updated parameter, and vc represents a parameter that decreases linearly from 1 to 0 as the number of iterations increases, r represents a random number in [0,1], p represents a dynamic threshold, and p = tanh(S(i)-DF), S(i) represents the fitness value of the individual particle, DF represents the currently obtained optimal fitness value of the individual particle, W represents the dynamic weight of the individual particle, and

[0114] Repeat the above steps until the maximum number of iterations is reached to output the optimal individual particle position, which is the optimal combination of support vector machine parameters. The support vector machine is improved based on the optimal combination of support vector machine parameters to obtain an improved support vector machine.

[0115] It should be noted that the support vector machine is a supervised learning model for classification and regression analysis. It separates data points of different categories by constructing an optimal hyperplane. In the support vector machine, the penalty coefficient is used to control the model's tolerance for classification errors. A larger C value will make the model pay more attention to the training data, which may lead to overfitting, while a smaller C value allows more misclassifications, which may lead to underfitting. The kernel function is a tool that maps the input space to a high-dimensional feature space, making it linearly separable in the high-dimensional space. The particle swarm optimization algorithm is an optimization algorithm based on swarm intelligence. It finds the optimal solution by simulating the foraging behavior of bird flocks. Each "particle" represents a potential solution, and the particle gradually approaches the optimal solution by updating its position and velocity.

[0116] In an optional embodiment, the image recognition model adopts Fast R-CNN.

[0117] It should be noted that Fast R-CNN is a deep learning model for target detection, which aims to improve detection speed and accuracy.

[0118] In an optional embodiment, the objective function for minimizing the allocation time is as follows:

[0119] T ST=min(max(T o =T p +T r ,o=1,2,...,n t ));

[0120] Among them, T ST represents the objective function of the shortest distribution time, that is, the total time for the last delivery device to complete the distribution task, T o It represents the time to complete the task of delivering the o-th delivery device, T p and T r It represents the total time for the delivery equipment to complete the picking and distribution tasks, n t Indicates the total number of delivered devices;

[0121] The constraints are as follows:

[0122]

[0123] in, Indicates shelf Z i Is it placed in V l Location, Indicates shelf Z i Whether it is delivered by R o allocation, m represents the number of free shelves, lg represents the total number of available positions on the free shelves, V l Indicates the placement position of the lth shelf in the storage area, It means that when the o-th delivery equipment performs the i-th assigned task, the picking work area P k Waiting time in queue, Indicates that the oth delivery device performs the ith assigned task and leaves the picking work area P k Entrance time, It indicates the time when the o-th delivery equipment arrives at the entrance of the replenishment workspace after executing the i-th assigned task.

[0124] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto, and various changes can be made within the knowledge scope of technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A warehouse automated inventory system integrating visual recognition, comprising an inventory matching unit (2), a feature extraction unit (6), a shelf classification unit (7), a goods identification unit (8) and a shelf allocation unit (9), characterized in that: A shelf coding unit (1), the shelf coding unit (1) is used to code a plurality of shelves of a target warehouse to obtain a shelf coding set corresponding to the target warehouse, wherein the shelf coding set includes the number of shelf rows, the number of shelf columns and the number of shelf layers, and transmit the shelf coding set corresponding to the target warehouse to an inventory matching unit (2); An image acquisition unit (3), the image acquisition unit (3) being used to acquire images of a plurality of shelves of the target warehouse by means of a high-definition camera at a fixed position, so as to obtain a shelf image set corresponding to the target warehouse, and transmit the shelf image set corresponding to the target warehouse to an image preprocessing unit (4) and a goods identification unit (8); An image preprocessing unit (4), the image preprocessing unit (4) receives the shelf image set corresponding to the target warehouse transmitted by the image acquisition unit (3), performs image preprocessing on the shelf image set corresponding to the target warehouse to obtain a standard shelf image set corresponding to the target warehouse, and transmits the standard shelf image set corresponding to the target warehouse to the image segmentation unit (5); An image segmentation unit (5) receives the standard shelf image set corresponding to the target warehouse transmitted by the image preprocessing unit (4), performs shelf image segmentation on the standard shelf image set to obtain a target shelf image set corresponding to the standard shelf image set, and transmits the target shelf image set corresponding to the standard shelf image set to a feature extraction unit (6).

2. The warehouse automated inventory counting system with integrated visual recognition according to claim 1, characterized in that: The feature extraction unit (6) receives the target shelf image set corresponding to the standard shelf image set transmitted by the image segmentation unit (5), and extracts features from each target shelf image in the target shelf image set corresponding to the standard shelf image set to obtain an image feature set corresponding to the target shelf image set, and transmits the image feature set corresponding to the target shelf image set to the shelf classification unit (7); The shelf classification unit (7) receives the image feature set corresponding to the target shelf image set transmitted by the feature extraction unit (6), and classifies the image feature set corresponding to the target shelf image set based on a trained shelf classification model to obtain the shelf type corresponding to the image feature set, wherein the shelf type includes a shelf with goods and a shelf without goods, and transmits the shelf type corresponding to the image feature set to the goods identification unit (8).

3. The warehouse automated inventory counting system with integrated visual recognition according to claim 2, characterized in that: The goods identification unit (8) receives the shelf image set corresponding to the target warehouse transmitted by the image acquisition unit (3) and the shelf type corresponding to the image feature set transmitted by the shelf classification unit (7), obtains a shelf image of a shelf type with goods corresponding to the image feature set based on the shelf image set corresponding to the target warehouse, performs image recognition on the shelf image of the shelf type with goods based on a trained image recognition model to obtain goods information corresponding to the shelf image of the shelf type with goods, updates the inventory table pre-stored in the database based on the goods information corresponding to the shelf image of the shelf type with goods and the shelf image of the shelf type without goods, to obtain a dynamic inventory table corresponding to the target warehouse, and transmits the dynamic inventory table corresponding to the target warehouse to the inventory matching unit (2); The inventory matching unit (2) receives the shelf code set corresponding to the target warehouse transmitted by the shelf coding unit (1) and the dynamic inventory table corresponding to the target warehouse transmitted by the goods identification unit (8), and matches the shelf code set corresponding to the target warehouse based on the dynamic inventory table corresponding to the target warehouse to obtain the shelf corresponding to the dynamic inventory table, displays light on the shelf corresponding to the dynamic inventory table that does not store goods, and transmits the information of the shelf corresponding to the dynamic inventory table that does not store goods to the shelf allocation unit (9); The shelf allocation unit (9) receives the shelf information without goods corresponding to the dynamic inventory table transmitted by the inventory matching unit (2), and constructs an objective function with the shortest allocation time based on the shelf information without goods corresponding to the dynamic inventory table, constructs constraint conditions based on the remaining free storage spaces on the shelf, and solves the objective function with the shortest allocation time through a linear programming algorithm based on the constraint conditions to obtain an allocation plan corresponding to the shelf information without goods corresponding to the dynamic inventory table.

4. The warehouse automated inventory counting system integrated with visual recognition according to claim 1, characterized in that: Performing shelf image segmentation on the standard shelf image set to obtain a target shelf image set corresponding to the standard shelf image set includes: The minimum and maximum grayscale values ​​of the standard shelf image are obtained, and a first segmentation threshold is calculated based on the minimum and maximum grayscale values. The calculation formula of the first segmentation threshold is as follows: Among them, T0 represents the first segmentation threshold, Z min Represents the minimum gray value, Z max Indicates the maximum gray value; Segmenting the standard shelf image into a first target shelf image and a first background image based on the first segmentation threshold; The average grayscale value of the first target shelf image and the average grayscale value of the first background image are obtained, and a second segmentation threshold is calculated based on the average grayscale value of the first target shelf image and the average grayscale value of the first background image. The second segmentation threshold calculation formula is as follows: Among them, T K represents the second segmentation threshold, H1 represents the average gray value of the first target standard shelf image, and H2 represents the average gray value of the first background standard shelf image; The first segmentation threshold is compared with the second segmentation threshold. If the first segmentation threshold is equal to the second segmentation threshold, the standard shelf image is segmented into a second target standard shelf image based on the second segmentation threshold. The second target standard shelf image is the final target standard shelf image. Otherwise, the second segmentation threshold is assigned to the first segmentation threshold, and the above operation is repeated.

5. The warehouse automated inventory counting system integrated with visual recognition according to claim 4, characterized in that: Performing image preprocessing on the shelf image set corresponding to the target warehouse to obtain a standard shelf image set corresponding to the target warehouse, including: Binarizing the shelf image to obtain a shelf binary image; Traversing each pixel point in the binary image of the shelf, when traversing to the current pixel point, judging whether the current pixel point is an isolated point by using a preset isolated point judgment rule; If the current pixel is an isolated point, the current pixel is deleted; otherwise, the next pixel of the current pixel is traversed until each pixel in the shelf binary image is traversed to obtain a standard shelf image; The above operation is repeated until all shelf images in the shelf image set are traversed to obtain a standard shelf image set corresponding to the target warehouse.

6. The warehouse automated inventory counting system integrated with visual recognition according to claim 5, characterized in that: The isolated point judgment rule includes a first isolated point judgment formula, a second isolated point judgment formula, a third isolated point judgment formula and a fourth isolated point judgment formula, and the first isolated point judgment formula is as follows: B(P0)=1; Wherein, P0 represents the current pixel point, and B(P0) represents the pixel value of the current pixel point; The second isolated point judgment formula is as follows: Where n represents the serial number of the pixel adjacent to the current pixel, N4 represents the first digital sequence, and N4 = {1, 2, 3, 4}, a 2n-1 Indicates whether the pixel value of the adjacent pixel numbered 2n-1 is 0, and The third isolated point judgment formula is as follows: Wherein, N8 represents a second digital sequence, and N8={1,2,...,8}; The fourth isolated point judgment formula is as follows: Among them, C n Indicates whether the pixel value of the adjacent pixel with sequence number n is 1, and 7. The warehouse automated inventory counting system integrated with visual recognition according to claim 2, characterized in that: Performing feature extraction on each target shelf image in the target shelf image set corresponding to the standard shelf image set to obtain an image feature set corresponding to the target shelf image set, including: Extracting the pixel area of ​​the target shelf image to obtain a first image feature corresponding to the target shelf image; Calculating the joint distribution of the grayscale values ​​of pixels in the target shelf image, and extracting the texture feature of the target shelf image based on the joint distribution of the grayscale values ​​of pixels in the target shelf image to obtain a second image feature corresponding to the target shelf image; Calculating the image gradient of the target shelf image to obtain edge features of the target shelf image in the horizontal and vertical directions, and using the edge features of the target shelf image in the horizontal and vertical directions as the third image features of the target shelf image; Traverse each pixel of the target shelf image, check whether the pixel value of each pixel is 1 and the mark of the pixel is not in the mark array, wherein the mark array is used to record the mark of the pixel whose pixel value is 1 of the traversed pixel, if the pixel currently traversed is in the mark array, traverse the neighborhood within a preset range of the pixel currently traversed, and record the mark of the pixel whose pixel value is 0 in the neighborhood within the preset range in the connected array, repeat the above operation until each pixel in the target shelf image is traversed to obtain multiple connected arrays, compare the number of elements in the multiple connected arrays with a preset threshold to obtain connected arrays greater than the preset threshold, count the number of connected arrays greater than the preset threshold, compare the number of connected arrays greater than the preset threshold with the preset number threshold to determine whether the target shelf image has connectivity, and use whether the target shelf image has connectivity as the fourth image feature of the target shelf image; The first image feature, the second image feature, the third image feature and the fourth image feature corresponding to the target shelf image are combined to obtain an image feature set corresponding to the target shelf image set.

8. The warehouse automated inventory counting system integrated with visual recognition according to claim 2, characterized in that: The shelf classification model adopts an improved support vector machine, and optimizes the parameters of the support vector machine based on a particle swarm optimization algorithm to obtain an improved support vector machine, wherein the parameters of the support vector machine include a penalty coefficient and a kernel function, wherein the parameters of the support vector machine are optimized based on a particle swarm optimization algorithm to obtain an improved support vector machine, including: Initializing the parameters of the particle swarm optimization algorithm, wherein the parameters of the particle swarm optimization algorithm include the population size N, the problem dimension D, the maximum number of iterations T, the upper bound ub of the search space, and the lower bound lb of the search space; Generate an initial particle population in the search space, calculate the fitness value of each individual particle in the particle population, and sort the fitness value of each individual particle in the particle population to update the best fitness value bF and the worst fitness value wF under the current number of iterations; The position of each individual particle in the particle population is disturbed, and the fitness value of the individual particle is recalculated. The disturbance formula is as follows: Among them, X(t+1) represents the position of the individual particle at the t+1th iteration, X b (t) represents the position of the best individual at the tth iteration, X A (t) and X B (t) represents two individuals randomly selected from the particle population at the tth iteration, vb represents a random number in [-a, a], a represents a dynamically updated parameter, and vc represents a parameter that decreases linearly from 1 to 0 as the number of iterations increases, r represents a random number in [0,1], p represents a dynamic threshold, and p = tanh(S(i)-DF), S(i) represents the fitness value of the individual particle, DF represents the currently obtained optimal fitness value of the individual particle, W represents the dynamic weight of the individual particle, and Repeat the above steps until the maximum number of iterations is reached to output the optimal individual particle position, which is the optimal combination of the parameters of the support vector machine. The support vector machine is improved based on the optimal combination of the parameters of the support vector machine to obtain an improved support vector machine.

9. The warehouse automated inventory counting system integrated with visual recognition according to claim 3, characterized in that: The image recognition model adopts Fast R-CNN.

10. The warehouse automated inventory counting system integrated with visual recognition according to claim 3, characterized in that: The objective function for minimizing the allocation time is as follows: T ST =min(max(T o =T p +T r ,o=1,2,..,n t )); Among them, T ST represents the objective function of the shortest distribution time, that is, the total time for the last delivery device to complete the distribution task, T o It represents the time to complete the task of delivering the o-th delivery device, T p and T r It represents the total time for the delivery equipment to complete the picking and distribution tasks, n t Indicates the total number of delivered devices; The constraints are as follows: in, Indicates shelf Z i Is it placed in V l Location, Indicates shelf Z i Whether it is delivered by R o allocation, m represents the number of free shelves, lg represents the total number of available positions on the free shelves, V l Indicates the placement position of the lth shelf in the storage area, It means that when the o-th delivery equipment performs the u-th distribution task, the picking work area P k Waiting time in queue, Indicates that the i-th delivery device performs the i-th assigned task and leaves the picking work area P k Entrance time, It indicates the time when the o-th delivery equipment arrives at the entrance of the replenishment workspace after executing the i-th assigned task.

Images