Machine vision-based new-generation intelligent inventory checking station inventory checking method

Abstract

The invention relates to the technical field of logistics equipment, in particular to a new-generation intelligent inventory checking station inventory checking method based on machine vision, and the method comprises the following steps: carrying out the morphological operation of a first azimuth picture set of each container, inputting the second orientation picture set of each cargo box after the second preprocessing and the morphological operation into a cargo box identification identification model, and calculating the whole stack prediction volume V1 of the cargos at the inventory taking station based on the identified cargo categories, the number of the cargo boxes loaded by the different categories of cargos and the accommodating volumes of the cargo boxes corresponding to the different categories of cargos; whether the whole stack of goods on the tray is in a complete shape or not is judged, and if the whole stack of goods on the tray is in the complete shape, the volume V2 of the whole stack of goods at the inventory taking station is calculated based on the pictures of the whole stack of goods stacked on the tray in different directions and the pixel proportion of the pictures of the whole stack of goods; on the basis of a comparison result between the volume V2 of the whole stack of the goods and the predicted volume V1 of the whole stack of the goods at the inventory taking station, the identified goods categories, the number of the containers loaded with the different categories of the goods and the accommodating volumes of the containers corresponding to the different categories of the goods, the actual categories and the actual number of the goods at the inventory taking station are determined; based on a comparison result of the determined actual category and the actual number of the goods at the inventory taking station and the actual category and the actual number of the goods stacked on the tray obtained after the RFID is read, the actual category and the actual number of the goods at the inventory taking station are rechecked for the first time; therefore, whether the goods stacked on the trays at the inventory taking station are missing or not can be judged.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of logistics equipment, in particular to a new generation of intelligent warehouse inventory station inventory method based on machine vision. BACKGROUND

[0002] In the production process of a tobacco factory, the goods are usually transported to a designated station by an AGV trolley, and then the goods are transported to the designated station. After the goods are transported to the designated station, the goods need to be inventoried. The existing inventory method for the goods stacked on the pallet mainly relies on manual inventory of the goods. Although this inventory method can inventory the stacked goods, the manual inventory method of the goods consumes a large amount of manpower and has low inventory efficiency.

[0003] Therefore, it is necessary to improve the traditional warehouse inventory method to solve the problem of consuming a large amount of manpower and low inventory efficiency caused by the manual inventory method of the goods.

[0004] SUMMARY

[0005] The present application relates to the technical field of logistics equipment, in particular to a new generation of intelligent warehouse inventory station inventory method based on machine vision.

[0006] To solve the above technical problems, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a new generation of intelligent warehouse inventory station inventory method based on machine vision, comprising the following steps:

[0008] S1, obtain different orientation whole stack photos of the goods stacked on the pallet at the warehouse inventory station, perform box boundary contour extraction and segmentation processing on the goods boxes in the whole stack photos after the first preprocessing, to obtain a plurality of orientation pictures containing a single box; wherein the plurality of orientation pictures containing a single box include a front view of each box and a top view of each box;

[0009] S2, find out the front view and top view of the box corresponding to the same type of goods from the plurality of orientation pictures containing a single box, and divide the front view and top view of the same box corresponding to the same type of goods into a group, to obtain a set of orientation pictures of each box, and perform morphological operation on the front view and top view of the box in the set of orientation pictures of each box, to obtain a set of orientation pictures of each box.

[0010] S3, input the second set of orientation pictures of each container after the second pre-processing into the pre-trained container identification recognition model to identify the goods category, the number of containers for different categories of goods, and the volume of the containers corresponding to different categories of goods, and calculate the predicted volume V1 of the goods at the inventory station based on the number of containers for different categories of goods and the volume of the containers corresponding to different categories of goods; wherein the container identification recognition model is constructed based on a deep learning model; the second set of orientation pictures of each container includes a front view of the container and a top view of the container, the front view of the container has a barcode associated with the goods category information and a specific identification word for each category of goods, and the top view of the container has a specific outer packaging diagram for each category of goods;

[0011] S4, obtain different orientation whole stack photos of the goods stacked on the pallet at the inventory station, and determine whether the whole stack of goods on the pallet is in a complete shape based on the different orientation whole stack photos of the goods stacked on the pallet; if the whole stack of goods on the pallet is in a complete shape, calculate the volume V2 of the whole stack of goods at the inventory station based on the different orientation whole stack photos of the goods stacked on the pallet and the pixel ratio of the whole stack photo; wherein the complete shape is a cuboid or a cube;

[0012] S5, based on the comparison result between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the inventory station, and the identified goods category, the number of containers for different categories of goods, and the volume of the containers corresponding to different categories of goods, determine the actual category and actual quantity of the goods at the inventory station;

[0013] S6, read the RFID electronic tag on the pallet of the stacked goods at the inventory station, and based on the comparison result between the real category and real quantity of the goods stacked on the pallet obtained indirectly after reading the RFID electronic tag and the actual category and actual quantity of the goods at the inventory station determined in S5, perform a first review on the actual category and actual quantity of the goods at the inventory station to determine whether the goods stacked on the pallet at the inventory station are missing; wherein the RFID is arranged at the bottom of the pallet, and the RFID electronic tag stores the information of the pallet code; the real category and real quantity of the goods stacked on the pallet obtained after reading the RFID electronic tag are the real category and real quantity of the goods corresponding to the pallet in the WMS system based on the read pallet code information.

[0014] Preferably, S1, obtain different orientation whole stack photos of the goods stacked on the pallet at the inventory station, and perform container boundary contour extraction and segmentation processing on the containers used for packaging the goods in the first pre-processed whole stack photo to obtain a plurality of orientation pictures containing a single container; wherein the plurality of orientation pictures containing a single container include a front view of each container and a top view of each container; comprising the following steps:

[0015] S1.1, two intelligent industrial cameras arranged at the disc storage station and in different directions are used to take pictures of the goods stacks stacked on the pallets at the disc storage station to obtain whole stack pictures of the goods in different directions; wherein the whole stack pictures of the goods in different directions include a whole stack picture of the goods in a first shooting direction and a whole stack picture of the goods in a second shooting direction; the whole stack picture of the goods in the second shooting direction is a top view of the whole stack of goods, and the whole stack picture of the goods in the first shooting direction is a front view of the whole stack of goods;

[0016] S1.2, the whole stack pictures of the goods in different directions are subjected to picture gray scale preprocessing to obtain the whole stack pictures of the goods after gray scale processing, and the picture gray scale conversion formula is as follows:

[0017] Gray = 0.2989 * R + 0.5870 * G + 0.1140 * B

[0018] wherein R represents the red component of each pixel point in the image, G represents the green component of each pixel point in the image, and B represents the blue component of each pixel point in the image; and Gray represents the converted gray scale value.

[0019] S1.3, the whole stack pictures of the goods after gray scale processing are subjected to Gaussian blur preprocessing to obtain the whole stack pictures of the goods after Gaussian blur processing, and the picture Gaussian blur formula is as follows:

[0020]

[0021] wherein G(x, y) represents the convolution kernel value generated by the Gaussian function; x and y are the distances between the center of the convolution kernel and the current pixel; and sigma represents the standard deviation, which controls the blur degree.

[0022] S1.4, based on an edge detector algorithm, the edges of each container in the whole stack pictures of the goods after Gaussian blur processing are subjected to edge enhancement processing to obtain the first preprocessed whole stack pictures; wherein the first preprocessed whole stack pictures are pictures in which the edges of each container in the whole stack pictures of the goods have been identified.

[0023] S1.5, based on a contour extraction algorithm, the contour of each container in the first preprocessed whole stack pictures is extracted and segmented to obtain multiple single-container direction pictures; wherein the single-container direction picture is a picture containing only one container; and the multiple single-container direction pictures include a top view of each container and a front view of each container.

[0024] Preferably, S2 finds the front view and the top view of the same category of goods from multiple orientation images containing a single container, and classifies the front view and the top view of the same container of the same category of goods into a group to obtain an orientation image set one of each container, and performs morphological operation on the front view and the top view of the container in the orientation image set one of each container to obtain an orientation image set two of each container, including the following steps:

[0025] S2.1, finds the top view and the front view of the same category of goods from multiple orientation images containing a single container to obtain the top view and the front view of the same category of goods corresponding to each container;

[0026] S2.2, finds the front view and the top view of the same container from the top view and the front view of the same category of goods corresponding to each container, and classifies the front view and the top view of the same container into a group to obtain an orientation image set one of each container; wherein the orientation image set one of each container includes the top view and the front view of the same container;

[0027] S2.3, first performs erosion morphological operation on the top view and the front view of the container in the orientation image set one of each container respectively, and then performs inflation morphological operation on the top view and the front view of the container in the orientation image set one of each container after the erosion operation to obtain an orientation image set two of each container, and the expression of the combination operation of image erosion and image inflation is as follows:

[0028]

[0029] Wherein, A represents the foreground pixel area of the input image; B represents the structure element, that is, the shape defining the morphological operation; increase the size of the foreground area, so that the structure element B covers the edge of the foreground pixel, and fills the broken area;

[0030] Or, first perform inflation morphological operation on the top view and the front view of the container in the orientation image set one of each container respectively, and then perform erosion morphological operation on the top view and the front view of the container in the orientation image set one of each container after the inflation operation to obtain an orientation image set two of each container, and the expression of the combination operation of image inflation and image erosion operation is as follows:

[0031]

[0032] Wherein, A represents the foreground pixel area of the input image; B represents the structure element, that is, the shape defining the morphological operation; increase the size of the foreground area, so that the structure element B covers the edge of the foreground pixel, and fills the broken area.

[0033] Preferably, the S3, the second pre-processed orientation picture set of each container is input into the pre-trained container identification recognition model to identify the goods category, the number of containers loaded with different categories of goods, and the volume of containers corresponding to different categories of goods, and based on the number of containers loaded with different categories of goods and the volume of containers corresponding to different categories of goods, the inventory station predicts the volume V1 of the whole pile of goods, including the following steps:

[0034] S3.1, Gaussian filter preprocessing is performed on the top view and side view of the container in the second pre-processed orientation picture set of each container to obtain the second pre-processed orientation picture set of each container, and the expression of the Gaussian filter operation is as follows:

[0035]

[0036] Wherein, G(x,y) is the convolution kernel value generated by the Gaussian function; x, y represents the distance between the convolution kernel center and the current pixel; σ represents the standard deviation, which controls the blurring degree, and the larger the value is, the stronger the blurring is;

[0037] S3.2, histogram equalization preprocessing is performed on the top view and front view of the container in the second pre-processed orientation picture set of each container to obtain the second pre-processed orientation picture set of each container, and the expression formula of the histogram equalization operation is as follows:

[0038]

[0039] Wherein, r represents the original gray level; s represents the mapped gray level; T(r) represents the cumulative distribution function (CDF); p(r') represents the probability density function of the gray level;

[0040] Or, adaptive histogram equalization preprocessing is performed on the top view and front view of the container in the second pre-processed orientation picture set of each container to obtain the second pre-processed orientation picture set of each container;

[0041] S3.3, the second pre-processed orientation picture set of each container is input into the pre-trained container identification recognition model to identify the goods category, the number of containers loaded with different categories of goods, and the volume of containers corresponding to different categories of goods; wherein, the container identification recognition model is used to identify the bar code, text and packaging picture on the outer wall of the container; the bar code on the outer wall of the container is associated with the volume of each container and the goods category information of each container; the second orientation picture set of each container includes a front view of the container and a top view of the container, the front view of the container has a bar code associated with the goods category information and a specific identification word for each category of goods, and the top view of the container has a specific outer packaging picture for each category of goods;

[0042] S3.4, based on the number of boxes of different categories of goods and the volume of the corresponding boxes of different categories of goods, calculate the sum of the volume of each category of goods, and add the calculated sum of the volume of each category of goods in turn to obtain the predicted volume V1 of the goods pallet at the warehouse station.

[0043] Preferably, the pre-trained box identification recognition model in S3.3 is obtained by the following steps:

[0044] S3.31, use an intelligent industrial camera to take pictures of the boxes used for each category of goods to obtain a picture set of boxes used for different categories of goods; wherein the outer wall of the box used for each category of goods is provided with a bar code associated with the goods category information, a specific identification word for each category of goods, and a specific outer packaging diagram for each category of goods;

[0045] S3.32, Gaussian filter preprocessing is performed on the box pictures corresponding to different categories in the box picture set to obtain a denoising-processed box picture set one, and the expression of the Gaussian filter operation is as follows:

[0046]

[0047] Wherein, G(x,y) is the convolution kernel value generated by the Gaussian function; x, y represents the distance between the convolution kernel center and the current pixel; σ represents the standard deviation, which controls the blurring degree, the larger the value is, the stronger the blurring is;

[0048] S3.33, histogram equalization preprocessing is performed on the box pictures corresponding to different categories in the denoising-processed box picture set one to obtain a box picture set two, and the expression formula of the histogram equalization operation is as follows:

[0049]

[0050] Wherein, r represents the original gray level; s represents the mapped gray level; T(r) represents the cumulative distribution function (CDF); p(r') represents the probability density function of the gray level;

[0051] S3.34, input the box picture set two into the deep learning model for training to obtain the trained box identification recognition model.

[0052] Preferably, S4, obtain different orientation pallet photos of the goods stacked on the pallet at the warehouse station, determine whether the pallet of goods is in a complete shape based on the different orientation pallet photos of the goods stacked on the pallet, if the pallet of goods is in a complete shape, calculate the volume V2 of the pallet of goods at the warehouse station based on the different orientation pallet photos of the goods stacked on the pallet and the pixel ratio of the pallet photo; wherein the complete shape is a cuboid or a cube, including the following steps:

[0053] S4.1, use the intelligent industrial camera arranged at three different orientations of the warehouse station to take photos of the goods stack on the pallet to obtain photos of the whole stack of goods from different orientations; wherein the photos of the whole stack of goods from different orientations include front view, top view and side view of the whole stack of goods;

[0054] S4.2, based on the photos of the whole stack of goods from different orientations, determine whether the whole stack of goods at the warehouse station is in a complete shape, if the whole stack of goods is in a complete shape, calculate the volume V2 of the whole stack of goods at the warehouse station based on the length, height and width of the whole stack of goods in the photos of the whole stack of goods from different orientations and the pixel ratio of the photos of the whole stack of goods, and the volume calculation formula of the whole stack of goods is as follows:

[0055] V2=S*L*H*P

[0056] wherein V2 represents the volume of the whole stack of goods; S represents the width of the whole stack of goods; L represents the length of the whole stack of goods; and P represents the pixel ratio of the photos of the whole stack of goods.

[0057] Preferably, S4, obtaining the photos of the whole stack of goods on the pallet at the warehouse station, based on the photos of the whole stack of goods on the pallet, determining whether the whole stack of goods on the pallet is in a complete shape, if the whole stack of goods on the pallet is in a complete shape, calculating the volume V2 of the whole stack of goods at the warehouse station based on the photos of the whole stack of goods on the pallet and the pixel ratio of the photos of the whole stack of goods, further comprising the following cases:

[0058] If the whole stack of goods on the pallet is not in a complete shape, the volume V2 of the whole stack of goods is calculated by using the volume calculation algorithm of the whole stack of goods; wherein the volume calculation algorithm of the whole stack of goods is any one of voxelization method, bounding box method or three-dimensional scanning volume algorithm.

[0059] Preferably, S5, based on the comparison result between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the warehouse station, the identified goods categories in S3, the number of boxes of different categories of goods and the containing volume of the boxes corresponding to different categories of goods, the actual categories and the actual quantity of goods at the warehouse station are determined, including the following steps:

[0060] S5.1, if the volume V2 of the whole stack of goods is equal to the predicted volume V1 of the whole stack of goods at the warehouse station, the actual categories of goods on the pallet are the identified categories of goods in S3; and the actual number of boxes of different categories of goods on the pallet is equal to the number of boxes of different categories of goods identified in S3.

[0061] S5.2, if the volume V2 of the whole stack of goods is much larger than the predicted volume V1 of the whole stack of goods at the warehouse station, then the volume V2 of the whole stack of goods is subtracted from the predicted volume V1 of the whole stack of goods at the warehouse station to determine the difference between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the warehouse station;

[0062] Based on the difference between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the warehouse station, the category of the goods corresponding to the difference between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the warehouse station and the number of different categories of goods are determined according to the containing volume of the corresponding goods box of different categories of goods.

[0063] The number of different categories of goods corresponding to the difference between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the warehouse station is added to the number of different categories of goods identified in S3 to obtain the actual number of different categories of goods on the pallet.

[0064] The category of the goods corresponding to the difference between the volume V2 of the whole stack of goods and the predicted volume V1 of the whole stack of goods at the warehouse station is added to the identified category of the goods in S3 to obtain the actual category of the goods on the pallet.

[0065] Preferably, S6, reading the RFID electronic tag on the pallet of the stacked goods at the warehouse station, comparing the real category of the goods and the real number of the goods indirectly obtained after reading the RFID electronic tag with the actual category of the goods and the actual number of the goods determined in S5, and first reviewing the actual category of the goods and the actual number of the goods at the warehouse station to determine whether the goods stacked on the pallet at the warehouse station are missing, including the following steps:

[0066] S6.1, reading the RFID electronic tag at the bottom of the pallet with the RFID reader-writer at the warehouse station to obtain the code corresponding to the pallet; wherein the RFID electronic tag at the bottom of the pallet stores the information of the pallet code;

[0067] S6.2, based on the obtained code corresponding to the pallet, searching for the real category of the goods and the real number of the goods on the pallet in the WMS system to obtain the real category of the goods and the real number of the goods on the pallet;

[0068] S6.3, comparing the real category of the goods on the pallet with the actual category of the goods at the warehouse station determined in S5; if the real category of the goods on the pallet is consistent with the actual category of the goods at the warehouse station determined in S5, the category of the goods stacked on the pallet is complete; if the real category of the goods on the pallet is not consistent with the actual category of the goods at the warehouse station determined in S5, the category of the goods stacked on the pallet is missing;

[0069] S6.4, comparing the real quantity of the goods on the pallet with the actual quantity of the goods at the palletizing station determined in S5; if the real quantity of the goods on the pallet is consistent with the actual quantity of the goods at the palletizing station determined in S5, the quantity of the goods stacked on the pallet is not missing; if the real quantity of the goods on the pallet is not consistent with the actual quantity of the goods at the palletizing station determined in S5, the quantity of the goods stacked on the pallet is missing.

[0070] Preferably, the S5, reading the RFID on the pallet of the stacked goods at the palletizing station, based on the comparison result of the real category and the real quantity of the goods stacked on the pallet obtained after reading and the actual category and the actual quantity of the goods at the palletizing station determined in S5, the first review of the actual category and the actual quantity of the goods at the palletizing station is carried out to determine whether the goods stacked on the pallet at the palletizing station is missing, and further comprising:

[0071] weighing the whole stack of the goods stacked on the pallet at the palletizing station by using the ground scale to obtain the actual weight of the whole stack of the goods;

[0072] based on the comparison result of the actual weight of the whole stack of the goods and the real weight of the whole stack of the goods obtained after reading the RFID on the pallet of the stacked goods at the palletizing station, the second review of the actual category and the actual quantity of the goods at the palletizing station is carried out; wherein the RFID is arranged at the bottom of the pallet, and the RFID also stores the weight information of the goods stacked on the pallet.

[0073] The present application proposes a new generation of intelligent palletizing station inventory device based on machine vision in the second aspect, comprising:

[0074] a whole stack of goods acquisition / segmentation processing module for acquiring different orientation whole stack photos of the goods stacked on the pallet at the palletizing station, performing box boundary contour extraction and segmentation processing on the goods boxes used for the goods packaging in the whole stack photos after the first preprocessing, to obtain a plurality of orientation pictures containing a single box; wherein the plurality of orientation pictures containing a single box include a front view of each box and a top view of each box;

[0075] a box picture classification processing module for finding out the front view and the top view of the box corresponding to the same category of goods from the plurality of orientation pictures containing a single box, and dividing the box front view and the box top view corresponding to the same box in the same category of goods into a group, to obtain a set of orientation pictures of each box one, and performing morphological operation on the box front view and the box top view in the set of orientation pictures of each box one, to obtain a set of orientation pictures of each box two;

[0076] The cargo whole stack predicted volume calculation processing module is configured to input the second set of orientation pictures of each container into a pre-trained container identification recognition model to identify the cargo categories, the number of containers for different categories of cargo, and the containing volume of the containers corresponding to different categories of cargo, and calculate the cargo whole stack predicted volume V1 at the inventory station based on the number of containers for different categories of cargo and the containing volume of the containers corresponding to different categories of cargo.

[0077] The cargo whole stack volume calculation processing module is configured to obtain different orientation whole stack photos of the cargo stacked on the pallet at the inventory station, determine whether the whole stack of the cargo on the pallet is in a complete shape based on the different orientation whole stack photos of the cargo stacked on the pallet, and if the whole stack of the cargo on the pallet is in a complete shape, calculate the cargo whole stack volume V2 at the inventory station based on the different orientation whole stack photos of the cargo stacked on the pallet and the pixel proportion of the cargo whole stack photo; wherein the complete shape is a cuboid or a cube.

[0078] The cargo category and cargo quantity determination module is connected to the cargo whole stack volume calculation processing module and is configured to determine the actual category of the cargo and the actual quantity of the cargo at the inventory station based on the comparison result between the cargo whole stack volume V2 and the cargo whole stack predicted volume V1 at the inventory station and the identified cargo categories, the number of containers for different categories of cargo, and the containing volume of the containers corresponding to different categories of cargo.

[0079] The cargo category and cargo quantity review module is configured to read the RFID electronic tag on the pallet of the stacked cargo at the inventory station, compare the real category of the cargo stacked on the pallet and the real quantity of the cargo obtained indirectly after reading the RFID electronic tag with the actual category of the cargo and the actual quantity of the cargo determined in S5, perform a first review on the actual category of the cargo and the actual quantity of the cargo at the inventory station to determine whether the cargo stacked on the pallet at the inventory station is missing, wherein the RFID is arranged at the bottom of the pallet, and the RFID electronic tag stores the information of the pallet code; the real category of the cargo stacked on the pallet and the real quantity of the cargo obtained after reading the RFID electronic tag are the real category of the cargo and the real quantity of the cargo corresponding to the pallet in the WMS system based on the read pallet code information after reading the RFID electronic tag with the RFID reader-writer.

[0080] Preferably, the new generation of intelligent warehouse station based on machine vision design device also comprises a display control integrated touch terminal, the display control integrated touch terminal is connected with the goods category and the goods quantity review module in the whole stack of goods, the display control integrated touch terminal is used to store the information whether the goods stacked on the pallet are missing, and the information whether the goods stacked on the pallet are missing is displayed.

[0081] The warehouse control management system is in communication connection with the display control integrated touch terminal through a communication network, and the central control management system is used to query the information whether the goods on the pallet are missing.

[0082] Compared with the prior art, the present application has at least one of the following beneficial effects:

[0083] 1、The new generation of intelligent warehouse station based on machine vision in the application, first, obtain the different direction whole stack photos of the goods stacked on the pallet at the warehouse station, and then carry out box boundary contour extraction and segmentation processing on the goods box used for packaging in the whole stack photo after the first preprocessing, so as to obtain a plurality of single box direction pictures, then find out the front view and the top view of the box corresponding to the goods of the same category from the plurality of single box direction pictures, and divide the front view and the top view of the box corresponding to the goods of the same category into a group, so as to obtain the direction picture set one of each box, and then carry out morphological operation on the front view and the top view of the box in the direction picture set one of each box, so as to obtain the direction picture set two of each box, then input the direction picture set two of each box after the second preprocessing into the pre-trained box identification model, so as to identify the goods category, the number of boxes of different goods categories and the containing volume of the boxes corresponding to different goods categories, and based on the number of boxes of different goods categories and the containing volume of the boxes corresponding to different goods categories, calculate the whole stack prediction volume V1 of the goods at the warehouse station, then obtain the different direction whole stack photos of the goods stacked on the pallet at the warehouse station, and based on the different direction whole stack photos of the goods stacked on the pallet, judge whether the whole stack of goods on the pallet is in a complete shape, if the whole stack of goods on the pallet is in a complete shape, then based on the different direction whole stack photos of the goods stacked on the pallet and the pixel proportion of the whole stack photo, calculate the whole stack volume V of the goods at the warehouse station. 2，Then, based on the comparison result between the whole stack volume V2 of the goods and the whole stack predicted volume V1 of the goods at the inventory station, and the identified goods categories, the number of boxes of different goods categories, and the containing volume of the boxes corresponding to different goods categories, the actual categories and the actual quantity of the goods at the inventory station are determined. Finally, the RFID on the pallet of the stacked goods at the inventory station is read, and based on the comparison result between the real categories and the real quantity of the goods stacked on the pallet obtained after reading and the actual categories and the actual quantity of the goods at the inventory station determined, the actual categories and the actual quantity of the goods at the inventory station are first reviewed to determine whether the goods stacked on the pallet at the inventory station are missing. In this way, manual counting is not required, and the problem of consuming a large amount of manpower and low efficiency of goods counting caused by manually counting the goods one by one when counting the goods is solved.

[0084] 2、The present application first reads the RFID on the pallet of the stacked goods at the inventory station, and then based on the comparison result between the real categories and the real quantity of the goods stacked on the pallet obtained after reading and the actual categories and the actual quantity of the goods at the inventory station determined, the actual categories and the actual quantity of the goods at the inventory station are first reviewed. Then, the whole stack of the goods stacked on the pallet at the inventory station is weighed by the ground scale, and based on the comparison result between the actual weight of the whole stack of the goods obtained after weighing and the real weight of the whole stack of the goods obtained after reading the RFID on the pallet of the stacked goods at the inventory station, the actual categories and the actual quantity of the goods at the inventory station are secondly reviewed. In this way, by reviewing the actual categories and the actual quantity of the goods at the inventory station twice, not only the accuracy of the inventory station counting the categories and the quantity of the goods is improved, but also the logistics personnel can understand the accurate goods situation. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 It is a new generation of intelligent inventory station counting method flowchart based on machine vision in the present application.

[0086] Figure 2 It is a part of the circuit connection diagram of the new generation of intelligent inventory station counting device based on machine vision in the present application.

[0087] Figure 3 It is a circuit connection diagram of the new generation of intelligent inventory station counting device based on machine vision in the present application. DETAILED DESCRIPTION

[0088] As Figures 1-3 shown, in order to make the purpose, technical scheme and advantages of the present application more clear and explicit, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0089] Embodiment one

[0090] In the production process of the cigarette factory, the goods are usually transported to the designated work station by AGV trolley, and then the goods are transported to the designated work station. After the goods are transported to the designated work station, the goods need to be counted. The existing counting method for the goods stacked on the pallet mainly relies on manual counting of the goods one by one. Although this counting method can count the stacked goods, the manual counting method of the goods one by one not only consumes a large amount of manpower, but also has low counting efficiency of the goods.

[0091] Therefore, the present application proposes a new generation of intelligent warehouse station counting method based on machine vision to solve the problem of consuming a large amount of manpower and low counting efficiency of goods caused by the manual counting method of the goods one by one when counting the goods.

[0092] Specifically, please refer to Figure 1 A new generation of intelligent warehouse station counting method based on machine vision, comprising the following steps:

[0093] Firstly, different orientation whole stack photos of the goods stacked on the pallet at the warehouse station are obtained, the box boundary contour extraction and segmentation processing of the goods box used for the goods packaging in the whole stack photo after the first preprocessing are performed to obtain a plurality of orientation pictures containing a single box, specifically as follows:

[0094] (1) Two intelligent industrial cameras arranged at different directions at the warehouse station are used to take photos of the goods stack stacked on the pallet at the warehouse station to obtain whole stack photos of the goods in different orientations; wherein the whole stack photos of the goods in different orientations include the whole stack photo of the goods in the first shooting direction and the whole stack photo of the goods in the second shooting direction; the whole stack photo of the goods in the second shooting direction is a top view of the goods stack, and the whole stack photo of the goods in the first shooting direction is a front view of the goods stack;

[0095] (2) The picture gray scale preprocessing is performed on the whole stack photos of the goods in different orientations to obtain the whole stack photos of the goods after gray scale processing, and the picture gray scale conversion formula is as follows:

[0096] Gray = 0.2989 * R + 0.5870 * G + 0.1140 * B

[0097] Wherein, R represents the red component of each pixel point in the image, G represents the green component of each pixel point in the image, and B represents the blue component of each pixel point in the image; Gray represents the converted gray value.

[0098] (3) Gaussian blur preprocessing is performed on the whole stack photos of the goods after gray scale processing to obtain the whole stack pictures of the goods after Gaussian blur processing, and the picture Gaussian blur formula is as follows:

[0099]

[0100] Wherein, G(x, y) represents the convolution kernel value generated by the Gaussian function; x, y is the distance between the center of the convolution kernel and the current pixel; σ represents the standard deviation, which controls the blur degree.

[0101] (4) Based on the edge detector algorithm, the edge of each container in the cargo stack picture after Gaussian blur processing is subjected to edge enhancement processing to obtain a first pre-processed stack photo; wherein the first pre-processed stack photo is a picture in which the edge of each container in the cargo stack picture has been identified.

[0102] (5) Based on the contour extraction algorithm, each container contour in the first pre-processed stack photo is extracted and segmented to obtain multiple single-container orientation pictures; wherein the single-container orientation picture is a picture with only one container; the multiple single-container orientation pictures include a top view of each container and an elevation view of each container.

[0103] Second step, from the multiple single-container orientation pictures, find the elevation view and top view of the containers corresponding to the same category of goods, and divide the elevation view and top view of the same container corresponding to the same category of goods into a group to obtain a set of orientation pictures of each container, and perform morphological operation on the elevation view and top view of each container in the set of orientation pictures of each container to obtain a set of orientation pictures of each container, as follows:

[0104] (1) From the multiple single-container orientation pictures, find the top view and elevation view of the containers corresponding to the same category of goods to obtain the top view and elevation view of the containers corresponding to each same category of goods.

[0105] Wherein, each same category of goods can be understood as multiple different categories of goods; the top view and elevation view of the containers corresponding to each same category of goods can be understood as the top view and elevation view of the containers corresponding to each same category of goods in multiple different categories of goods.

[0106] It should be noted that since each outer wall of the containers corresponding to the same category of goods has a certain identification, when finding the top view and elevation view of the containers corresponding to the same category of goods, the top view and elevation view corresponding to the same category of goods can be found according to the identification on each outer wall of the containers corresponding to the same category of goods.

[0107] (2), from the top view and the front view of each same category of goods corresponding to the container, find the front view and the top view of the same container, and divide the front view and the top view of the same container into a group to obtain the orientation picture set one of each container; wherein the orientation picture set one of each container includes the same container top view and the same container front view.

[0108] It should be noted that since each container corresponding to the same category of goods has a number on the outer wall for distinguishing the container, when finding the front view and the top view of the same container, the front view and the top view of the same container can be found according to the number on the outer wall of each container corresponding to the same category of goods, and then the front view and the top view of the same container are divided into a group to obtain the orientation picture set one of each container.

[0109] (3), first, the container top view and the container front view in the orientation picture set one of each container are respectively subjected to erosion morphological operation, and then the container top view and the container front view in the orientation picture set one of each container after the erosion operation are subjected to inflation morphological operation to obtain the orientation picture set two of each container, and the expression of the picture erosion and the picture inflation combined operation is as follows:

[0110]

[0111] Wherein, A represents the foreground pixel region of the input image; B represents the structure element, that is, the shape defining the morphological operation; increase the size of the foreground region, so that the structure element B covers the edge of the foreground pixel, and fills the broken area;

[0112] Or, first, the container top view and the container front view in the orientation picture set one of each container are respectively subjected to inflation morphological operation, and then the container top view and the container front view in the orientation picture set one of each container after the inflation operation are subjected to erosion morphological operation to obtain the orientation picture set two of each container, and the expression of the picture inflation and the picture erosion operation is as follows:

[0113]

[0114] Wherein, A represents the foreground pixel region of the input image; B represents the structure element, that is, the shape defining the morphological operation; increase the size of the foreground region, so that the structure element B covers the edge of the foreground pixel, and fills the broken area.

[0115] Third step, input the second pre-processed orientation picture set two of each container into the pre-trained container identification recognition model to identify the goods category, the number of containers loaded with different categories of goods and the containing volume of the containers corresponding to different categories of goods, and based on the number of containers loaded with different categories of goods and the containing volume of the containers corresponding to different categories of goods, calculate the warehouse inventory station whole pile prediction volume V1, as follows:

[0116] (1) Gaussian filter preprocessing is performed on the container top view and container side view in the second orientation picture set of each container to obtain the second pre-processed orientation picture set two of each container, and the expression of the Gaussian filter operation is as follows:

[0117]

[0118] Where G(x, y) is the convolution kernel value generated by the Gaussian function; x, y represent the distance between the center of the convolution kernel and the current pixel; and sigma represents the standard deviation, which controls the blurring degree, and the larger the value, the stronger the blurring;

[0119] (2) Histogram equalization preprocessing is performed on the container top view and container front view in the second pre-processed orientation picture set two of each container to obtain the second pre-processed orientation picture set two of each container, and the expression of the histogram equalization operation is as follows:

[0120]

[0121] Where r represents the original gray level; s represents the mapped gray level; T(r) represents the cumulative distribution function (CDF); and p(r') represents the probability density function of the gray level;

[0122] Alternatively, adaptive histogram equalization preprocessing can also be performed on the container top view and container front view in the second pre-processed orientation picture set two of each container to obtain the second pre-processed orientation picture set two of each container.

[0123] (3) The second pre-processed orientation picture set two of each container is input into the pre-trained container identification recognition model to identify the goods category, the number of containers loaded with different categories of goods and the containing volume of the containers corresponding to different categories of goods; wherein the container identification recognition model is used to identify the barcodes, characters and packaging pictures on the outer wall of the container; the barcodes on the outer wall of the container are respectively associated with the volume of each container and the goods category information of each container; the second orientation picture set two of each container includes a container front view and a container top view, the container front view has a barcode associated with the goods category information and a specific identification word for each category of goods, and the container top view has a specific outer packaging diagram for each category of goods.

[0124] (4) Calculate the sum of the volume of each category of goods based on the number of boxes of different categories of goods and the containing volume of the corresponding boxes of different categories of goods, and sequentially add the calculated sum of the volume of each category of goods to obtain the predicted volume V1 of the whole stack of goods at the warehouse station.

[0125] Fourth step, obtain the different orientation whole stack photos of the goods stacked on the pallet at the warehouse station, and determine whether the whole stack of goods on the pallet is in a complete shape based on the different orientation whole stack photos of the goods stacked on the pallet. If the whole stack of goods on the pallet is in a complete shape, calculate the volume V2 of the whole stack of goods at the warehouse station based on the different orientation whole stack photos of the goods stacked on the pallet and the pixel ratio of the whole stack photo of the goods, as follows:

[0126] (1) Use the intelligent industrial cameras arranged in three different orientations at the warehouse station to take photos of the whole stack of goods stacked on the pallet to obtain different orientation whole stack photos of the goods; wherein the different orientation whole stack photos of the goods include a front view of the whole stack of goods, a top view of the whole stack of goods, and a side view of the whole stack of goods.

[0127] (2) Determine whether the whole stack of goods at the warehouse station is in a complete shape based on the different orientation whole stack photos of the goods. If the whole stack of goods is in a complete shape, calculate the volume V2 of the whole stack of goods at the warehouse station based on the length of the whole stack of goods, the height of the whole stack of goods, the width of the whole stack of goods, and the pixel ratio of the different orientation whole stack photos of the goods, as follows:

[0128] V2 = S * L * H * P

[0129] Wherein, V2 represents the volume of the whole stack of goods; S represents the width of the whole stack of goods; L represents the length of the whole stack of goods; and P is the pixel ratio of the whole stack photo of the goods.

[0130] It should be noted that in the process of obtaining the different orientation whole stack photos of the goods stacked on the pallet at the warehouse station, determining whether the whole stack of goods on the pallet is in a complete shape based on the different orientation whole stack photos of the goods stacked on the pallet, and calculating the volume V2 of the whole stack of goods at the warehouse station based on the different orientation whole stack photos of the goods stacked on the pallet and the pixel ratio of the whole stack photo of the goods, the following conditions are also included:

[0131] If the whole stack of goods on the pallet is not in a complete shape, the volume V2 of the whole stack of goods is calculated using the whole stack volume algorithm; wherein the whole stack volume algorithm is any one of voxelization method, bounding box method, or three-dimensional scanning volume algorithm.

[0132] Or, the non-intact cargo pallet is regarded as a complete shape, and the volume of the non-intact cargo pallet is calculated by first calculating the volume of the complete cargo pallet, then calculating the volume of the missing goods in the non-intact cargo pallet, and finally subtracting the volume of the missing goods in the non-intact cargo pallet from the volume of the complete cargo pallet. The difference is the volume of the non-intact cargo pallet.

[0133] It should be noted that the non-intact shape is an irregular shape, i.e., not a rectangular solid shape.

[0134] In the fifth step, based on the comparison result between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station and the identified cargo categories, the number of boxes of different categories of goods and the volume of the corresponding boxes of different categories of goods, the actual categories and actual quantities of the goods at the warehouse station are determined, as follows:

[0135] (1) If the cargo pallet volume V2 is equal to the predicted cargo pallet volume V1 at the warehouse station, the actual categories of goods on the pallet are the identified categories of goods in the third step, and the actual number of boxes of different categories of goods on the pallet is equal to the number of boxes of different categories of goods identified in the third step.

[0136] (2) If the cargo pallet volume V2 is much larger than the predicted cargo pallet volume V1 at the warehouse station, the difference between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station is determined to determine the difference between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station.

[0137] Based on the difference between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station, the categories and quantities of different categories of goods corresponding to the difference between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station are determined according to the volume of the corresponding boxes of different categories of goods.

[0138] The number of different categories of goods corresponding to the difference between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station is added to the number of boxes of different categories of goods identified in the third step to obtain the actual number of boxes of different categories of goods on the pallet.

[0139] The categories of goods corresponding to the difference between the cargo pallet volume V2 and the predicted cargo pallet volume V1 at the warehouse station are added to the identified categories of goods in the third step to obtain the actual categories of goods on the pallet.

[0140] The sixth step is to read the RFID electronic tag on the pallet of the stacked goods at the disc storage station, and based on the comparison result of the real category and the real quantity of the goods stacked on the pallet obtained indirectly after reading the RFID electronic tag and the actual category and the actual quantity of the goods at the disc storage station determined in S5, the actual category and the actual quantity of the goods at the disc storage station are first reviewed to determine whether the goods stacked on the pallet at the disc storage station are missing, which is as follows:

[0141] (1) The RFID reader and writer at the disc storage station is used to read the RFID electronic tag at the bottom of the pallet to obtain the code corresponding to the pallet; wherein the RFID electronic tag at the bottom of the pallet stores the information of the pallet code.

[0142] (2) Based on the code corresponding to the pallet obtained, the real category and the real quantity of the goods corresponding to the pallet are found in the WMS system to obtain the real category and the real quantity of the goods on the pallet.

[0143] (3) The real category of the goods on the pallet is compared with the actual category of the goods at the disc storage station determined in S5; if the real category of the goods on the pallet is consistent with the actual category of the goods at the disc storage station determined in S5, the category of the goods stacked on the pallet is complete; if the real category of the goods on the pallet is inconsistent with the actual category of the goods at the disc storage station determined in S5, the category of the goods stacked on the pallet is missing.

[0144] (4) The real quantity of the goods on the pallet is compared with the actual quantity of the goods at the disc storage station determined in S5; if the real quantity of the goods on the pallet is consistent with the actual quantity of the goods at the disc storage station determined in S5, the quantity of the goods stacked on the pallet is not missing; if the real quantity of the goods on the pallet is inconsistent with the actual quantity of the goods at the disc storage station determined in S5, the quantity of the goods stacked on the pallet is missing.

[0145] The technical scheme of the embodiment of the present application first acquires different orientation whole stack photos of the goods stacked on the pallet at the warehouse station, and performs box boundary contour extraction and segmentation processing on the goods boxes used by the goods in the whole stack photo after the first preprocessing, to obtain a plurality of orientation pictures containing a single box, then finds out the front view and the top view of the box corresponding to the same category of goods from the plurality of orientation pictures containing a single box, and divides the box front view and the box top view corresponding to the same box in the same category of goods into a group, to obtain an orientation picture set one of each box, and performs morphological operation on the box front view and the box top view in the orientation picture set one of each box, to obtain an orientation picture set two of each box, then inputs the orientation picture set two of each box after the second preprocessing into a pre-trained box identification recognition model, to identify the goods category, the number of boxes of different category goods, and the containing volume of the box corresponding to different category goods, and based on the number of boxes of different category goods and the containing volume of the box corresponding to different category goods, calculates the whole stack prediction volume V1 of the goods at the warehouse station, then acquires different orientation whole stack photos of the goods stacked on the pallet at the warehouse station, and based on the different orientation whole stack photos of the goods stacked on the pallet, judges whether the whole stack of goods on the pallet is in a complete shape, if the whole stack of goods on the pallet is in a complete shape, then based on the different orientation whole stack photos of the goods stacked on the pallet and the pixel proportion of the whole stack photo of the goods, calculates the whole stack volume V2 of the goods at the warehouse station 2， Then based on the comparison result between the whole stack volume V2 and the whole stack prediction volume V1 of the goods at the warehouse station, and the identified goods category, the number of boxes of different category goods, and the containing volume of the box corresponding to different category goods, the actual category and the actual quantity of the goods at the warehouse station are determined, finally the RFID on the pallet of the stacked goods at the warehouse station is read, based on the comparison result between the real category and the real quantity of the goods stacked on the pallet obtained after reading and the actual category and the actual quantity of the goods at the warehouse station determined in S5, the actual category and the actual quantity of the goods at the warehouse station are first reviewed to judge whether the goods stacked on the pallet at the warehouse station are missing, in this way, without manual counting method, the problem of consuming a large amount of manpower and low efficiency of goods counting caused by manually counting the goods one by one when counting the goods is solved.

[0146] It should be noted that the machine vision of the new generation of intelligent inventory station inventory method based on machine vision in the present application refers to the following in the present scheme: S1, obtaining different orientation whole stack photos of the goods stacked on the pallet at the inventory station, performing box boundary contour extraction and segmentation processing on the goods boxes used for the goods packaging in the whole stack photos after the first preprocessing, to obtain a plurality of orientation pictures containing a single box; wherein the plurality of orientation pictures containing a single box include a front view of each box and a top view of each box; S2, finding out the front view and top view of the box corresponding to the same category of goods from the plurality of orientation pictures containing a single box, and dividing the box front view and box top view corresponding to the same box in the same category of goods into a group to obtain a set of orientation pictures of each box, and performing morphological operation on the box front view and box top view in the set of orientation pictures of each box to obtain a set of orientation pictures of each box; S3, inputting the set of orientation pictures of each box after the second preprocessing into a pre-trained box identification recognition model to identify the goods category, the number of boxes of different category goods and the containing volume of the box corresponding to different category goods, and calculating the whole stack prediction volume V1 of the goods at the inventory station based on the number of boxes of different category goods and the containing volume of the box corresponding to different category goods; wherein the box identification recognition model is constructed based on a deep learning model; the set of orientation pictures of each box includes a box front view and a box top view, the box front view has a barcode associated with the goods category information and a specific identification word for each category of goods, and the box top view has a specific outer packaging diagram for each category of goods; S4, obtaining different orientation whole stack photos of the goods stacked on the pallet at the inventory station, judging whether the whole stack of goods on the pallet is in a complete shape based on the different orientation whole stack photos of the goods stacked on the pallet, if the whole stack of goods on the pallet is in a complete shape, then calculating the whole stack volume V2 of the goods at the inventory station based on the different orientation whole stack photos of the goods stacked on the pallet and the pixel ratio of the whole stack photo; wherein the complete shape is a cuboid or a cube; S5, based on the comparison result between the whole stack volume V2 and the whole stack prediction volume V1 of the goods at the inventory station, and the identified goods category, the number of boxes of different category goods and the containing volume of the box corresponding to different category goods, the actual category and the actual quantity of the goods at the inventory station are determined.

[0147] Embodiment two

[0148] In the production process of a tobacco factory, the goods are usually first transported to a designated station by an AGV trolley, and then the goods are transported to a designated station. After the goods are transported to the designated station, the goods need to be inventoried. The existing inventory method for the goods stacked on the pallet mainly relies on manual inventory of the goods one by one. Although this inventory method can complete the inventory of the stacked goods, relying on manual inventory of the goods one by one not only consumes a large amount of manpower, but also has low inventory efficiency.

[0149] Therefore, the present application proposes a new generation of intelligent inventory station inventory device based on machine vision in the first aspect to solve the problem of consuming a large amount of manpower and low efficiency of goods inventory caused by the way of manually inventorying goods one by one when inventorying goods.

[0150] Specifically, please refer to Figure 2 A new generation of intelligent inventory station inventory device based on machine vision, comprising a goods stack acquisition / separation processing module, a carton picture classification processing module, a goods stack predicted volume calculation processing module, a goods stack volume calculation processing module, a goods category and quantity determination module in the goods stack, and a goods category and quantity review module in the goods stack; wherein the goods stack acquisition / separation processing module is used to acquire different orientation stack photos of goods stacked on the pallet at the inventory station, and to perform carton boundary contour extraction and separation processing on the cartons used for the goods packaging in the first preprocessed stack photo, so as to obtain a plurality of orientation photos containing a single carton; wherein the plurality of orientation photos containing a single carton include an elevation view of each carton and a plan view of each carton;

[0151] The carton picture classification processing module is connected with the goods stack acquisition / separation processing module, and is used to find out the elevation view and the plan view of the carton corresponding to the same category of goods from the plurality of orientation photos containing a single carton, and to divide the carton elevation view and the carton plan view corresponding to the same carton in the same category of goods into a group, so as to obtain an orientation photo set one of each carton, and to perform morphological operation on the carton elevation view and the carton plan view in the orientation photo set one of each carton, so as to obtain an orientation photo set two of each carton;

[0152] The goods stack predicted volume calculation processing module is connected with the carton picture classification processing module, and is used to input the second preprocessed orientation photo set two of each carton into a pre-trained carton identification recognition model, so as to identify the goods category, the number of cartons containing different categories of goods, and the containing volume of the cartons corresponding to different categories of goods, and to calculate the goods stack predicted volume V1 at the inventory station based on the number of cartons containing different categories of goods and the containing volume of the cartons corresponding to different categories of goods; wherein the carton identification recognition model is constructed based on a deep learning model; the orientation photo set two of each carton includes a carton elevation view and a carton plan view, the carton elevation view has a barcode associated with the goods category information and a specific identification word of each category of goods, and the carton plan view has a specific outer packaging diagram of each category of goods;

[0153] The cargo whole stack volume calculation processing module is configured to acquire different orientation whole stack photos of the cargo stacked on the pallet at the warehouse inventory station, determine whether the whole stack of the cargo on the pallet is in a complete shape based on the different orientation whole stack photos of the cargo stacked on the pallet, calculate the whole stack volume V2 of the cargo at the warehouse inventory station based on the different orientation whole stack photos of the cargo stacked on the pallet and the pixel proportion of the cargo whole stack photo if the whole stack of the cargo on the pallet is in the complete shape; and the complete shape is a cuboid or a cube.

[0154] The cargo whole stack cargo category and cargo quantity determination module is configured to determine the actual category and actual quantity of the cargo at the warehouse inventory station based on the comparison result between the whole stack volume V2 and the predicted volume V1 of the cargo whole stack at the warehouse inventory station and the recognized cargo category, the number of boxes of different categories of cargo and the containing volume of the corresponding boxes of different categories of cargo.

[0155] The cargo whole stack cargo category and cargo quantity review module is configured to read the RFID electronic tag on the pallet of the stacked cargo at the warehouse inventory station, perform a first review on the actual category and actual quantity of the cargo at the warehouse inventory station based on the comparison result between the real category and real quantity of the cargo stacked on the pallet indirectly obtained after reading the RFID electronic tag and the actual category and actual quantity of the cargo at the warehouse inventory station determined in S5, to determine whether the cargo stacked on the pallet at the warehouse inventory station is missing; wherein the RFID is arranged at the bottom of the pallet, and the RFID electronic tag stores the information of the pallet code; the real category and real quantity of the cargo stacked on the pallet obtained after reading the RFID electronic tag are the real category and real quantity of the cargo corresponding to the pallet in the WMS system based on the pallet code information read by the RFID reader-writer.

[0156] The technical scheme of the embodiment of the present application first acquires different orientation whole stack photos of the goods stacked on the pallet at the warehouse station through a goods whole stack acquisition / segmentation processing module, performs box boundary contour extraction and segmentation processing on the goods boxes used for packaging the goods in the first preprocessed whole stack photos, to obtain a plurality of orientation pictures containing a single box, then finds out the front view and the top view of the box corresponding to the same type of goods from the plurality of orientation pictures containing a single box through a box picture classification processing module, and divides the box front view and the box top view corresponding to the same box in the same type of goods into a group, to obtain an orientation picture set one of each box, performs morphological operation on the box front view and the box top view in the orientation picture set one of each box, to obtain an orientation picture set two of each box, then inputs the second preprocessed orientation picture set two of each box into a pre-trained box identification recognition model through a goods whole stack volume prediction calculation processing module, to identify the goods type, the number of boxes containing different types of goods, and the containing volume of the boxes corresponding to different types of goods, and based on the number of boxes containing different types of goods and the containing volume of the boxes corresponding to different types of goods, calculates the goods whole stack volume prediction V1 at the warehouse station, then acquires different orientation whole stack photos of the goods stacked on the pallet at the warehouse station through a goods whole stack volume calculation processing module, judges whether the whole stack of goods on the pallet is in a complete shape based on the different orientation whole stack photos of the goods stacked on the pallet, if the whole stack of goods on the pallet is in a complete shape, calculates the goods whole stack volume V2 at the warehouse station based on the different orientation whole stack photos of the goods stacked on the pallet and the pixel proportion of the goods whole stack photo, then determines the actual type of goods and the actual quantity of goods at the warehouse station based on the comparison result between the goods whole stack volume V2 and the goods whole stack volume prediction V1 at the warehouse station, and the identified goods type, the number of boxes containing different types of goods, and the containing volume of the boxes corresponding to different types of goods, through a goods whole stack goods type and quantity determination module, finally reads the RFID electronic tag on the pallet of the stacked goods at the warehouse station through a goods whole stack goods type and quantity review module, compares the real type of goods and the real quantity of goods of the goods stacked on the pallet indirectly obtained after reading the RFID electronic tag with the actual type of goods and the actual quantity of goods at the warehouse station determined in S5, to perform the first review on the actual type of goods and the actual quantity of goods at the warehouse station, to judge whether the goods stacked on the pallet at the warehouse station are missing; wherein the RFID is arranged at the bottom of the pallet, and the RFID electronic tag stores the information of the pallet code.The real category and the real quantity of the goods stacked on the pallet obtained after reading the RFID electronic tag are the real category and the real quantity of the goods corresponding to the pallet in the WMS system based on the pallet code information read by the RFID reader after reading the RFID electronic tag by the RFID reader. In this way, manual counting is not required, and the problem of consuming a large amount of manpower and low efficiency of goods counting caused by manually counting goods one by one is solved.

[0157] Please refer to Figure 3 In order to facilitate the query of the quantity and category of the goods at the intelligent warehouse station, the new generation of intelligent warehouse station counting device based on machine vision further comprises a display control integrated touch terminal and a warehouse control management system. The display control integrated touch terminal is connected with the goods category and quantity review module in the whole stack of goods, is used for storing information about whether the goods stacked on the pallet at the warehouse station are missing, and displays the information about whether the goods stacked on the pallet are missing.

[0158] The warehouse control management system is in communication connection with the display control integrated touch terminal through a communication network, and the central control management system is used for querying the information about whether the goods on the pallet are missing.

[0159] Although the present application has been described with reference to the explanatory embodiments thereof, it is to be understood that many other modifications and implementations can be devised by those skilled in the art which will fall within the principles and spirit of the application. More specifically, various modifications and improvements can be made to the components of the subject combination layout and / or the layout within the scope of the present application, drawings and claims. In addition to the modifications and improvements to the components and / or layout, other uses will be apparent to those skilled in the art.

Claims

1. A new generation of intelligent inventory station method based on machine vision, characterized by: include: S1. Obtain photos of the entire stack of goods on pallets at different angles at the inventory station. Extract and segment the outline of the boxes used for packaging the goods in the first pre-processed photos of the entire stack to obtain multiple orientation images containing individual boxes. Among them, the multiple orientation images containing individual boxes include the front view and the top view of each box. S2. From multiple orientation images containing a single cargo box, find the front view and top view of the cargo box corresponding to the same category of goods, and group the front view and top view of the cargo box corresponding to the same cargo box in the same category of goods into a group to obtain the first orientation image set of each cargo box. Then, perform morphological operations on the front view and top view of the cargo box in the first orientation image set of each cargo box to obtain the second orientation image set of each cargo box. S3. Input the second set of orientation images of each cargo box after the second preprocessing into the pre-trained cargo box identification model to identify the cargo category, the number of cargo boxes for different categories of cargo, and the capacity of the cargo boxes corresponding to different categories of cargo. Based on the number of cargo boxes for different categories of cargo and the capacity of the cargo boxes corresponding to different categories of cargo, calculate the predicted volume V1 of the entire stack of cargo at the inventory station. The cargo box identification model is built based on a deep learning model. The second set of orientation images of each cargo box includes a front view and a top view of the cargo box. The front view of the cargo box has a barcode associated with the cargo category information and a specific identification word for each category of cargo. The top view of the cargo box has an outer packaging image specific to each category of cargo. S4. Obtain photos of the entire stack of goods piled on pallets at different angles at the inventory station. Based on the photos of the entire stack of goods piled on pallets at different angles, determine whether the entire stack of goods on the pallet is a complete shape. If the entire stack of goods on the pallet is a complete shape, calculate the volume V2 of the entire stack of goods at the inventory station based on the photos of the entire stack of goods piled on pallets at different angles and the pixel ratio of the photos of the entire stack of goods. Wherein, the complete shape is a cuboid or a cube. S5. Based on the comparison results between the total stack volume V2 and the predicted total stack volume V1 at the inventory station, as well as the identified cargo categories, the number of boxes for different cargo categories, and the capacity of the corresponding boxes for different cargo categories, determine the actual categories and quantities of cargo at the inventory station. S6. Read the RFID tags on the pallets of stacked goods at the inventory station. Based on the comparison between the actual category and quantity of goods stacked on the pallets indirectly obtained after reading the RFID tags and the actual category and quantity of goods at the inventory station determined in S5, perform the first verification of the actual category and quantity of goods at the inventory station to determine whether any goods stacked on the pallets at the inventory station are missing. The RFID tag is located on the bottom of the pallet and stores the pallet code information. The actual category and quantity of goods stacked on the pallet obtained after reading the RFID tags are obtained by using an RFID reader to read the RFID tags and then searching for the actual category and quantity of the goods corresponding to the goods on the pallet in the WMS system based on the read pallet code information.

2. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: Step S1 involves acquiring photos of stacked goods on pallets at different angles at the inventory check station, extracting and segmenting the boundary contours of the boxes used for packaging the goods in the first pre-processed photos to obtain multiple orientation images containing individual boxes; wherein, the multiple orientation images containing individual boxes include a front view and a top view of each box; including the following steps: S1.

1. Two intelligent industrial cameras arranged at the inventory station and in different directions are used to take pictures of the stack of goods piled on pallets at the inventory station to obtain photos of the entire stack of goods from different angles; wherein, the photos of the entire stack of goods from different angles include photos of the entire stack of goods from the first shooting direction and photos of the entire stack of goods from the second shooting direction; the photos of the entire stack of goods from the second shooting direction are top views of the entire stack of goods, and the photos of the entire stack of goods from the first shooting direction are front views of the entire stack of goods. S1.

2. Perform grayscale preprocessing on photos of the entire stack of goods from different angles to obtain grayscale-processed photos of the entire stack of goods. The grayscale conversion formula is as follows: Gray=0.2989×R+0.5870×G+0.1140×B Where R represents the red component of each pixel in the image, G represents the green component of each pixel in the image, and B represents the blue component of each pixel in the image; Gray represents the converted grayscale value. S1.

3. Perform Gaussian blur preprocessing on the grayscale photos of the entire stack of goods to obtain a Gaussian blurred image of the entire stack of goods. The Gaussian blur formula for the image is as follows: Where G(x, y) represents the convolution kernel value generated by the Gaussian function; x and y are the distances between the center of the convolution kernel and the current pixel; σ represents the standard deviation, which controls the degree of blur. S1.

4. Based on the edge detector algorithm, the edges of each box in the Gaussian blurred image of the entire stack of goods are subjected to strong edge processing to obtain the first pre-processed image of the entire stack; wherein, the first pre-processed image of the entire stack is the image in which the edges of each box in the image of the entire stack of goods have been identified; S1.5 Based on the contour extraction algorithm, the contour of each carton in the first preprocessed stack photo is extracted and segmented to obtain multiple orientation images containing a single carton; wherein, the orientation image containing a single carton is an image with only one carton; the multiple orientation images containing a single carton include a top view and a front view of each carton.

3. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: S2 involves identifying the front and top views of cargo boxes corresponding to the same category of goods from multiple location images containing individual cargo boxes. The front and top views of the same cargo box within the same category are grouped together to obtain a location image set one for each cargo box. Morphological operations are then performed on the front and top views of the cargo boxes in location image set one to obtain a location image set two for each cargo box. This includes the following steps: S2.1 From multiple orientation maps containing a single cargo box, find the top view and front view of the cargo box corresponding to the same category of goods, so as to obtain the top view and front view of the cargo box corresponding to each category of goods. S2.2 From the top view and front view of the cargo box corresponding to each type of goods, find the front view and top view of the same cargo box, and group the front view and top view of the same cargo box into a group to obtain the first set of orientation images for each cargo box; wherein, the first set of orientation images for each cargo box includes the top view and the front view of the same cargo box. S2.3 First, perform an erosion morphological operation on the top view and front view of each cargo box in the first set of orientation images. Then, perform a dilation morphological operation on the top view and front view of each cargo box in the first set of orientation images after the erosion operation to obtain the second set of orientation images for each cargo box. The expression for the combined image erosion and image dilation operation is as follows: Where A represents the foreground pixel region of the input image; B represents the structuring element, that is, the shape that defines the morphological operation; increasing the size of the foreground region makes the structuring element B cover the edge of the foreground pixels, filling in the broken area; Alternatively, first perform a dilation morphological operation on the top view and front view of each cargo box in the first set of orientation images, then perform an erosion morphological operation on the top view and front view of each cargo box in the first set of orientation images after the dilation operation, to obtain the second set of orientation images for each cargo box. The expressions for the image dilation and image erosion operations are as follows: Where A represents the foreground pixel region of the input image; B represents the structuring element, that is, the shape that defines the morphological operation; increasing the size of the foreground region makes the structuring element B cover the edge of the foreground pixels, filling in the broken area.

4. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: S3 involves inputting the second set of pre-processed location images of each cargo box into a pre-trained cargo box identification model to identify the cargo category, the number of cargo boxes containing different categories of cargo, and the capacity of the cargo boxes corresponding to different categories of cargo. Based on the number of cargo boxes containing different categories of cargo and the capacity of the cargo boxes corresponding to different categories of cargo, the predicted volume V1 of the entire stack of cargo at the inventory station is calculated, including the following steps: S3.

1. Perform Gaussian filtering preprocessing on the top view and side view of each cargo box in the second set of orientation images to obtain the second set of orientation images for each cargo box after noise reduction. The expression for the Gaussian filtering operation is as follows: Where G(x,y) is the convolution kernel value generated by the Gaussian function; x and y represent the distance between the center of the convolution kernel and the current pixel; σ represents the standard deviation, which controls the degree of blur; the larger the value, the stronger the blur. S3.

2. Perform histogram equalization preprocessing on the top-view and front-view images of each cargo box in the second set of orientation images after noise reduction preprocessing to obtain the second set of orientation images for each cargo box after the second preprocessing. The formula for histogram equalization is as follows: Where r represents the original gray level; s represents the mapped gray level; T(r) represents the cumulative distribution function (CDF); and p(r') represents the probability density function of the gray level. Alternatively, adaptive histogram equalization preprocessing can be performed on the top-view and front-view images of each cargo box in the second set of orientation images after noise reduction preprocessing to obtain the second set of orientation images of each cargo box after the second preprocessing. S3.3 Input the second set of orientation images of each cargo box after the second preprocessing into the pre-trained cargo box identification recognition model to identify the cargo category, the number of cargo boxes containing different categories of cargo, and the capacity of the cargo boxes corresponding to different categories of cargo. The cargo box identification recognition model is used to identify the barcodes, text, and packaging images on the outer wall of the cargo box. The barcodes on the outer wall of the cargo box are associated with the capacity of each cargo box and the cargo category information of each cargo box. The second set of orientation images of each cargo box includes a front view and a top view of the cargo box. The front view of the cargo box has a barcode associated with the cargo category information and a specific identification text for each category of cargo. The top view of the cargo box has an outer packaging image specific to each category of cargo. S3.4 Based on the number of boxes for different categories of goods and the capacity of the corresponding boxes for different categories of goods, calculate the sum of the volumes of each category of goods, and then add the calculated sums of the volumes of each category of goods in sequence to obtain the predicted volume V1 of the entire stack of goods at the inventory station.

5. The design method for a new generation of intelligent inventory control station based on machine vision according to claim 4, characterized in that: The pre-trained cargo container identification model in S3.3 is obtained through the following steps: S3.

31. Use an intelligent industrial camera to take pictures of the boxes used for each type of goods to obtain a collection of pictures of boxes used for different types of goods; wherein, the outer wall of the box used for each type of goods is equipped with a barcode associated with the goods category information, a specific identification word for each type of goods, and a specific outer packaging image for each type of goods. S3.

32. Perform Gaussian filtering preprocessing on the cargo box images corresponding to different categories in the cargo box image set to obtain the first set of cargo box images after noise reduction. The expression for the Gaussian filtering operation is as follows: Where G(x,y) is the convolution kernel value generated by the Gaussian function; x and y represent the distance between the center of the convolution kernel and the current pixel; σ represents the standard deviation, which controls the degree of blur; the larger the value, the stronger the blur. S3.

33. Perform histogram equalization preprocessing on the cargo box images corresponding to different categories in the first set of cargo box images after noise reduction preprocessing to obtain the second set of cargo box images. The formula for the histogram equalization operation is as follows: Where r represents the original gray level; s represents the mapped gray level; T(r) represents the cumulative distribution function (CDF); and p(r') represents the probability density function of the gray level. S3.

34. Input the second set of cargo box images into the deep learning model for training to obtain the trained cargo box identification model.

6. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: Step S4 involves acquiring photos of the entire stack of goods piled on pallets at the inventory station from different angles. Based on these photos, it is determined whether the entire stack of goods on the pallet is a complete shape. If the entire stack is a complete shape, the volume V2 of the entire stack of goods at the inventory station is calculated based on the photos of the entire stack from different angles and the pixel ratio of the photos. The complete shape is defined as a cuboid or cube. This step includes the following steps: S4.1 Use three intelligent industrial cameras arranged at three different locations at the inventory station to take pictures of the stack of goods on the pallets to obtain photos of the entire stack of goods from different perspectives; among them, the photos of the entire stack of goods from different perspectives include a front view, a top view, and a side view of the entire stack of goods. S4.

2. Based on photos of the entire stack of goods from different orientations, determine whether the stack of goods at the inventory station is a complete shape. If the stack is a complete shape, calculate the volume V2 of the entire stack of goods at the inventory station based on the length, height, width, and pixel ratio of the stack in the photos from different orientations. The formula for calculating the volume of the entire stack of goods is as follows: V2 = S * L * H * P Where V2 represents the volume of the entire stack of goods; S represents the width of the entire stack of goods; L represents the length of the entire stack of goods; and P represents the percentage of pixels in the image of the entire stack of goods.

7. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: S4, acquiring photos of the entire stack of goods piled on pallets at different angles at the inventory station, determining whether the entire stack of goods on the pallet is a complete shape based on the photos of the entire stack of goods piled on the pallet at different angles and the pixel ratio of the photos of the entire stack of goods, further includes the following cases in calculating the volume V2 of the entire stack of goods at the inventory station: If the entire stack of goods on the pallet is not a complete shape, the volume V2 of the entire stack of goods is calculated using the entire stack volume algorithm; wherein, the entire stack volume algorithm can be any one of the voxelization method, the bounding box method, or the 3D scanning volume algorithm.

8. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: S5, based on the comparison between the total stack volume V2 and the predicted total stack volume V1 at the inventory station, and the cargo categories identified in S3, the number of boxes for each cargo category, and the capacity of the corresponding boxes for each cargo category, determines the actual categories and quantities of goods at the inventory station, including the following steps: S5.1 If the total stack volume V2 is equal to the predicted total stack volume V1 at the inventory station, then the actual goods category on the pallet is the goods category identified in S3; the actual number of boxes of different goods category on the pallet is equal to the number of boxes of different goods category identified in S3. S5.2 If the total volume of the goods stack V2 is much larger than the predicted volume of the goods stack V1 at the inventory station, then the difference between the total volume of the goods stack V2 and the predicted volume of the goods stack V1 at the inventory station is calculated to determine the difference between the total volume of the goods stack V2 and the predicted volume of the goods stack V1 at the inventory station. Based on the difference between the total volume of goods stack V2 and the predicted volume of goods stack V1 at the inventory station, and according to the capacity of the cargo boxes corresponding to different categories of goods, the categories of goods and the quantities of different categories of goods corresponding to the difference between the total volume of goods stack V2 and the predicted volume of goods stack V1 at the inventory station are determined. Add the number of different types of goods corresponding to the difference between the total volume of the goods stack V2 and the predicted volume of the goods stack V1 at the inventory station to the number of boxes of different types of goods identified in S3 to obtain the actual number of boxes of different types of goods on the pallet. The categories of goods corresponding to the difference between the total volume of the goods stack V2 and the predicted volume of the goods stack V1 at the inventory station are combined with the categories of goods already identified in S3 to obtain the actual categories of goods on the pallet.

9. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: Step S6 involves reading the RFID tags on the pallets of stacked goods at the inventory station. Based on the comparison between the actual category and quantity of goods stacked on the pallets indirectly obtained after reading the RFID tags and the actual category and quantity of goods at the inventory station determined in Step S5, a first verification of the actual category and quantity of goods at the inventory station is performed to determine whether any goods stacked on the pallets at the inventory station are missing. This includes the following steps: S6.1 Use the RFID reader at the pallet station to read the RFID electronic tag on the bottom of the pallet to obtain the code corresponding to the pallet; wherein, the RFID electronic tag on the bottom of the pallet stores the pallet code information; S6.

2. Based on the obtained code corresponding to the pallet, search for the actual category and quantity of the goods on the pallet in the WMS system to obtain the actual category and quantity of the goods on the pallet. S6.3 Compare the actual category of goods on the pallet with the actual category of goods at the inventory station identified in S5; if the actual category of goods on the pallet matches the actual category of goods at the inventory station identified in S5, then the categories of goods stacked on the pallet are complete; if the actual category of goods on the pallet does not match the actual category of goods at the inventory station identified in S5, then the categories of goods stacked on the pallet are missing. S6.4 Compare the actual quantity of goods on the pallet with the actual quantity of goods at the inventory station determined in S5; if the actual quantity of goods on the pallet matches the actual quantity of goods at the inventory station determined in S5, then the quantity of goods stacked on the pallet is not missing; if the actual quantity of goods on the pallet does not match the actual quantity of goods at the inventory station determined in S5, then the quantity of goods stacked on the pallet is missing.

10. The next-generation intelligent inventory station method based on machine vision according to claim 1, characterized in that: S5, after reading the RFID tags on the pallets of stacked goods at the inventory station, and comparing the actual category and quantity of the goods stacked on the pallets obtained after reading with the actual category and quantity of the goods at the inventory station determined in S5, performs a first verification of the actual category and quantity of the goods at the inventory station to determine whether any goods stacked on the pallets at the inventory station are missing, it also includes: The weighing scale is used to weigh the entire stack of goods on pallets at the warehouse station to obtain the actual weight of the entire stack of goods. Based on the comparison between the actual weight of the entire stack of goods and the actual weight of the entire stack of goods obtained by reading the RFID tags on the pallets of stacked goods at the inventory station, a second verification is performed on the actual category and quantity of goods at the inventory station. The RFID tags are located at the bottom of the pallets and also store the weight information of the goods stacked on the pallets.

Images