Intelligent identification system for the location of palletized products in the warehouse
By using a smart recognition system that combines gravity sensors and cameras with convolutional neural networks in the warehouse, the problem of automatically identifying the location of palletized products has been solved, achieving efficient positioning for intelligent warehousing and outbound operations.
Patent Information
- Application Number
- CN202411285128.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-13
AI Technical Summary
The location of palletized products in the warehouse is difficult to identify automatically, resulting in low picking efficiency. In addition, the location needs to be manually recorded when the products are put into storage, which also affects efficiency.
Employing gravity sensors, cameras, and intelligent recognition systems, the system uses gravity signals to activate the camera to capture images and combines convolutional neural networks to identify the type and model of electronic components, thereby achieving intelligent warehousing and outbound positioning.
It improves warehouse picking and receiving efficiency, reduces reliance on warehouse staff's professional knowledge, and ensures accurate product location.
Smart Images

Figure CN118811337B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to intelligent recognition technology, and in particular to an intelligent recognition system for the position of tray-packed products in a warehouse. Background Art
[0002] For processing companies that generate circuit boards, various types of electronic components need to be welded on the circuit boards, such as resistors, capacitors, inductors, diodes, transistors and IC chips, etc., and these electronic components include many types, such as resistors, each resistance value represents a model. In order to meet the production of circuit boards with different needs, companies need to store a large number of these electronic components in warehouses. In order to facilitate the storage of electronic components, at present, manufacturers generally place electronic components on tapes with several grooves that match the size of the electronic components, and wind them on a reel roller to form a reel product. Customers who purchase these products absorb them and weld them on the circuit boards when producing circuit boards.
[0003] Since electronic components such as resistors, capacitors, inductors, diodes, transistors and IC chips are attached to tape and rolled into reels, their appearance does not differ much. Therefore, for manufacturers of circuit boards, the requirements for warehouses are relatively high. When they put these products into storage, they not only need to record the type (capacitor or inductor), model (such as resistors, each resistance specification represents a model) and quantity of electronic components attached to each roll of products, but also record the location of the product on the shelf to facilitate pickup during subsequent production.
[0004] At present, the specifications of the shelves in the warehouse are the same, that is, each shelf and each grid on it are the same. If the workers are not particularly familiar with the warehouse, it will take a certain amount of time to get the products when picking up the trayed products, which makes the picking efficiency low. In addition, since some trayed products contain thousands of electronic components, if they are not used up in one production, they cannot be placed in the position they choose when they are put back. They need to be registered again in the warehouse before they can be stored in the shelf position recorded in the warehouse to facilitate the next pick-up. Summary of the invention
[0005] In view of the above-mentioned deficiencies in the prior art, the intelligent identification system for the location of packaged products in the warehouse provided by the present invention solves the problem in the prior art that the shelves cannot automatically indicate the location of the packaged products to be picked up, resulting in low picking efficiency.
[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:
[0007] Provided is a system for intelligently identifying the location of palletized products in a warehouse, comprising:
[0008] A number of shelves, each with a first flash lamp installed on the side wall. The shelves are divided into multiple layers by a placement board, and each layer is divided into several accommodation grooves for accommodating tray-mounted products by a horizontal partition and a vertical partition. A second flash lamp and a gravity sensor are respectively installed on the side wall and inside of each accommodation groove;
[0009] A number of cameras, the number of which is the same as that of the shelves. A slide rail is installed on the top wall of the warehouse directly above each shelf, and a lifting and translation mechanism for driving the camera to translate and lift is installed on the slide rail;
[0010] A control module, one is installed on each shelf, and it is connected to the camera, lifting mechanism, first flash lamp, and all second flash lamps and gravity sensors of the corresponding shelf;
[0011] The control module starts the lifting and translation mechanism to drive the camera to translate and lift according to the gravity signal uploaded by the gravity sensor, so that the camera is directly facing the accommodation groove where the gravity sensor uploading the gravity signal is located, and starts the camera to capture the image of the tray-mounted product;
[0012] When the control module receives the pick-up requirement sent by the server, it turns on the first flash lamp and the second flash lamp of the accommodation groove where the tray-mounted product to be extracted is located; when the gravity signal of the corresponding gravity sensor disappears, it turns off the first flash lamp and the second flash lamp;
[0013] A server, which is connected to all control modules, is used to receive the images of tray-mounted products uploaded by the control modules, extract the contour images and contour dimensions of the electronic components in the images of tray-mounted products, and input the contour images and contour dimensions into the trained neural network to identify the types and models of the electronic components; and bind the types and models of the electronic components to the control module that uploads the images of tray-mounted products;
[0014] When the server receives the pick-up requirement uploaded by the pick-up personnel, it queries the control module associated with the types and models of the electronic components in the pick-up requirement, and sends the pick-up requirement to the control module.
[0015] Further, the lifting and translation mechanism includes a hub motor and a mounting block. The mounting block is installed on the output shaft of the hub motor through a bearing; an electric turntable is fixedly installed on the lower surface of the mounting block, an L-shaped mounting rod is connected to the electric turntable, and an electric push rod is fixed at the other end of the L-shaped mounting rod. The camera is installed at the end of the electric push rod;
[0016] The hub motor, electric turntable, and electric push rod are all electrically connected to the control module on the corresponding shelf; the slide rail includes a "C"-shaped skateboard with a length greater than or equal to the length of the shelf. Sealing plates are provided at both ends of the skateboard. The top horizontal plate of the skateboard is fixed to the top wall of the warehouse, and a baffle is fixed to the side of the bottom horizontal plate. The baffle and the skateboard cooperate to form a chute for the hub motor to slide.
[0017] Furthermore, the method for extracting the outline image and outline size of the electronic component in the tray-packed product image includes:
[0018] A1. Read the tape image containing the electronic components on the reel-packed product, search for an image region in the reel-packed product image that is similar to the tape image, and crop the reel-packed product image, retaining the image region that is similar to the tape image as the image to be identified;
[0019] A2, filtering the image to be identified, and then performing image enhancement operation on the filtered image to obtain a denoised and enhanced pre-processed image;
[0020] A3, using an edge detection algorithm to perform edge detection on the preprocessed image to obtain multiple connected domains, and according to the number of pixels in the connected domains, retaining multiple connected domains whose difference in the number of pixels is less than a preset threshold as contour images of multiple electronic components;
[0021] A4. Extract the contour lines of the electronic components from each contour image, and calculate the distance between adjacent pixels on the contour to obtain the perimeter of each contour image, and use it as the contour size.
[0022] Furthermore, the expression for filtering the image to be recognized is:
[0023]
[0024]
[0025] in, The filtered image Q In Location The pixel value of; z is the normalization factor; For the image to be recognized P In Location The pixel value of i For x Relative coordinates on the axis x The offset of j For y Relative coordinates on the axis y The offset of is the Gaussian kernel function; σ is the standard deviation of the Gaussian kernel; e is the natural logarithm; is the regularization weight; is the regularization parameter; For the image to be recognized P exist( x + i , y +j ) position gradient; is the gradient vector L 2 norm; For The pixel value variance of the local area centered at ; For the image to be recognized P The average variance of is a constant.
[0026] Furthermore, the trained neural network is a convolutional neural network, and the method for training it includes:
[0027] B1. Obtain a sample set and divide the sample set into a training set and a verification set; the sample set includes a number of contour image samples of electronic components with marked categories and sizes;
[0028] B2. Preprocessing the contour image samples of all electronic components in the training set and the validation set, wherein the preprocessing includes at least filtering and image enhancement operations;
[0029] B3. Use the preprocessed training set to train the convolutional neural network, calculate the difference between the output of the convolutional neural network and the true label through forward propagation, and update the network weights through back propagation to minimize the loss function;
[0030] B4, using the preprocessed validation set to validate the trained convolutional neural network. When the performance of the convolutional neural network on the validation set reaches the preset performance index, the training of the convolutional neural network is completed. Otherwise, return to step B3;
[0031] The convolutional neural network includes at least one convolution layer for extracting image features, at least one activation layer for nonlinear mapping, at least one pooling layer for reducing feature dimensions, and two fully connected layers for making classification decisions on size and category respectively; the convolution layer, activation layer, pooling layer and fully connected layer are connected in sequence.
[0032] Furthermore, the loss function is expressed as:
[0033]
[0034]
[0035]
[0036]
[0037] in, is the total loss function; are weight parameters respectively; is the loss function based on classification difficulty; is the cross entropy loss function; is the classification loss function; C is the total number of categories of electronic components; c is a categorical variable; k is the index of the electronic component contour image sample in the training set; N is the total number of electronic component contour image samples in the training set; For the k Electronic component contour image samples belong to the category c The predicted probability of For the k Electronic component contour image samples belong to the category c The true label of is the dynamic adjustment factor; For the k The first of the electronic component contour image samples m Features M is the total number of features included in the electronic component outline image sample; m is the characteristic variable; To take the absolute value; For the k The difficulty coefficient of the electronic component outline image samples; τ is the preset difficulty coefficient; is the sample classification accuracy; is an exponential function; γ is the adjustment factor.
[0038] Furthermore, the electronic component is one of a resistor, a capacitor, an inductor, a diode, a triode and an IC chip, and each electronic component includes several types.
[0039] Furthermore, the bottom surface of each receiving slot is also provided with a friction wheel connected to a micro motor and rotating the tray-packed product under the action of friction; the micro motor is electrically connected to a control module on the shelf where it is located; the height of the transverse partition is less than the radius length of the tray-packed product;
[0040] The coiled product includes a central roller, the length of which is equal to the width of the tape, and sector plates with the central axis of the central roller as the center are fixedly connected at equal intervals at both ends of the central roller; an observation window is formed between two adjacent sector plates, and the starting point of the tape winding on the central roller is located in the area where the observation window is located; the electronic components are arranged in the card slots on the tape at equal intervals.
[0041] Furthermore, the intelligent identification system for the location of the trayed products in the warehouse also includes calculating the total number of electronic components attached to the tape of the trayed products:
[0042] The control module starts the micro motor to rotate the tray-packed product according to the gravity signal uploaded by the gravity sensor, and starts the lifting and translation mechanism to make the camera located in front of the receiving slot corresponding to the gravity sensor. The camera first collects the image of the tray-packed product from the front of the receiving slot, and then rotates the camera with the electric turntable to collect images of the tray-packed product in the rotating state at multiple angles; the collected images are sent to the server through the control module;
[0043] The server performs filtering and image enhancement operations on the images of the tray-packed product from multiple angles and the images of the tray-packed product collected from the front of the receiving slot, and then uses the enhanced images to perform three-dimensional reconstruction of the tray-packed product to obtain a three-dimensional image of the tray-packed product; uses an edge detection algorithm to detect the side of the three-dimensional image to obtain a side profile image, and calculates the distance between adjacent pixel points on the profile image to obtain the length of the tape on the tray-packed product;
[0044] The server uses an edge detection algorithm to perform edge detection on the enhanced image of the disk-packed product image collected from the front of the receiving slot to obtain multiple connected domains. According to the number of pixels in the connected domains, multiple connected domains whose difference in the number of pixels is less than a preset threshold are retained, and the distance between two adjacent connected domains is calculated as the distance between two adjacent electronic components on the tape.
[0045] The total number of electronic components on the tape is calculated using the length of the tape and the distance between two adjacent electronic components, and the total number of electronic components is bound to the control module that uploads the disk-mounted product image.
[0046] Furthermore, the edge detection algorithm is a Canny algorithm, a Sobel operator or a Prewitt operator.
[0047] Compared with the prior art, the present invention has the following beneficial effects:
[0048] When the product is exported, the pick-up personnel send the product and model to the server. The server can query the information stored inside to find the location of the goods to be picked up, and send this information to the corresponding control module. After receiving the information, the control module can accurately know the location of the storage slot of the goods to be picked up, and then start the first flash light on the shelf and the flash light on the storage slot. In this way, after the pick-up personnel enter the warehouse, they can quickly find the corresponding shelf through the first flash light, and then find the trayed product to be picked up through the second flash light on the shelf. By locating the location of the trayed product in this way, it can help warehouse personnel quickly find the required products and improve the efficiency of picking up.
[0049] After sensing gravity information, the intelligent recognition system of this solution indicates that a tray-packed product has been placed here, and then starts the camera to collect images. Through image analysis, the outline image and outline size of the electronic components on the tray-packed product can be obtained first. Then, combined with the neural network, the type and model of the tray-packed product placed in the receiving slot where the gravity sensor is located can be accurately known.
[0050] This solution can realize intelligent warehousing of products through the above method. When products are put into storage, relevant personnel do not need to place the products in a specific position, and can place them in any free position on the shelf. The cooperation of gravity sensors, control modules and servers can identify the types and models of electronic components on tray-packed products and record them, so as to realize intelligent warehousing. In this way, relevant personnel do not need to have strong professional knowledge of identifying electronic components, thereby improving warehousing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 This is the principle block diagram of the intelligent identification system for the location of palletized products in the warehouse.
[0052] Figure 2 This is a schematic diagram of the structure of the shelves in the intelligent identification system for the location of palletized products in the warehouse.
[0053] Figure 3 This is a schematic diagram of the structure of the lifting and translation mechanism in the intelligent identification system for the position of palletized products in the warehouse.
[0054] Figure 4 Schematic diagram of the structure after the tape wrapped around the reel product is removed.
[0055] Among them, 1. Shelf; 11. First flash light; 12. Storage board; 13. Horizontal partition; 14. Longitudinal partition; 15. Receiving slot; 16. Second flash light; 17. Gravity sensor; 2. Camera; 3. Control module; 4. Slide rail; 41. Baffle; 5. Lifting and translation mechanism; 51. Hub motor; 52. Mounting block; 53. Electric turntable; 54. L-shaped mounting rod; 55. Electric push rod; 6. Tray-packaged products; 61. Center roller; 62. Fan-shaped plate. DETAILED DESCRIPTION
[0056] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0057] like Figure 1As shown, the intelligent identification system for the position of tray products in the warehouse of this solution includes several shelves 1, several cameras 2, several control modules 3 and a server. As Figure 2 As shown, a first flash lamp 11 is provided on the side wall of each shelf 1. The shelf 1 is divided into multiple layers by a placement board 12, and each layer is divided into several accommodation grooves 15 for accommodating tray products 6 by a transverse partition 13 and a longitudinal partition 14. A second flash lamp 16 and a gravity sensor 17 are respectively provided on the side wall and inside of each accommodation groove 15.
[0058] In this solution, the shelf 1 is a double-sided shelf, that is, tray products 6 can be placed on both sides of the shelf 1. After being divided into multiple layers, each layer is first divided into two parts in the length direction by the longitudinal partition 14, and then the two parts of each layer are divided into several accommodation grooves 15 by multiple transverse partitions 13.
[0059] The number of cameras 2 in this solution is the same as the number of shelves 1. A slide rail 4 is provided on the top wall of the warehouse directly above each shelf 1, and a lifting and translation mechanism 5 for driving the camera 2 to translate and lift is installed on the slide rail 4; as Figure 3 As shown, the lifting and translation mechanism 5 in this solution includes a hub motor 51 and a mounting block 52. The mounting block 52 is installed on the output shaft of the hub motor 51 through a bearing; an electric turntable 53 is fixedly installed on the lower surface of the mounting block 52, an L-shaped mounting rod 54 is connected to the electric turntable 53, and an electric push rod 55 is fixed at the other end of the L-shaped mounting rod 54. The camera 2 is installed at the end of the electric push rod 55.
[0060] The hub motor 51, the electric turntable 53 and the electric push rod 55 are all electrically connected to the control module 3 on the corresponding shelf 1; the slide rail 4 includes a "C"-shaped slide plate with a length greater than or equal to the length of the shelf 1. Sealing plates are provided at both ends of the slide plate ( Figure 3 The shown lifting and translation mechanism省去了两侧的封板), the top horizontal plate of the slide plate is fixed on the top wall of the warehouse, and a baffle 41 is fixed on the side of the bottom horizontal plate. The baffle 41 and the slide plate cooperate to form a chute for the hub motor 51 to slide.
[0061] In order to collect images of tray products in different accommodation grooves 15 on both sides of the shelf 1, in this solution, the guide rail is arranged directly above the shelf 1, and then combined with the L-shaped mounting rod 54, which can make the camera 2 deviate from the top central axis of the shelf 1 and be located on the side of the shelf 1. In this way, through the cooperation of the electric turntable 53 and the L-shaped mounting rod 54, the commutation of the camera 2 can be realized, so as to realize the collection of images on both sides of the shelf 1. The length of the electric push rod 55 can be set according to the height of the shelf 1, so that the camera 2 can collect images of tray products 6 in all accommodation grooves 15 between the bottom layer and the top layer of the shelf 1.
[0062] A control module 3 is provided on each shelf 1, which can be set on the top of the shelf 1, and is connected to the camera 2, lifting mechanism, first flash light 11 and all second flash lights 16 and gravity sensor of the corresponding shelf 1; when the gravity sensor is installed, it is necessary to set the shelf 1 information corresponding to the gravity sensor and the specific receiving slot 15 information in the shelf 1 in the controller module.
[0063] The control module 3 starts the lifting and translation mechanism 5 to translate and lift the camera 2 according to the gravity signal uploaded by the gravity sensor, so that the camera 2 faces the receiving slot 15 where the gravity sensor that uploads the gravity signal is located, and starts the camera 2 to capture the image of the tray-packed product.
[0064] When the control module 3 receives the pickup request sent by the server, it turns on the first flash light 11 and the second flash light 16 of the receiving slot 15 where the picked tray product 6 is located; when the gravity signal of the corresponding gravity sensor disappears, the first flash light 11 and the second flash light 16 are turned off.
[0065] The server is connected to all the control modules 3, and is used to receive the disk-packaged product images uploaded by the control modules 3, extract the outline images and outline dimensions of the electronic components in the disk-packaged product images, and input the outline images and outline dimensions into the trained neural network to identify the types and models of the electronic components; and bind the types and models of the electronic components to the control modules 3 that uploaded the disk-packaged product images; to facilitate quick search of the shelves 1 and receiving slots 15 corresponding to the disk-packaged products 6 when picking up the goods later.
[0066] When the server receives the pickup request uploaded by the pickup personnel, it queries the control module 3 associated with the electronic component type and model in the pickup request, and sends the pickup request to the control module 3.
[0067] In one embodiment of the present invention, a method for extracting a contour image and contour size of an electronic component in a tray-packed product image includes:
[0068] A1, reading the tape image containing the electronic components on the tray-packed product 6, searching for an image region in the tray-packed product image that is similar to the tape image, and cropping the tray-packed product image, retaining the image region that is similar to the tape image as the image to be identified;
[0069] A2. Filter the image to be identified, and then perform image enhancement on the filtered image to obtain a denoised and enhanced pre-processed image. The expression for filtering the image to be identified is:
[0070]
[0071]
[0072] in, The filtered image Q In Location The pixel value of; z is the normalization factor; For the image to be recognized P In Location The pixel value of i For x Relative coordinates on the axis x The offset of j For y Relative coordinates on the axis y The offset of is the Gaussian kernel function; σ is the standard deviation of the Gaussian kernel; e is the natural logarithm; is the regularization weight; is the regularization parameter; For the image to be recognized P exist( x + i , y + j ) position gradient; is the gradient vector L 2 norm; For The pixel value variance of the local area centered at ; For the image to be recognized P The average variance of is a constant.
[0073] A3. Use an edge detection algorithm to perform edge detection on the preprocessed image to obtain multiple connected domains. According to the number of pixels in the connected domains, multiple connected domains whose difference in the number of pixels is less than a preset threshold are retained as contour images of multiple electronic components; the edge detection algorithm can be a Canny algorithm, a Sobel operator, or a Prewitt operator.
[0074] A4. Extract the contour lines of the electronic components from each contour image, and calculate the distance between adjacent pixels on the contour to obtain the perimeter of each contour image, and use it as the contour size.
[0075] During implementation, the scheme preferably uses a convolutional neural network as the trained neural network, and the method for training the convolutional neural network includes:
[0076] B1. Obtain a sample set and divide the sample set into a training set and a verification set; the sample set includes a number of contour image samples of electronic components with marked categories and sizes;
[0077] B2. Preprocessing the contour image samples of all electronic components in the training set and the validation set, wherein the preprocessing includes at least filtering and image enhancement operations;
[0078] B3. Use the preprocessed training set to train the convolutional neural network, calculate the difference between the output of the convolutional neural network and the true label through forward propagation, and update the network weights through back propagation to minimize the loss function; the expression of the loss function is:
[0079]
[0080]
[0081]
[0082]
[0083] in, is the total loss function; are weight parameters respectively; is the loss function based on classification difficulty; is the cross entropy loss function; is the classification loss function; C is the total number of categories of electronic components; c is a categorical variable; k is the index of the electronic component contour image sample in the training set; N is the total number of electronic component contour image samples in the training set; For the k Electronic component contour image samples belong to the category c The predicted probability of For the k Electronic component contour image samples belong to the category c The true label of is the dynamic adjustment factor; For the k The first of the electronic component contour image samples m Features M is the total number of features included in the electronic component outline image sample; m is the characteristic variable; To take the absolute value; For the k The difficulty coefficient of the electronic component outline image samples; τ is the preset difficulty coefficient; is the sample classification accuracy; is an exponential function; γ is the adjustment factor.
[0084] B4, using the preprocessed validation set to validate the trained convolutional neural network. When the performance of the convolutional neural network on the validation set reaches the preset performance index, the training of the convolutional neural network is completed. Otherwise, return to step B3;
[0085] The convolutional neural network includes at least one convolution layer for extracting image features, at least one activation layer for nonlinear mapping, at least one pooling layer for reducing feature dimensions, and two fully connected layers for making classification decisions on size and category respectively; the convolution layer, activation layer, pooling layer and fully connected layer are connected in sequence.
[0086] The electronic component of this solution is one of a resistor, a capacitor, an inductor, a diode, a triode and an IC chip, and each electronic component includes several models.
[0087] The bottom surface of each receiving slot 15 is also provided with a friction wheel connected to a micro motor and rotating with the trayed product 6 under the action of friction; the micro motor is electrically connected to the control module 3 on the shelf 1 where it is located; the height of the transverse partition 13 is less than the radius length of the trayed product 6; the setting of the height of the transverse partition 13 can ensure that the camera 2 can completely capture the tape winding situation at each observation window when capturing the side image of the trayed product 6. The micro motor and the friction wheel are not shown in the figure.
[0088] like Figure 4 As shown, the disk product 6 includes a central roller 61, the length of which is equal to the width of the tape, and sector plates 62 are fixedly connected at equal intervals to both ends of the central roller 61 with the central axis of the central roller 61 as the center of the circle; an observation window is formed between two adjacent sector plates 62, and the starting point of the tape winding on the central roller 61 is located in the area where the observation window is located; the electronic components are arranged in the card slots on the tape at equal intervals.
[0089] The above structure of the coiled product 6 and the unique setting of the starting winding of the tape in this solution can facilitate the camera 2 to accurately know where the starting point of the tape is when collecting the side image of the coiled product 6, so as to ensure that a relatively accurate tape length is obtained during subsequent image recognition.
[0090] When winding the tape, the end of the tape should be located at the observation window as much as possible. If not, an image of the front side of the disc-packaged product 6 while it is rotating can be captured. In this way, the end position of the tape can be determined by combining the side image and the front image, so as to accurately determine the length of the tape.
[0091] The intelligent position identification system of the tray-packed product 6 in the warehouse also includes calculating the total number of electronic components attached to the tape of the tray-packed product 6:
[0092] The control module 3 starts the micro motor to rotate the tray-packed product 6 according to the gravity signal uploaded by the gravity sensor, and starts the lifting and translation mechanism 5 to make the camera 2 located in front of the receiving slot 15 corresponding to the gravity sensor. The camera 2 first collects the image of the tray-packed product from the front of the receiving slot 15, and then rotates the camera 2 through the electric turntable 53 to collect images of the tray-packed product 6 in the rotating state at multiple angles; the collected images are sent to the server through the control module 3;
[0093] The server performs filtering and image enhancement operations on the images of the tray-packed product 6 at multiple angles and the images of the tray-packed product collected from the front of the receiving slot 15, and then uses the enhanced images to perform three-dimensional reconstruction of the tray-packed product 6 to obtain a three-dimensional image of the tray-packed product 6; uses an edge detection algorithm to detect the side of the three-dimensional image to obtain a side profile image, and calculates the distance between adjacent pixel points on the profile image to obtain the length of the tape on the tray-packed product 6;
[0094] The server performs edge detection on the enhanced image of the disk-packed product image collected from the front of the receiving slot 15 using an edge detection algorithm to obtain multiple connected domains. According to the number of pixels in the connected domains, multiple connected domains whose difference in the number of pixels is less than a preset threshold are retained, and the distance between two adjacent connected domains is calculated as the distance between two adjacent electronic components on the tape.
[0095] The total number of electronic components on the tape is calculated using the length of the tape and the distance between two adjacent electronic components, and the total number of electronic components is bound to the control module 3 that uploads the disk-mounted product image.
[0096] When the gravity sensor of this solution senses the change in gravity, the micro motor and the friction wheel can be used to rotate the tray-packed product 6. At this time, the image of the tray-packed product 6 facing the camera and the side can be collected, and three-dimensional reconstruction can be performed in combination with the collected images. After a series of image processing, the number of electronic components on the tray-packed product 6 can be obtained. This solution can achieve fully automated warehousing by counting the number of electronic components and identifying the types and signals that can be achieved before, which can greatly improve the efficiency of warehousing and the accuracy of the records of warehousing products.
[0097] In summary, the location intelligent identification system of this solution can realize intelligent warehousing, which can reduce the professional knowledge and ability of warehouse warehousing personnel and improve warehousing efficiency; when the product is out of the warehouse (picked up), the location of the trayed product 6 can be reminded to facilitate the picking up personnel to quickly find the location of the product, thereby improving the picking up efficiency.
Claims
1. Intelligent identification system for the location of products in trays in the warehouse, It is characterized in that include: A plurality of shelves, each of which is provided with a first flashing light on its side wall, the shelves being divided into multiple layers by a storage board, each layer being divided into multiple receiving slots for receiving trayed products by a transverse partition and a longitudinal partition, and each receiving slot being provided with a second flashing light and a gravity sensor on its side wall and inside; A number of cameras, the number of which is the same as the number of shelves, a slide rail is provided on the top wall of the warehouse directly above each shelf, and a lifting and translation mechanism is installed on the slide rail for moving and lifting the camera; A control module, one of which is provided on each shelf and is connected to the camera, the lifting mechanism, the first flash light and all the second flash lights and the gravity sensor of the corresponding shelf; The control module starts the lifting and translation mechanism to translate and lift the camera according to the gravity signal uploaded by the gravity sensor, so that the camera faces the receiving slot where the gravity sensor that uploads the gravity signal is located, and starts the camera to capture the image of the tray-packed product; When the control module receives the pickup request sent by the server, the first flash light and the second flash light of the receiving slot where the picked-up tray product is located are turned on; when the gravity signal of the corresponding gravity sensor disappears, the first flash light and the second flash light are turned off; The server is connected to all the control modules and is used to receive the disk-packed product image uploaded by the control module, extract the outline image and outline size of the electronic component in the disk-packed product image, and input the outline image and outline size into the trained neural network to identify the type and model of the electronic component; and bind the type and model of the electronic component to the control module that uploaded the disk-packed product image; When the server receives the pickup request uploaded by the pickup personnel, it queries the control module associated with the type and model of the electronic components in the pickup request, and sends the pickup request to the control module; The method for extracting the outline image and outline size of the electronic component in the tray-packed product image includes: A1. Read the tape image containing the electronic components on the reel-packed product, search for an image region in the reel-packed product image that is similar to the tape image, and crop the reel-packed product image, retaining the image region that is similar to the tape image as the image to be identified; A2, filtering the image to be identified, and then performing image enhancement operation on the filtered image to obtain a denoised and enhanced pre-processed image; A3, using an edge detection algorithm to perform edge detection on the preprocessed image to obtain multiple connected domains, and according to the number of pixels in the connected domains, retaining multiple connected domains whose difference in the number of pixels is less than a preset threshold as contour images of multiple electronic components; A4. According to each contour image, the contour lines of the electronic components are extracted, and the perimeter of each contour image is obtained by calculating the distance between adjacent pixel points on the contour, and the perimeter is used as the contour size; The expression for filtering the image to be recognized is: in, The filtered image Q In Location The pixel value of; z is the normalization factor; For the image to be recognized P In Location The pixel value of i For x Relative coordinates on the axis x The offset of j For y Relative coordinates on the axis y The offset of is the Gaussian kernel function; σ is the standard deviation of the Gaussian kernel; e is the natural logarithm; is the regularization weight; is the regularization parameter; For the image to be recognized P exist( x + i , y + j ) position gradient; is the gradient vector L 2 norm; For The pixel value variance of the local area centered at ; For the image to be recognized P The average variance of is a constant; The trained neural network is a convolutional neural network, and the method for training it includes: B1. Obtain a sample set and divide the sample set into a training set and a verification set; the sample set includes a number of contour image samples of electronic components with marked categories and sizes; B2. Preprocess the contour image samples of all electronic components in the training set and the validation set. The preprocessing includes at least filtering and image enhancement operations; B3. Use the preprocessed training set to train the convolutional neural network. Calculate the difference between the output of the convolutional neural network and the true label through forward propagation, and update the network weights through backpropagation to minimize the loss function; B4. Use the preprocessed validation set to verify the trained convolutional neural network. When the performance of the convolutional neural network on the validation set reaches the preset performance index, the training of the convolutional neural network is completed. Otherwise, return to step B3; The convolutional neural network includes at least one convolutional layer for extracting image features, at least one activation layer for non-linear mapping, at least one pooling layer for reducing the feature dimension, and two fully connected layers for classifying and making decisions on size and category respectively. The convolutional layer, activation layer, pooling layer, and fully connected layer are connected in sequence; The expression of the loss function is: in, is the total loss function; are weight parameters respectively; is the loss function based on classification difficulty; is the cross entropy loss function; is the classification loss function; C is the total number of categories of electronic components; c is a categorical variable; k is the index of the electronic component contour image sample in the training set; N is the total number of electronic component contour image samples in the training set; For the k Electronic component contour image samples belong to the category c The predicted probability of For the k Electronic component contour image samples belong to the category c The true label of is the dynamic adjustment factor; For the k The first of the electronic component contour image samples m Features M is the total number of features included in the electronic component outline image sample; m is the characteristic variable; To take the absolute value; For the k The difficulty coefficient of the electronic component outline image samples; τ is the preset difficulty coefficient; is the sample classification accuracy; is an exponential function; γ is the adjustment factor.
2. The intelligent identification system for the position of disk-shaped products in the warehouse according to claim 1, characterized in that the lifting and translation mechanism includes a hub motor and a mounting block. The mounting block is installed on the output shaft of the hub motor through a bearing; an electric turntable is fixedly installed on the lower surface of the mounting block. An L-shaped mounting rod is connected to the electric turntable, and an electric push rod is fixed at the other end of the L-shaped mounting rod. The camera is installed at the end of the electric push rod; the hub motor, the electric turntable, and the electric push rod are all electrically connected to the control module on the corresponding shelf; the slide rail includes a "C"-shaped slide plate with a length greater than or equal to the length of the shelf. Sealing plates are provided at both ends of the slide plate. The top horizontal plate of the slide plate is fixed to the top wall of the warehouse, and a baffle is fixed to the side of the bottom horizontal plate. The baffle and the slide plate cooperate to form a chute for the hub motor to slide.
3. The intelligent identification system for the position of disk-shaped products in the warehouse according to claim 1 or 2, characterized in that the electronic component is one of a resistor, a capacitor, an inductor, a diode, a triode, and an IC chip, and each type of electronic component includes several models.
4. The intelligent identification system for the position of disk-shaped products in the warehouse according to claim 1 or 2, characterized in that a friction wheel connected to a micro motor and rotating the disk-shaped product under the action of friction is further provided on the bottom surface of each receiving groove; the micro motor is electrically connected to the control module on its corresponding shelf; the height of the horizontal partition is less than the radius length of the disk-shaped product; the disk-shaped product includes a central roll. The length of the central roll is equal to the width of the tape. Sector plates centered on the central axis of the central roll are fixedly connected to both ends of the central roll at equal intervals. An observation window is formed between adjacent two sector plates. The winding starting point of the tape on the central roll is located in the area where the observation window is located; the electronic components are arranged at equal intervals in the card slots on the tape.
5. The intelligent identification system for the position of disk-shaped products in the warehouse according to claim 4, characterized in that it further includes calculating the total number of electronic components attached to the tape of the disk-shaped product: The control module starts the micro motor to rotate the tray-packed product according to the gravity signal uploaded by the gravity sensor, and starts the lifting and translation mechanism to make the camera located in front of the receiving slot corresponding to the gravity sensor. The camera first collects the image of the tray-packed product from the front of the receiving slot, and then rotates the camera with the electric turntable to collect images of the tray-packed product in the rotating state at multiple angles; the collected images are sent to the server through the control module; The server performs filtering and image enhancement operations on the images of the tray-packed product from multiple angles and the images of the tray-packed product collected from the front of the receiving slot, and then uses the enhanced images to perform three-dimensional reconstruction of the tray-packed product to obtain a three-dimensional image of the tray-packed product; uses an edge detection algorithm to detect the side of the three-dimensional image to obtain a side profile image, and calculates the distance between adjacent pixel points on the profile image to obtain the length of the tape on the tray-packed product; The server uses an edge detection algorithm to perform edge detection on the enhanced image of the disk-packed product image collected from the front of the receiving slot to obtain multiple connected domains. According to the number of pixels in the connected domains, multiple connected domains whose difference in the number of pixels is less than a preset threshold are retained, and the distance between two adjacent connected domains is calculated as the distance between two adjacent electronic components on the tape. The total number of electronic components on the tape is calculated using the length of the tape and the distance between two adjacent electronic components, and the total number of electronic components is bound to the control module that uploads the disk-mounted product image.
6. The intelligent identification system for the location of tray-packed products in the warehouse according to claim 1 or 5, It is characterized in that The edge detection algorithm is a Canny algorithm, a Sobel operator or a Prewitt operator.
Citation Information
Patent Citations
Energy management system
US20020010518A1
Planogram Matching
US20170178060A1
Systems and methods for inventory management
US20230020026A1
