Warehouse in-out rechecking management system based on automatic visual scanning

Abstract

The invention discloses a warehouse in-out rechecking management system based on automatic visual scanning, which relates to the technical field of warehouse rechecking management, and comprises a pose acquisition unit, a visual calibration unit, a cargo reading unit, a record management unit, a warehouse-in rechecking unit and a warehouse-out rechecking unit, the pose acquisition unit is used for acquiring a color image and a depth image of a target cargo in a to-be-identified area so as to obtain pose information of the cargo, and the visual calibration unit is used for obtaining a relation matrix of a camera coordinate system and a mechanical arm coordinate system by calculating a rotation matrix and a translation matrix. And space coordinates in an image collected by the camera are converted into a mechanical arm coordinate system. The automatic visual scanning and mechanical arm technology are combined, the intelligent level in the warehouse management process can be greatly improved, the method is more suitable for actual complex warehouse scenes, the cargo warehouse-in and warehouse-out speed can be increased, and the cargo checking accuracy can also be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of warehouse review management, and in particular to a warehouse entry and exit review management system based on automatic visual scanning. Background Art

[0002] The main responsibilities of the warehouse entry and exit review include reviewing the goods before they leave the warehouse, quality inspection, sealing, providing warehouse exit data, coordinating with other departments, warehouse safety management, handling abnormal situations, warehouse sorting and cleaning, etc. The reviewer needs to review the goods before they leave the warehouse, check the type, quantity, specifications and other information of the goods for consistency with the order or delivery note, and ensure the accuracy and completeness of the goods; At present, most reviews are carried out manually, which not only fails to improve the intelligence of the warehouse management process, but also wastes human resources and increases labor costs. In addition, the number of goods stored in the warehouse is large and diverse, which is not suitable for complex warehousing scenarios. When dealing with the review management of goods entering and leaving the warehouse, the error rate is still high, and it is difficult to ensure the stability and accuracy of the entire goods entering and leaving the warehouse. In addition, it takes extra time to review the goods after they are stored in the cargo space and before they are shipped out, which reduces the efficiency of goods entering and leaving the warehouse. The quantity verification during the review process is also prone to errors and difficult to verify quickly. Moreover, during the storage of goods in the warehouse, it is not guaranteed that the loss of goods will be known in time, and warehouse management is prone to confusion. Summary of the invention

[0003] The purpose of the present invention is to provide a warehouse entry and exit review management system based on automatic visual scanning to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a warehouse entry and exit review management system based on automatic visual scanning, comprising: A posture acquisition unit, which is used to acquire a color image and a depth image of the target goods in the area to be identified, so as to obtain the posture information of the goods; A visual calibration unit, wherein the visual calibration unit obtains a relationship matrix between a camera coordinate system and a robot coordinate system by calculating a rotation matrix and a translation matrix, and converts the spatial coordinates in an image captured by the camera into the robot coordinate system; A cargo reading unit, which obtains cargo data in the camera coordinate system based on cargo positioning information, and controls the robotic arm to capture images of individual cargoes based on the cargo in the robotic arm coordinate system, obtains high-definition images, and then analyzes cargo barcodes and cargo name information on the images, and obtains cargo quantity information; Record management unit, which checks the whole pile of goods before entering the warehouse, verifies whether the quantity of goods is consistent with the purchase order, checks whether the packaging is intact, and confirms that the barcode information of the goods is accurate before processing them for entry into the warehouse. During the entry process, an information verification form is prepared in real time and input into the entire system, the areas in the warehouse are divided, and the shelf information, cargo location information, inventory information and equipment information of the warehouse are managed; The warehouse entry verification unit is used to associate the robot arm with the system controller. The robot arm is equipped with a high-definition camera. The scanning range of the high-definition camera includes the goods themselves and the barcode information of the cargo location. When the goods are in and out of the warehouse, the robot arm obtains the information verification form and reads the cargo information. The high-definition camera and the visual calibration unit automatically scan the cargo barcode to allocate the cargo location, and verify the allocated cargo location and the quantity of the goods; The outbound review unit checks the location of the goods through the information verification form when executing the outbound review. The robotic arm receives the outbound instruction issued by the system and takes pictures of the goods at the target location. During this process, the robotic arm conducts a review and inspection again through the visual calibration unit, and matches and verifies the read barcode of the goods with the data on the information verification form. When the read goods information matches the information recorded in the information verification form, the goods are shipped out normally, and after the goods are completely shipped out, the inbound and outbound records in the information verification form are changed to the outbound status.

[0005] Preferably, the posture acquisition unit includes an image calibration module, an image correction module and a registration and alignment module, and the image calibration module uses The black and white chessboard is used as the calibration board. The size of each square on the chessboard is , manually change the calibration plate posture within the camera's field of view, obtain calibration images of the calibration plate at multiple different angles, the image correction module grayscales the obtained calibration image, then detects corner points on the checkerboard grid on the calibration plate, and finally finds sub-pixel corner points based on the original corner points. The sub-pixel corner points are real coordinates obtained by iterative calculation using the least squares method. The camera intrinsic parameter matrix and distortion coefficient are obtained according to the sub-pixel corner point coordinates, and the obtained intrinsic parameter matrix and distortion coefficient are saved in txt file format for later call, which are used to correct the image captured by the camera. The registration and alignment module uses an RGB lens to capture color images and an IR lens to capture depth images. The color images are used for target detection, and the depth images are used to measure the depth data of the detected target objects. The color image and the depth image are registered and aligned using the registration algorithm to obtain the conversion relationship between the color image and the depth image, so that the color image and the depth image overlap.

[0006] Preferably, the visual calibration unit includes the following execution steps: Suppose the point set in the camera coordinate system is , the point set in the robot coordinate system is , and obtain the rotation matrix between the two point sets and translation matrix , establish the following formula: ; First, calculate the center point of the point set in the camera and robot coordinate systems. The calculation formula is: ; in, is the point in the camera coordinate system, represents the point in the robot coordinate system, represents a point in the coordinate system, Respectively represent the three-axis coordinates of the coordinate system, and Respectively represent the center point of the point set in the camera and robot coordinate systems, represents the number of points in the coordinate system, Indicates the point number, Represents a transpose operation; Next, calculate the optimal rotation matrix , by calculating the center point of the point set in the camera and robot coordinate system, a new point set is generated and , the point set is re-centered as: ; Then calculate the covariance matrix between the point sets : ; The covariance matrix Decompose the feature matrix by singular value decomposition and the feature matrix , and by the formula Obtain the rotation matrix between point sets ; Finally, calculate the translation matrix : ; By finding the rotation matrix and translation matrix Get the relationship matrix between the camera coordinate system and the robotic arm coordinate system , the relationship matrix The formula is: .

[0007] Preferably, the cargo reading unit includes an information acquisition module and a cargo positioning module. The information acquisition module identifies the type of cargo according to the barcode, and performs image filtering and image thresholding on the barcode, uses Gaussian filtering to filter out Gaussian noise in the image, and performs image thresholding on the filtered image to obtain a binary image, thereby separating the target image from the background. The cargo positioning module obtains the positioning information of the cargo through a multi-angle template matching positioning method.

[0008] Preferably, the record management unit includes an information setting module, which is used to define the entry information of the information verification form and the goods barcode, the information verification form includes a storage location table corresponding to the goods and the allocated storage locations, a quantity table of different categories of goods to be stored, a task order table for storing task orders, an information table for recording basic product parameters, and goods in and out of the warehouse records, and the barcode information of each commodity records the product information and weight information of the commodity.

[0009] Preferably, the record management unit also includes a warehouse management module, which is used to manage shelf information, cargo location information, inventory information and equipment information. Shelf information management is to configure corresponding parameter information for each shelf and bind it to the corresponding area. Cargo location information management is to bind each cargo location to the corresponding shelf, and provide a cargo location barcode that records the cargo location number and the spatial coordinate information. Inventory information management is to record and display the inventory data in the warehouse, and display the details of the stored goods in each cargo location in the entire system. Equipment information management is to uniformly manage the intelligent storage equipment in the warehouse.

[0010] Preferably, the warehousing review unit includes an warehousing execution module, a goods allocation module and an information matching module. When the warehousing execution module executes the warehousing review, the warehouse staff first places the goods on the warehousing platform according to the prompt of the LED screen on the conveyor of the warehousing platform. After confirming that the warehousing operation is to be executed through the barcode recognition of the goods, the RGV transports the goods to the conveyor at the corresponding lane entrance. After the goods are placed on the conveyor at the lane entrance, the conveyor transports them to the warehousing entrance. During the warehousing process of the goods, the visual calibration unit performs a review and inspection to ensure that the goods are correctly warehousing. After the goods allocation module obtains the barcode information of the goods , associate the barcode information with the information verification form, so as to allocate the cargo location according to the data recorded in the barcode information, and record the allocated cargo location in the information verification form. The information matching module places the goods at the position allocated in the information verification form, and at the same time scans the cargo location barcode information to confirm whether the allocated cargo location matches the barcode information marked on the cargo location. If it matches, the goods entry and exit records in the information verification form are set to have been entered into the warehouse. Otherwise, the cargo location is reallocated and stored according to the reallocated cargo location. If it still does not match, an alarm is issued. The entire system is connected to the mobile terminal network to facilitate notification of the administrator for processing.

[0011] Preferably, the warehousing review unit also includes a quantity verification module, a real-time update module and an inventory management module. The quantity verification module synchronously records the corresponding quantity information of the goods entering the warehouse, which is used to verify the quantity of goods when entering the warehouse. After the real-time update module allocates a cargo space, it eliminates the allocated cargo space in the information verification form, and after a cargo space is vacated, it adds the vacated cargo space to the information verification form. The inventory management module is used to manage inventory information. When the goods enter the warehouse, the goods entry and exit records on the information verification form are in the warehouse status. When the goods leave the warehouse, the goods entry and exit records on the information verification form are in the warehouse status. The allocated cargo spaces in the warehouse are scanned regularly according to the information verification form. If there are vacancies in the allocated cargo spaces or non-vacant unallocated cargo spaces, the report system will notify the administrator.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention combines automatic visual scanning with robotic arm technology, which can greatly improve the level of intelligence in the warehouse management process and is more suitable for actual complex warehouse scenes. It can not only improve the rate of goods in and out of the warehouse, but also increase the accuracy of goods verification. The posture information of the goods is obtained through the posture acquisition unit, which is convenient for calibration of individual goods. The conversion relationship between the camera and the robotic arm coordinate system is established through the visual calibration unit, and the spatial coordinates in the image collected by the camera are converted to the robotic arm coordinate system to ensure the stability and accuracy of the entire goods in and out of the warehouse; 2. The present invention uses a cargo reading unit to accurately capture cargo images based on cargo positioning information and the cargo in the obtained robotic arm coordinate system to control the robotic arm, thereby ensuring that the cargo barcode can be correctly and clearly identified, and avoiding failure to scan the cargo barcode during cargo stacking. The record management unit follows up in real time based on the content in the information verification form and synchronizes it to the system, as well as divides the cargo location area, manages equipment and inventory in advance, which is convenient for subsequent intelligent allocation of cargo locations. The warehousing review unit can ensure that the cargo is accurately stored in the designated location in the warehouse, and the status of the cargo can be learned in time, thereby improving the subsequent outbound delivery rate. The quantity of the cargo can be verified again during the warehousing process, and the possibility of cargo loss or over-warehousing is reduced during storage in the warehouse. The outbound delivery review unit performs a review and inspection again, without spending extra time. The inspection is performed based on the information verification form, thereby reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of the overall system structure provided by an embodiment of the present invention; Figure 2 A module block diagram of a posture acquisition unit provided in an embodiment of the present invention; Figure 3A module block diagram of a cargo reading unit and a record management unit provided in an embodiment of the present invention; Figure 4 A module block diagram of a warehousing review unit provided in an embodiment of the present invention.

[0014] In the figure: 1. Posture acquisition unit; 101. Image calibration module; 102. Image correction module; 103. Registration and alignment module; 2. Visual calibration unit; 3. Goods reading unit; 301. Information acquisition module; 302. Goods positioning module; 4. Record management unit; 401. Information setting module; 402. Warehouse management module; 5. Warehouse entry review unit; 501. Warehouse entry execution module; 502. Goods allocation module; 503. Information matching module; 504. Quantity verification module; 505. Real-time update module; 506. Inventory management module; 6. Outbound review unit. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] See also Figures 1 to 4 The present invention provides a technical solution: a warehouse entry and exit review management system based on automatic visual scanning, comprising: A posture acquisition unit 1 is used to acquire a color image and a depth image of the target goods in the area to be identified, so as to obtain the posture information of the goods; Visual calibration unit 2, which obtains the relationship matrix between the camera coordinate system and the robot coordinate system by calculating the rotation matrix and the translation matrix, and converts the spatial coordinates in the image collected by the camera into the robot coordinate system; The cargo reading unit 3 obtains cargo data in the camera coordinate system based on cargo positioning information, and controls the robotic arm to complete image capture of a single cargo based on the cargo in the robotic arm coordinate system, obtains a high-definition image, and then analyzes the cargo barcode and cargo name information on the image, and obtains cargo quantity information; Record management unit 4: Record management unit 4 will check the whole pile of goods before entering the warehouse, check whether the quantity of goods is consistent with the purchase order, check whether the packaging is intact, and confirm that the barcode information of the goods is accurate before processing them for entry into the warehouse. During the entry process, an information verification sheet will be prepared in real time and input into the entire system, the areas in the warehouse will be divided, and the shelf information, cargo location information, inventory information and equipment information of the warehouse will be managed; The warehouse entry verification unit 5 is used to associate the robot arm with the system controller. The robot arm is equipped with a high-definition camera. The scanning range of the high-definition camera includes the goods themselves and the barcode information of the cargo location. When the goods are in and out of the warehouse, the robot arm obtains the information verification form and reads the cargo information. The high-definition camera and the visual calibration unit 2 automatically scan the cargo barcode to allocate the cargo location, and check the allocated cargo location and the quantity of the goods. The outbound review unit 6 checks the location of the goods through the information verification sheet when performing the outbound review. The robot arm receives the outbound instruction issued by the system and takes pictures of the goods at the target location. During this process, the robot arm conducts a review and inspection again through the visual calibration unit 2, and matches and checks the read barcode of the goods with the data on the information verification sheet. When the read goods information matches the information recorded in the information verification sheet, the goods are shipped out normally, and after the goods are completely shipped out, the inbound and outbound records in the information verification sheet are changed to the outbound status; The posture acquisition unit 1 includes an image calibration module 101, an image correction module 102 and a registration and alignment module 103. The image calibration module 101 uses The black and white chessboard is used as the calibration board. The size of each square on the chessboard is , manually change the calibration plate posture within the camera's field of view, obtain calibration images of the calibration plate at multiple different angles, the image correction module 102 grayscales the obtained calibration image, then detects corner points on the checkerboard grid on the calibration plate, and finally finds sub-pixel corner points based on the original corner points. The sub-pixel corner points are real coordinates obtained by iterative calculation using the least squares method. The camera intrinsic parameter matrix and distortion coefficient are obtained according to the sub-pixel corner point coordinates, and the obtained intrinsic parameter matrix and distortion coefficient are saved in a txt file format for later use, which are used to correct the image captured by the camera. The registration and alignment module 103 uses an RGB lens to capture a color image and an IR lens to capture a depth image. The color image is used for target detection, and the depth image is used to measure the depth data of the detected target object. The color image and the depth image are registered and aligned using a registration algorithm to obtain a conversion relationship between the color image and the depth image, so that the color image and the depth image overlap. The specific registration algorithm is: ; ; The following relationship is obtained through the above formula: ; ; in, is a point in the robot coordinate system, Represents the corresponding points in the RGB lens coordinate system, Represents the corresponding points in the pixel coordinate system of the two-dimensional RGB image, Represents the transformation matrix from the RGB lens coordinate system to the two-dimensional pixel coordinate system, represents the corresponding point in the IR lens coordinate system, Represents the corresponding points in the pixel coordinate system of the two-dimensional depth image, Represents the transformation matrix from the IR lens coordinate system to the two-dimensional depth image coordinate system, and They represent the rotation matrix and translation vector in the RGB lens coordinate system respectively. and They represent the rotation matrix and translation vector in the IR lens coordinate system respectively. represents the intermediate parameter, and They represent the rotation matrix and translation vector respectively; The visual calibration unit 2 includes the following execution steps: Suppose the point set in the camera coordinate system is , the point set in the robot coordinate system is , and obtain the rotation matrix between the two point sets and translation matrix , establish the following formula: ; First, calculate the center point of the point set in the camera and robot coordinate systems. The calculation formula is: ; in, is the point in the camera coordinate system, represents the point in the robot coordinate system, represents a point in the coordinate system, Respectively represent the three-axis coordinates of the coordinate system, and Respectively represent the center point of the point set in the camera and robot coordinate systems, represents the number of points in the coordinate system, Indicates the point number, Represents a transpose operation; Next, calculate the optimal rotation matrix , by calculating the center point of the point set in the camera and robot coordinate system, a new point set is generated and , the point set is re-centered as: ; Then calculate the covariance matrix between the point sets : ; The covariance matrix Decompose the feature matrix by singular value decomposition and the feature matrix , and by the formula Obtain the rotation matrix between point sets , singular value decomposition is an important matrix decomposition in county algebra; Finally, calculate the translation matrix : ; By finding the rotation matrix and translation matrix Get the relationship matrix between the camera coordinate system and the robotic arm coordinate system , the relationship matrix The formula is: ; The cargo reading unit 3 includes an information acquisition module 301 and a cargo positioning module 302. The information acquisition module 301 identifies the type of cargo according to the barcode of the cargo, and performs image filtering and image thresholding on the barcode. Gaussian filtering is used to filter out Gaussian noise of the image. The filtered image is thresholded to obtain a binary image, thereby separating the target image from the background. The image thresholding operation is to select a global threshold, set the pixels with grayscale values ​​greater than the threshold to 255, and set the pixels with grayscale values ​​less than the threshold to 0; The cargo positioning module 302 obtains cargo positioning information by a multi-angle template matching positioning method. The multi-angle template matching positioning method is as follows: Find the positioning coordinates with the highest matching degree with the template image in the image to be detected, so that the template image In the image to be detected The image pixel area is translated in a sliding window manner. Suppose the image to be detected The area covered by the template image is , by comparing the template image With the image to be detected The template matching process is completed by the similarity of the sliding window area, and the formula is: ; in, represents the template image, represents the image to be detected, Represents a template image In the image to be detected Pixel area The similarity of Represents a template image All pixels in the area, and Respectively expressed in pixels Image to be detected at position and template image The average gray value of Represented in the template image The image to be detected in all pixels of the area Template Image The area covered, and Both represent the number of images to be detected. represents the absolute value, Represents pixel points; The record management unit 4 includes an information setting module 401, which is used to define the input information of the information verification form and the goods barcode. The information verification form includes a table of goods and allocated cargo locations, a table of quantities of goods of different categories to be stored, a task order table for storing task orders, an information table for recording basic parameters of products, and a record of goods entering and leaving the warehouse, and the barcode information of each commodity records the product information and weight information of the commodity; The record management unit 4 also includes a warehouse management module 402, which is used to manage shelf information, cargo location information, inventory information and equipment information. Shelf information management is to configure corresponding parameter information for each shelf and bind it with the area to which it belongs. Cargo location information management is to bind each cargo location with the shelf to which it belongs, and a cargo location barcode is provided to record the cargo location number and the spatial coordinate information. Inventory information management is to record and display the inventory data in the warehouse, and to display the details of the stored goods in each cargo location in the entire system. Equipment information management is to uniformly manage the intelligent storage equipment in the warehouse. The warehousing review unit 5 includes an warehousing execution module 501, a goods allocation module 502 and an information matching module 503. When the warehousing execution module 501 performs warehousing review, the warehouse staff first places the goods on the warehousing platform according to the prompt of the LED screen on the conveyor of the warehousing platform. After confirming that the warehousing operation is to be executed through the barcode recognition of the goods, the RGV transports the goods to the conveyor at the corresponding lane entrance. After the goods are placed on the conveyor at the lane entrance, the conveyor transports them to the warehousing entrance. During the warehousing process, the visual calibration unit 2 is used to perform a review and inspection to ensure that the goods are correctly warehousing. The goods allocation module 502 obtains the barcode of the goods. After receiving the information, the barcode information is associated with the information verification form, so as to allocate the cargo location according to the data recorded in the barcode information, and the allocated cargo location is recorded in the information verification form. The information matching module 503 places the goods at the position allocated in the information verification form, and at the same time scans the cargo location barcode information to confirm whether the allocated cargo location matches the barcode information marked on the cargo location. If it matches, the goods entry and exit records in the information verification form are set to have been entered into the warehouse. Otherwise, the cargo location is reallocated and stored according to the reallocated cargo location. If it still does not match, an alarm is issued. The entire system is connected to the mobile terminal network to facilitate notification of the administrator for processing; The warehousing review unit 5 also includes a quantity verification module 504, a real-time update module 505 and an inventory management module 506. The quantity verification module 504 synchronously records the corresponding quantity information of the goods entering the warehouse, which is used to verify the quantity of goods when entering the warehouse. After the real-time update module 505 allocates a cargo space, it eliminates the allocated cargo space in the information verification form, and after a cargo space is vacated, it adds the vacated cargo space in the information verification form. The inventory management module 506 is used to manage inventory information. When the goods are put into the warehouse, the goods entry and exit records on the information verification form are in the warehouse entry status. When the goods are shipped out of the warehouse, the goods entry and exit records on the information verification form are in the warehouse exit status. The allocated cargo spaces in the warehouse are scanned regularly according to the information verification form. If there are vacancies in the allocated cargo spaces or non-vacant unallocated cargo spaces, the report system notifies the administrator.

[0017] Working principle: When the present invention is used, the color image and the depth image are collected by the camera through the posture acquisition unit 1, and the two are aligned to obtain the posture information of the goods. The center point of the point set in the camera and robot coordinate system is calculated by the visual calibration unit 2, and the rotation matrix and translation matrix between the two point sets are obtained to obtain the relationship matrix between the camera coordinate system and the robot coordinate system. The spatial coordinates in the image collected by the camera are converted to the robot coordinate system. The target image is separated from the background by the cargo reading unit 3, and the positioning information of the cargo is obtained by a multi-angle template matching positioning method. The positioning information of the cargo is integrated to obtain the data of the cargo in the camera coordinate system, and according to the cargo in the obtained robot coordinate system, the robot is controlled to complete the image shooting of a single cargo to obtain a high-definition image, and then the cargo barcode and cargo name information on the image are analyzed, and the quantity information of the cargo is obtained; The goods arriving at the site are checked for entry into the warehouse through the record management unit 4. After checking whether the quantity of the goods is consistent with the purchase order, checking whether the packaging is intact, and confirming that the barcode information of the goods is accurate, the goods are processed for entry into the warehouse. During the process of goods entering the warehouse, an information verification form is prepared in real time, and the barcode of the goods is automatically scanned to obtain information, and then input into the entire system. The shelf information, cargo location information, inventory information and equipment information of the warehouse are managed through the warehouse management module 402 for easy management and control. During the process of goods entering the warehouse, the entry review unit 5 conducts a review and inspection through the visual calibration unit 2 to ensure that the goods are correctly entered into the warehouse. The cargo allocation module 502 obtains the barcode information of the cargo, associates the barcode information with the information verification form, and allocates the cargo location according to the data recorded in the barcode information, and records the allocated cargo location in the information verification form. The information matching module 503 places the cargo at the position allocated in the information verification form, and at the same time scans the cargo location barcode information to confirm whether the allocated cargo location matches the barcode information marked on the cargo location. If it matches, the cargo entry and exit record in the information verification form is set to have been entered into the warehouse. Otherwise, the cargo location is reallocated and stored according to the reallocated cargo location. If it still does not match, an alarm is issued. The entire system and the mobile terminal The terminal network is connected to facilitate notification of the administrator for processing. The quantity information of the goods is synchronously recorded through the quantity verification module 504 for checking the quantity of the goods when entering the warehouse. After the real-time update module 505 allocates a cargo space, the allocated cargo space is eliminated in the information verification form, and after a cargo space is vacated, the vacated cargo space is added to the information verification form. The inventory management module 506 is used to manage inventory information. When the goods are entered into the warehouse, the goods entry and exit records on the information verification form are in the warehouse entry state. When the goods are out of the warehouse, the goods entry and exit records on the information verification form are in the outbound state. The allocated cargo space in the warehouse is scanned regularly according to the information verification form. If there is any inventory, If there is a vacancy in the allocated storage location or a non-vacancy in the unallocated storage location, the reporting system will notify the administrator. When executing the outbound delivery, the outbound delivery review unit 6 will check the location of the goods through the information verification form. The robotic arm will receive the outbound delivery instruction issued by the system and take pictures of the goods in the target storage location. During this process, the robotic arm will conduct a review and inspection again through the visual calibration unit 2, and match the read barcode of the goods with the data on the information verification form. When the goods to be shipped out are consistent with the information recorded in the information verification form, the goods will be shipped out normally, and after the goods are completely shipped out, the inbound and outbound records in the information verification form will be changed to the outbound status.

[0018] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0019] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The warehouse in and out review management system based on automatic visual scanning is characterized by: include: A posture acquisition unit (1), the posture acquisition unit (1) being used to acquire a color image and a depth image of target goods in a to-be-identified area, thereby obtaining posture information of the goods; A visual calibration unit (2), wherein the visual calibration unit (2) obtains a relationship matrix between a camera coordinate system and a robot coordinate system by calculating a rotation matrix and a translation matrix, and converts the spatial coordinates in an image captured by the camera into the robot coordinate system; A cargo reading unit (3), wherein the cargo reading unit (3) obtains cargo data in a camera coordinate system based on cargo positioning information, and controls the robotic arm to complete image capture of a single cargo based on the cargo in the robotic arm coordinate system, thereby obtaining a high-definition image, and then analyzes cargo barcode and cargo name information on the image, thereby obtaining cargo quantity information; A record management unit (4), wherein the record management unit (4) performs a warehouse entry inspection on the entire pile of goods, checks whether the quantity of the goods is consistent with the purchase order, checks whether the packaging is intact, and confirms that the barcode information of the goods is accurate before proceeding to the warehouse. During the warehouse entry process, an information verification sheet is prepared in real time and input into the entire system, the areas within the warehouse are divided, and the shelf information, cargo location information, inventory information and equipment information of the warehouse are managed; An incoming warehouse verification unit (5), the incoming warehouse verification unit (5) is used to associate a mechanical arm with a system controller, the mechanical arm is provided with a high-definition camera, the scanning range of the high-definition camera includes the goods themselves and the barcode information of the cargo location, when the goods are in or out of the warehouse, the mechanical arm obtains the information verification sheet and reads the cargo information, automatically scans the cargo barcode through the high-definition camera and the visual calibration unit (2) to allocate the cargo location, and verifies the allocated cargo location and the quantity of the goods; The outbound review unit (6) checks the location of the goods through the information verification sheet when executing the outbound review. The robot arm receives the outbound instruction issued by the system and photographs the goods at the target location. During this process, the robot arm performs a review check again through the visual calibration unit (2) and matches the read barcode of the goods with the data on the information verification sheet. When the read goods information matches the information recorded in the information verification sheet, the goods are shipped out normally, and after the goods are completely shipped out, the inbound and outbound records in the information verification sheet are changed to the outbound status.

2. The warehouse entry and exit review management system based on automatic visual scanning according to claim 1 is characterized by: The posture acquisition unit (1) comprises an image calibration module (101), an image correction module (102) and a registration and alignment module (103). The image calibration module (101) uses The black and white chessboard is used as the calibration board. The size of each square on the chessboard is , manually changing the position of the calibration plate within the camera's field of view, obtaining calibration images of the calibration plate at multiple different angles, the image correction module (102) graying the obtained calibration image, then detecting corner points of the checkerboard grid points on the calibration plate, and finally finding sub-pixel corner points based on the original corner points, the sub-pixel corner points are real number coordinates obtained by iterative calculation using the least squares method, obtaining the camera's intrinsic parameter matrix and distortion coefficients according to the sub-pixel corner point coordinates, and saving the obtained intrinsic parameter matrix and distortion coefficients in a txt file format for later use, and used to correct the image captured by the camera, the registration and alignment module (103) uses an RGB lens to capture a color image and uses an IR lens to capture a depth image, the color image is used for target detection, the depth image is used to measure the depth data of the detected target object, and the color image and the depth image are registered and aligned using a registration algorithm to obtain a conversion relationship between the color image and the depth image, so that the color image and the depth image overlap.

3. The warehouse entry and exit review management system based on automatic visual scanning according to claim 1 is characterized by: The visual calibration unit (2) comprises the following execution steps: Suppose the point set in the camera coordinate system is , the point set in the robot coordinate system is , and obtain the rotation matrix between the two point sets and translation matrix , establish the following formula: ; First, calculate the center point of the point set in the camera and robotic arm coordinate systems. The calculation formula is: ； in, is the point in the camera coordinate system, represents the point in the robot coordinate system, represents a point in the coordinate system, Respectively represent the three-axis coordinates of the coordinate system, and Respectively represent the center point of the point set in the camera and robotic arm coordinate systems, represents the number of points in the coordinate system, Indicates the point number, Represents a transpose operation; Next, calculate the optimal rotation matrix , by calculating the center point of the point set in the camera and robot coordinate system, a new point set is generated and , the point set is re-centered as: ； Then calculate the covariance matrix between the point sets : ； The covariance matrix Decompose the feature matrix by singular value decomposition and the feature matrix , and by the formula Obtain the rotation matrix between point sets ; Finally, calculate the translation matrix : ； By finding the rotation matrix and translation matrix Get the relationship matrix between the camera coordinate system and the robotic arm coordinate system , the relationship matrix The formula is: .

4. The warehouse entry and exit review management system based on automatic visual scanning according to claim 1 is characterized by: The cargo reading unit (3) comprises an information acquisition module (301) and a cargo positioning module (302). The information acquisition module (301) identifies the type of cargo according to the barcode of the cargo, and performs image filtering and image thresholding on the barcode, uses Gaussian filtering to filter out Gaussian noise in the image, performs image thresholding on the filtered image to obtain a binary image, thereby separating the target image from the background. The cargo positioning module (302) obtains cargo positioning information by using a multi-angle template matching positioning method.

5. The warehouse entry and exit review management system based on automatic visual scanning according to claim 1 is characterized by: The record management unit (4) comprises an information setting module (401), wherein the information setting module (401) is used to define the input information of the information verification form and the goods barcode, wherein the information verification form comprises a table of goods and allocated goods locations, a table of quantities of goods of different categories to be stored, a task order table for storing task orders, an information table for recording basic parameters of products, and a record of goods entering and leaving the warehouse, and the barcode information of each good records the product information and weight information of the good.

6. The warehouse entry and exit review management system based on automatic visual scanning according to claim 5 is characterized by: The record management unit (4) also includes a warehouse management module (402), which is used to manage shelf information, cargo location information, inventory information and equipment information. Shelf information management is to configure corresponding parameter information for each shelf and bind it with the corresponding area. Cargo location information management is to bind each cargo location with the corresponding shelf, and provide a cargo location barcode that records the cargo location number and the spatial coordinate information. Inventory information management is to record and display the inventory data in the warehouse, and display the detailed information of the stored goods in each cargo location in the entire system. Equipment information management is to uniformly manage the intelligent storage equipment in the warehouse.

7. The warehouse entry and exit review management system based on automatic visual scanning according to claim 1 is characterized by: The warehousing review unit (5) comprises a warehousing execution module (501), a goods allocation module (502) and an information matching module (503). When the warehousing execution module (501) performs warehousing review, the warehouse staff first places the goods on the warehousing platform according to the prompt of the LED screen on the conveyor of the warehousing platform. After confirming that the warehousing operation is to be executed through the barcode recognition of the goods, the RGV transports the goods to the conveyor at the corresponding lane entrance. After the goods are placed on the conveyor at the lane entrance, the conveyor transports them to the warehousing entrance. During the warehousing process of the goods, the visual calibration unit (2) performs a review and inspection to ensure that the goods are correctly warehousing. The goods allocation module (502) ) After obtaining the barcode information of the goods, the barcode information is associated with the information verification form, so as to allocate the cargo location according to the data recorded in the barcode information, and the allocated cargo location is recorded in the information verification form. The information matching module (503) places the goods at the position allocated in the information verification form, and at the same time scans the cargo location barcode information to confirm whether the allocated cargo location matches the barcode information marked on the cargo location. If it matches, the goods entry and exit record in the information verification form is set to have been entered into the warehouse. Otherwise, the cargo location is reallocated and stored according to the reallocated cargo location. If it still does not match, an alarm is issued. The entire system is connected to the mobile terminal network to facilitate notification of the administrator for processing.

8. The warehouse entry and exit review management system based on automatic visual scanning according to claim 7 is characterized by: The warehousing verification unit (5) further comprises a quantity verification module (504), a real-time update module (505) and an inventory management module (506). The quantity verification module (504) synchronously records the corresponding quantity information of the goods entering the warehouse, and is used to verify the quantity of the goods when entering the warehouse. After the real-time update module (505) allocates a cargo space, it eliminates the allocated cargo space in the information verification form, and after a cargo space is vacated, it adds the vacated cargo space in the information verification form. The inventory management module (506) is used to manage inventory information. When the goods enter the warehouse, the goods entry and exit records on the information verification form are in the warehouse entry status. When the goods leave the warehouse, the goods entry and exit records on the information verification form are in the warehouse exit status. The allocated cargo space in the warehouse is scanned regularly according to the information verification form. If there is a vacancy in the allocated cargo space or a non-vacancy in the unallocated cargo space, the system is reported to notify the administrator.

Images