Method, device, computer equipment and storage medium for checking goods out of warehouse

Abstract

The present application relates to a method, device, computer equipment, storage medium and computer program product for cargo outbound verification. The method comprises: when cargo is detected to arrive at an outbound platform, an outbound image of the cargo is obtained, the cargo is placed on a pallet, and an identification mark is provided on the outer surface of each pallet; according to the identification mark of the identified pallet, an inbound image collected when the cargo is inbound is obtained; target recognition is performed on the inbound image, cargo in the inbound image is extracted, and a first cargo image is obtained; target recognition is performed on the outbound image, cargo in the outbound image is extracted, and a second cargo image is obtained; feature matching is performed on the first cargo image and the second cargo image, and if the match is successful, it is determined that the outbound verification is successful. The use of this method can effectively avoid the occurrence of inconsistent outbound cargo due to verification errors, maintain the normal operation of the automated warehousing and logistics system, and improve production efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of warehousing and logistics, and in particular to a method, device, computer equipment, storage medium and computer program product for checking the goods leaving a warehouse. Background Art

[0002] With the development of warehouse logistics management technology, automated warehouse logistics management has become one of the basic management methods for many industrial production enterprises. In the automated warehouse logistics system, pallets with uniform specifications and unique identification are often used to carry goods. The pallets and goods are kept in the shelves as a whole. The handling mechanism generally uses AGV trolleys, manual transport vehicles, conveyor belts, stackers, etc. to carry and transport pallets carrying goods to realize the automated transportation of goods.

[0003] In industrial production, for practical management needs, the goods carried on the pallet must be consistent when entering and leaving the warehouse. In traditional technology, the WCS system is usually used to verify the cargo information of the incoming and outgoing goods through the unique identification of the pallet to ensure the consistency of the incoming and outgoing goods. However, in the actual production process, the goods may not be restored in time after manual operation after entering the warehouse, or there may be a collision during the warehousing process, resulting in inconsistent cargo shapes. If only the WCS system is used to verify the cargo information, verification errors are prone to occur, resulting in the inconsistency between the outgoing and incoming goods, causing the automated warehousing and logistics system to operate in a disorderly manner and reduce production efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for verifying whether the incoming and outgoing goods are consistent with each other in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a method for checking the goods leaving a warehouse, the method comprising:

[0006] When it is detected that the goods arrive at the outbound platform, an outbound image of the goods is obtained, wherein the goods are placed on a pallet, and an identification mark is provided on the outer surface of each pallet;

[0007] According to the identified identification mark of the pallet, an entry image of the goods collected when entering the warehouse is obtained;

[0008] Performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image; performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image;

[0009] The first cargo image and the second cargo image are feature matched, and if the match is successful, it is determined that the outbound inspection is successful.

[0010] In one embodiment, the acquiring of the storage image captured when the goods are stored in storage according to the identified identification mark of the pallet includes:

[0011] According to the identified identification mark of the pallet, the storage information stored when the goods are stored is obtained;

[0012] According to the identification mark, obtaining the outbound information of the goods;

[0013] Performing basic verification on the incoming warehouse information and the outgoing warehouse information;

[0014] If the basic verification is successful, the storage image captured when the goods are stored is obtained.

[0015] In one embodiment, the step of performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image, and performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image, includes:

[0016] Inputting the warehouse entry image into a pre-trained image target detection model for target recognition, extracting a first key feature of the goods in the warehouse entry image, and obtaining a first goods image containing the first key feature;

[0017] The outbound image is input into the image target detection model for target recognition, and a second key feature of the goods in the outbound image is extracted to obtain a second goods image containing the second key feature.

[0018] In one embodiment, the key feature includes at least one of an appearance outline, a trademark, an icon, text, and a barcode of the goods in the image; and the feature matching of the first goods image and the second goods image includes:

[0019] Performing an image alignment operation on the first cargo image and the second cargo image to obtain an aligned first cargo image and a second cargo image;

[0020] Feature matching is performed on an attribute of a first key feature in the aligned first image and an attribute of a second key feature in the aligned second image.

[0021] In one embodiment, the method further comprises:

[0022] If the matching is unsuccessful, obtaining unmatched key features in the first cargo image and the second cargo image;

[0023] Determine whether each of the unmatched key features meets the preset feature exclusion rules. If so, determine that the outbound verification is successful.

[0024] In one embodiment, when the goods are detected to arrive at the outbound platform, the outbound image of the goods is collected, including:

[0025] When it is detected that the goods arrive at the outbound platform, a time series image of the outbound goods collected within a preset time period is obtained;

[0026] Performing a scoring operation on the outbound time series images according to a preset scoring rule;

[0027] The image with the highest score in the warehouse-out time series images is determined as the warehouse-out image of the goods.

[0028] In a second aspect, the present application also provides a cargo outbound inspection device, the device comprising:

[0029] An outbound image acquisition module is used to acquire an outbound image of the goods when it is detected that the goods arrive at the outbound platform, the goods are placed on a pallet, and an identification mark is provided on the outer surface of each pallet;

[0030] A storage image acquisition module, used to acquire a storage image captured when the goods are stored according to the identified identification mark of the pallet;

[0031] An image target recognition module is used to perform target recognition on the incoming image, extract the goods in the incoming image, and obtain a first goods image, and to perform target recognition on the outgoing image, extract the goods in the outgoing image, and obtain a second goods image;

[0032] The feature matching module is used to perform feature matching on the first cargo image and the second cargo image. If the matching is successful, it is determined that the warehouse exit verification is successful.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0035] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0036] The above-mentioned cargo outbound verification method, device, computer equipment, storage medium and computer program product, when detecting that the cargo arrives at the outbound platform, collects the outbound image of the cargo, and obtains the inbound image collected when the cargo enters the warehouse according to the identification mark of the pallet holding the cargo. Perform target recognition on the inbound image, extract the cargo in the inbound image, and obtain the first cargo image; perform target recognition on the outbound image, extract the cargo in the outbound image, and obtain the second cargo image. Perform feature matching on the first cargo image collected at the time of entry and the second cargo image collected at the time of exit, and when it is determined that the cargo at the time of entry is consistent with the cargo at the time of exit, that is, when the match is successful, the cargo is then shipped out, which effectively avoids the inconsistency of the outbound cargo caused by verification errors, maintains the normal operation of the automated warehousing and logistics system, and improves production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an application environment diagram of a cargo outbound verification method in one embodiment;

[0038] Figure 2 A schematic diagram of a flow chart of a method for checking the goods leaving a warehouse in one embodiment;

[0039] Figure 3 It is a flowchart of the steps of obtaining the storage image collected when the goods are stored according to the identification mark of the identified pallet in one embodiment;

[0040] Figure 4 is a schematic diagram of a first cargo image in one embodiment;

[0041] Figure 5 It is a flowchart of the step of collecting the outbound image of goods when it is detected that the goods arrive at the outbound platform in one embodiment;

[0042] Figure 6 It is a flow chart of a method for checking the goods leaving the warehouse in another embodiment;

[0043] Figure 7 It is an application environment diagram of a cargo outbound verification method in another embodiment;

[0044] Figure 8 A schematic diagram of the output result of a failed goods delivery inspection in an embodiment;

[0045] Fig. 9 It is a structural block diagram of a device for checking the goods leaving the warehouse in one embodiment;

[0046] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The cargo outbound verification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the cargo verification system 102 communicates with the image acquisition device 104 through the network. The data storage system 106 can store the data that the cargo verification system 102 needs to process. The data storage system can be integrated on the cargo verification system 102, or it can be placed on the cloud or other network servers. When it is detected that the cargo arrives at the outbound platform, the cargo verification system 102 obtains the outbound image of the cargo collected by the image acquisition device 104, wherein the cargo is placed on a pallet, and an identification mark is set on the outer surface of each pallet. The cargo verification system 102 obtains the incoming image of the cargo collected when the cargo enters the warehouse from the data storage system 106 according to the identification mark of the identified pallet; performs target recognition on the incoming image, extracts the cargo in the incoming image, obtains the first cargo image, performs target recognition on the outbound image, extracts the cargo in the outbound image, and obtains the second cargo image; performs feature matching on the first cargo image and the second cargo image, and if the match is successful, it is determined that the outbound verification is successful. Among them, the cargo verification system 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc., and can also be implemented by an independent server or a server cluster composed of multiple servers. The image acquisition device 104 has the function of continuously acquiring images.

[0049] In one embodiment, Figure 2 As shown, a method for checking the goods leaving the warehouse is provided, and the method is applied to Figure 1 Taking the cargo verification system 102 in the example as an example, the following steps are included:

[0050] Step 202: When it is detected that the goods arrive at the outbound platform, an outbound image of the goods is obtained. The goods are placed on a pallet, and an identification mark is set on the outer surface of each pallet.

[0051] Among them, the outbound platform is a platform provided for checking the outbound goods when the goods are outbound. The outbound platform is equipped with necessary image acquisition equipment, goods outbound sensing equipment and pallet identification equipment. It can be understood that when the warehouse control system issues a goods outbound order, the transportation equipment transports the goods to the outbound platform for outbound verification, and the goods can only be outbound after the outbound verification is confirmed to be successful.

[0052] The outbound image of the goods is an image captured by an image acquisition device installed at the outbound platform when the goods are transported to the outbound platform. It can be understood that the outbound image can be an image of the goods captured by the image acquisition device of the outbound platform controlled by the goods verification system after the goods arrive at the outbound platform, or it can be an image of the goods captured by an image acquisition device obtained within a preset time interval after the goods verification system receives information that the goods have arrived at the outbound platform.

[0053] Among them, the pallet is a medium that transforms static goods into dynamic goods, a cargo platform, and an active platform, or a movable ground. Even if the goods are placed on the ground and lose their flexibility, they will immediately become active and become flexible mobile goods once they are loaded on the pallet. The outer surface of each pallet, such as the side, will be provided with a unique identification mark, such as a pallet code. The warehouse control system manages the automated logistics operations of the goods according to the unique identification mark of the pallet. It can be understood that in the embodiment of the present application, the goods are all placed on the pallet for transportation and verification.

[0054] Specifically, the goods are placed on a pallet. After receiving the outbound instruction sent by the warehouse control system, the transportation equipment transports the pallet carrying the goods to the outbound platform. When it is detected that the goods are transported to the outbound platform by the transportation equipment, the cargo verification system obtains the outbound image corresponding to the goods.

[0055] In one of the embodiments, a sensor is provided on the outbound platform for detecting whether the goods have arrived at the outbound platform. When the sensor detects that the goods have arrived at the outbound platform, a goods arrival signal is sent to the warehouse control system. After receiving the signal, the warehouse control system sends a goods arrival signal to the outbound platform to the goods verification system, notifying the goods verification system that the goods have arrived at the outbound platform.

[0056] Step 204, obtaining a storage image captured when the goods are stored in the storage according to the identified identification mark of the pallet.

[0057] Among them, the entry image is the image captured by the image acquisition device of the entry platform when the goods that need to be shipped out are entering the warehouse. It can be understood that after the entry image corresponding to the goods entering the warehouse is acquired, the cargo verification system will bind the entry image and the pallet code of the cargo pallet and store them in the data storage system of the cargo verification system, so as to facilitate quick search and use in subsequent comparison. It can be understood that the entry platform and the shipment platform are two different verification platforms.

[0058] Specifically, according to the identified identification mark of the pallet, the cargo verification system searches the data storage system for the storage image of the cargo corresponding to the identification mark that was captured when the cargo was stored.

[0059] In one of the embodiments, a scanner is provided on the outbound platform for scanning and identifying the pallet identification mark.

[0060] In one embodiment, since the pallet is recycled in the automated material system, one identification mark in the cargo verification system may correspond to multiple cargo entry images. At this time, according to the identified identification mark of the pallet, the captured entry image of the cargo entry is obtained, including: according to the identified identification mark of the pallet, the time when the corresponding entry image is captured is obtained, and the entry image with the smallest time interval between the capture time and the current time is determined as the cargo entry image.

[0061] Step 206, performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image, performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image.

[0062] Among them, object recognition is the operation of identifying and marking the goods in the image.

[0063] Specifically, the cargo verification system performs target recognition on the incoming image, identifies and extracts the cargo in the outgoing image, and obtains a first cargo image; and performs the same target recognition operation on the outgoing image, identifies and extracts the cargo in the outgoing image, and obtains a second cargo image. It can be understood that the target recognition process can use a dedicated target recognition algorithm or a pre-trained target detection model.

[0064] Step 208: feature matching is performed between the first cargo image and the second cargo image. If the matching is successful, it is determined that the outbound inspection is successful.

[0065] Specifically, the cargo verification system performs feature matching on the first cargo image obtained based on the warehouse entry image and the second cargo image obtained based on the warehouse exit image. If the match is successful, it is considered that the cargo to be shipped out is consistent with the cargo when it was shipped in. After the cargo is shipped in, there is no loss, transfer, drop, collision damage, etc. At this time, it is determined that the cargo outbound verification is successful, and the subsequent outbound operation can be performed. It can be understood that feature matching can be feature point matching in the image, cargo contour feature matching, or cargo key point area matching, etc., and this application does not limit this.

[0066] In the above-mentioned cargo outbound verification method, when it is detected that the cargo arrives at the outbound platform, the outbound image of the cargo is collected, and the inbound image collected when the cargo enters the warehouse is obtained according to the identification mark of the pallet holding the cargo. The inbound image is subjected to target recognition, and the cargo in the inbound image is extracted to obtain a first cargo image; the outbound image is subjected to target recognition, and the cargo in the outbound image is extracted to obtain a second cargo image. The first cargo image collected when entering the warehouse is matched with the second cargo image collected when leaving the warehouse for feature matching, and when it is determined that the cargo at the time of entering the warehouse is consistent with the cargo at the time of leaving the warehouse, that is, when the match is successful, the cargo is then shipped out of the warehouse, which effectively avoids the inconsistency of the outbound cargo caused by the verification error, maintains the normal operation of the automated warehousing and logistics system, and improves the production efficiency.

[0067] In one embodiment, in order to eliminate the influence of objects piled up outside the platform, a region of interest (ROI) is set for each platform, wherein the ROI setting method can be manual setting or automatic setting.

[0068] Specifically, when setting the ROI manually, a slightly larger maximum field of view than the pallet and cargo is selected as the ROI; when setting the ROI automatically, a large number of images of pallet cargo taken at each station are collected, and the location of the cargo in the image is identified by an algorithm. After integration, the edge positions of all cargo in the same station are obtained [x_roi_min, y_roi_min, x_roi_max, y_roi_max], that is, [minimum x pixel value, minimum y pixel value, maximum x pixel value, maximum y pixel value], and then a redundant threshold α is set, and the final ROI can be set to [x_roi_min-α, y_roi_min-α, x_roi_max+α, y_roi_max+α]. It can be understood that cargo identification can be accomplished by using a recognition algorithm or a recognition model.

[0069] In one embodiment, Figure 3 As shown, according to the identified identification mark of the pallet, the warehouse entry image collected when the goods are put into the warehouse is obtained, including:

[0070] Step 302, based on the identified identification mark of the pallet, obtain the storage information stored when the goods are stored.

[0071] Among them, the warehousing information is the cargo information uploaded when the cargo is put into storage. The warehousing information may include the cargo pallet identification mark, cargo category, barcode information, warehousing time, etc.

[0072] Specifically, when goods enter the warehouse, the warehouse control system will bind the goods entry information, the goods entry image and the identification mark of the goods pallet together and send them to the goods verification system for storage. When it is detected that the goods have arrived at the outbound platform, the goods verification system obtains the entry information corresponding to the identification mark from the data storage system according to the identification mark of the pallet.

[0073] Step 304, obtaining the goods' outbound information according to the identification mark.

[0074] The outbound information is the cargo information uploaded when the cargo is outbound, and the outbound information may include the cargo category, barcode information, outbound destination, etc. of the cargo.

[0075] Specifically, the cargo verification system obtains the cargo delivery information corresponding to the identification mark according to the identification mark. It is understandable that the cargo delivery information can be generated and sent to the cargo verification system at the same time when the warehouse control system generates the cargo delivery instruction, or can be sent to the cargo verification system in real time according to the identification mark when the warehouse control system determines that the cargo has arrived at the delivery platform.

[0076] Step 306, perform basic verification on the incoming and outgoing information.

[0077] Among them, the incoming information and outgoing information of the same goods differ only in a few parameters such as the outgoing destination, and other parameters related to the goods themselves, such as the type of goods, barcode information, etc., are the same. Therefore, the goods verification system can perform basic verification based on the information with the same parameters in the outgoing information and the incoming information to determine whether the outgoing goods and the goods corresponding to the acquired incoming information are the same goods.

[0078] Specifically, the cargo verification system performs basic verification on the incoming information and the outgoing information according to the same parameters in the incoming information and the outgoing information.

[0079] Step 308: If the basic verification is successful, the storage image captured when the goods are stored is obtained.

[0080] Specifically, if the basic verification is successful, it means that the current outbound goods and the incoming goods corresponding to the incoming information are the same goods. Therefore, the incoming image corresponding to the incoming information is the image captured when the current outbound goods enter the warehouse, and the image is obtained as the incoming image for subsequent image matching verification. If the basic verification is unsuccessful, it means that the current outbound goods and the incoming goods corresponding to the incoming information are not the same goods. This may be caused by manual interference or control errors of the warehouse control system. At this time, the cargo verification system directly determines that the outbound verification has failed and notifies manual intervention.

[0081] In this embodiment, a basic check is performed in advance on the outbound information of the goods obtained based on the pallet identification mark and the inbound information of the goods. Only when the basic check is successful will the operation steps of obtaining the inbound image corresponding to the inbound information and matching it with the outbound image of the goods be performed. If the basic check is unsuccessful, the goods verification system will directly determine that the outbound verification has failed, and there is no need to perform subsequent image matching operations, which can greatly reduce the amount of calculation in the outbound verification process and improve the efficiency of the outbound verification.

[0082] In one embodiment, performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image, and performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image, include:

[0083] Inputting the incoming image into a pre-trained image target detection model for target recognition, extracting the first key feature of the goods in the incoming image, and obtaining a first goods image containing the first key feature;

[0084] The outbound image is input into the image target detection model for target recognition, and the second key feature of the goods in the outbound image is extracted to obtain a second goods image containing the second key feature.

[0085] The target detection model is a pre-trained model used to identify and mark targets contained in an image. Specifically, the target detection model performs target recognition on the input image and outputs an image with key features marked.

[0086] Among them, the key feature is a feature in the image that can be used to identify the consistency of the goods. By matching the key features, it can be determined whether the goods contained in the two images are the same goods. It can be understood that the key feature in the same image can be one or more, and the number of key features is set according to actual conditions, and this application does not limit this.

[0087] Take the case of tobacco boxes containing tobacco to be shipped out as an example. Each brand of tobacco has its own tobacco box. The brand name, barcode and other features corresponding to the brand on the surface of the tobacco box are identified as key features. The deep learning model is trained with a large amount of training data to obtain the final image object detection model. The image object detection model can identify and mark features such as brand names and barcodes on the surface of the tobacco box in the image, and output an image with marked key features.

[0088] Specifically, the cargo verification system inputs the acquired cargo entry image into the image target detection model, and the image target detection model performs target recognition on the entry image, identifies and marks the key features of the cargo in the entry image as the first key features, and outputs the first cargo image containing the first key features. The acquired cargo exit image is input into the image target detection model, and the image target detection model performs target recognition on the exit image, identifies and marks the key features of the cargo in the exit image as the second key features, and outputs the second cargo image containing the second key features.

[0089] In this embodiment, the key features in the goods in-and-out images are respectively identified and marked by a pre-trained image target detection model, and the in-warehouse image containing the first key feature is output as the first goods image, and the out-warehouse image containing the second key feature is output as the second goods image. When the first image and the second image are subsequently feature matched, the first key feature of the first image can be directly matched with the second key feature of the second image, so that the entire out-warehouse verification process is faster and more accurate.

[0090] In one embodiment, the key feature includes at least one of an appearance outline, a trademark, an icon, text, and a barcode of the goods in the image.

[0091] Performing feature matching on the first cargo image and the second cargo image includes: performing an image alignment operation on the first cargo image and the second cargo image to obtain aligned first cargo image and second cargo image; and performing feature matching on an attribute of a first key feature in the aligned first image and an attribute of a second key feature in the aligned second image.

[0092] The attribute of the key feature includes at least one of the content and area of ​​the key feature. For example, when the key feature is a pattern or text, the attribute of the key feature can be the content of the key feature, and whether the feature matches is determined based on whether the content is consistent; when the key feature is a feature marked by a rectangular box, the attribute of the key feature is the area of ​​the key feature, and whether the feature matches is determined based on the intersection over union (IOU) of the feature matrix.

[0093] Specifically, an image alignment operation is performed on the first cargo image and the second cargo image according to the key features identified and marked by the image object detection model to obtain the aligned first cargo image and the second cargo image. Feature matching is performed on the attributes of the first key feature in the aligned first image and the attributes of the second key feature in the aligned second image. It can be understood that the image alignment operation can be implemented by an image alignment algorithm or model.

[0094] In one embodiment, the key feature includes an outline of the goods in the image and a barcode, and the key feature is marked by a rectangular frame. Performing an image alignment operation on the first goods image and the second goods image according to the key feature includes: performing an alignment operation on the first goods image and the second goods image according to the coordinates of the key feature rectangular frame.

[0095] Taking the case where the outgoing goods are in the shape of boxes as an example, the incoming images captured are input into the image object detection model. The image object detection model marks the key features, i.e., the box outline and the barcode, with a rectangular frame and outputs the first goods image. The first goods image is shown in FIG. Figure 4 As shown, the rectangular boxes corresponding to A, B, C, and D in the image are marked with the key features of the box outline, and the rectangular boxes corresponding to E, F, G, and H are marked with the key features of the barcode. The positions of all recognized rectangular boxes determine the pixel positions of the image area occupied by the goods, that is, [the minimum x coordinate of the incoming image (in_x_min), the minimum y coordinate of the incoming image (in_y_min), the maximum x coordinate of the incoming image (in_x_max), the maximum y coordinate of the incoming image (in_y_max)], and similarly obtain the pixel positions of the area occupied by the goods in the outgoing image, that is, [the minimum x coordinate of the outgoing image (out_x_min), the minimum y coordinate of the outgoing image (out_y_min), the maximum x coordinate of the outgoing image (out_x_max), the maximum y coordinate of the outgoing image (out_y_max)]. Calculate the average of the two corresponding coordinates in the two sets of coordinates, and obtain the average coordinates [align the minimum x coordinate (scaled_x_min), align the minimum y coordinate (scaled_y_min), align the maximum x coordinate (scaled_x_max), align the maximum y coordinate (scaled_y_max)], and scale the corresponding coordinate values ​​of the first cargo image and the second cargo image to the size of the average coordinates. Obtain the aligned first cargo image and the second cargo image. And align all the rectangular boxes of the key targets in the first cargo image and the second cargo image to the size of the aligned rectangular box.

[0096] Assuming that the coordinates of any key feature rectangular box in the first cargo image are marked as the coordinates of the upper left corner and the lower right corner, that is, [in_x1, in_y1, in_x2, in_y2], the coordinates of the projection to the aligned key feature rectangular box are converted as follows:

[0097] k_x=(scaled_x_max–scaled_x_min) / (in_x_max–in_x_min);

[0098] k_y=(scaled_y_max–scaled_y_min) / (in_y_max–in_y_min);

[0099] scaled_in_x1=(in_x1–in_x_min)*k_x;

[0100] scaled_in_y1=(in_y1–in_y_min)*k_y;

[0101] scaled_in_x2=(in_x2–in_x_min)*k_x;

[0102] scaled_in_y2=(in_y2–in_y_min)*k_y;

[0103] After conversion, the coordinates of the key feature rectangular box in the first cargo image are expressed as [scaled_in_x1, scaled_in_y1, scaled_in_x2, scaled_in_y2]. Similarly, the coordinates of the key feature rectangular box in the second cargo image can be calculated as [scaled_out_x1, scaled_out_y1, scaled_out_x2, scaled_out_y2].

[0104] At this time, the attribute of the key feature is the area of ​​the rectangular frame. The attribute of the first key feature of the aligned first image is matched with the attribute of the second key feature in the aligned second image, including: calculating the area of ​​each key feature rectangular frame according to the coordinates of the aligned key feature rectangular frame, and calculating the intersection and union ratio of the area of ​​each first key feature rectangular frame and each second key feature rectangular frame. The obtained area intersection and union ratios are arranged in order from large to small, and the key feature group with the largest intersection and union ratio and the intersection and union ratio that meets the preset threshold is selected and identified as the matching key feature.

[0105] In the above embodiment, by aligning the first cargo image output by the image object detection model with the second cargo image, the attribute of the first key feature of the aligned first image is matched with the attribute of the second key feature in the aligned second image. This can effectively reduce the influence of errors caused by external factors such as the image acquisition angle on the feature matching results, and improve the accuracy of the warehouse inspection process.

[0106] In one embodiment, the cargo outbound verification method further includes:

[0107] If the match is unsuccessful, the unmatched key features in the first cargo image and the second cargo image are obtained; it is determined whether each unmatched key feature meets the preset feature exclusion rule, and if so, it is determined that the outbound verification is successful.

[0108] Among them, the unmatched key features are key features that cannot correspond to each other in the first cargo image and the second cargo image. When there are unmatched key features, it means that the cargo in the first cargo image, i.e., the incoming cargo, is different from the cargo in the second cargo image, i.e., the outgoing cargo.

[0109] Among them, the preset feature exclusion rule is a feature exclusion rule set to reduce the feature matching error caused by target detection error, and is used to exclude unmatched key features that meet the preset rules.

[0110] Specifically, since the inbound image acquisition device and the outbound image acquisition device are two different devices and are set at two different stations, it is inevitable that there is an error in the shooting angle. At the same time, when the image target detection model outputs the image after target recognition, there is also a certain probability that there will be a recognition error. Therefore, in order to reduce the error in the feature matching result caused by the target detection error, the preset feature exclusion rule is pre-set according to the actual situation of the model and the image acquisition area. When there are unmatched key features that are not matched successfully, it is determined whether each unmatched key feature meets the preset feature exclusion rule. If it is satisfied, it means that the failure of the key feature to match successfully is caused by the target detection error, so the key feature is excluded from the unmatched key features. If all the unmatched key features meet the preset feature exclusion rule, it means that the goods in the first cargo image and the second cargo image are consistent at this time, that is, the inbound cargo and the outbound cargo are consistent, and it is determined that the outbound verification is successful. If there are key features in the unmatched key features that cannot meet the preset feature exclusion rule, it means that the inbound cargo and the outbound cargo are inconsistent at this time, and the inbound cargo has changed after entering the warehouse. At this point, the cargo verification system will determine that the outbound verification has failed, identify the key features that ultimately did not match or the cargo corresponding to the key features in the input image, and generate a cargo verification anomaly alarm that is returned to the warehouse control system.

[0111] In this embodiment, by using preset feature exclusion rules to screen again the key features that have not been successfully matched, it is possible to effectively avoid errors in feature matching results caused by target detection errors, thereby effectively improving the accuracy of the warehouse inspection process.

[0112] In one of the embodiments, the preset feature exclusion rule includes at least one of the target detection confidence being lower than a preset confidence threshold or the cargo coverage area being higher than a preset area threshold.

[0113] Specifically, the target detection confidence is information about whether the recognition result given by the image target detection model is accurate. The closer the target detection confidence is to 1, the higher the accuracy of the recognition result obtained by the model. The closer the target detection confidence is to 0, the less accurate the recognition result obtained by the model. Therefore, if the target detection confidence corresponding to the unmatched key feature is lower than the preset confidence threshold, it is considered that the recognition of the key feature is inaccurate at this time, and its failure to match is most likely caused by recognition errors. Therefore, this key feature is excluded from the unmatched key features through the preset feature exclusion rules.

[0114] The cargo coverage area refers to the area of ​​a certain cargo covered by other items in the image. In actual production, the cargo carried by a pallet is generally multi-layered. Since the image acquisition devices and platforms corresponding to the in-and-out images are different, some cargo may be captured by the image acquisition device when entering the warehouse due to external factors such as shooting angles. When the uncovered area of ​​the cargo reaches a certain value, there is a chance that it will be recognized as a key feature by the image target detection model. If the area of ​​the cargo that can be collected is less than a certain value when leaving the warehouse, it cannot be recognized by the image target detection model. This key feature will be classified as an unmatched key feature, which may result in an unsuccessful outbound verification. Therefore, this key feature is excluded from the unmatched key features through the preset feature exclusion rules to improve the accuracy of the outbound verification results.

[0115] It can be understood that the preset confidence threshold and the preset area threshold can be set according to actual conditions. In this embodiment, the preset confidence threshold is set to 0.9, and the preset area threshold is set to 0.5.

[0116] In one embodiment, Figure 5 As shown, when it is detected that the goods arrive at the outbound platform, the outbound image of the goods is collected, including the following steps:

[0117] Step 502: when it is detected that the goods arrive at the outbound platform, a time series image of the outbound goods collected within a preset time period is obtained.

[0118] The preset time period is a preset time period during which the image acquisition device continuously acquires images of the outbound goods. The image acquisition device corresponds to a fixed image acquisition area. During the operation of the automated logistics transportation system, the image acquisition device continuously acquires images of the fixed image acquisition area to obtain a sequence of images within a certain time period.

[0119] Specifically, due to the common delay problem in the communication process, when the cargo verification system detects that the cargo has arrived at the outbound platform, it is very likely that the cargo has actually passed the outbound platform, or has passed the best image acquisition point of the outbound platform. If the cargo verification system obtains the image acquired by the image acquisition device in real time at this time, it is very likely that the image quality does not meet the requirements for feature matching. Therefore, when the cargo verification system detects that the cargo has arrived at the outbound platform, it obtains the outbound time series images of the cargo continuously acquired by the image acquisition device within a preset time period. Taking the image acquisition device acquiring 10 frames of images in 1 second as an example, when the cargo verification system detects that the cargo has arrived at the outbound platform, it acquires 10 frames of images acquired by the image acquisition device 1 second before the current time as the outbound time series images of the cargo.

[0120] Step 504: perform scoring operations on the outbound time series images according to preset scoring rules.

[0121] The preset scoring rules are rules for evaluating the quality of image capture. According to the preset scoring rules, corresponding scores can be determined for the outbound time series images. It is understandable that the preset scoring rules can be pre-set according to the actual application scenario and stored in the data storage system of the cargo verification system.

[0122] Specifically, the cargo verification system scores the acquired outbound time series images according to a preset scoring rule to obtain a score value corresponding to each outbound time series image.

[0123] In one embodiment, the preset scoring rule is set according to the center coordinates of the entire area of ​​the goods and the center coordinates of the screen. Specifically, the score of each outbound sequence image is determined according to the position distance between the center coordinates of the entire area of ​​the goods in each outbound sequence image and the center coordinates of the image screen. The closer the center coordinates of the entire area of ​​the goods are to the center coordinates of the image screen, the higher the score.

[0124] Step 506: determine the image with the highest score in the warehouse-out time series images as the warehouse-out image of the goods.

[0125] Specifically, the obtained image scores are arranged in order from high to low, and the image with the highest corresponding score is selected to be determined as the outbound image of the goods.

[0126] In this embodiment, by scoring the continuously acquired time series images, the image with the highest score is selected as the outbound image of the goods, thereby ensuring the quality of the subsequent target recognition images, further improving the accuracy of target detection, and thus improving the accuracy of the outbound verification results.

[0127] In one embodiment, Figure 6 As shown, a method for checking the goods out of the warehouse is provided, which can be applied in Figure 7 In the application environment shown, the goods in this embodiment are taken as boxed goods.

[0128] Among them, it is assumed that there is a gantry on the periphery of the entry and exit platform, and the gantry beam is located directly above the middle of the entry and exit platform, perpendicular to the running direction of the conveyor belt. The snapshot camera is hoisted on the gantry beam, and the camera is installed directly above the center point of the platform, higher than the maximum height of the pallet and the goods stacked to avoid collision with the transported goods, and the camera is set to shoot vertically downward. The photoelectric sensor is installed directly below the entry and exit platform, measuring vertically upward to detect whether the pallet cargo has entered the platform. It can be understood that a pallet code scanning device is also provided on the platform to scan the pallet code of the pallet. The pallet code scanning device can be set at any position on the platform as long as the pallet code can be scanned. The pallet code scanning device in this embodiment is not shown in the figure.

[0129] In order to eliminate the influence of the accumulation outside the platform, the area of ​​interest is set as a fixed image acquisition area on the platform. The camera continuously captures the pallet cargo passing through the fixed image acquisition area and collects images.

[0130] Specifically, the WCS (warehouse control system) sends a cargo entry instruction to the conveyor belt. The conveyor belt responds to the cargo entry instruction and transports the pallet carrying the cargo to the entry platform. When the cargo passes through the fixed image collection area of ​​the camera, the camera will continuously capture the cargo to obtain a time series image of the entry. At the same time, the photoelectric sensor sends a sensing signal to the WCS system. After receiving the sensing signal, the WCS system obtains the pallet code scanned by the pallet code scanning device, determines the cargo entry information of the cargo based on the pallet code, generates the cargo entry arrival instruction based on the cargo entry information and the pallet code, and sends it to the cargo verification system.

[0131] The cargo verification system receives the arrival instruction of the incoming cargo and obtains the incoming time series images collected by the camera within a preset time period. The incoming time series images are a collection of images of the cargo from entering the fixed image collection area to leaving the fixed image collection area. For example, if the camera collects 10 frames of images in 1 second, the cargo verification system receives the arrival instruction of the incoming cargo and obtains 50 frames of images collected by the camera within 5 seconds as the incoming time series images.

[0132] The cargo verification system calculates the score of the cargo image entering the platform based on the cargo position detected in each incoming time series image. If the center coordinates of the entire cargo area are (x0, y0), the closer (x0, y0) is to the center of the image, the higher the score. The cargo image with the highest score is selected for image preprocessing and determined as the cargo incoming image. The cargo incoming image is bound to the incoming cargo information and pallet code and stored in the data storage system for subsequent cargo outbound verification.

[0133] When the WCS system sends a cargo outbound instruction to the conveyor belt, the conveyor belt responds to the cargo outbound instruction and transports the pallet carrying the cargo to the outbound platform. The photoelectric sensor sends a sensing signal to the WCS system. After receiving the sensing signal, the WCS system obtains the pallet code scanned by the pallet code scanning device, determines the cargo outbound information based on the pallet code, generates the cargo outbound arrival instruction based on the cargo outbound information and the pallet code, and sends it to the cargo verification system.

[0134] The cargo verification system obtains the incoming cargo information with the same pallet code closest to the current time from the data storage system based on the pallet code in the outgoing cargo arrival instruction, and verifies the outgoing cargo information with the incoming cargo information in terms of cargo category, barcode information, and entry time. If the verification fails, the outgoing cargo verification exception information is output and sent to the WCS system.

[0135] If the inspection is successful, the incoming image bound with the incoming goods information is obtained, and the collected outgoing image of the goods is obtained. It can be understood that the outgoing image collection process of the goods is basically the same as the incoming image collection process, which will not be repeated here.

[0136] The cargo verification system inputs the incoming image and the outgoing image into the pre-trained image target detection model respectively. The image target detection model outputs the first cargo image after marking the key features such as the cargo box outline and barcode in the incoming image with a rectangular frame, and outputs the second cargo image after marking the key features such as the cargo box outline and barcode in the outgoing image with a rectangular frame. The first cargo image and the second cargo image are aligned in size, and each key feature in the aligned first cargo image is matched with each key feature in the aligned second cargo image respectively. The area intersection and union ratio between the rectangular frames of each key feature is calculated, and the two key features with the largest area intersection and union ratio and meeting the preset area intersection and union ratio threshold are determined as matching key features. If a key feature in the first cargo image or the second cargo image cannot be matched with other key features to obtain an area intersection and union ratio that meets the preset area intersection and union ratio threshold, the key feature is determined to be an unmatched key feature. It is determined whether the unmatched key feature meets the preset exclusion rule. If it does, the key feature is excluded from the unmatched key features. After all unmatched key features have been excluded and screened, if there are still unmatched key features, it is determined that the goods have failed the outbound inspection, and the final unmatched key features or the goods corresponding to the key features are marked in the input image, such as Figure 8As shown, there are unmatched key features I and J in the incoming image, and unmatched key features K, L, M, and N in the outgoing image. A cargo verification abnormality alarm is generated based on the output image and returned to the warehouse control system. If there are no unmatched key features after screening, it is determined that the outgoing cargo verification is successful, and the subsequent cargo outgoing steps are executed.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a cargo outbound verification device for implementing the cargo outbound verification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the cargo outbound verification device provided below can refer to the limitations of the cargo outbound verification method above, and will not be repeated here.

[0139] In one embodiment, Fig. 9 As shown, a device 900 for checking goods out of warehouse is provided, comprising: an outbound image acquisition module 901, an inbound image acquisition module 902, an image target recognition module 903 and a feature matching module 904, wherein:

[0140] The outbound image acquisition module 901 is used to acquire the outbound image of the goods when it is detected that the goods arrive at the outbound platform. The goods are placed on a pallet, and an identification mark is set on the outer surface of each pallet.

[0141] The storage-in image acquisition module 902 is used to acquire the storage-in image captured when the goods are stored according to the identified identification mark of the pallet.

[0142] The image target recognition module 903 is used to perform target recognition on the incoming image, extract the goods in the incoming image, and obtain the first goods image, and perform target recognition on the outgoing image, extract the goods in the outgoing image, and obtain the second goods image.

[0143] The feature matching module 904 performs feature matching on the first cargo image and the second cargo image. If the matching is successful, it is determined that the outbound inspection is successful.

[0144] The above-mentioned cargo outbound verification device obtains the incoming image captured when the cargo enters the warehouse according to the identification mark of the cargo pallet. The incoming image is subjected to target recognition, and the cargo in the incoming image is extracted to obtain a first cargo image; the outbound image is subjected to target recognition, and the cargo in the outbound image is extracted to obtain a second cargo image. The first cargo image captured when entering the warehouse is matched with the second cargo image captured when leaving the warehouse for feature matching, and the cargo is shipped out when it is determined that the cargo at the time of entry is consistent with the cargo leaving the warehouse, that is, when the match is successful, thereby effectively avoiding the inconsistency of the outbound cargo caused by the verification error, maintaining the normal operation of the automated warehousing and logistics system, and improving the production efficiency.

[0145] In one embodiment, the entry image acquisition module is also used to: obtain the entry information stored when the goods enter the warehouse based on the identification mark of the identified pallet; obtain the outbound information of the goods based on the identification mark; perform basic verification on the entry information and the outbound information; if the basic verification is successful, obtain the entry image collected when the goods enter the warehouse.

[0146] In one embodiment, the image target recognition module is further used to: input the incoming image into a pre-trained image target detection model for target recognition, extract the first key feature of the goods in the incoming image, and obtain a first goods image containing the first key feature; input the outgoing image into the image target detection model for target recognition, extract the second key feature of the goods in the outgoing image, and obtain a second goods image containing the second key feature.

[0147] In one embodiment, the feature matching module is further used to: perform an image alignment operation on the first cargo image and the second cargo image to obtain the aligned first cargo image and the second cargo image; and perform feature matching on the attributes of the first key feature in the aligned first image and the attributes of the second key feature in the aligned second image.

[0148] In one embodiment, the device for verifying the goods leaving the warehouse further includes: an unmatched key feature screening module, which is used to obtain unmatched key features in the first cargo image and the second cargo image if the matching is unsuccessful; determine whether each unmatched key feature satisfies a preset feature exclusion rule, and if so, determine that the warehouse verification is successful.

[0149] In one embodiment, the outbound image acquisition module is also used to: when it is detected that the goods have arrived at the outbound platform, acquire the outbound time series images of the goods collected within a preset time period; perform scoring operations on the outbound time series images according to preset scoring rules; and determine the image with the highest score in the outbound time series images as the outbound image of the goods.

[0150] Each module in the above-mentioned goods outbound verification device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.

[0151] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as storage images, storage information, outbound images, outbound information, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for checking the outbound delivery of goods is implemented.

[0152] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In one embodiment, a computer device is provided, which may be the cargo verification system in the present application, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0154] When it is detected that the goods arrive at the outbound platform, the outbound image of the goods is obtained. The goods are placed on a pallet, and an identification mark is set on the outer surface of each pallet;

[0155] According to the identified identification mark of the pallet, the warehouse entry image captured when the goods are put into the warehouse is obtained;

[0156] Performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image; performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image;

[0157] The first cargo image and the second cargo image are feature matched, and if the match is successful, it is determined that the warehouse exit verification is successful.

[0158] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0159] According to the identification mark of the identified pallet, the storage information stored when the goods are put into storage is obtained;

[0160] Obtain the outbound information of the goods according to the identification mark;

[0161] Perform basic verification on incoming and outgoing information;

[0162] If the basic verification is successful, the incoming images captured when the goods enter the warehouse are obtained.

[0163] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0164] Inputting the incoming image into a pre-trained image target detection model for target recognition, extracting the first key feature of the goods in the incoming image, and obtaining a first goods image containing the first key feature;

[0165] The outbound image is input into the image target detection model for target recognition, and the second key feature of the goods in the outbound image is extracted to obtain a second goods image containing the second key feature.

[0166] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0167] Performing an image alignment operation on the first cargo image and the second cargo image to obtain aligned first cargo image and second cargo image;

[0168] Feature matching is performed on the attributes of the first key feature in the aligned first image and the attributes of the second key feature in the aligned second image.

[0169] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0170] If the matching is unsuccessful, obtaining the unmatched key features in the first cargo image and the second cargo image;

[0171] Determine whether each unmatched key feature meets the preset feature exclusion rules. If so, determine that the outbound verification is successful.

[0172] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0173] When it is detected that the goods arrive at the outbound platform, the outbound time series images of the goods collected within a preset time period are obtained;

[0174] Scoring the outbound time series images according to the preset scoring rules;

[0175] The image with the highest score in the outbound time series images is determined as the outbound image of the goods.

[0176] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0177] When it is detected that the goods arrive at the outbound platform, the outbound image of the goods is obtained. The goods are placed on a pallet, and an identification mark is set on the outer surface of each pallet;

[0178] According to the identified identification mark of the pallet, the warehouse entry image captured when the goods are put into the warehouse is obtained;

[0179] Performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image; performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image;

[0180] The first cargo image and the second cargo image are feature matched, and if the match is successful, it is determined that the warehouse exit verification is successful.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] According to the identification mark of the identified pallet, the storage information stored when the goods are put into storage is obtained;

[0183] Obtain the outbound information of the goods according to the identification mark;

[0184] Perform basic verification on incoming and outgoing information;

[0185] If the basic verification is successful, the incoming images captured when the goods enter the warehouse are obtained.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0187] Inputting the incoming image into a pre-trained image target detection model for target recognition, extracting the first key feature of the goods in the incoming image, and obtaining a first goods image containing the first key feature;

[0188] The outbound image is input into the image target detection model for target recognition, and the second key feature of the goods in the outbound image is extracted to obtain a second goods image containing the second key feature.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0190] Performing an image alignment operation on the first cargo image and the second cargo image to obtain aligned first cargo image and second cargo image;

[0191] Feature matching is performed on the attributes of the first key feature in the aligned first image and the attributes of the second key feature in the aligned second image.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] If the matching is unsuccessful, obtaining the unmatched key features in the first cargo image and the second cargo image;

[0194] Determine whether each unmatched key feature meets the preset feature exclusion rules. If so, determine that the outbound verification is successful.

[0195] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0196] When it is detected that the goods arrive at the outbound platform, the outbound time series images of the goods collected within a preset time period are obtained;

[0197] Scoring the outbound time series images according to the preset scoring rules;

[0198] The image with the highest score in the outbound time series images is determined as the outbound image of the goods.

[0199] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0200] When it is detected that the goods arrive at the outbound platform, the outbound image of the goods is obtained. The goods are placed on a pallet, and an identification mark is set on the outer surface of each pallet;

[0201] According to the identified identification mark of the pallet, the warehouse entry image captured when the goods are put into the warehouse is obtained;

[0202] Performing target recognition on the incoming image, extracting the goods in the incoming image, and obtaining a first goods image; performing target recognition on the outgoing image, extracting the goods in the outgoing image, and obtaining a second goods image;

[0203] The first cargo image and the second cargo image are feature matched, and if the match is successful, it is determined that the warehouse exit verification is successful.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0205] According to the identification mark of the identified pallet, the storage information stored when the goods are put into storage is obtained;

[0206] Obtain the outbound information of the goods according to the identification mark;

[0207] Perform basic verification on incoming and outgoing information;

[0208] If the basic verification is successful, the incoming images captured when the goods enter the warehouse are obtained.

[0209] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0210] Inputting the incoming image into a pre-trained image target detection model for target recognition, extracting the first key feature of the goods in the incoming image, and obtaining a first goods image containing the first key feature;

[0211] The outbound image is input into the image target detection model for target recognition, and the second key feature of the goods in the outbound image is extracted to obtain a second goods image containing the second key feature.

[0212] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0213] Performing an image alignment operation on the first cargo image and the second cargo image to obtain aligned first cargo image and second cargo image;

[0214] Feature matching is performed on the attributes of the first key feature in the aligned first image and the attributes of the second key feature in the aligned second image.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0216] If the matching is unsuccessful, obtaining the unmatched key features in the first cargo image and the second cargo image;

[0217] Determine whether each unmatched key feature meets the preset feature exclusion rules. If so, determine that the outbound verification is successful.

[0218] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0219] When it is detected that the goods arrive at the outbound platform, the outbound time series images of the goods collected within a preset time period are obtained;

[0220] Scoring the outbound time series images according to the preset scoring rules;

[0221] The image with the highest score in the outbound time series images is determined as the outbound image of the goods.

[0222] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0224] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for checking the goods leaving the warehouse, characterized in that: The method comprises: When it is detected that the goods arrive at the outbound platform, an outbound image of the goods is obtained, wherein the goods are placed on a pallet, and an identification mark is provided on the outer surface of each pallet; According to the identified identification mark of the pallet, an entry image of the goods collected when entering the warehouse is obtained; Inputting the warehouse entry image into a pre-trained image target detection model for target recognition, extracting a first key feature of the goods in the warehouse entry image, and obtaining a first goods image containing the first key feature; the first goods image is a warehouse entry image containing the first key feature; The outbound image is input into the image target detection model for target recognition, and the second key feature of the goods in the outbound image is extracted to obtain a second goods image containing the second key feature; the second goods image is an outbound image containing the second key feature; the key feature includes at least one of the appearance outline trademark, icon, text and barcode of the goods in the image, and the key feature is marked by a rectangular frame; Performing an image alignment operation on the first cargo image and the second cargo image to obtain an aligned first cargo image and a second cargo image; Calculating an area intersection-and-union ratio of a key feature rectangular frame of a first key feature in the aligned first image and a key feature rectangular frame of a second key feature in the aligned second image; determining whether the first cargo image and the second cargo image are matched successfully according to the area intersection-and-union ratio; If the match is successful, the outbound verification is determined to be successful; If the matching is unsuccessful, obtaining unmatched key features in the first cargo image and the second cargo image; Determine whether each of the unmatched key features meets the preset feature exclusion rules. If so, determine that the outbound verification is successful; the preset feature exclusion rules include that the cargo coverage area is higher than the preset coverage area threshold.

2. The method according to claim 1, characterized in that The step of obtaining the storage image captured when the goods are stored in storage according to the identified identification mark of the pallet includes: According to the identified identification mark of the pallet, the storage information stored when the goods are stored is obtained; According to the identification mark, obtaining the outbound information of the goods; Performing basic verification on the incoming warehouse information and the outgoing warehouse information; If the basic verification is successful, the storage image captured when the goods are stored is obtained.

3. The method according to claim 1, characterized in that When it is detected that the goods arrive at the outbound platform, collecting the outbound image of the goods includes: When it is detected that the goods arrive at the outbound platform, a time series image of the outbound goods collected within a preset time period is obtained; Performing a scoring operation on the outbound time series images according to a preset scoring rule; The image with the highest score in the warehouse-out time series images is determined as the warehouse-out image of the goods.

4. The method according to claim 3, characterized in that The preset scoring rule includes determining the score of each of the outbound time series images according to the position distance between the center coordinates of the entire area of ​​the goods in each of the outbound time series images and the center coordinates of the image screen. The closer the center coordinates of the entire area of ​​the goods are to the center coordinates of the image screen, the higher the score.

5. A cargo outbound inspection device, characterized in that: The device comprises: An outbound image acquisition module is used to acquire an outbound image of the goods when it is detected that the goods arrive at the outbound platform, the goods are placed on a pallet, and an identification mark is provided on the outer surface of each pallet; A storage image acquisition module, used to acquire a storage image captured when the goods are stored according to the identified identification mark of the pallet; An image target recognition module is used to input the incoming image into a pre-trained image target detection model for target recognition, extract a first key feature of the goods in the incoming image, and obtain a first goods image containing the first key feature; input the outgoing image into the image target detection model for target recognition, extract a second key feature of the goods in the outgoing image, and obtain a second goods image containing the second key feature; the key feature includes at least one of the appearance outline trademark, icon, text and barcode of the goods in the image, and the key feature is marked by a rectangular frame; a feature matching module, configured to perform an image alignment operation on the first cargo image and the second cargo image to obtain the aligned first cargo image and the second cargo image; calculate an area intersection and union ratio of a key feature rectangular frame of a first key feature in the aligned first image and a key feature rectangular frame of a second key feature in the aligned second image; determine whether the first cargo image and the second cargo image are successfully matched according to the area intersection and union ratio; if the match is successful, determine that the warehouse exit verification is successful; The unmatched key feature screening module is used to obtain the unmatched key features in the first cargo image and the second cargo image if the match is unsuccessful; determine whether each of the unmatched key features meets the preset feature exclusion rules, and if so, determine that the outbound verification is successful; the preset feature exclusion rules include that the cargo coverage area is higher than the preset coverage area threshold.

6. The device according to claim 5, characterized in that The entry image acquisition module is also used to: obtain the entry information stored when the goods are entered into the warehouse according to the identified identification mark of the pallet; obtain the exit information of the goods according to the identification mark; and perform basic verification on the entry information and the exit information; If the basic verification is successful, the storage image captured when the goods are stored is obtained.

7. The device according to claim 5, characterized in that The outbound image acquisition module is used to: when it is detected that the goods arrive at the outbound platform, obtain the outbound time series images of the goods collected within a preset time period; score the outbound time series images according to preset scoring rules; and determine the image with the highest score in the outbound time series images as the outbound image of the goods.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images