A processing method and imaging device for logistics objects

Abstract

The present application provides a method for processing logistics objects and an imaging device. In the present application, a first perspective image including at least one logistics object is obtained, and descriptive information of at least part of the logistics object in the at least one logistics object is obtained. The logistics status of at least one logistics object is determined based on the first perspective image of the at least one logistics object and / or the descriptive information of at least part of the logistics object in the at least one logistics object. Through the present application, the accuracy of determining the logistics status of the logistics object can be improved by combining the perspective image of the logistics object and the descriptive information of the logistics object. For example, in the scenario of security inspection of logistics objects, the accuracy of determining whether the logistics object is a prohibited logistics object can be improved.

Description

Technical Field

[0001] The present application relates to the field of logistics, and in particular to a processing method and imaging device for logistics objects. Background Art

[0002] Currently, express parcels can be inspected using X-ray-based security inspection machines. For example, when an express parcel enters the security inspection machine, the security inspection machine can take an X-ray image of the express parcel in real time and perform intelligent recognition of the X-ray image to determine whether the express parcel includes prohibited items.

[0003] However, the X-ray images taken by the security inspection machine lose many personalized features of the objects in the express package. Therefore, it is easy to make misjudgments when determining whether the express package includes prohibited items based on the X-ray images. For example, it is easy to mistake a metal glue gun with a wrench (used for decoration, not prohibited item) as a real gun (contraband), resulting in a low accuracy rate in determining whether the express package includes contraband. Summary of the invention

[0004] In order to improve the accuracy of determining whether an express package includes prohibited items, the present application shows a logistics object processing method and imaging device.

[0005] In a first aspect, the present application provides a method for processing a logistics object, the method comprising:

[0006] Acquire a first perspective image including at least one logistics object, and acquire description information of at least part of the at least one logistics object;

[0007] The logistics status of the at least one logistics object is determined according to the first perspective image and / or the description information.

[0008] In an optional implementation, determining the logistics state of the at least one logistics object according to the first perspective image and / or the description information includes:

[0009] Determine whether the at least one logistics object is damaged according to the first perspective image and the description information.

[0010] In an optional implementation, determining the logistics state of the at least one logistics object according to the first perspective image and / or the description information includes:

[0011] Determine whether an item in the at least one logistics object is lost according to the first perspective image and the description information.

[0012] In an optional implementation, the obtaining description information of at least part of the at least one logistics object includes:

[0013] determining a prohibited candidate logistics object among the at least one logistics object according to the first perspective image;

[0014] Obtaining description information of the candidate logistics object;

[0015] In an optional implementation, determining the logistics state of the at least one logistics object according to the first perspective image and / or the description information includes:

[0016] Verify whether the candidate logistics object is a prohibited logistics object according to the description information.

[0017] In an optional implementation, the obtaining the description information of the candidate logistics object includes:

[0018] intercepting a second perspective image of the candidate logistics object in the first perspective image;

[0019] The description information of the candidate logistics object is acquired according to the second perspective image.

[0020] In an optional implementation, the acquiring description information of the candidate logistics object according to the second perspective image includes:

[0021] In the correspondence between the physical image of the logistics object and the description information of the logistics object, searching for the physical image of the candidate logistics object according to the second perspective image;

[0022] The description information corresponding to the physical image of the candidate logistics object is searched in the corresponding relationship.

[0023] In an optional implementation, the method further includes:

[0024] Before acquiring the first perspective image, acquiring a physical image of the candidate logistics object, the physical image including identification information of the candidate logistics object set on the surface of the candidate logistics object;

[0025] extracting identification information of the candidate logistics object from the physical object image;

[0026] Acquire description information of the candidate logistics object from the cloud according to the identification information;

[0027] The physical image of the candidate logistics object and the description information of the candidate logistics object are stored in the corresponding relationship.

[0028] In an optional implementation, the method further includes:

[0029] In the case that it has been verified whether the candidate logistics object is a prohibited logistics object based on the description information, the physical image of the candidate logistics object and the description information of the candidate logistics object are deleted in the corresponding relationship.

[0030] In a second aspect, the present application shows an imaging device, the imaging device comprising:

[0031] A first acquisition module is used to acquire a first perspective image including at least one logistics object, and a second acquisition module is used to acquire description information of at least part of the at least one logistics object;

[0032] A determination module is used to determine the logistics status of the at least one logistics object according to the first perspective image and / or the description information.

[0033] In an optional implementation, the determining module includes:

[0034] The first determining unit is used to determine whether the at least one logistics object is damaged according to the first perspective image and the description information.

[0035] In an optional implementation, the determining module includes:

[0036] A second determination unit is used to determine whether an item in the at least one logistics object is lost according to the first perspective image and the description information.

[0037] In an optional implementation, the second acquisition module includes:

[0038] a third determining unit, configured to determine a prohibited candidate logistics object in the at least one logistics object according to the first perspective image;

[0039] An acquisition unit, used to acquire description information of the candidate logistics object;

[0040] In an optional implementation, the determining module includes:

[0041] A verification unit is used to verify whether the candidate logistics object is a prohibited logistics object according to the description information.

[0042] In an optional implementation, the acquiring unit includes:

[0043] a capture subunit, configured to capture a second perspective image of the candidate logistics object in the first perspective image;

[0044] An acquisition subunit is used to acquire description information of the candidate logistics object according to the second perspective image.

[0045] In an optional implementation, the acquisition subunit is specifically used to: search for the physical image of the candidate logistics object according to the second perspective image in the correspondence between the physical image of the logistics object and the descriptive information of the logistics object; and search for the descriptive information corresponding to the physical image of the candidate logistics object in the correspondence.

[0046] In an optional implementation, the acquisition subunit is also used to: before acquiring the first perspective image, acquire a physical image of the candidate logistics object, the physical image including identification information of the candidate logistics object set on the surface of the candidate logistics object; extract the identification information of the candidate logistics object from the physical image; acquire description information of the candidate logistics object from the cloud based on the identification information; and store the physical image of the candidate logistics object and the description information of the candidate logistics object in the corresponding relationship.

[0047] In an optional implementation, the acquisition subunit is further used to: when it has been verified whether the candidate logistics object is a prohibited logistics object based on the description information, delete the physical image of the candidate logistics object and the description information of the candidate logistics object in the corresponding relationship.

[0048] In a third aspect, the present application shows an electronic device, the electronic device comprising:

[0049] processor;

[0050] a memory for storing processor-executable instructions;

[0051] Wherein, the processor is configured to execute the logistics object processing method as described in the first aspect.

[0052] In a fourth aspect, the present application illustrates a non-temporary computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the method for processing logistics objects as described in the first aspect.

[0053] In a fifth aspect, the present application illustrates a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the method for processing logistics objects as described in the first aspect.

[0054] Compared with the prior art, the embodiments of the present application have the following advantages:

[0055] In the present application, a first perspective image including at least one logistics object is obtained, and descriptive information of at least part of the logistics object in the at least one logistics object is obtained. The logistics state of the at least one logistics object is determined based on the first perspective image of the at least one logistics object and / or the descriptive information of at least part of the logistics object in the at least one logistics object. Through the present application, the accuracy of determining the logistics state of the logistics object can be improved by combining the perspective image of the logistics object and the descriptive information of the logistics object.

[0056] For example, in a scenario where security checks are performed on logistics objects, personalized features of the logistics objects may be lost in the first perspective image. Therefore, errors may occur in determining a prohibited candidate logistics object in at least one logistics object based on the first perspective image. For example, a logistics object that is not actually prohibited may be determined as a prohibited logistics object, which may reduce the accuracy of determining whether a logistics object is a prohibited logistics object.

[0057] However, through the present application, after determining a prohibited candidate logistics object in at least one logistics object according to the first perspective image, the description information of the candidate logistics object can also be obtained. The description information of the candidate logistics object can sometimes reflect whether the candidate logistics object is a prohibited item, etc. For example, if the candidate logistics object is a prohibited item, the description information in the logistics object sometimes contains relevant descriptions about the candidate logistics object being a prohibited item, etc. If the candidate logistics object is not a prohibited item, the description information of the logistics object often does not contain relevant descriptions about the candidate logistics object being a prohibited item, etc. Therefore, it is possible to verify whether the candidate logistics object is a prohibited logistics object according to the description information of the candidate logistics object, and as a final result, for example, the logistics object that is misjudged and is actually not prohibited can be screened out from the prohibited candidate logistics objects determined according to the first perspective image, thereby improving the accuracy of determining whether the logistics object is a prohibited logistics object. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of a method for processing a logistics object shown in an exemplary embodiment of the present application.

[0059] Figure 2 It is a flowchart of a method for processing a logistics object shown in an exemplary embodiment of the present application.

[0060] Figure 3 It is a flowchart of a method for processing a logistics object shown in an exemplary embodiment of the present application.

[0061] Figure 4 It is a structural block diagram of an imaging device shown in an exemplary embodiment of the present application.

[0062] Figure 5It is a schematic diagram of the structure of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0064] Reference Figure 1 , shows a flow chart of a method for processing a logistics object of the present application, which may include:

[0065] In step S101, a first perspective image including at least one logistics object is acquired, and description information of at least part of the at least one logistics object is acquired.

[0066] In the present application, when it is necessary to determine the logistics status of at least one logistics object, for example, it is necessary to perform security inspection on at least one logistics object (to detect whether the logistics object includes contraband, etc.), or to determine whether at least one logistics object is damaged, or to determine whether items in at least one logistics object are lost, etc., the user can place at least one logistics object in an imaging device (such as a security inspection machine, etc.) so that the imaging device captures a first perspective image including the at least one logistics object.

[0067] The first perspective image may include an X-ray image or the like.

[0068] Logistics objects may include items that require logistics operations, for example, after a buyer purchases a product sold by a seller, the seller needs to mail the product to the buyer through a courier company. In this case, the product can be regarded as a logistics object. The seller can mail the product to the buyer in the form of a courier package through a courier company. The courier package has a unique logistics identifier, such as a courier number, etc., which is equivalent to the product having a unique logistics identifier.

[0069] In one possible case, the executor of this step may be a security inspection machine, that is, the present application improves the security inspection machine, so that after capturing a first perspective image including at least one logistics object, the security inspection machine may execute step S102.

[0070] In another possible case, the executor of this step may be an electronic device (not a security inspection machine), that is, the present application does not improve the security inspection machine, and the electronic device is communicatively connected to the security inspection machine. After capturing a first perspective image including at least one logistics object, the security inspection machine may send the captured first perspective image including at least one logistics object to the electronic device, and the electronic device may receive the first perspective image sent by the security inspection machine, and then may execute step S102.

[0071] In step S102, the logistics state of at least one logistics object is determined according to the first perspective image of at least one logistics object and / or the description information of at least part of the logistics object in at least one logistics object.

[0072] The logistics status of the logistics object includes, but is not limited to: whether the logistics object is a prohibited item, whether the logistics object is damaged, and whether the items in the logistics object are lost, etc. Of course, other statuses may also be included, and the application will not give examples one by one.

[0073] Among them, how to determine the logistics status of at least one logistics object based on the first perspective image of at least one logistics object and / or the description information of at least part of the logistics object in at least one logistics object can be specifically referred to the embodiments shown later and will not be described in detail here.

[0074] In the present application, a first perspective image including at least one logistics object is obtained, and descriptive information of at least part of the logistics object in the at least one logistics object is obtained. The logistics state of the at least one logistics object is determined based on the first perspective image of the at least one logistics object and / or the descriptive information of at least part of the logistics object in the at least one logistics object. Through the present application, the accuracy of determining the logistics state of the logistics object can be improved by combining the perspective image of the logistics object and the descriptive information of the logistics object.

[0075] In one embodiment of the present application, when it is necessary to perform security inspection on at least one logistics object (to detect whether the logistics object includes contraband, etc.), when obtaining the description information of at least part of the logistics objects in the at least one logistics object, refer to Figure 2 , which can be implemented according to the following process, including:

[0076] In step S201, a prohibited candidate logistics object is determined in at least one logistics object according to a first perspective image.

[0077] In one embodiment of the present application, when a first perspective image including at least one logistics object is obtained, image analysis can be performed on the first perspective image to determine the contraband of at least one logistics object. Wherein, when a logistics object is a contraband, it can be determined that the logistics object is a prohibited logistics object. Contraband can include guns, flammable and explosive items, and items with strong magnetism, etc. Contraband can be set according to actual security requirements, and this application does not limit this.

[0078] In one embodiment of the present application, the first perspective image at least includes the shape outline of at least one logistics object.

[0079] Thus, in one method, the perspective images of the contraband items already on the market (including the shape and outline of the contraband items, etc.) can be statistically calculated in advance.

[0080] Thus, in this step, when a prohibited candidate logistics object is determined in at least one logistics object based on the first perspective image, for the perspective image of any logistics object included in the first perspective image, the similarity between the second perspective image of the logistics object and the perspective images of the prohibited items already existing on the market can be calculated. When the similarity between the perspective image of the logistics object and the perspective image of any prohibited item already existing on the market is greater than a preset threshold, the logistics object can be determined as a prohibited candidate logistics object.

[0081] If the similarity between the perspective image of the logistics object and the perspective image of any prohibited item already existing on the market is less than or equal to a preset threshold, the logistics object can be determined as a non-prohibited logistics object (i.e., a safe logistics object).

[0082] The preset threshold value may be determined according to actual conditions, for example, 90%, 85%, or 80%, etc. The present application does not limit the specific value.

[0083] In addition, the specific calculation method for calculating the similarity between two perspective images can refer to the currently existing method, and the present application does not limit the specific calculation method of the similarity.

[0084] In another embodiment of the present application, an image recognition model may be trained in advance, so that when determining a prohibited candidate logistics object in at least one logistics object based on the first perspective image, the image recognition model may be used.

[0085] The training method of the image recognition model includes: obtaining at least one training data, the training data including: a sample perspective image of a sample logistics object and annotated prohibited data of the sample logistics object. The model is trained using the at least one training data until the parameters in the model converge to obtain the image recognition model.

[0086] Among them, marking prohibited data is used to indicate whether the sample logistics object is a prohibited logistics object or a non-prohibited logistics object.

[0087] The model can include xtranse CV and others.

[0088] In this way, when a prohibited candidate logistics object is determined in at least one logistics object based on the first perspective image, the first perspective image can be input into the trained image recognition model to obtain the prohibited candidate logistics object output by the image recognition model.

[0089] Through the embodiment of the present application, by using at least one training data to train the image recognition model, the image recognition model can learn the image features of the perspective image of the prohibited logistics object and the image features of the perspective image of the non-prohibited logistics object. In this way, the image recognition model can determine the image features of the perspective image of each logistics object in the first perspective image, and when the image features of the perspective image of a logistics object meet the image features of the perspective image of the prohibited logistics object, the logistics object can be determined as a candidate prohibited logistics object.

[0090] In step S202, the description information of the candidate logistics object is obtained.

[0091] In the present application, the description information of the logistics object may include description information in text form, description information in voice form, description information in picture form, description information in video form, etc.

[0092] The description information of the logistics object may include a detailed introduction to the logistics object, etc.

[0093] For example, when the logistics object is a commodity that can be sold, the detailed introduction of the commodity includes the introduction of the commodity by the seller of the commodity, and may also include the evaluation of the commodity by the majority of buyers.

[0094] The seller's introduction to the product may include an introduction to the product's functions, an introduction to the product's appearance, and an introduction to the category of the product.

[0095] The specific method of obtaining the description information of the candidate logistics object can be found in the embodiments shown later, which will not be described in detail here.

[0096] Accordingly, when determining the logistics status of at least one logistics object based on the first perspective image and the description information, it is possible to verify whether the candidate logistics object is a prohibited logistics object based on the description information of the candidate logistics object.

[0097] In one embodiment of the present application, the description information of the candidate logistics object may sometimes reflect whether the candidate logistics object is a prohibited item, etc.

[0098] For example, in one possible case, if a logistics object is a prohibited item, the description information in the logistics object sometimes contains prohibited keywords (keywords used to indicate prohibited items), such as "guns", "alcohol", and "magnetic".

[0099] In addition, if a logistics object is not a prohibited item, prohibited keywords will often not exist in the description information of the logistics object.

[0100] Therefore, when verifying whether the candidate logistics object is a prohibited logistics object based on the description information of the candidate logistics object, in one way, it can be determined whether there are prohibited keywords in the description information of the candidate logistics object. If there are prohibited keywords in the description information of the candidate logistics object, it can be explained that the candidate logistics object is a prohibited item, and then the candidate logistics object can be determined to be a prohibited logistics object. If there are no prohibited keywords in the description information of the candidate logistics object, it can be explained that the candidate logistics object is not a non-prohibited item, and then the candidate logistics object can be determined to be not a prohibited logistics object.

[0101] In this way, for any logistics object, it is assumed that the logistics object is actually a non-prohibited logistics object, so that the description information of the logistics object often does not contain prohibited keywords, etc.

[0102] However, after capturing a perspective image including the logistics object, if the logistics object is mistakenly identified as a prohibited logistics object due to reasons such as the shape of the perspective image being similar to the image of a prohibited item, the logistics object can be verified as a non-prohibited logistics object through the above-mentioned method with the aid of the description information of the logistics object, thereby avoiding misjudgment of the logistics object as much as possible.

[0103] However, sometimes, misjudgment may still occur in the above manner, for example, a candidate logistics object that is actually not prohibited may still be misjudged as a prohibited candidate object.

[0104] For example, suppose that the candidate logistics object is not actually a prohibited item, that is, the logistics object is actually a non-prohibited logistics object, but the description information of the candidate logistics object includes a sentence "It is forbidden to be placed together with magnetic items (magnetism is a prohibited keyword, etc.) to avoid demagnetization". This sentence does not indicate that the candidate logistics object is magnetic, but indicates how to avoid demagnetization of the candidate logistics object.

[0105] However, in the above method, it will be determined that the prohibited keyword "magnetism" exists in the description information of the candidate logistics object, so the candidate logistics object is verified to be a prohibited logistics object, which leads to a misjudgment.

[0106] Therefore, in order to avoid the above-mentioned misjudgment as much as possible, in another embodiment of the present application, a prohibited verification model can be trained in advance. In this way, when verifying whether a candidate logistics object is a prohibited logistics object based on the description information of the candidate logistics object, the prohibited verification model can be used.

[0107] The training method of the prohibited inspection model includes: obtaining at least one training data, the training data including: sample description information of the sample logistics object and the labeled prohibited data of the sample logistics object. The model is trained using the at least one training data until the parameters in the model converge to obtain the prohibited inspection model.

[0108] The marked prohibited data is used to indicate whether the sample logistics object is a prohibited logistics object or a non-prohibited logistics object. Where the marked prohibited data is used to indicate that the sample logistics object is a prohibited logistics object, the marked prohibited data may also include a prohibited level, such as charged, prohibited, and special goods.

[0109] The model can include faster-rcnn and the like.

[0110] In this way, when verifying whether a candidate logistics object is a prohibited logistics object based on its description information, the description information can be input into the trained prohibited logistics object verification model to obtain the verification result of whether the candidate logistics object is a prohibited logistics object output by the prohibited logistics object verification model.

[0111] Through the embodiments of the present application, the semantics in the description information can be identified based on the prohibited verification model, for example, it can be determined whether the prohibited keywords in the description information of the candidate logistics object are used to indicate the attributes of the candidate logistics object, etc., so that it is possible to accurately verify whether the candidate logistics object is a prohibited logistics object based on the description information of the candidate logistics object to reduce the probability of misjudgment.

[0112] The personalized features of the logistics object may be lost in the first perspective image. Therefore, errors may occur in determining a prohibited candidate logistics object in at least one logistics object based on the first perspective image. For example, a logistics object that is not actually prohibited may be determined as a prohibited logistics object, which may reduce the accuracy of determining whether the logistics object is a prohibited logistics object.

[0113] However, through the present application, after determining a prohibited candidate logistics object in at least one logistics object according to the first perspective image, the description information of the candidate logistics object can also be obtained. The description information of the candidate logistics object can sometimes reflect whether the candidate logistics object is a prohibited item, etc. For example, if the candidate logistics object is a prohibited item, the description information in the logistics object sometimes contains relevant descriptions about the candidate logistics object being a prohibited item, etc. If the candidate logistics object is not a prohibited item, the description information of the logistics object often does not contain relevant descriptions about the candidate logistics object being a prohibited item, etc. Therefore, it is possible to verify whether the candidate logistics object is a prohibited logistics object according to the description information of the candidate logistics object, and as a final result, for example, the logistics object that is misjudged and is actually not prohibited can be screened out from the prohibited candidate logistics objects determined according to the first perspective image, thereby improving the accuracy of determining whether the logistics object is a prohibited logistics object.

[0114] Furthermore, in order to facilitate security personnel to know which logistics objects are prohibited logistics objects, in another embodiment of the present application, the first perspective image can also be displayed on the screen, and the prohibited logistics objects verified on the first perspective image can be circled.

[0115] In this way, the security personnel can see the first perspective image on the screen, and see the prohibited logistics objects circled and checked on the first perspective image. In this way, the security personnel can know which logistics object among the at least one logistics object under security inspection is a prohibited logistics object based on the circled mark, and then can find the physical prohibited logistics object among the physical objects of at least one logistics object based on the outline of the prohibited logistics object circled on the first perspective image.

[0116] Among them, when the marked prohibited data is used to indicate that the sample logistics object is a prohibited logistics object or a non-prohibited logistics object, the prohibited verification model will output the verification result that the candidate logistics object is a prohibited logistics object or a non-prohibited logistics object.

[0117] When the marked prohibited data also includes prohibited levels (for example, charged, prohibited, and magnetic, etc.), if the prohibited verification model verifies that the candidate logistics object is a prohibited logistics object, the prohibited verification model can output the prohibited level of the candidate logistics object in addition to the verification result that the candidate logistics object is a prohibited logistics object.

[0118] In which, when the prohibited items verification model also outputs prohibited items levels, the verified prohibited items levels can also be displayed on the screen so that security personnel can determine which level of prohibited items the verified prohibited items belong to.

[0119] In another embodiment of the present application, see Figure 3, in step S202, comprising:

[0120] In step S301, a second perspective image of the candidate logistics object is intercepted from the first perspective image.

[0121] In the present application, after determining the candidate logistics object in the first perspective image in step S201, the image capture technology can be used to capture the second perspective image of the candidate logistics object in the first perspective image. The specific capture method can refer to the existing capture method and will not be described in detail here.

[0122] In step S302, description information of the candidate logistics object is obtained according to the second perspective image of the candidate logistics object.

[0123] In one embodiment of the present application, before acquiring the first perspective image, a physical image of the candidate logistics object can be acquired, the physical image including identification information of the candidate logistics object set on the surface of the candidate logistics object. Then, the identification information of the candidate logistics object can be extracted based on the physical image of the candidate logistics object, and the identification information of the candidate logistics object can be cached.

[0124] Thus, in one method, when obtaining the description information of the candidate logistics object based on the second perspective image of the candidate logistics object, the cached identification information of the candidate logistics object can be obtained based on the second perspective image of the candidate logistics object, and the description information of the candidate logistics object can be searched in the cloud in real time based on the identification information of the candidate logistics object.

[0125] However, in the above method, the description information of the candidate logistics object is not obtained until a prohibited candidate logistics object is determined in at least one logistics object based on the first perspective image (after step S201). The process of searching for the description information of the candidate logistics object in the cloud in real time based on the identification information of the candidate logistics object takes some time, which reduces the efficiency of obtaining the description information of the candidate logistics object.

[0126] Therefore, in order to improve the efficiency of obtaining the description information of the candidate logistics object. In another embodiment of the present application, this step can be implemented by the following process, including:

[0127] 3021. In the correspondence between the physical image of the logistics object and the description information of the logistics object, search for the physical image of the candidate logistics object according to the second perspective image.

[0128] In this application, a correspondence between a physical image of a logistics object and the description information of the logistics object can be created in advance. The specific creation process includes:

[0129] 11) Before acquiring the first perspective image, a physical image of the candidate logistics object is acquired, the physical image including identification information of the candidate logistics object set on the surface of the candidate logistics object.

[0130] For example, before acquiring the first perspective image, a camera may be used to capture a physical image of the candidate logistics object, and the physical image of the candidate logistics object may be cached.

[0131] For example, a camera is arranged above the entrance of the security inspection machine, and the camera can capture the physical image of the candidate logistics object when the candidate logistics object enters the security inspection machine from the entrance.

[0132] 12) Extract identification information of the candidate logistics object from the physical image of the candidate logistics object.

[0133] In the present application, a logistics waybill is set on the surface of the candidate logistics object, and the logistics waybill is set with identification information of the candidate logistics object. The identification information of the candidate logistics object includes the express number, shopping order number or product name of the candidate logistics object, etc. The identification information of different logistics objects is different.

[0134] Among them, image analysis technology can be used to extract identification information of candidate logistics objects from physical images of candidate logistics objects. This application does not limit the specific image analysis technology.

[0135] 13) Obtain description information of the candidate logistics object from the cloud based on the identification information of the candidate logistics object.

[0136] In this application, the cloud can store description information of each logistics object, etc.

[0137] For example, an electronic device may send a request to the cloud, the request carries the identification information of a candidate logistics object, and the request is used to request the description information of the candidate logistics object. The cloud receives the request, extracts the identification information of the candidate logistics object from the request, obtains the description information of the stored candidate logistics object based on the identification information of the candidate logistics object, and then sends the description information of the obtained candidate logistics object to the electronic device. The electronic device then receives the description information of the candidate logistics object sent by the cloud.

[0138] 14) In the correspondence between the physical image of the logistics object and the descriptive information of the logistics object, the acquired physical image of the candidate logistics object and the acquired descriptive information of the candidate logistics object are stored.

[0139] For example, in one example, the photographed physical image of the candidate logistics object and the extracted description information of the candidate logistics object can be composed of corresponding table items and stored in the corresponding relationship between the physical image of the logistics object and the description information of the logistics object.

[0140] Thus, in step S3021, the similarity between each physical image in the corresponding relationship and the second perspective image can be calculated, and the physical image with the highest similarity to the second perspective image is used as the physical image of the candidate logistics object. The specific calculation method for calculating the similarity between the two images can use an existing calculation method, and this application does not limit the specific calculation method.

[0141] 3022. In the correspondence between the physical image of the logistics object and the descriptive information of the logistics object, search for descriptive information corresponding to the physical image of the candidate logistics object.

[0142] In an embodiment of the present application, before capturing the second perspective image of the candidate logistics object in the first perspective image (before step S301), the description information of the candidate logistics object can be obtained and the obtained description information of the candidate logistics object can be cached. In this way, when it is necessary to obtain the description information of the candidate logistics object later, the cached description information of the candidate logistics object can be directly obtained, and there is no need to search for the description information of the candidate logistics object in the cloud in real time according to the identification information of the candidate logistics object. This is equivalent to saving the process of "searching for the description information of the candidate logistics object in the cloud in real time according to the identification information of the candidate logistics object" which takes some time, thereby improving the efficiency of obtaining the description information of the candidate logistics object.

[0143] Furthermore, in the case where it has been verified whether the candidate logistics object is a prohibited logistics object based on the description information of the candidate logistics object, the security check on the candidate logistics object is terminated, and thereafter it may no longer be necessary to continue to conduct security checks on the candidate logistics object again in the manner of the present application. In this way, the physical image of the candidate logistics object and the description information of the candidate logistics object stored in the correspondence between the physical image of the logistics object and the description information of the logistics object often no longer have security check-related functions. Therefore, in order to save storage space, in the case where it has been verified whether the candidate logistics object is a prohibited logistics object based on the description information of the candidate logistics object, the physical image of the candidate logistics object and the description information of the candidate logistics object may be deleted in the correspondence between the physical image of the logistics object and the description information of the logistics object.

[0144] In one embodiment of the present application, the logistics object may be damaged during the logistics transportation process. For example, assuming that the logistics object includes fragile items, such as glass or ceramics, the fragile items may be damaged due to vibration or collision during the logistics transportation process.

[0145] If the recipient of the logistics object receives a damaged logistics object, the subjective feeling of the recipient of the logistics object will be reduced. Therefore, in order to avoid the recipient of the logistics object receiving a damaged logistics object, it is possible to detect in advance whether the logistics object is damaged.

[0146] For example, it may be determined whether at least one logistics object is damaged according to the first perspective image of at least one logistics object and the description information of at least one logistics object.

[0147] For example, the second perspective image of each logistics object can be intercepted in the first perspective image. Then, for any one of the at least one logistics object, the description information of the logistics object can be obtained based on the second perspective image of the logistics object. The specific acquisition method can refer to step S302 and will not be described in detail here. The description information of the logistics object may include description information in image form, etc. For example, the description information of the logistics object may be the appearance image of the logistics object, etc., and then it can be determined whether the logistics object is damaged based on the second perspective image of the logistics object and the appearance image of the logistics object. The same is true for each of the other logistics objects in the at least one logistics object.

[0148] Among them, the damage recognition model can be trained in advance, so that when determining whether the logistics object is damaged based on the second perspective image of the logistics object and the appearance image of the logistics object, the damage recognition model can be used.

[0149] The training method of the damage recognition model includes: obtaining at least one training data, the training data including: a sample perspective image of the sample logistics object, a sample appearance image of the sample logistics object, and annotated damage data of the sample logistics object. The model is trained using the at least one training data until the parameters in the model converge to obtain the damage recognition model.

[0150] The sample perspective image of the sample logistics object may include a perspective image of a damaged sample logistics object, and the sample appearance image of the sample logistics object includes an appearance image of an undamaged sample logistics object. The sample logistics object in the sample perspective image and the sample logistics object in the sample appearance image are the same sample logistics object.

[0151] The marked damage data is used to indicate the damaged position of the sample logistics object in the sample perspective image, etc.

[0152] In this way, when determining whether the logistics object is damaged based on the second perspective image of the logistics object and the appearance image of the logistics object, the second perspective image of the logistics object and the appearance image of the logistics object can be input into the trained damage recognition model to obtain the result output by the damage recognition model indicating whether the logistics object is damaged.

[0153] In one embodiment of the present application, a logistics object may include multiple items, and the logistics object may lose at least one item during the logistics transportation process. Therefore, it is necessary to discover the logistics object that has lost the item as early as possible so as to handle the loss event of the logistics object as early as possible. For example, a new and complete logistics object should be reissued to the recipient of the logistics object as soon as possible so that the recipient of the logistics object can obtain the complete logistics object.

[0154] In order to find out as early as possible whether there is a logistics object with lost items, it can be determined whether the items in at least one logistics object are lost based on the first perspective image of at least one logistics object and the description information of at least one logistics object.

[0155] For example, the second perspective image of each logistics object can be intercepted in the first perspective image. Then, for any one of the at least one logistics object, the description information of the logistics object can be obtained based on the second perspective image of the logistics object. The specific acquisition method can refer to step S302 and will not be described in detail here. The description information of the logistics object may include description information in image form, etc. For example, the description information of the logistics object may be an appearance image of the logistics object, etc. The appearance image of the logistics object includes each item in the logistics object, and then it can be determined whether the items in the logistics object are lost based on the second perspective image of the logistics object and the appearance image of the logistics object. The same is true for each of the other logistics objects in the at least one logistics object.

[0156] Among them, the lost recognition model can be trained in advance, so that when determining whether the items in the logistics object are lost based on the second perspective image of the logistics object and the appearance image of the logistics object, the lost recognition model can be used.

[0157] The training method of the loss recognition model includes: obtaining at least one training data, the training data including: a sample perspective image of a sample logistics object, a sample appearance image of a sample logistics object, and annotated loss data of the sample logistics object. The model is trained using the at least one training data until the parameters in the model converge to obtain the loss recognition model.

[0158] The sample perspective image of the sample logistics object may include a perspective image of the sample logistics object with at least one item missing, and the sample appearance image of the sample logistics object includes an appearance image of the sample logistics object without any item missing. The sample logistics object in the sample perspective image and the sample logistics object in the sample appearance image are the same sample logistics object.

[0159] The annotated lost data is used to indicate whether there are lost items in the sample logistics object in the sample perspective image and the location of the lost items in the sample appearance image.

[0160] In this way, when determining whether an item in a logistics object is lost based on the second perspective image of the logistics object and the appearance image of the logistics object, the second perspective image of the logistics object and the appearance image of the logistics object can be input into a trained loss recognition model to obtain a result output by the loss recognition model indicating whether the item in the logistics object is lost.

[0161] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the present application.

[0162] Reference Figure 4 , shows a structural block diagram of an embodiment of an imaging device of the present application, which may specifically include the following modules:

[0163] A first acquisition module 11 is used to acquire a first perspective image including at least one logistics object, and a second acquisition module 12 is used to acquire description information of at least part of the at least one logistics object;

[0164] The determination module 13 is used to determine the logistics status of the at least one logistics object according to the first perspective image and / or the description information.

[0165] In an optional implementation, the determining module includes:

[0166] The first determining unit is used to determine whether the at least one logistics object is damaged according to the first perspective image and the description information.

[0167] In an optional implementation, the determining module includes:

[0168] A second determination unit is used to determine whether an item in the at least one logistics object is lost according to the first perspective image and the description information.

[0169] In an optional implementation, the second acquisition module includes:

[0170] a third determining unit, configured to determine a prohibited candidate logistics object in the at least one logistics object according to the first perspective image;

[0171] An acquisition unit, used to acquire description information of the candidate logistics object;

[0172] In an optional implementation, the determining module includes:

[0173] A verification unit is used to verify whether the candidate logistics object is a prohibited logistics object according to the description information.

[0174] In an optional implementation, the acquiring unit includes:

[0175] a capture subunit, configured to capture a second perspective image of the candidate logistics object in the first perspective image;

[0176] An acquisition subunit is used to acquire description information of the candidate logistics object according to the second perspective image.

[0177] In an optional implementation, the acquisition subunit is specifically used to: search for the physical image of the candidate logistics object according to the second perspective image in the correspondence between the physical image of the logistics object and the descriptive information of the logistics object; and search for the descriptive information corresponding to the physical image of the candidate logistics object in the correspondence.

[0178] In an optional implementation, the acquisition subunit is also used to: before acquiring the first perspective image, acquire a physical image of the candidate logistics object, the physical image including identification information of the candidate logistics object set on the surface of the candidate logistics object; extract the identification information of the candidate logistics object from the physical image; acquire description information of the candidate logistics object from the cloud based on the identification information; and store the physical image of the candidate logistics object and the description information of the candidate logistics object in the corresponding relationship.

[0179] In an optional implementation, the acquisition subunit is further used to: when it has been verified whether the candidate logistics object is a prohibited logistics object based on the description information, delete the physical image of the candidate logistics object and the description information of the candidate logistics object in the corresponding relationship.

[0180] In the present application, a first perspective image including at least one logistics object is obtained, and descriptive information of at least part of the logistics object in the at least one logistics object is obtained. The logistics state of the at least one logistics object is determined based on the first perspective image of the at least one logistics object and / or the descriptive information of at least part of the logistics object in the at least one logistics object. Through the present application, the accuracy of determining the logistics state of the logistics object can be improved by combining the perspective image of the logistics object and the descriptive information of the logistics object.

[0181] For example, in a scenario where security checks are performed on logistics objects, personalized features of the logistics objects may be lost in the first perspective image. Therefore, errors may occur in determining a prohibited candidate logistics object in at least one logistics object based on the first perspective image. For example, a logistics object that is not actually prohibited may be determined as a prohibited logistics object, which may reduce the accuracy of determining whether a logistics object is a prohibited logistics object.

[0182] However, through the present application, after determining a prohibited candidate logistics object in at least one logistics object according to the first perspective image, the description information of the candidate logistics object can also be obtained. The description information of the candidate logistics object can sometimes reflect whether the candidate logistics object is a prohibited item, etc. For example, if the candidate logistics object is a prohibited item, the description information in the logistics object sometimes contains relevant descriptions about the candidate logistics object being a prohibited item, etc. If the candidate logistics object is not a prohibited item, the description information of the logistics object often does not contain relevant descriptions about the candidate logistics object being a prohibited item, etc. Therefore, it is possible to verify whether the candidate logistics object is a prohibited logistics object according to the description information of the candidate logistics object, and as a final result, for example, the logistics object that is misjudged and is actually not prohibited can be screened out from the prohibited candidate logistics objects determined according to the first perspective image, thereby improving the accuracy of determining whether the logistics object is a prohibited logistics object.

[0183] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0184] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, the electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes a server, a gateway, a sub-device, etc., and the sub-device is an Internet of Things device or other device.

[0185] The embodiments of the present disclosure may be implemented as an apparatus configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the apparatus may include electronic devices such as servers (clusters), terminal devices such as IoT devices, and the like.

[0186] Figure 5 An exemplary apparatus 1300 that can be used to implement various embodiments described in this application is schematically shown.

[0187] For one embodiment, Figure 5An exemplary apparatus 1300 is shown having one or more processors 1302, a control module (chip set) 1304 coupled to at least one of the (one or more) processors 1302, a memory 1306 coupled to the control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1306.

[0188] The processor 1302 may include one or more single-core or multi-core processors, and the processor 1302 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1300 can be used as a server device such as a gateway described in the embodiments of the present application.

[0189] In some embodiments, the device 1300 may include one or more computer-readable media (e.g., memory 1306 or NVM / storage device 1308) having instructions 1314 and one or more processors 1302 configured to execute the instructions 1314 in combination with the one or more computer-readable media to implement a module to perform the actions described in the present disclosure.

[0190] For one embodiment, the control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1302 and / or any suitable device or component in communication with the control module 1304 .

[0191] The control module 1304 may include a memory controller module to provide an interface to the memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0192] The memory 1306 may be used, for example, to load and store data and / or instructions 1314 for the device 1300. For one embodiment, the memory 1306 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1306 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0193] For one embodiment, control module 1304 may include one or more input / output controllers to provide an interface to NVM / storage device 1308 and input / output device(s) 1310 .

[0194] For example, NVM / storage 1308 may be used to store data and / or instructions 1314. NVM / storage 1308 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0195] NVM / storage device 1308 may include storage resources that are physically part of the device on which apparatus 1300 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1308 may be accessed via input / output device(s) 1310 over a network.

[0196] (One or more) input / output devices 1310 may provide an interface for the apparatus 1300 to communicate with any other appropriate device, and the input / output device 1310 may include a communication component, a phonetic component, a sensor component, etc. The network interface 1312 may provide an interface for the apparatus 1300 to communicate through one or more networks, and the apparatus 1300 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0197] For one embodiment, at least one of the processor(s) 1302 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1304. For one embodiment, at least one of the processor(s) 1302 may be packaged together with the logic of one or more controllers of the control module 1304 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the control module 1304. For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the control module 1304 to form a system-on-chip (SoC).

[0198] In various embodiments, the device 1300 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 1300 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0199] An embodiment of the present application provides an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the electronic device to execute a method for processing a logistics object as described in one or more of the present application.

[0200] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0201] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0202] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0205] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0206] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0207] The above is a detailed introduction to a logistics object processing method and imaging device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing a logistics object, characterized in that: The method comprises: Acquire a first perspective image including at least one logistics object, and acquire description information of at least part of the at least one logistics object; determining a logistics state of the at least one logistics object according to the first perspective image and the description information; The obtaining description information of at least part of the at least one logistics object includes: determining a prohibited candidate logistics object among the at least one logistics object according to the first perspective image; Obtaining description information of the candidate logistics object; The determining the logistics state of the at least one logistics object according to the first perspective image and the description information includes: Verifying whether the candidate logistics object is a prohibited logistics object according to the description information; The verifying whether the candidate logistics object is a prohibited logistics object according to the description information includes: inputting the description information into a trained prohibited logistics object verification model, and obtaining a verification result of whether the candidate logistics object is a prohibited logistics object output by the prohibited logistics object verification model.

2. The method according to claim 1, characterized in that The determining the logistics state of the at least one logistics object according to the first perspective image and the description information includes: Determine whether the at least one logistics object is damaged according to the first perspective image and the description information.

3. The method according to claim 1, characterized in that The determining the logistics state of the at least one logistics object according to the first perspective image and the description information includes: Determine whether an item in the at least one logistics object is lost according to the first perspective image and the description information.

4. The method according to claim 1, characterized in that: The obtaining the description information of the candidate logistics object includes: intercepting a second perspective image of the candidate logistics object in the first perspective image; The description information of the candidate logistics object is acquired according to the second perspective image.

5. The method according to claim 4, characterized in that The acquiring description information of the candidate logistics object according to the second perspective image includes: In the correspondence between the physical image of the logistics object and the description information of the logistics object, searching for the physical image of the candidate logistics object according to the second perspective image; The description information corresponding to the physical image of the candidate logistics object is searched in the corresponding relationship.

6. The method according to claim 5, characterized in that The method further comprises: Before acquiring the first perspective image, acquiring a physical image of the candidate logistics object, the physical image including identification information of the candidate logistics object set on the surface of the candidate logistics object; extracting identification information of the candidate logistics object from the physical object image; Acquire description information of the candidate logistics object from the cloud according to the identification information; The physical image of the candidate logistics object and the description information of the candidate logistics object are stored in the corresponding relationship.

7. The method according to claim 5 or 6, characterized in that: The method further comprises: In the case that it has been verified whether the candidate logistics object is a prohibited logistics object based on the description information, the physical image of the candidate logistics object and the description information of the candidate logistics object are deleted in the corresponding relationship.

8. An imaging device, characterized in that: The imaging device comprises: A first acquisition module is used to acquire a first perspective image including at least one logistics object, and a second acquisition module is used to acquire description information of at least part of the at least one logistics object; a determination module, configured to determine a logistics state of the at least one logistics object according to the first perspective image and the description information; The second acquisition module includes: a third determining unit, configured to determine a prohibited candidate logistics object in the at least one logistics object according to the first perspective image; An acquisition unit, used to acquire description information of the candidate logistics object; The determination module comprises: A verification unit, configured to verify whether the candidate logistics object is a prohibited logistics object according to the description information; The verification unit is specifically used to: input the description information into a trained prohibited logistics object verification model to obtain a verification result of whether the candidate logistics object output by the prohibited logistics object verification model is a prohibited logistics object.

Images