Parcel location services and routing

Abstract

Methods, systems, and computer-readable storage media for identifying and routing a first package having a corrupted shipping tag within a shipping network are described. First data of a first package having a corrupted tag is obtained. Second data for the plurality of packages within the transport network is obtained. A particular parcel that matches the first parcel is identified from the plurality of parcels. Identification is performed by applying one or more identification algorithms to the first data and the second data. Applying the identification algorithms includes obtaining a subset of possible matches from the plurality of packages as an output from each algorithm, the possible matches including packages identified by the one or more identification algorithms as having one or more characteristics similar to the first package. The first package is routed over the transport network using information from the tag of the particular package.

Description

Technical Field

[0001] This disclosure relates generally to data processing. Background Art

[0002] Package delivery companies and carriers may manage millions of packages per month. Package processing is performed to confirm postage data, check the security of packages, and ensure that packages match labels, among other examples. Deliveries may be organized according to transportation plans between different locations within a delivery network. Different transportation hubs or facilities may be defined within a delivery network to support the delivery and pickup of packages. Different transportation hubs are connected by different modes, and the transfer of packages may be run according to a predetermined plan or on demand. Transportation modes may include trains, airplanes, ferries, cars, trucks, ships, and boats, among other exemplary transportation modes. A package may be routed from an initial delivery location through one or more other intermediate locations within a delivery network to reach a destination location defined for the package. Typically, a package includes a label that includes shipping data such as the name and address of the sender and recipient, a routing code for notifying how the package is to be routed, and potentially other information. Summary of the invention

[0003] Implementations of the present disclosure relate to identification and routing of a package that has a damaged shipping label and is being shipped through a shipping network to an appropriate destination location. The identification can be based on analysis of data collected for the package and for other packages that have been scanned at other locations in the shipping network (e.g., the package's label is scanned by a machine or other automated tool for collecting a digital footprint of the package) and have not yet been identified as arriving at their destination point. In some implementations, the data collected for other packages can be collected through automated scanning by a machine or through user interaction with a tool to collect data for the package.

[0004] An example method may include operations such as: obtaining first data of a first package within a delivery network, wherein the first package includes a label that is damaged and prevents automatic identification of the first package based on the label; obtaining second data of multiple packages within the delivery network; identifying a specific package from the multiple packages that matches the first package by applying one or more identification algorithms to the first data and the second data, wherein applying the one or more identification algorithms includes obtaining a subset of possible matches from the multiple packages as an output from each algorithm, the possible matches including packages identified by the one or more identification algorithms as having one or more characteristics similar to the first package; and using information from the label of the specific package to route the first package through the delivery network.

[0005] In some implementations, identifying the specific package includes identifying the specific package that matches the first package based on providing an image of the first package and images of the subset of possible matches as input to a machine learning model trained for image recognition.

[0006] In some implementations, identifying the particular package includes evaluating the first data and the second data by applying a first algorithm of the one or more identification algorithms. Applying the first algorithm may include matching a first portion of the first data with a first portion of the second data, both first portions being associated with a first characteristic of a set of characteristics of the first package. In response to the matching, a first subset of the plurality of packages that substantially matches the first package may be determined based on the first characteristic. In response to determining that the first subset of the plurality of packages includes a set of packages within a predetermined threshold number, the first subset is provided as an input to a machine learning model to identify a particular package from the subset that matches the first package. The machine learning model may be trained to identify the particular package based on image recognition.

[0007] In some implementations, applying one or more identification algorithms includes applying a plurality of identification algorithms in a sequence, wherein at each step in the sequence, a corresponding data set of the second data from the plurality of packages is evaluated, the corresponding data set being associated with: i) a corresponding characteristic being evaluated, and ii) a previously determined subset of packages from the plurality of packages that match the first package at a previous step in the sequence based on a previously evaluated characteristic of the first package. In some implementations, at each step in the sequence, a characteristic of the first package is evaluated. The characteristic may be selected from the group consisting of: package weight, package color, package scanning order, package size, external surface markings, and an expected package delivery route. The corresponding characteristic is selected to determine an identification algorithm for evaluating the first data and the second data based on the corresponding characteristic.

[0008] In some implementations, each identification algorithm is arranged in an order according to a different computational complexity, which can be based at least on the number of packages provided as input to the identification algorithm from a previous iteration. In some implementations, the identification algorithms are sorted in order according to increasing computational complexity.

[0009] In some implementations, the first data and the second data are collected for a set of characteristics of packages routed through a delivery network, and the first data and the second data are associated with respective locations at the delivery network. In some implementations, applying the one or more identification algorithms includes evaluating the first data and the second data based on the respective one or more characteristics of the first package. Based on the evaluation of the first data and the second data, a set of packages matching the first package based on the respective one or more characteristics is determined as a set of candidate packages.

[0010] In some implementations, applying one or more identification algorithms includes applying a package weight analysis, which includes: comparing the weight of a first package included in the first data with the weights of other packages included in the second data; and excluding from the subset of possible matches any packages within the multiple packages that have a corresponding weight that does not match the weight of the first package within a weight threshold range defined around the weight of the first package.

[0011] In some implementations, applying one or more identification algorithms includes applying a package scanning order analysis, which includes: comparing an order in which a first package is loaded into a shipping vehicle or shipping container with an order in which other packages in a plurality of packages are unloaded from the shipping vehicle or shipping container; and based on the order in which the first package is loaded into the shipping vehicle or shipping container, excluding from a subset of possible matches any packages in the plurality of packages that have a corresponding unloading order that is not within a threshold range of an expected unloading order for the first package.

[0012] In some implementations, applying the one or more identification algorithms includes applying a package color analysis, the package color analysis including: comparing a first set of colors present in the image of the first package with sets of colors present in images of other packages in the plurality of packages; and excluding from the subset of possible matches any packages in the plurality of packages having a corresponding set of colors that exceeds a threshold similarity value to the first set of colors present in the image of the first package. In some implementations, comparing the first set of colors present in the image of the first package with sets of colors present in images of other packages in the plurality of packages includes: determining a Bhattacharyya distance between pixel values ​​of the image of the first package and corresponding pixel values ​​of the images of the other packages. Excluding from the subset of possible matches any packages in the plurality of packages having a corresponding set of colors that exceeds a threshold similarity value to the first set of colors present in the image of the first package includes: excluding from the subset of possible matches any packages in the plurality of packages having a corresponding Bhattacharyya distance greater than the threshold Bhattacharyya distance.

[0013] In some implementations, comparing the first set of colors present in the image of the first package to sets of colors present in images of other packages in the plurality of packages includes identifying regions within the images of the other packages that contain package labels and ignoring pixels in the regions when performing the comparison.

[0014] In some implementations, comparing the first set of colors present in the image of the first package to sets of colors present in images of other packages of the plurality of packages includes: identifying label regions within the images of the other packages that include package labels; and comparing corresponding label regions within each of the images of the other packages to a region within the image of the first package, the region including a location on the first package that is similarly positioned to a location of the corresponding label region, thereby determining whether the region within the image of the first package includes any damage markings.

[0015] In some implementations, applying the one or more identification algorithms includes applying a package size analysis, the package size analysis including: comparing a first set of package sizes determined from the image of the first package with a set of package sizes determined from images of other packages in the plurality of packages; and excluding from the subset of possible matches any packages in the plurality of packages having a corresponding set of package sizes that exceeds a threshold size similarity value for the first set of package sizes determined from the image of the first package. In some implementations, the package size analysis may also include: determining a first set of package sizes from the image of the first package by projecting a set of three-dimensional package models onto the image of the first package to identify a best fit between one of the package models and the first package in the image of the first package; and applying the dimensions of the one of the package models as the first set of package sizes.

[0016] In some implementations, applying the one or more identification algorithms includes applying a distinguishing feature analysis, the distinguishing feature analysis including: identifying a first set of one or more features on the first package from the image of the first package, the features including one or more of a mark, a scratch, a hole, or text; comparing the first set of one or more features to images of other packages in the plurality of packages; and excluding from the subset of possible matches any package in the plurality of packages that does not have at least one of the one or more features detected on the first package.

[0017] In some implementations, applying one or more identification algorithms includes applying a shipping routing analysis, which includes: obtaining the location of a shipping facility within a shipping network where a first package is identified as having a damaged label; determining possible delivery methods for other packages in a plurality of packages; and excluding from a subset of possible matches any package in the plurality of packages that does not include a location within a corresponding expected delivery method.

[0018] In some implementations, determining the expected delivery pattern for the other packages includes determining the other packages from the plurality of packages as packages associated with corresponding locations of a delivery facility within the delivery network, the corresponding location of the delivery facility having a direct delivery connection to the location of the first package, or having a direct delivery connection to a single intermediate location between the corresponding location of the packages from the other packages and the location of the first package.

[0019] In some examples, the delivery network is a network graph including a plurality of nodes representing locations for delivering and picking up packages, wherein the network graph includes a plurality of directional edges representing connections between pairs of locations, wherein based on the network graph, a plurality of routes connecting at least some of the nodes are defined.

[0020] Another second example of the method may include operations such as: obtaining first data of a first package within a delivery network, wherein the first package includes a label that is damaged and prevents automatic identification of the first package based on the label; obtaining second data of multiple packages within the delivery network; identifying a specific package from the multiple packages by: applying multiple identification algorithms to the first data and the second data in order to obtain a subset of possible matches from the multiple packages as an output from each algorithm, the possible matches including packages identified by the multiple identification algorithms as having one or more characteristics similar to the first package; and providing an image of the first package from the first data and an image of the subset of possible matches determined after applying the multiple identification algorithms as input to a machine learning model, identifying a specific package that matches the first package; and using information from the label of the specific package to route the first package through the delivery network.

[0021] In some implementations, the example second method may

[0022] Other implementations of this aspect include corresponding systems, apparatus, and computer programs configured to perform the actions of the method encoded on a computer storage device.

[0023] The present disclosure also provides a computer-readable storage medium coupled to one or more processors and having instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform operations according to an implementation of the method provided herein.

[0024] The present disclosure also provides a system for implementing the method provided herein. The system includes one or more processors, and a computer-readable storage medium coupled to the one or more processors, on which instructions are stored, and when executed by the one or more processors, the one or more processors perform operations according to the implementation of the method provided herein.

[0025] It should be understood that the method according to the present disclosure may include any combination of aspects and features described herein. That is, the method according to the present disclosure is not limited to the combination of aspects and features specifically described herein, but also includes any combination of aspects and features provided.

[0026] These and other implementations may each provide one or more advantages. The implementation reduces the computing resources used to identify packages with damaged labels by efficiently processing the data of the packages collected at the transportation network. Using the technology of the implementation of the present disclosure, the data collection of packages within the transportation or delivery network can be enhanced by implementing different sensors and computers to obtain relevant data in a timely manner for identifying packages with damaged labels to be routed through the network. According to the implementation of the present disclosure, the identification of a specific package is performed faster because a combination of one or more identification algorithms can be implemented. Different identification algorithms can be executed sequentially based on a dynamic evaluation of the computational cost of applying these algorithms to specific data obtained at the network for the package. Therefore, the identification implemented provides useful information from the label of a specific package to route the package through the delivery network in a faster, cheaper and quite flexible manner.

[0027] The details of one or more implementations of the present disclosure are set forth in the drawings and the description below. Other features and advantages of the present disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Depicted is an example transportation network that may be used to route packages in accordance with implementations of the present disclosure.

[0029] Figure 2A is a flow chart illustrating an example process for routing a package according to an implementation of the present disclosure.

[0030] Figure 2B is a flow chart illustrating an example process for matching a package with a damaged label to an image of the package before the label was damaged, according to an implementation of the present disclosure.

[0031] Figure 3 Depicted is a conveyor belt for transporting products at a delivery hub portion of a delivery network where data for packages may be collected in accordance with implementations of the present disclosure.

[0032] Figure 4 is a block diagram of an example scheme for arranging packages in a shipping vehicle according to an implementation of the present disclosure.

[0033] Figure 5 is a block diagram of an exemplary system for identifying and routing packages with damaged labels according to the present disclosure.

[0034] Figure 6 is a flow chart illustrating an example process for identifying a package based on applying one or more identification algorithms in a sequential manner according to an implementation of the present disclosure.

[0035] Figure 7 is a schematic diagram of an example computer system that can be used to perform implementations of the present disclosure.

[0036] Like reference numbers in the various drawings represent like elements. DETAILED DESCRIPTION

[0037] Implementations of the present disclosure involve data processing associated with identifying a package with a damaged shipping label among multiple other packages within a shipping network that have not yet been routed to a final shipping destination. Specifically, the process disclosed herein matches a package with a damaged shipping label to an image of the package before damage occurred. The matched image can then be used to extract shipping data from the undamaged label in the image.

[0038] In some implementations, a transportation network may be organized to facilitate the transportation of packages between transportation hubs (e.g., locations or sorting facilities) that are intermediate or final destinations for packages mailed by customers. In some implementations, when a package is provided for routing through the transportation network, the package is marked and a route is defined according to transportation conditions defined by the shipper, or otherwise generated for the package (e.g., routing requirements, price criteria, availability of transportation modes, etc. generated by a software application). In some implementations, the transportation network is a delivery network of a logistics operator that mails and delivers packages (e.g., parcels, letters, boxes, etc.) through defined delivery hubs within the delivery network.

[0039] In some implementations, a package to be shipped can be received at a package station of a transportation network and directed to a destination address (e.g., a shipping hub or other location that can be reached by transfer from a shipping hub of the network). The package can be tagged with data related to the package and indexed with shipping data for routing. In some implementations, a shipping label can include information that a carrier needs to provide to transport a package from its origin to its destination. The shipping label can include a scannable barcode with a tracking number that enables the carrier to provide delivery updates. In some implementations, a shipping label can be different from a mailing label, which only includes the name and address of the recipient. Typically, a unique shipping label is required for shipping, and the shipping label can be tracked during the routing of the package at different locations of the carrier. In some implementations, data on the progress of the shipment can be collected as the package is routed through the carrier's transportation network. For example, images of the package can be collected, the package can be weighed, the package can be recorded in a database, and contextual information for a specific shipment can be collected (e.g., the order in which the packages are organized in a specific vehicle or vessel, and the identification on the vehicle or vessel used to carry the package is arranged).

[0040] In some implementations, packages may be mailed through one or more transportation hubs of the network and with the aid of multiple modes of transportation and according to a predetermined schedule of transportation vessels (e.g., cars, buses, trucks, trains, airplanes, ships, etc.). For example, two transportation hubs may be connected by one or more modes of transportation (e.g., trucks only, or trucks and airplanes). The transportation may be performed according to a predetermined schedule of transportation between the first hub and the second hub (e.g., one flight per day, or trucks departing from locations at 0800 and 160000 each day).

[0041] In some implementations, when a package is received for transport at a shipping facility or location, it can be tagged, and the tag can be used to route the package during the shipping process. For example, the tag can include information about the recipient and sender, the type of delivery (e.g., urgent, fragile, priority, etc.), and information about the shipping route. The route can define the shipping hubs through which the package is routed before it reaches the final destination. In some examples, the route can also indicate the type of transportation between locations (e.g., the mode of transportation). In some implementations, the tag can include specific data about the package (e.g., the type of goods being transported, the priority of transportation, etc.). When the package is routed, the tag is reviewed at each shipping hub to record the progress of the package through the shipping network. In this way, tracking of the progress of the package can be recorded and provided to the sender or recipient upon request. In some cases, the label may be damaged (e.g., partially or completely scratched, broken, blurred, torn, removed, etc.), and such damage may affect the identification of the package and how to determine whether to continue to transport the package through the shipping network. In some implementations, according to implementations of the present disclosure, a package may be identified by using one or more identification mechanisms to identify the package from other packages that have been recorded as being shipped through a transportation network.

[0042] In some implementations, data of packages recorded at a transport hub in a delivery network that have not yet arrived at the final destination of the package can be collected and used to identify packages with damaged labels. Packages with damaged labels are expected to have been scanned at a previous location on their routing (at least when they were initially received for delivery) where the package had an intact shipping label. In some implementations, data of packages within a threshold distance from the location of the package with the damaged label (e.g., initially identified as having a damaged label) can be collected and used to apply one or more recognition algorithms to recognize images of packages corresponding to the package with the damaged label before the label was damaged. For example, the recognition algorithm attempts to match the package with the damaged label with images of other packages based on package characteristics other than the label to recognize images of the package taken before the label was damaged. In some implementations, recognition can be based on different recognition criteria, including size, weight, location, visual appearance (e.g., color, markings, patterns, etc.), probability of arriving at a specific location, contextual information of the delivery, etc.

[0043] Figure 1 An example transportation network 100 that may be used to route packages in accordance with implementations of the present disclosure is depicted. Transportation network 100 is a logistics network that includes multiple locations, hubs, or facilities and vehicles or other transportation units for transporting packages therebetween. Transportation network 100 may include or be associated with a data network 106 that includes one or more data collection components at different locations in the logistics network. Data included at or associated with the transportation network 106 may be collected based on sensors and computers configured for the packages being tracked. The collection of data may be performed automatically or based on a user or external application interactively inputting data related to packages transported within the transportation network.

[0044] The computer system 108 includes one or more server devices and a database (e.g., processor, memory). In some implementations, the computer system 108 may store data and logic for processing data related to the transportation network 100. The computer system 108 may process requests for dispatching packages through the transportation hubs of the transportation network 100, and may monitor the progress of package transportation. In some cases, the computer system 108 may obtain data from each or some transportation hubs in the transportation network 100 and one or more vehicles providing transportation services. In some implementations, the data obtained may include, but is not limited to, text data, label data, image data, video capture data, other types of sensor data, or combinations thereof. In some implementations, a user may interact with the computer system 108 through a device to send and receive information related to one or more packages routed through the transportation network 100, or information related to services provided by the transportation network 100 (e.g., status checks, delivery estimates, etc.).

[0045] In some implementations, the device may communicate with the computer system 108 via the network 106. In some examples, the device may be any suitable type of computing device, including, but not limited to, a desktop computer, a laptop computer, a handheld computer, a tablet computer, a personal digital assistant (PDA), a cellular phone, a network appliance, a smart device, a network-connected scanner, or a suitable combination of any two or more of these or other data processing devices. In some implementations, the network 106 may include a large computer network, such as a local area network (LAN), a wide area network (WAN), the Internet, a cellular network, a telephone network (e.g., a PSTN), or a suitable combination thereof, that connects any number of communication devices, mobile computing devices, fixed computing devices, and server systems.

[0046] In some implementations, the computer system 108 includes at least one server and at least one data storage. Figure 1 In the example of FIG. 1 , computer system 108 is intended to represent various forms of servers, including but not limited to web servers, application servers, proxy servers, network servers, and / or server pools. In general, a computer system accepts requests for application services and provides such services to any number of devices (e.g., devices on network 106). In some implementations, computer system 108 is a cloud platform that deploys cloud applications for managing and storing data related to transportation network 100.

[0047] In some implementations, the transportation network 100 includes a plurality of transportation hubs (or locations or nodes) that can be defined in a graph structure to support transportation from a starting destination to a final destination. The hubs of the transportation network include locations A, B, C, D, G, F, H, and I. In some implementations, additional hubs can be defined in the network. The hubs in the network 100 can be connected by edges representing transportation routes between the hubs, such as Figure 1 In some implementations, two hub locations can be connected with one or more types of transportation (eg, trucks and / or airplanes), where each mode of transportation can be associated with a schedule for operating the transportation.

[0048] In some implementations, location A 105 may be a receiving hub where package 130 is received for delivery by the sender to a destination location. For example, the destination location may be a hub of transportation network 105, such as location H 135. In another example, based on the destination location defined for package 130, it may be determined that the package may be routed through locations C, D, F, and H to reach a final destination location that is not part of a hub defined in transportation network 100 (e.g., a private address).

[0049] In some implementations, when package A 130 is received for transport to its target destination (e.g., location H 135), a route for dispatching package 130 may be defined. For example, the route may be defined based on the initial delivery requirements provided for the package from the sender. Package 130 may be transferred through a route including transport hubs A, C, D, F, and H. The mode of transport connection between location A and location C 110 may be a truck, where package 130 is placed on the truck along with other packages transported from location A to location C according to a placement order. In some implementations, the placement order of packages (e.g., packages received from a conveyor belt) on the truck at location A may be recorded. For example, the determination of the order of the locations included in the route may be performed manually or automatically by an implemented logistics software service. In some implementations, the determination of the route may be based on the shipping request of package A and the priority defined for the shipping. In some implementations, if there are two options for transportation modes (such as truck and airplane options) between two nodes (e.g., between C and D), then if the shipping request is for regular non-priority delivery, then package A may be routed by truck. In those cases, data of the package may be collected at two locations while package A is transported from C to D. The collected data may be data from a label of the package 130 (e.g., before any damage to the label), as well as image data collected at that location (e.g., on a conveyor belt), or contextual data related to the order of the packages being placed inside the truck at location C and being brought outside the truck at location D.

[0050] In some implementations, where, for example, package 130 is sent to destination H, package 130 is inspected at each location during a defined route through transportation network 100, and data is collected at each location. Figure 3 The data collection may be performed as described above, where the data of the package is collected based on sensor data collected from a camera located in close proximity to a conveyor belt, or other sensors (such as scale sensors located at the conveyor belt). In some other examples, the data collection of the package may be performed while the package is being loaded onto a truck or other vehicle. For example, the data of the package may be collected as described above. Figure 4 Data for arranging packages at the truck is collected as described.

[0051] In some implementations, the label of the package 130 may be obstructed or damaged while the package 130 is being routed from its starting location A to its destination location H. For example, the label may be partially or fully scratched, may be covered with some substance that may make the label illegible, may be removed, or may be covered with another label that makes the label unreadable. For example, if it is determined that the package 130 has a damaged label at location D, then a determination of how to continue routing the package from location D to its target destination may be performed based on data that has been collected about packages that have been transported through the transportation network 100 but have not yet arrived at their final destination as expected.

[0052] In some implementations, if a package is identified as having a damaged label at a first location, data of other packages collected at other locations of the transportation network 100 may be collected. The other locations may be locations that are reachable within a time period that could reasonably be expected to have reached the first location, or locations that are reachable based on a planned transportation route to the first location since the package with the damaged label was discovered. In some implementations, packages that have been tracked at other locations may be used as a candidate set of packages that include images of packages that match the currently processed package with the damaged label. It is understood that the package with the damaged label was scanned and processed at a previous location before it was discovered at the first location. Therefore, the collected data of previously processed packages may be used as a basis for executing one or more identification algorithms to determine which previously processed package matches the package with the damaged label. The package with the damaged label may then be routed to its destination based on routing and / or address information associated with the matching package.

[0053] In some implementations, different locations within the transportation network 100 may be equipped with sensors and processing tools and techniques to track information for each package handled at a location. In some implementations, the tracking information may be recorded in a database, such as a database portion of the computer system 108. The tracking information may be organized for different locations, for different types of packages, and may include timestamp data, weight data, priority data, status data, routing data, final destination data, and sender and recipient data, among other example data. In some implementations, data for each package transported through the transportation network 100 may be collected as a single unit with an indication of the progress of the transport.

[0054] In some implementations, data collected about packages routed through transportation network 100 may be used to provide notification services to end users or services, or to support identification of packages that cannot be identified (e.g., due to damaged labels).

[0055] In some implementations, if package A 130 is identified as a package with a damaged label at location D 115, data of packages that have been tracked at the transportation network can be collected. Such data can be collected for packages that meet one or more criteria, which can be predetermined or dynamically interpreted. For example, data of packages tracked in a computer system 108 associated with the transportation network 100 at a point in time that is within a preset amount of time (e.g., within the previous 2 to 4 hours, within the previous 10 to 20 hours, etc.), or within a current geographic distance from location D, or based on other predetermined criteria that can be used to dynamically determine which packages can be used for initial evaluation. In some implementations, the predetermined criteria can be defined as requirements that can be used to compare locations that are immediately adjacent to location D, which can be dynamically evaluated to determine which packages are relevant to the evaluation and identification of package A with a damaged label at location D. The criteria can be defined based on the type of vehicle that package A may have been delivered by, and the expected time period for delivering the package from a location connected to location D that provides a transportation mode corresponding to that type of vehicle.

[0056] In some implementations, criteria may be defined to filter locations connected to the package's location D 115 that have the same type of transportation. For example, if package A 130 has been delivered by airplane, locations connected to the transportation mode "airplane" may be determined, such as locations F and C, and based on the distances between D and C and D and F, time periods during which packages at the locations may have been scanned and transferred may be determined. In this example, if the travel time between the two locations is approximately 2 hours, and flights are scheduled every 12 hours, packages that have been scanned between 14 and 16 hours prior to the time package 130 was identified as having a damaged label may be identified and used as an initial set of packages to be evaluated to determine whether a matching package in the subset may be identified as package 130.

[0057] In some implementations, the initial set of potentially matching packages may be defined too restrictively. Such a definition may result in difficulty identifying correctly matching packages. For example, defining a higher precision to limit the set of packages that can match a package with a damaged label may result in an inability to determine a match. In such a case, the provision of filtering may be modified to include more packages for evaluation that may be possible matches to the first package. Thus, in the event that a matching package cannot be determined with a desired level of certainty based on the initial set of packages, a new definition of the initial set of packages may be determined.

[0058] In some implementations, the example of defining an initial set of packages based on a time period before identifying package A 130 may be construed as an example and may be combined with other criteria that may be used to determine an initial set. Additionally, a new initial set may be determined, where such a new set may be determined based on the same or different criteria used to identify packages. Where the same criteria are used, the criteria may be customized to a different granularity to allow more packages to fall into the initial set, and packages that have already been evaluated may be re-evaluated with the new initial set, or excluded from the evaluation.

[0059] In some implementations, package A 130 may not have been tracked for the type of transportation being delivered, and then locations within a threshold distance (e.g., time or geographic distance) may be determined to collect data for tracked packages at those locations. In some other cases, the initial set may include all packages tracked in the transportation system. Additionally, the initial set may be defined based on other criteria, such as the visual appearance of the package (e.g., a red package). In some implementations, the initial set may be defined based on other example criteria or a combination of criteria, and the initial set may be iteratively defined based on different criteria until a matching package is determined.

[0060] Figure 2Ais a flow chart illustrating an example process 200 for routing a package with a damaged shipping label. Process 200 may be executed by one or more computing systems, such as Figure 1 Computer system 108.

[0061] In some implementations, a package may be transported through a transportation network, which may be a delivery network for handling the delivery of packages between an originating location and a destination location. While transporting the package, the package may move through one or more locations where data about the package may be collected. In some implementations, such data may include, but is not limited to, timestamp data, physical characteristics of the package (e.g., weight, color), image data, and video data.

[0062] At 205, first data of a first package within a delivery network is obtained. The first package includes a label that is damaged and prevents automatic identification of the first package based on the label. The first data may include, but is not limited to, an image of the package with the damaged label, a weight of the first package, a size of the first package, a shape of the first package, dimensions of the first package, a previous location of the first package within the delivery network (e.g., a package drop or sorting facility, if known), a current location of the first package within the delivery network, and a location of the first package within a sequence of packages unloaded from a delivery vehicle. The first data may be obtained, for example, from one or more sensors, computers, or databases within the delivery network.

[0063] At 210, second data of a plurality of packages within a delivery network is obtained. The plurality of packages are selected as potential matches for a package with a damaged label (i.e., a first package). And, the second data includes characteristics of each package in the plurality of packages that can help identify a match with the first package. The second data includes data associated with the second package obtained before the damage of the first package was discovered, for example, data obtained at a transportation network location (e.g., a package drop or sorting facility) that the package passed earlier in its corresponding delivery route. For example, the second data may include, but is not limited to, an image of the package, a weight of the package, a size of the package, a shape of the package, a dimension of the package, a previous location of the package within the delivery network (e.g., a package drop or sorting facility), a current location of the package within this delivery network, and a location of the package within the sequence of packages unloaded from a delivery vehicle. The second data may be obtained, for example, from one or more sensors, computers, or databases within the delivery network.

[0064] In some implementations, a plurality of packages associated with the obtained second data can be determined to be packages transported through the delivery network that have not yet arrived at their intended destinations. For example, the plurality of packages includes packages that were scanned as departing from a facility (e.g., a hub or location) and were not scanned as arriving with the package at a destination or intermediate location of the routing trip.

[0065] In some implementations, a plurality of packages may be packages that were recorded or scanned at a departure location at the delivery network, but were not recorded or scanned as arriving as expected. For example, a set of packages may be recorded as departing from location C 110 and expected to arrive at location D 115 within 4 hours. For example, the travel time between C and D is taken to be 3.3 hours on average, and a buffer period may be added to determine 4 hours as the expected travel time. In some other examples, the expected travel time may be determined based on applying a prediction algorithm, which may be trained based on historical data collected from package shipments operating at the delivery network. In those examples and with reference to Figure 1 In the example transportation network 100, if a package is scanned at location C as departing from location D, but has not yet arrived at location D within the expected travel time, the package may be included in the multiple packages as a potential match with the first package that was determined to have a damaged label at location D 115.

[0066] In some implementations, the second data obtained is for a plurality of packages defined as an initial set of packages to be used for applying the identification algorithm. Figure 1 In some implementations, the initial set of packages can be defined based on identification criteria associated with requirements, such as a time period for scanning a package before identifying a first package and obtaining first data, a distance between the location of the first package and other locations (hubs or facilities) of the delivery network where one or more packages are recorded (or scanned).

[0067] In some implementations, multiple packages may be defined as packages that match the first package and that meet the reasonable expected package criteria based on a likelihood of having arrived at the location where the first package was found to have a damaged shipping label. For example, if package X was scanned at location I one minute before the first package was found to have a damaged label at location A (which is remote from location X), and there was no reasonable expectation that package X might have been transported from location I to location A, then package X would not be a possible match for the first package and would not be included in the multiple packages associated with the obtained second data at 220.

[0068] At 220, a particular package among a plurality of packages is identified. This is accomplished by applying one or more identification algorithms (see below) to the Figure 2BIn some implementations, applying one or more identification algorithms includes obtaining a subset of possible matches from the plurality of packages as an output from each algorithm. In some implementations, the possible matches include packages identified by the one or more identification algorithms as having one or more characteristics similar to the first package.

[0069] In some implementations, the set of algorithms can be applied one after another in a sequential manner. The identification algorithm reduces the number of packages that fall within the criteria for a possible match to the first package, and thus, the set of possible matches is sequentially reduced by each algorithm to determine one or more matches associated with a high probability of being the first package before the shipping label was damaged.

[0070] In some implementations, identifying a particular package includes evaluating the first data and the second data by applying one of the one or more identification algorithms. In some implementations, the determination of which algorithm to apply can be performed dynamically based on the characteristics of the first data and the second data. In some implementations, the second data includes image data with a lower resolution, and the use of image processing techniques to compare the first data and the second data can be performed using fewer computing resources. In these cases, selecting an algorithm based on image comparison can be selected as the first algorithm to be applied to identify a particular package. For example, an image comparison can be performed to compare the size of a package, the color of a package, specific visual features of a package, and machine learning recognition techniques to determine a match with a first package from multiple other packages.

[0071] In some implementations, applying the first algorithm (e.g., package weight comparison, scan order analysis, package color comparison, package size comparison, feature analysis, routing analysis) can include matching a first portion of the first data with a first portion of the second data, where both first portions are associated with a first characteristic from a set of characteristics of the first package. For example, the first portion is associated with a color of the package.

[0072] In some implementations, in response to performing a match based on a first characteristic of the first package, a first subset of the plurality of packages can be determined. The first subset can substantially match the first package based on the first characteristic.

[0073] In some implementations, in response to determining that a first subset of the plurality of packages includes a set of packages within a predetermined threshold number (e.g., defined as the number of packages or defined as a percentage of the number of packages from the initial set), the first subset can be provided as input to a second subsequent algorithm. In some implementations, the second subsequent algorithm can be a machine learning model that identifies a specific package from the subset that matches the first package. In some implementations, the machine learning model is trained to identify a specific package based on image recognition. When performing package identification based on image recognition, the machine learning model can provide a score of similarity between two or more images. In some cases, a high similarity score can be determined when comparing two images representing the same package, but the package lacks a label on one image and the package does not lack a label on the other image. In some cases, machine learning logic can be implemented to calculate a score of a predetermined number of pairwise comparisons of the image of the first package without the label with the corresponding images of the image set of other packages (e.g., a percentage of all or all images of the other packages).

[0074] In some implementations, the comparison can be performed on portions of the image (e.g., images of the package from different perspectives) in stages and / or batches. If it is determined that the similarity score is above a predetermined threshold, the image of the package corresponding to the similarity score can be output and provided for verification and / or routing of the first package that lacks the label. In such an implementation, the remaining images of the other packages can be omitted from processing the machine learning model because the candidates that match the first package without the label are output. For example, if it is later determined that the candidate package does not match the first package (e.g., based on user verification), the subsequent application of the machine learning model can be performed based on the images of those remaining packages that have not yet been evaluated. In some implementations, a comparison is performed between the image of the package with the missing label and the image of each of the other packages. In such an implementation, a match can be determined based on the pairwise comparison with the highest score.

[0075] In some implementations, an algorithm from the one or more identification algorithms can then be selected and applied to the second data and the plurality of packages in a particular order. In some implementations, the order can be provided as a preset order or an order dynamically determined based on the type of data, the amount of data, the number of packages, and the rules and precision applied for identification, among other examples.

[0076] In some implementations, one or more identification algorithms may be determined to be associated with corresponding one or more weighted scores, the weighted scores representing expected gains from running the corresponding algorithms. In some cases, the weighted scores may be determined based on historical data collected over time to apply the algorithms to identify, for example, a package in a particular shipping or transportation network.

[0077] In some implementations, weighted scores may be determined as needed when determining which algorithm to apply, and the weighted scores may be used to prioritize the order in which the algorithms are applied. For example, a higher weighted score indicates that the associated algorithm has a greater potential to significantly (or substantially) reduce the evaluation (or candidate) set of packages to potentially match the first package without a label. In some cases, even if the computational complexity of a particular algorithm is low, a different, more computationally complex algorithm may be applied first because the more computationally complex algorithm may reduce the set of packages being evaluated by a greater amount, thereby potentially reducing the total number of identification algorithms that must be applied to reduce the set of packages being evaluated to below a threshold size. In some implementations, this preference for more computationally complex algorithms may be based on an evaluation of historical results and performance measurements of all algorithms and the performance of the algorithms in reducing the size of the set of packages being evaluated. Thus, algorithms from one or more algorithms may be evaluated to determine a weighted score for each algorithm corresponding to their efficiency in reducing the size of the set of packages being evaluated to determine the packages that match the first package without a label.

[0078] For example, if a package weight analysis algorithm can generally reduce the set of candidate packages by 50%, and a package size analysis can reduce the set of candidate packages by 90%, then the package size analysis can be associated with a higher weighted score than the package weight analysis algorithm, even though the package size analysis requires more computational resources than the package weight analysis. In some implementations, the weighted score can be based on a combination of the computational complexity of the associated identification algorithm and the amount by which the algorithm can reduce the size of the set of packages being evaluated.

[0079] In some implementations, applying one or more identification algorithms includes applying multiple identification algorithms in order. For example, the order may be similar to that described with respect to Figure 2B The discussed algorithm is described in the order in which it is run. In some implementations, at each step in the order, a corresponding set of data of the second data from the plurality of packages is evaluated. The corresponding set can be associated with: i) the corresponding characteristic being evaluated, and ii) a previously determined subset of packages from the plurality of packages that matched the first package at a previous step in the order based on the previously evaluated characteristic of the first package.

[0080] In some implementations, at each step in the sequence, a characteristic of the first package is evaluated. The characteristics may include, but are not limited to, package weight, package color, package scanning order, package size, external surface markings, and expected package delivery route, among other examples. In some implementations, the corresponding characteristic may be selected to determine an identification algorithm for evaluating the first data and the second data based on the corresponding characteristic.

[0081] In some implementations, the identification algorithms that may be applied may have relatively different computational complexities and resource requirements for their operation. Such characteristics of the algorithms may be based on the type of identification algorithms that may be arranged in an order according to different computational complexities based at least on the number of packages provided as input to the identification algorithms from a previous iteration. In some implementations, one or more identification algorithms may be evaluated and a computational cost may be assigned to each algorithm each time the application of one of the algorithms is determined, wherein the algorithm with the relatively lowest computational cost may be applied to determine a subsequent subset of packages that may be used to identify a first package or as an initial set for subsequent application of another identification algorithm. In some implementations, the identification algorithms may be applied in order according to increasing computational complexity.

[0082] In some implementations, after the identification algorithm has been executed, identifying the particular package from the plurality of packages can be performed based on providing one or more images of the first package and an image of each package in the subset of possible matches as input to a machine learning model trained for image recognition. In some of these cases, the subset of possible matches is determined based on an initial set of packages defined as the plurality of packages, or based on filtering the initial set of packages based on another identification algorithm (e.g., color matching, size matching, weight matching, context matching, etc.).

[0083] At 225, the first package is routed to its final destination through a shipping network using information from the label of the particular package identified at 220. In some implementations, the label information can be obtained from an image of the particular package that includes the label. In some implementations, the label information can be obtained from a database record associated with the particular package (e.g., a database record indicating a destination address).

[0084] Figure 2B is shown as from Figure 2A Flowchart of example steps performed as a portion of step 220. Figure 2B The sub-process presented above is based on Figure 2A The first data and the second data obtained are executed as described in the above, and a method for running Figure 2A 220 in an example process flow.

[0085] The obtained second data is for a plurality of packages that are scanned at the departure location and are not identified as being delivered at their intended destination. In some implementations, the plurality of packages are a subset of all scanned packages that have not yet been identified as having arrived, which also meet the filtering criteria to reduce the subset of packages within the following distances (e.g., geographic distance or transportation time when considering a transportation schedule): a higher probability associated with a previous location where the first package was previously scanned, a distance associated with a higher probability. For example, a package that was scanned two hours ago at a location four hours away from the location of the first package is associated with a lower probability of matching the first package. For example, if such a package was scanned two hours ago at a location with a direct transportation route to the location of the first package, but the travel time between the two locations is four hours, the package will not be a possible match for the first package, and its data will be excluded from the second data. In some implementations, the process may also consider the transportation schedule between locations in the delivery network. For example, if a package was scanned two hours ago at a location with a direct transportation route to the location of the first package, the package may be a possible match for the first package if the travel time between the locations is less than two hours. However, if the shipping schedule between the locations does not plan any transportation routes between the two locations during that time, then the package will not be a possible match. In other words, only packages that have a reasonable probability of being located at the location of the first package at the same time as the damaged label on the first package was found, based on the travel time and / or shipping schedule, are included in the plurality of packages used to obtain the second data.

[0086] In some implementations, initial filtering criteria may be defined for determining a plurality of packages for obtaining the second data. Such filtering may define a threshold travel time between a first package and a scanning location of another package (e.g., scanned at a packing station with a direct connection to the location of the package, scanned at a network location within a 200-mile radius, scanned at a packing station that is connected using existing transportation modes and can be reached within 12 hours based on an operating schedule, etc.).

[0087] At 231, a first identification algorithm is applied. For example, the first identification algorithm can be a package weight analysis that includes comparing a weight of a first package included in the first data with weights of other packages included in the second data. In some implementations, the weight analysis is used to exclude from the plurality of packages any packages from the plurality of packages that have a corresponding weight that does not match the weight of the first package within a weight threshold range (e.g., a weight range value or a percentage range around a weight).

[0088] At 237a, the subset of packages determined to match the first package by applying the first algorithm at 231 is compared to a threshold number of packages to determine which identification algorithm to apply to identify a specific package that matches the first package from the plurality of packages. For example, if a weight analysis is performed and the number of packages in the subset is two, where the threshold is defined as 2 or 3 or more, the subset of two packages and corresponding data from the second data can be provided to 282 to perform a machine learning identification analysis on the image of the first package and the images of the remaining two packages in the subset. It will be appreciated that these thresholds are examples and other thresholds can be configured for the operation of identification. In addition and for example, the thresholds can be provided as user input and can be obtained at each step, rather than being predetermined for the method 220 as a whole.

[0089] In some implementations, if the subset defined after running the first identification algorithm does not meet the criteria 237a, then the subset can be provided at 232 for running a second identification algorithm.

[0090] At 232, a subset of packages that meet the identification criteria for the first identification algorithm at 231 is provided for use in a second identification algorithm. For example, such an identification algorithm may be a package scan order analysis. Applying the package scan order analysis may include comparing the order in which a first package is loaded into a delivery vehicle or a delivery container with the order in which other packages in a plurality of packages are unloaded from the delivery vehicle or the delivery container. In such an example, based on the analysis, any package within the plurality of packages having a corresponding unloading order that is not within a threshold range of an expected unloading order for the first package is excluded. The expected unloading order may be based on the order in which the first package is loaded into the delivery vehicle or the delivery container. For example, when packages are loaded on a truck or other mode of transportation, they may be loaded in a particular order in which the packages can be tracked and stored. In the event that the packages are unloaded in a different order than the order in which they were loaded, identifying the packages based solely on the order may be problematic. However, a certain variation in the loading and unloading order may be expected and acceptable in order to determine with a higher probability that a package loaded at a given location is likely to have been unloaded at a location within a range of locations around its loading location (or in the reverse order). In some implementations, the identification algorithm may be used as described with respect to FIG. Figure 3 and Figure 4 Tracking of loading and unloading positions is performed as described.

[0091] In some implementations, the result set of packages after applying the package scan order analysis can be evaluated based on criterion 237b. Criterion 237b is substantially similar to criterion 237a and evaluates whether the number of packages in the determined subset is below or equal to a threshold. If the subset is below or equal to the threshold (e.g., 5 packages, otherwise 1% of the packages from the initial package set used at the first algorithm), the result subset from the second algorithm can be provided to 238 for machine learning identification analysis, as previously discussed.

[0092] In some implementations, if the subset defined after running the first identification algorithm does not meet the criteria 237a, then the subset can be provided at 232 for running a second identification algorithm.

[0093] At 233, the result subset of the packages after applying the second identification algorithm 232 can be provided for applying a third identification algorithm. For example, the third identification algorithm can be a package color analysis. In some implementations, the package color analysis can include comparing a first set of colors present in the image of the first package to a set of colors present in the image of the packages provided as the subset of the result set from 232. Based on the package color analysis, any packages within the plurality of packages having a corresponding set of colors that exceeds a threshold similarity value to the first set of colors present in the image of the first package are excluded.

[0094] In some implementations, when comparing images of a first package with images of other packages from the relevant subset for the identification algorithm, color comparison can be based on color captured at a granular level by segmenting the package from the background in the relevant image and creating a histogram of the colors present in that area of ​​the image. In some implementations, the histogram can be normalized to correct for different package sizes and different distances from the camera that has acquired the image of the package to the location of the package when the photo was taken. These distributions of pixel values ​​can be compared in a variety of ways (including based on Bhattacharyya distances) so that it can be determined that the images of the package with the smallest (or sufficiently small) Bhattacharyya distance are likely to be the same package.

[0095] In some implementations, comparing the first set of colors present in the image of the first parcel to the sets of colors present in the images of other parcels of the subset provided for running the third algorithm (e.g., the result set of the second algorithm) can include determining a Bhattacharyya distance between pixel values ​​of the image of the first parcel and corresponding pixel values ​​of the images of other parcels from the subset. In some implementations, excluding from the subset of possible matches any parcels for use in the algorithm that have a corresponding Bhattacharyya distance to the first parcel that is greater than a threshold Bhattacharyya distance can be performed.

[0096] In some implementations, comparing the first set of colors present in the image of the first package to the sets of colors present in images of other packages of the associated subset of packages includes identifying regions within the images of the other packages that contain package labels and ignoring pixels in the regions when performing the comparison.

[0097] In some implementations, comparing the first set of colors present in the image of the first package to the sets of colors present in the images of other packages of the associated subset of packages includes identifying label regions within the images of the other packages that include package labels. In some implementations, the corresponding label regions within each image of the other packages are compared to a region within the image of the first package that includes a location on the first package that is similarly positioned to the location of the corresponding label region to determine whether the region within the image of the first package includes any damage markings. For example, the package color analysis may include using coloring differences within the images to identify relative locations of shipping labels on packages within the associated subset of packages. The same relative locations are then examined using image processing techniques in the image of the first package (i.e., the package with the damaged label) to determine whether the relative locations include damage (e.g., discoloration, missing pieces / damaged labels, etc.) that indicates the label is located at a similar location on the first package. If the first package has markings, i.e., a damaged label located in the same relative region as the shipping label in the image of the other package, then there is a likelihood that the other package matches the package with the damaged label. In other words, the package color analysis algorithm may identify the locations of the shipping labels on the images of the packages in the relevant subset and use those locations to perform targeted image analysis of the images of the package missing the label to determine if the damaged shipping label is in or present in the same area. For example, if the label is identified in the far upper right corner on the 8×12 inch side of the brown box with a seam running horizontally through the middle of the side, and the image of the first package (with the damaged label) shows a white spot with jagged edges (e.g., indicating that the label is torn off) in the upper right corner of the 8×12 inch side with a seam running horizontally through the middle of the side, then a possible match for the first package may be indicated.

[0098] In some implementations, an image of a package may include a label, and another image of a package may not have a label. When performing a color comparison, the images may be segmented, and areas including the label within one of the images may be ignored in both images to compare package colors that are expected to be similar.

[0099] In some implementations, where one image of a package includes a label and one image does not include a label or the label is damaged, the area where the label is located in one of the images can be compared to the area in the other image. In some implementations, the area around the label in both images (whether it is present, damaged, or absent) can include distinguishing features that can be associated with a higher probability of matching the package.

[0100] At 237c, check whether the result subset after applying the third identification algorithm is below the threshold. If the subset has fewer or an equal number of packages than the number of packages corresponding to the threshold, the subset is provided to 238 for performing machine learning identification analysis.

[0101] If the subset determined from the third identification algorithm 233 is greater than the threshold defined at 237c, then the subset is provided for a fourth identification algorithm. For example, the fourth identification algorithm may be a package size analysis. In some implementations, the package size analysis may include comparing a first set of package sizes determined from the image of the first package with a set of package sizes determined from images of other packages of the associated subset of packages for the fourth identification algorithm (e.g., package size analysis). In some implementations, any packages within the subset of packages having a corresponding package size that exceeds a threshold size similarity value for the first set of package sizes determined from the image of the first package may be excluded to determine the resulting subset. In some implementations, the threshold size similarity value may be based on an absolute cutoff value or a percentage cutoff, or according to a rule that relies on both.

[0102] In some implementations, the package dimension analysis includes determining a first set of package dimensions from the image of the first package by projecting a set of three-dimensional package models onto the image of the first package to identify a best fit between one of the package models and the first package within the image of the first package. The dimensions of one of the package models can be applied as the first set of package dimensions.

[0103] Based on excluding packages according to the implementation of the fourth identification algorithm, the result subset is evaluated in comparison with the threshold at 237d. If the subset is less than or equal to the threshold, identification can be performed by the machine learning identification algorithm at 238, as previously discussed. If the subset is greater than the threshold, subsequent identification algorithms can be executed (e.g., in a defined order).

[0104] like Figure 2B As shown, a subsequent identification algorithm may be applied after the fourth algorithm at 235, and a result subset of applying such an algorithm may be compared to a threshold at 237e. In addition, if the number of packages in the result set is above the threshold, further identification of the result set may be performed based on the subsequent algorithm at 236. Figure 2B As shown, at 235, the identification algorithm that may be applied is a distinguishing feature analysis, and at 236, the algorithm is a shipping route analysis.

[0105] In some implementations, applying the distinguishing feature analysis includes: identifying a first set of one or more features on the first package from the image of the first package. The features may include one or more of a mark, a scratch, a hole, or text, etc. Comparing the first set of one or more features to images of other packages of the algorithm's subset of related packages (e.g., provided as a result set from the fourth identification algorithm 234). Eliminating any packages within the subset of packages from the analysis that do not have at least one of the one or more features detected on the first package, and determining a result subset of packages that match the first package.

[0106] In some implementations, the subsequent identification algorithm applied at 236 can be a shipping route analysis. The shipping route analysis can include obtaining a location of a shipping facility within a shipping network where the first package was found to have a damaged label. Possible modes of delivery for other packages in the subset of packages for identification can be determined. Based on the analysis, any packages within the subset of packages that do not include a location within the corresponding possible delivery mode can be excluded. In some implementations, the locations of the shipping network (e.g., shipping facilities) for the other packages are determined, and the determined locations are filtered to those locations that have a direct connection to the location of the first package, or those locations that are directly connected to a single intermediate location between the corresponding location of the package and the location of the first package.

[0107] For example, if a package's delivery route should go through locations A->B->C->D->E, where the package was successfully scanned at location C (and it was last successfully scanned at C), and the package is found at location X, where X is a location with 4 intermediate transfers from C, then the package identified at C is less likely to match that package than another package identified at location Y, where Y is a location with a single intermediate transfer from C.

[0108] In some implementations, the shipping routing analysis can be performed based on the structure of a shipping network. In some implementations, the shipping network is a graph network that includes connections between locations (also referred to as hubs or facilities or nodes) associated with direction, type (e.g., transportation mode), cost, and length, and other connection characteristics.

[0109] In some implementations, the delivery network is a network graph including a plurality of nodes representing locations for delivery and pickup of packages. The network graph includes a plurality of directional edges representing connections between pairs of locations. The edges may also be associated with cost values ​​that may represent expected transit times (e.g., which may be a predicted value of transit times calculated based on evaluating historical data, or may be an average of the most recent predetermined number of transit times or the most recent transit time, among other examples). Based on the network graph, a plurality of routes connecting at least some of the nodes are defined.

[0110] For example, a delivery route for a package may be defined as comprising transportation according to the following chain of nodes from a graph delivery network structure: A->B->C->D->E. In this example, the package was last successfully scanned at location C 48 hours ago. If a package is found at X (which is 5 trip segments from C), but the time cost along the graph is 24 hours, then it is plausible (or possible) that the package is at X. However, even if Y is only one trip segment from C, if it takes 96 hours to get from C to Y, then the package is almost certainly not at Y. By evaluating the likelihood that a package scanned at another location matches a package with a damaged label, the set of candidate packages to be further evaluated can be reduced, and image processing techniques can be run based on the data of the packages in the candidate set to determine which is the package with the damaged label.

[0111] In some implementations, the algorithm can be changed such as Figure 2B For example, as previously discussed, the type of algorithm to be subsequently applied can be dynamically determined, for example, based on a decreasing order of computational complexity and resource requirements of the algorithm. Algorithms can be subsequently applied to reduce the number of potential candidate packages that will match the first package. According to implementations of the present disclosure, a subset of packages from an initially defined initial set of packages can be provided for machine learning identification based on an image of the package and an image of the first package.

[0112] Figure 3 is a conveyor belt 300 for transporting products at a delivery hub portion of a delivery network, wherein data of packages is collected, according to an implementation of the present disclosure.

[0113] In some cases, the conveyor belt 300 is installed at a location of the transportation network, for example, at a delivery facility. The conveyor belt 300 can be used to distribute packages received at a location to an exit point at the location, reaching multiple other locations connected to the location by various types of transportation. In some implementations, when a package is received at a location (e.g., from another location, as part of the route of the package to its destination), the package is scanned. There may be multiple ways to perform the scan. In some implementations, the conveyor belt can accommodate an arch 312, which is positioned above the conveyor belt 300 and is configured with multiple cameras 315, 310 and 325, which are configured to capture images of packages transported by the conveyor belt 300. The camera can be positioned in such a configuration that images can be taken from different angles. In some implementations, the package may include a label. In some cases, based on the images collected for the transmitted package, the label data of the package can be processed and the data of the package can be updated. The collection of the image data can also be associated with a timestamp to identify the time when the package was scanned.

[0114] In some implementations, the conveyor belt 300 may include Figure 1 The image data may be collected at one or more locations of the transportation network 100 of the embodiment of the present invention, and the data collected by the conveyor belt may be recorded at the computer system 108 for the scanned package. Such image data may be used to identify the package determined to have a damaged label. For example, if the package is determined to have a damaged label at another location, the data collected at the conveyor belt at a particular location may be used later. The other location may be a location closer to the location of the package with the damaged label and may be a possible location in the expected delivery route of the package with the damaged label.

[0115] In some implementations, the conveyor belt 300 can be provided with scale sensors 320 that can be distributed along the moving belt and can be used to weigh the packages. In some implementations, the packages can be positioned on the conveyor belt 300 to support weighing. For example, the packages can be positioned on one line but not on another line to enable measurement using a scale. In some implementations, the packages can be weighed on a scale external to the conveyor belt 300 before being positioned on the conveyor belt 300.

[0116] In some implementations, data collected from the conveyor belt can be used as Figure 2A A portion of the second data obtained at operation 210 of , and is also used throughout this disclosure and for example in Figure 2B The identification algorithm described in .

[0117] Figure 4is a block diagram of an example scheme 400 for arranging packages in a shipping vehicle according to an implementation of the present disclosure.

[0118] In some implementations, at a location in a transportation network (such as a shipping facility), the transport network can be configured to receive the transported goods by moving the transported goods from a conveyor belt 410 (e.g., substantially similar to Figure 3 The package is transferred from the conveyor belt 300 to a truck 405 (or other means of transportation) for processing.

[0119] In some implementations, the truck 405 can be provided with a camera 420 that can be positioned to capture packages entering the truck 405 through the rear door. The camera 420 can capture images of the packages being loaded (or unloaded if the truck is at the destination location). The camera 420 can also collect data for locating the order in which the packages are located within the truck 405. However, other sensors or tools can also be used to track the order in which the packages are loaded and unloaded.

[0120] Figure 5 is a block diagram of an exemplary system 500 for identifying and routing packages with damaged labels according to the present disclosure.

[0121] In some implementations, the example system 500 may be substantially similar to Figure 1 The computer system 108 of the present invention can be provided. The system 500 can be associated with a transportation or delivery network and can be provided with a data storage for storing data of packages distributed through the network. In some implementations, the system 500 can be provided with logic implemented to identify and route a package when the package is determined to have a damaged label.

[0122] In some implementations, the system 500 can be configured to operate as described herein and, for example, in Figure 2A and 2B The method of identifying the package in question.

[0123] In some implementations, data of identified packages 530 may be stored. The data includes data of packages that were scanned at a location (or facility) in the shipping network when shipping was requested and also scanned at the destination location when delivered. In addition, data of unidentified packages 540 is stored. Those are packages that were identified at a location within the transportation network but could not be identified and determined how to be dispatched. For example, these packages include packages with damaged labels, such as those ... Figure 1 , Figure 2A and Figure 2B described.

[0124] In some implementations, package data 515 is also stored for different locations within the transportation network. Such data is for identified or unidentified packages. The data collected for such packages may include information such as the sender and recipient of the package, tracking of the package's route or the package's expected route, shipping costs, the weight of the package, the inclusion of the package in different modes of transportation (e.g., including contextual information for arranging the package in a particular mode of transportation), and image data of the package (e.g., from a camera such as Figure 3 The images collected by the conveyor belt 300, or from such as Figure 4 The images collected by the transport vehicle of the truck 405, or images obtained in other ways), are other example data types of packages.

[0125] System 500 includes package data processing system 510 including implemented logic to identify unidentified packages from unidentified packages 540 based on applying identification algorithm(s) portion of identification logic 502. Identification logic 502 may include instructions for processing data from stored data at 515, 530, and 540 to identify packages that match the identified packages, e.g., as described with respect to Figure 2A and / or Figure 2B as described.

[0126] In some implementations, package data processing system 510 can receive input from user 505. The received input can include input for executing the identification algorithm, such as the identification of the algorithm to be used, the order in which the algorithms are run, and the properties of the algorithm runs, as well as other user input related to identifying the package and routing the package based on the identification.

[0127] Figure 6 is a flow chart illustrating an example method 600 for identifying a package based on applying one or more identification algorithms in a sequential manner according to an implementation of the present disclosure. In some implementations, the method 600 may be at least partially similar to Figure 2A Identification method 200.

[0128] In some implementations, database 602 stores data about packages routed through the delivery network. The package data may be similar to that in Figure 5515, 530 and 540 of the package. Database 602 includes package transportation data 603, which includes data of the recipient and sender of the package and / or routing data of the package. Package transportation data 603 can also include collected sensor data, such as image data collected at the location where the package is transferred. Package location data 604 including time data identifying when the package was scanned at different locations on the route of the package to its destination can also be stored for the package. The data of the package can be organized in multiple database tables, where the packages can be identified by identification numbers that uniquely distinguish them.

[0129] At 607, first data for a first package 605 is obtained. The first package 605 may be a package that has a damaged label and cannot be routed to its destination location because it cannot be identified. At 612, second data for a package 610 is obtained. The package 610 may be determined to be a package that has been sent through the shipping network but has not been scanned upon delivery. The second data for the package 610 may be obtained from the database 602. The second data obtained at 612 may be similar to the data described in Figure 2A The second data obtained at 210 is substantially the same. Package 610 can be determined to be a package scanned at an expected time and a location adjacent to the location of first package 605.

[0130] In some implementations, operations 607 and 612 can be performed substantially in parallel or sequentially. At 615, the first data and the second data are evaluated. The type of data collected can be determined. For example, it can be determined whether image data is part of the acquired data and whether such image data is of high resolution or low resolution. At 620, a current set of candidate packages to be evaluated is determined to determine a match for first package 605. The current set of candidate packages can be all or a portion of package 607. In some implementations, the current candidate set can be determined based on package 610, wherein package 610 can be evaluated to exclude packages having a location and / or time of the most recent scan that is associated with a lower expectation of a match for first package 605.

[0131] At 625, it is determined that an available identification algorithm is to be applied to identify a match for the first package 605. In some implementations, a list of identification algorithms may be defined or provided as input 627. In some implementations, the available identification algorithms may include information about Figure 2B Different identification algorithms are discussed.

[0132] At 630, it is determined whether the available algorithms include a weight identification algorithm (eg, corresponding to Figure 2B600 ). If the weight identification algorithm is part of the available identification algorithms, then at 640, the current candidate set is filtered to determine a new candidate set that includes only packages within a threshold range of the weight of the first package 605. At 650, the weight identification algorithm is removed from the list of available identification algorithms to be applied to identify the first package 605. Method 600 proceeds to 615, where the first and second data are evaluated. For example, only data related to the filtered candidate package set is determined as a relevant portion from the second data for further processing. Method 600 can iterate on steps 615 to 630. After the first application of the weight identification algorithm, method 600 can evaluate at 630 that the weight identification algorithm is not available and continue at 655. The computational cost of each algorithm portion in the algorithm portion of the list of available algorithms can be determined. The computational cost can be determined based on at least the size of the current candidate package set (as previously determined at 640 after applying the weight identification algorithm). At 660, an algorithm is selected from the list of available algorithms. The selected algorithm has the lowest computational cost. At 670, the algorithm selected at 660 is applied to determine a new candidate set, and the algorithm that has been applied is removed from the list of available algorithms. The process may proceed at 615 to evaluate the relevant data corresponding to the newly determined candidate set, and process the data of the set according to a new identified algorithm that may be selected based on the computational cost of the remaining algorithms. In some implementations, the computational cost may vary based on the type and quality of the first and second data, as well as the number of packages 612 and the number of packages in the new candidate set. Thus, each time a new candidate set is determined for application of a subsequent algorithm, the computational cost of the remaining available algorithms may change.

[0133] In some implementations, applying one or more identification algorithms (e.g., weight identification algorithm 630, or other identification algorithms selected at 660) includes evaluating at least a portion of first data 607 and second data 612 based on corresponding one or more characteristics of first package 605. Based on the evaluation of the first data and the second data, a set of packages is determined as a candidate set of packages that match the first package based on the corresponding one or more characteristics.

[0134] Reference now Figure 7, a schematic diagram of an example computing system 700 is provided. The system 700 can be used for the operations described in association with the implementations described herein. For example, the system 700 can be included in any or all of the server components discussed herein. The system 700 includes a processor 710, a memory 720, a storage device 730, and an input / output device 740. The components 710, 720, 730, 740 are interconnected using a system bus 750. The processor 710 is capable of processing instructions for execution within the system 700. In some implementations, the processor 710 is a single-threaded processor. In some implementations, the processor 710 is a multi-threaded processor. The processor 710 is capable of processing instructions stored in the memory 720 or on the storage device 730 to display graphical information for a user interface on the input / output device 740.

[0135] The memory 720 stores information within the system 700. In some implementations, the memory 720 is a computer-readable medium. In some implementations, the memory 720 is a volatile memory unit. In some implementations, the memory 720 is a non-volatile memory unit. The storage device 730 can provide large-capacity storage for the system 700. In some implementations, the storage device 730 is a computer-readable medium. In some implementations, the storage device 730 can be a hard disk device, an optical disk device, or a tape device. The input / output device 740 provides input / output operations for the system 700. In some implementations, the input / output device 740 includes a keyboard and / or a pointing device. In some specific implementations, the input / output device 740 includes a display unit for displaying a graphical user interface.

[0136] The features described may be implemented in digital electronic circuits, or in computer hardware, firmware, software, or in a combination thereof. The apparatus may be implemented in a computer program product tangibly embodied in an information carrier (e.g., in a machine-readable storage device for execution by a programmable processor), and the method steps may be performed by a programmable processor executing a program of instructions to perform the functions described by operating on input data and generating output. The features described may advantageously be implemented in one or more computer programs executable on a programmable system comprising at least one programmable processor coupled to receive data and instructions from and send data and instructions to a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used directly or indirectly in a computer to perform some activity or bring about some result. A computer program may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0137] For example, suitable processors for executing a program of instructions include both general-purpose and special-purpose microprocessors, as well as the sole processor or one of multiple processors of any kind of computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer may include a processor for executing instructions and one or more memories for storing instructions and data. Typically, a computer may also include one or more mass storage devices for storing data files, or be operatively coupled to communicate with them; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into an ASIC (Application Specific Integrated Circuit).

[0138] To provide for interaction with a user, these features may be implemented on a computer having a display device (such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (such as a mouse or trackball) through which the user can provide input to the computer.

[0139] These features can be implemented in a computer system that includes a back-end component (such as a data server), or includes a middleware component (such as an application server or an Internet server), or includes a front-end component (such as a client computer with a graphical user interface or an Internet browser), or any combination thereof. The components of the system can be connected by any form or medium of digital data communication (such as a communication network). Examples of communication networks include, for example, LANs, WANs, and the computers and networks that form the Internet.

[0140] A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a network (such as the one described). The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0141] Furthermore, the logic flows depicted in the accompanying drawings do not require the particular order or sequential sequence shown to achieve the desired results. In addition, other steps may be provided, or steps may be eliminated from the described flows, and other components may be added to or removed from the described systems. Therefore, other implementations are within the scope of the appended claims.

[0142] A number of implementations of the present disclosure have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of the present disclosure. Therefore, other implementations are within the scope of the appended claims.

Claims

1. A computer-implemented method executed by one or more processors, the method comprising: obtaining first data of a first package within a delivery network, wherein the first package includes a label, the label being damaged and preventing automatic identification of the first package based on the label; Obtaining second data of a plurality of packages in the delivery network; identifying a particular package from the plurality of packages that matches the first package by applying one or more identification algorithms to the first data and the second data, wherein applying the one or more identification algorithms comprises obtaining a subset of possible matches from the plurality of packages as an output from each algorithm, the possible matches comprising packages identified by the one or more identification algorithms as having one or more characteristics similar to the first package; and Information from the label of the particular package is used to route the first package through the delivery network.

2. The method according to claim 1, wherein: Identifying the specific package includes: The particular package that matches the first package is identified based on providing the image of the first package and the images of the subset of possible matches as input to a machine learning model trained for image recognition.

3. The method of claim 1 , wherein identifying the particular package comprises evaluating the first data and the second data by applying a first algorithm of the one or more identification algorithms, wherein applying the first algorithm comprises: matching a first portion of the first data with a first portion of the second data, both first portions being associated with a first characteristic in the characteristic set of the first package; and In response to the matching, determining a first subset of the plurality of packages that substantially matches the first package based on the first characteristic, In response to determining that the first subset of the plurality of packages includes a set of packages within a predetermined threshold number, the first subset is provided as input to a machine learning model to identify the specific package from the subset that matches the first package, wherein the machine learning model is trained to identify the specific package based on image recognition.

4. The method of claim 1 , wherein applying the one or more identification algorithms comprises applying a plurality of identification algorithms in a sequence, wherein at each step in the sequence a respective set of data of the second data from the plurality of packages is evaluated, the respective set being associated with: i) a respective characteristic being evaluated, and ii) a previously determined subset of packages from the plurality of packages that matched the first package at a previous step in the sequence based on a previously evaluated characteristic of the first package.

5. The method according to claim 4, wherein: At each step in the sequence, a characteristic of the first package is evaluated, wherein the characteristic is selected from the group consisting of: package weight, package color, package scanning order, package size, external surface markings, and expected package delivery route, and wherein a corresponding characteristic is selected to determine an identification algorithm for evaluating the first data and the second data based on the corresponding characteristic.

6. The method according to claim 4, in, Each identification algorithm is arranged in the order according to a different computational complexity based at least on the number of packages provided as input to the identification algorithm from a previous iteration.

7. The method according to claim 6, wherein: The identification algorithms are ordered according to increasing computational complexity.

8. The method of claim 1, wherein the first data and the second data are collected for a set of characteristics of packages routed through the delivery network, and the first data and the second data are associated with corresponding locations at the delivery network.

9. The method of claim 8, wherein applying the one or more identification algorithms comprises evaluating the first data and the second data based on corresponding one or more characteristics of the first package, and wherein based on the evaluation of the first data and the second data, a set of packages that match the first package is determined as a candidate set of packages based on the corresponding one or more characteristics.

10. The method of claim 1, wherein applying the one or more identification algorithms comprises applying a package weight analysis, the package weight analysis comprising: comparing the weight of the first package included in the first data with the weights of other packages included in the second data; and Any package within the plurality of packages having a corresponding weight that does not match the weight of the first package within a weight threshold range defined around the weight of the first package is excluded from the subset of possible matches.

11. The method of claim 1 , wherein applying the one or more identification algorithms comprises applying a package scanning sequence analysis, the package scanning sequence analysis comprising: comparing an order in which the first package was loaded into a shipping vehicle or shipping container with an order in which other packages in the plurality of packages were unloaded from the shipping vehicle or shipping container; and Based on the order in which the first package was loaded into a shipping vehicle or shipping container, any package within the plurality of packages having a corresponding unloading order that is not within a threshold range of an expected unloading order for the first package is excluded from the subset of possible matches.

12. The method of claim 1, wherein applying the one or more identification algorithms comprises applying a package color analysis, the package color analysis comprising: comparing a first set of colors present in the image of the first package to sets of colors present in images of other packages in the plurality of packages; and Any package within the plurality of packages having a corresponding set of colors that exceeds a threshold similarity value to the first set of colors present within the image of the first package is excluded from the subset of possible matches.

13. The method according to claim 12, wherein: Comparing the first set of colors present within the image of the first package with sets of colors present within images of other packages of the plurality of packages includes: determining a Bhattacharyya distance between pixel values ​​of the image of the first package and corresponding pixel values ​​of the images of the other packages; Wherein, excluding from the subset of possible matches any package within the multiple packages having a corresponding color set that exceeds a threshold similarity value of the first color set present in the image of the first package includes: excluding from the subset of possible matches any package within the multiple packages having a corresponding Bhattacharyya distance greater than a threshold Bhattacharyya distance.

14. The method according to claim 12, wherein: Comparing the first set of colors present within the image of the first package to sets of colors present within images of other packages of the plurality of packages includes identifying regions within the images of the other packages that contain package labels and ignoring pixels in the regions when performing the comparison.

15. The method according to claim 12, wherein: Comparing the first set of colors present in the image of the first package with sets of colors present in images of other packages in the plurality of packages includes: identifying label areas within images of other packages that include package labels; and The corresponding label area within each of the images of the other packages is compared with an area within the image of the first package, the area including a location on the first package that is similarly positioned to the location of the corresponding label area, and thereby determining whether the area within the image of the first package includes any damage marks.

16. The method of claim 1, wherein applying the one or more identification algorithms comprises applying a package dimensional analysis, the package dimensional analysis comprising: comparing a first set of package dimensions determined from the image of the first package to a set of package dimensions determined from images of other packages in the plurality of packages; and Any packages within the plurality of packages having a corresponding set of package dimensions that exceeds a threshold dimension similarity value for the first set of package dimensions determined from the image of the first package are excluded from the subset of possible matches.

17. The method according to claim 16, wherein: The package size analysis also includes: determining the first set of package dimensions from the image of the first package by projecting a set of three-dimensional package models onto the image of the first package to identify within the image of the first package a best fit between one of the package models and the first package; and The size of the one of the package models is applied as the first set of package sizes.

18. The method according to claim 1, wherein: Applying the one or more identification algorithms includes applying a distinguishing feature analysis, including: identifying a first set of one or more features on the first package from the image of the first package, the features comprising one or more of a mark, a scratch, a hole, or text; comparing the first set of one or more features to images of other packages in the plurality of packages; and Any package within the plurality of packages that does not have at least one of the one or more features detected on the first package is excluded from the subset of possible matches.

19. The method of claim 1, wherein applying the one or more identification algorithms comprises applying a shipping routing analysis, comprising: obtaining a location of a delivery facility within a delivery network wherein the first package is identified as having a damaged label; determining possible delivery methods for other packages in the plurality of packages; and Any package within the plurality of packages that does not include the location within the corresponding intended delivery method is excluded from the subset of possible matches.

20. The method according to claim 19, wherein: Determine the expected delivery method for other packages including: The other packages from the multiple packages are determined as packages associated with corresponding locations of a delivery facility within a delivery network, and the corresponding location of the delivery facility has a direct delivery connection with the location of the first package, or has a direct delivery connection with a single intermediate location between the corresponding location of packages from the other packages and the location of the first package.

21. A method according to claim 1, wherein the delivery network is a network graph including a plurality of nodes representing locations for delivering and picking up packages, wherein the network graph includes a plurality of directional edges representing connections between pairs of locations, wherein based on the network graph, a plurality of routes connecting at least some of the nodes are defined.

22. A system comprising: one or more computing devices; and One or more computer-readable storage devices coupled to the one or more computing devices and having instructions stored thereon that, when executed by the one or more computing devices, cause the one or more computing devices to perform operations comprising: obtaining first data of a first package within a delivery network, wherein the first package includes a label that is damaged and prevents automatic identification of the first package based on the label; Obtaining second data of a plurality of packages in the delivery network; identifying a particular package from the plurality of packages that matches the first package by applying one or more identification algorithms to the first data and the second data, wherein applying the one or more identification algorithms comprises obtaining a subset of possible matches from the plurality of packages as an output from each algorithm, the possible matches comprising packages identified by the one or more identification algorithms as having one or more characteristics similar to the first package; and Information from the label of the particular package is used to route the first package through the delivery network.

23. A computer-implemented method executed by one or more processors, the method comprising: obtaining first data of a first package within a delivery network, wherein the first package includes a label that is damaged and prevents automatic identification of the first package based on the label; obtaining second data of a plurality of packages within the delivery network; Identify a specific package from multiple packages by following these steps: applying a plurality of identification algorithms to the first data and the second data in order to obtain a subset of possible matches from the plurality of packages as an output from each algorithm, the possible matches comprising packages identified by the plurality of identification algorithms as having one or more characteristics similar to the first package; and identifying the particular package that matches the first package based on providing as input to a machine learning model an image of the first package and the subset of images of possible matches from the first data determined after applying a plurality of identification algorithms; and Information from the label of the particular package is used to route the first package through the delivery network.

Images