Parcel association method and device, electronic device, and storage medium

Abstract

Embodiments of the present application provide a package association method and device, electronic equipment and storage medium, belonging to the technical field of logistics sorting. The method comprises: obtaining a lost package; obtaining loss information of the lost package, determining a target temporary storage duration of the lost package according to the loss information; within the target temporary storage duration, performing a re-association operation on each frame of image collected by a camera until the lost package is successfully associated with a package; wherein the re-association operation comprises: predicting a search range of the lost package in an execution frame image performing the re-association operation according to the loss information, performing package detection on the execution frame image based on the search range and determining candidate associated packages of the lost package, and calculating a matching score between each candidate associated package and the lost package; according to the matching score, determining a target associated package of the lost package among the candidate associated packages. The embodiments of the present application can perform re-association on the lost package.

Description

Technical Field

[0001] This application relates to the field of logistics sorting technology, and in particular to a parcel association method and apparatus, electronic equipment and storage medium. Background Technology

[0002] In automated logistics sorting systems, parcels are scanned by mainline scanning equipment to collect label information before falling into the corresponding sorting chute. To achieve fully automated and intelligent management, the parcels must maintain an accurate association with the label information after entering the chute. This allows backend personnel or automated equipment to identify the identity information of stacked parcels without repeated scanning, effectively improving sorting efficiency and reducing logistics error rates caused by misidentification or mispacking.

[0003] Currently, related technologies typically employ the deployment of multiple 2D or 3D vision cameras along the chute to achieve end-to-end identity binding of packages from the main line and chute to the final cage loading stage through real-time multi-view tracking. This process mainly includes two core components: continuous tracking within a single camera's field of view and identity relay across camera fields of view. In the single-camera continuous tracking phase, packages often fail to track due to severe occlusion or temporary removal from the camera's field of view (i.e., target loss). However, related technologies lack a mechanism for re-associating lost packages. Summary of the Invention

[0004] The main objective of this application is to propose a package association method, apparatus, electronic device, and storage medium that can re-associate lost packages.

[0005] To achieve the above objectives, a first aspect of this application proposes a package association method, which is applied to package tracking within a single camera. The method includes: Retrieve lost packages, wherein the lost packages are target packages that have not been successfully associated, as determined based on the current frame image and / or previous multiple frame images captured by the camera; For each lost package, the loss information of the lost package is obtained, and the target temporary storage time of the lost package is determined based on the loss information; wherein, the loss information includes the status information of the lost package in the last frame before it was lost; During the target storage period, a reassociation operation is performed sequentially on each frame of image captured by the camera until the lost package is successfully associated with the package. The reassociation operation includes: Based on the loss information, predict the search range of the lost package in the execution frame image where the reassociation operation is performed. Based on the search range, perform package detection on the execution frame image and determine the candidate associated packages of the lost package. Calculate the matching score between each candidate associated package and the lost package. Based on the matching score, the target associated package of the lost package is determined among the candidate associated packages, wherein the matching score of the target associated package is higher than a preset matching threshold and not lower than the matching score of other candidate associated packages.

[0006] To achieve the above objectives, a second aspect of this application provides a package association device for package tracking within a single camera, the device comprising: A lost package acquisition unit is used to acquire lost packages, wherein the lost package is a target package that has not been successfully associated, determined based on the current frame image and / or previous multiple frame images captured by the camera. The temporary storage duration determination unit is used to obtain the loss information of each lost package and determine the target temporary storage duration of the lost package based on the loss information; wherein, the loss information includes the status information of the lost package in the last frame before it was lost; The reassociation unit is used to sequentially perform a reassociation operation on each frame of image captured by the camera during the target temporary storage period until the lost package is successfully associated with the package. The reassociation operations include: Based on the loss information, predict the search range of the lost package in the execution frame image where the reassociation operation is performed. Based on the search range, perform package detection on the execution frame image and determine the candidate associated packages of the lost package. Calculate the matching score between each candidate associated package and the lost package. Based on the matching score, the target associated package of the lost package is determined among the candidate associated packages, wherein the matching score of the target associated package is higher than a preset matching threshold and not lower than the matching score of other candidate associated packages.

[0007] To achieve the above objectives, a third aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in any one of the embodiments of the first aspect.

[0008] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the embodiments of the first aspect.

[0009] The package association method, apparatus, electronic device, and storage medium proposed in this application first acquire the lost package, then determine the target temporary storage time based on the loss information of the lost package, and finally, within the target temporary storage time, perform a re-association operation on the lost package based on each frame of image captured by the camera. Specifically, the re-association operation is implemented through the following reasoning steps: first, candidate associated packages for the lost package are determined within the search range; second, the matching score between the candidate associated packages and the lost package is calculated; finally, the candidate associated package with the highest matching score is selected, and if its matching score is greater than a preset matching threshold, it is determined as the target associated package for the lost package. Therefore, this application embodiment achieves the re-association of lost packages by constructing a reasoning link of "candidate screening - matching score calculation - threshold decision". Attached Figure Description

[0010] Figure 1 This is a flowchart of one embodiment of the package association method provided in this application; Figure 2 This is a flowchart of an embodiment of determining the target temporary storage duration provided in this application; Figure 3 This is a flowchart illustrating an embodiment of determining the search range provided in this application. Figure 4 This is a schematic diagram of the package association device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] In automated logistics sorting systems, parcels are scanned by mainline scanning equipment to collect label information before falling into the corresponding sorting chute. To achieve fully automated and intelligent management, the parcels must maintain an accurate association with the label information after entering the chute. This allows backend personnel or automated equipment to identify the identity information of stacked parcels without repeated scanning, effectively improving sorting efficiency and reducing logistics error rates caused by misidentification or mispacking.

[0013] Currently, related technologies typically employ the deployment of multiple 2D or 3D vision cameras along the chute to achieve end-to-end identity binding of packages from the main line and chute to the final cage loading stage through real-time multi-view tracking. This process mainly includes two core components: continuous tracking within a single camera's field of view and identity relay across camera fields of view. In the single-camera continuous tracking stage, although package tracking can be achieved through multi-feature fusion matching, this method still has the following shortcomings: 1. Lack of a systematic management mechanism after target loss: When the package is severely obscured or temporarily out of sight, the current multi-feature fusion matching method may fail, and there is no clear state transition and processing flow after failure, which may lead to the target ID (such as tracking identifier) ​​being terminated or recreated incorrectly.

[0014] 2. Inaccurate identification when lost packages reappear: When a package reappears after disappearing due to obstruction, it's impossible to determine if it's the same package, lacking motion prediction and search mechanisms to assist in re-identification. Furthermore, when packages of the same type have high visual similarity, relying solely on appearance features can easily lead to misidentification.

[0015] 3. The impact of similar package density on association is not considered: In high-density similar package scenarios, the probability of false association increases significantly; there is a lack of a mechanism to perceive the number of packages within the search range; and no density-adaptive matching threshold adjustment strategy has been established.

[0016] 4. Lack of accuracy verification after association: Identity is confirmed immediately after re-association, lacking a fault tolerance mechanism. Successful matching of a single frame does not guarantee correct long-term association; it cannot handle associations that are correct in the short term but incorrect in the long term.

[0017] Based on this, embodiments of this application provide a package association method and apparatus, electronic device and storage medium, which can re-associate lost packages.

[0018] The package association method provided in this application relates to the field of logistics sorting technology. The package association method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, barcode scanner, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application implementing the package association method, but is not limited to the above forms.

[0019] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0020] Please see Figure 1 , Figure 1 This is an optional flowchart of the package association method provided in the embodiments of this application. In some embodiments of this application, the package association method is applied to package tracking within a single camera. Figure 1 The specific methods may include, but are not limited to, steps S101 to S103.

[0021] Step S101: Obtain the lost package, wherein the lost package is a target package that has not been successfully associated, determined based on the current frame image and / or previous multiple frame images captured by the camera. Step S102: For each lost package, obtain the loss information of the lost package and determine the target temporary storage time of the lost package based on the loss information; wherein, the loss information includes the status information of the lost package in the last frame before it was lost; Step S103: During the target temporary storage period, perform the reassociation operation on each frame of image captured by the camera in sequence until the lost package is successfully associated with the package. The reassociation operations include: Based on the loss information, predict the search range of the lost package in the execution frame image when performing the re-association operation. Based on the search range, perform package detection on the execution frame image and determine the candidate associated packages of the lost package. Calculate the matching score between each candidate associated package and the lost package. Based on the matching score, the target associated package of the lost package is determined from each candidate associated package. The matching score of the target associated package is higher than the preset matching threshold and not lower than the matching scores of other candidate associated packages.

[0022] In this embodiment of the application, a package tracking state machine is constructed. As shown in Table 1 below, the package tracking state machine can include the following six states: Table 1. Example of Package Tracking Device Status

[0023] Referring to Table 1, the package tracking state machine constructed in this embodiment of the application achieves global state management of the package from its first appearance to its final departure from sight by defining six discrete and mutually exclusive tracking states. The six states include: newly registered state, active tracking state, temporarily lost state, re-associated state, verification passed state, and permanently lost state.

[0024] Furthermore, as shown in Table 2 below, the embodiments of this application define the transition paths and triggering conditions between the above six states.

[0025] Table 2 Example Table of State Transition Conditions

[0026] Based on this, the registration status will be explained first. In some embodiments, before step S101, the method provided in this application may further include the following steps: Perform package detection on the current frame image captured by the camera to identify the packages to be registered in the current frame image; If the location of the package to be registered in the current frame image is within the boundary of the camera detection area, and the package to be registered does not match the already registered package target, the package to be registered will be registered as a new package target.

[0027] In this embodiment, cameras deployed at predetermined positions along the sorting chute first acquire the current frame image in real time. The current frame image refers to the most recently acquired frame of image data in the time series, used for comparison and analysis with the previous frame or historical frame images to achieve the detection and tracking of moving targets. The current frame image includes, but is not limited to, RGB images, depth images, or RGB-D images. Based on this, a package detection operation is performed on the current frame image using a pre-trained target detection model. The package detection operation aims to identify all visual instances that may belong to packages in the current frame image and output the detection result for each instance. The detection result includes at least the position and size information of the target detection box in the image coordinate system, as well as the corresponding detection confidence score. The target detection box is usually represented in the form of a rectangle, and its position information includes the pixel coordinates of the upper left and lower right corners of the rectangle, or the center point coordinates combined with the width and height. The detection confidence score is used to characterize the probability that the detection result belongs to a real package, and the value range is usually from 0 to 1, with a higher value indicating a more reliable detection result. In one feasible implementation, the object detection model can be constructed using a deep learning-based object detection algorithm, such as the YOLO series algorithms. After obtaining all detection results in the current frame image through the object detection model, for each detection result, its corresponding image region is considered as a potential package to be registered. It is understood that when a package first enters the camera's field of view, it has not yet been assigned a globally unique tracking identifier (TrackID), and an initial profile needs to be established for it through a subsequent registration process. Through the above method, this embodiment of the application can achieve real-time perception and preliminary localization of packages within the chute, laying a data foundation for subsequent identity registration and continuous tracking.

[0028] For each identified package to be registered, its target detection bounding box information in the current frame image can be further extracted. The center or bottom center coordinates of the target detection bounding box are then transformed from the image coordinate system to the world coordinate system based on the camera's intrinsic and extrinsic parameters to obtain the package's position coordinates in the actual physical space. Since the cameras on the sorting chute are typically deployed on fixed supports, their intrinsic and extrinsic parameters are predetermined through calibration during system installation; therefore, the above coordinate transformation relationship is deterministic and reusable. After obtaining the position coordinates of the package to be registered, it can be determined whether these coordinates fall within the boundary range of the camera's detection area. The boundary range of the camera's detection area refers to a pre-defined spatial region boundary within the camera's effective field of view, used to define which locations belong to the effective tracking range of the current camera. Detection results outside this range are ignored or handled by other cameras. In other words, the boundary range serves the purpose that a package is considered valuable for tracking only when it appears within this boundary range; if the package has moved to the edge of the detection area or even completely left the area, it may be about to enter the field of view of the next camera or has already left the chute, and is no longer suitable as a registration target for the current camera. Introducing this boundary range determination during the registration phase can effectively avoid false registrations caused by imaging distortion or partial truncation at the edge of the field of view. It also prevents targets already registered on other cameras from being duplicated in the current camera. Specifically, the boundary range determination process involves comparing the position coordinates of the package to be registered with the geometric constraints of the boundary range. If the constraints are met, the package is determined to be within the boundary range of the camera's detection area; otherwise, the package is ignored, and the registration process is not triggered.

[0029] After determining that the package to be registered is within the boundary of the camera detection area, it is also necessary to verify whether the package to be registered matches any registered package targets. Registered package targets refer to packages that have completed registration in the tracking system and have been assigned a globally unique tracking identity (TrackID). Their identity information is stored in the system's database and can be accessed by subsequent matching and verification operations. The purpose of this verification operation is to prevent the same physical package from being registered repeatedly in a short period of time, thereby avoiding identity conflicts and resource waste. Specifically, the appearance features and / or location information of the package to be registered can be compared with all registered package targets. If the location distance between any registered package target and the package to be registered is less than a preset distance threshold, and / or the appearance similarity is higher than a preset similarity threshold, then the package to be registered is determined to match the registered target, and the registration operation is not performed. Conversely, if no registered package target meets the above matching conditions with the package to be registered, then the package to be registered is determined to not match any of the registered package targets. If a package to be registered is determined to be within the boundary but does not match any already registered package targets, it can be registered as a new package target, meaning a globally unique tracking identifier is assigned to it. Understandably, tracking identifiers are typically in incrementing integer form to ensure that each package target has a unique and identifiable TrackID within the entire tracking system. After registration, as shown in Table 3, information about the newly registered package target can be collected and stored in the database as a basis for subsequent tracking and matching.

[0030] Table 3. Example of Registration and Information Collection

[0031] Once the registration process is complete, the newly registered package will enter the new registration state and, according to the state transition rules shown in Table 2, will be transferred to the active tracking state (i.e., New→Active) when the conditions are met, after which continuous tracking can be performed.

[0032] By setting the above registration conditions, the system can quickly respond to newly entered packages while effectively suppressing duplicate and erroneous registrations, thus improving the robustness and resource utilization efficiency of the tracking system.

[0033] In step S101 of some embodiments, lost packages are first identified. A lost package can refer to a target package that, based on the current frame image and / or previous multiple frames captured by the camera, has been registered in the tracking system but has not yet been successfully associated with any detection result. Specifically, during package tracking, the system performs a matching operation between the detection result in each frame image and the registered package targets. This matching operation can employ a multi-feature fusion matching method, that is, extracting multi-dimensional features (including but not limited to motion features and appearance features) of the package indicated by the detection result, comparing them with the corresponding features of each registered package target, calculating the comprehensive matching score between the two, and determining the matching pair with the highest score as the association relationship.

[0034] A registered package target (such as a package target with TrackID=100) is considered lost if it meets any of the following conditions. In this case, the package target's status is transitioned from active tracking to temporarily lost (Active → Lost), the package target is marked as lost, and stored in a temporary list, awaiting subsequent re-association attempts: Condition 1: Consecutive Frame Count Determination: If a registered package target cannot establish a valid association with any detection result in multiple consecutive frames, that is... .in, This indicates the number of consecutive frames in which the target package has not been matched. The threshold value can be set to any integer value between 3 and 5 frames, without limitation. This judgment condition is based on the following criteria: During normal package tracking, since the camera continuously acquires images at a high frame rate, the same package will not disappear into thin air between adjacent frames. If a match cannot be found in multiple consecutive frames, it is highly likely that the package has been severely obscured or has left the current camera's field of view.

[0035] Condition 2: Detection Confidence Determination: Although a detection result matching the package target exists in a certain frame (such as the current frame), the detection confidence of this result is lower than the preset confidence threshold, i.e. .in, This represents the detection confidence score of the target detection model outputting the detection box corresponding to the current detection result. This confidence score is used to characterize the probability that the detection result belongs to a real package. The value range is usually from 0 to 1. The higher the value, the more reliable the detection result. This is the confidence threshold (e.g., set to 0.5, with no specific limit). This judgment condition is based on the following criteria: When the detection confidence is low, it indicates that the target detection model is not confident enough in determining the presence of a package in the area. The detection result is likely a false detection caused by noise, shadows, or partial occlusion. If a low-confidence detection result is forcibly associated with a package target in this case, it may lead to incorrect matching. Therefore, it is more prudent to consider such situations as target loss.

[0036] Condition 3: Occlusion Area Determination: The registered package target is severely occluded in multiple consecutive frames, i.e. .in, This indicates the occlusion ratio of the registered package target in the current frame, specifically defined as the ratio of the overlap area between the target detection box of this package and the target detection boxes of adjacent packages to the total area of ​​the target detection box itself. The value ranges from 0 to 1. A value of 0 indicates no obstruction, while a value of 1 indicates complete obstruction. This indicates the number of consecutive frames during which the package target meets the occlusion ratio threshold. The preset occlusion ratio threshold, This is a preset threshold for the number of frames of continuous occlusion. This judgment condition is set based on the following criteria: When a package is severely occluded by adjacent packages, although the target detection model may still output a detection result, the appearance features corresponding to the detection result are severely incomplete. If matching is continued based on these incomplete features, it is very easy to lead to false associations. Therefore, after the occlusion exceeds a certain time, the target is actively switched to a lost state, and then retrieved by motion feature prediction and subsequent re-association mechanism, which can achieve a more robust tracking effect.

[0037] It is understandable that a lost package is not the same as a package that has physically left the field of view. A lost package may be temporarily unrecognizable by the detection model due to reasons such as being obscured by other packages, moving too fast causing image blur, or being at the edge of the camera's field of view. Therefore, the system will not immediately release the tracking resources it occupies, but will keep it in a temporary list with a certain waiting time, hoping that the association can be restored when the lost package reappears in a subsequent frame. In this way, the embodiments of this application can promptly identify the target of the lost package and provide a clear target object for subsequent re-association operations.

[0038] For lost packages that have entered the Lost state, the loss context information shown in Table 4 below can be collected and recorded for use in subsequent reassociation operations.

[0039] Table 4 Example of Lost Context Information

[0040] In step S102 of some embodiments, for each lost package obtained based on step S101 (such as each package target recorded in the temporary storage list), the loss information of the lost package can be obtained, and the target temporary storage duration of the lost package can be calculated based on the loss information. The loss information includes the state information of the lost package in the last frame before it was lost. The state information can refer to the motion state data of the lost package at the moment of the last frame before it was lost, such as the position information and speed information of the lost package, specifically including the information shown in Table 4. The target temporary storage duration can refer to the maximum waiting time set for the lost package. Within this time window, the system will continuously attempt to re-associate the lost package. If the lost package is not found after the time window expires (i.e., the association and matching are still unsuccessful), the system will release the tracking resources it occupies and transfer the target state to a permanently lost state (i.e., Lost → Deleted). Releasing tracking resources can mean that the system removes the lost package from the temporary storage list, clears the memory space it occupies (including identity identifiers, appearance features, motion features, etc.), and stops performing subsequent matching calculations and re-association attempts for the target. The tracking identity identifier of the target then becomes invalid.

[0041] Understandably, excessively long storage times increase computational overhead and the risk of false associations, while excessively short storage times may lead to premature deletion of the target. Therefore, in this embodiment, the target storage time is not a fixed value, but is dynamically determined adaptively based on the lost package's loss information (such as motion state and location information). Specifically, the dynamic determination principle can be: the faster the speed, the shorter the storage time (fast targets may have already left the camera's field of view); the closer the location is to the camera detection boundary, the shorter the storage time (boundary targets may have already left the camera's field of view).

[0042] Reference Figure 2 In some embodiments, the loss information includes the status information of the lost package in the last frame before it was lost, and the status information includes the location information and speed information of the lost package. Step S102, "determining the target temporary storage duration of the lost package based on the loss information," may include, but is not limited to, steps S201 to S204.

[0043] Step S201: Calculate the speed influence factor based on the speed information and the preset speed threshold; Step S202: Determine the location of the lost package in the last frame before it was lost, and the boundary distance between it and the nearest boundary in the camera detection area, based on the location information; Step S203: Calculate the boundary influence factor based on the boundary distance and the area size of the detection region; Step S204: Calculate the target temporary storage time based on the boundary influence factor, velocity influence factor, and preset basic temporary storage time.

[0044] In step S201 of some embodiments, the velocity information of the lost package in the last frame before it was lost is first read from the loss information corresponding to the lost package (as shown in Table 4), and the velocity influence factor is calculated based on the velocity information and a preset velocity threshold. Here, the velocity information refers to the motion velocity vector of the lost package at the last frame before it was lost, which includes the velocity magnitude (i.e.,...). ) and velocity direction (i.e. Two components, typically calculated by the system based on the position differences of multiple consecutive frames prior to the loss. Velocity threshold (denoted as...) The speed threshold is a pre-defined benchmark value used to distinguish between the normal and rapid movement of the package. The value of the speed threshold can be set according to the actual application scenario and is not limited thereto. Specifically, the speed influence factor can be calculated using the following formula 1. .

[0045] (Formula 1) Speed ​​Influence Factor This parameter characterizes the degree to which speed affects the temporary storage time. Its core design logic is that the faster a package moves, the farther it travels within the same time window, and the more likely it is to have left the effective field of view of the current camera. Therefore, it should be allocated a shorter temporary storage time to avoid prolonged ineffective waiting. Calculated using Formula 1, when the package is stationary, the speed magnitude is 0, and the speed influence factor is 1.0, indicating that the speed factor does not reduce the temporary storage time, and the lost package can retain the full basic temporary storage time. When the package moves at a speed threshold, the speed magnitude equals the speed threshold, and the speed influence factor is 0.2, indicating that the speed factor reduces the temporary storage time to 20% of the basic value. When the package speed is between these two values, the speed influence factor changes linearly between 0.2 and 1.0. The speed influence factor... The lower limit of 0.2 serves to prevent the temporary storage time from becoming too short to zero during extremely high-speed movement, ensuring that there is at least a certain time window for lost packages that are moving rapidly but have not yet left the field of vision to complete re-association. Through the above method, the embodiments of this application realize the adaptive adjustment of the temporary storage time to the target's movement speed, so that high-speed targets have a shorter waiting time and low-speed targets have a longer waiting time, thereby allocating system resources more rationally.

[0046] In step S202 of some embodiments, the position information of the lost package in the last frame before it was lost can also be read from the loss information, and the boundary distance between the position of the lost package in the last frame before it was lost and the nearest boundary in the camera detection area can be determined based on the position information. The position information can refer to the spatial coordinates of the lost package in the world coordinate system at the time of the last frame before it was lost (i.e.,...). The camera detection area refers to the effective field of view coverage of a single 3D camera on a sorting chute. It corresponds to a fixed two-dimensional or three-dimensional spatial region in the world coordinate system, determined by the camera's field of view, mounting height, mounting angle, lens distortion parameters, and the physical structure of the chute. In practical applications, the camera detection area is usually defined as a polygonal region in the world coordinate system, with each edge of the polygon constituting the boundary of the detection area. Boundary distance refers to the geometric distance between the position of the lost package in the last frame before loss and the nearest boundary of the camera detection area. Boundary distance measures how close the target is to the edge of the field of view at the time of loss and is an important basis for determining whether the target may have left the field of view.

[0047] In step S203 of some embodiments, after determining the boundary distance, as shown in Formula 2 below, the boundary distance can be used to determine the boundary distance. and the area size of the detection area Calculate the boundary influence factor .

[0048] (Formula 2) Among them, the area size This is a geometric parameter characterizing the overall size of the camera's detection area, and its specific value depends on the shape and size of the detection area. For example, when the detection area is rectangular, the area size can be the length of the rectangle's diagonal or the average of the long and short sides. When the detection area is polygonal, the area size can be the diameter of the polygon's circumcircle or the maximum span on the area's plane. Boundary Influence Factor This parameter characterizes the influence of boundary distance on the temporary storage duration. Its core design logic is as follows: the closer the target's position is to the field of view boundary at the time of loss, the more likely it has already left or is about to leave the current camera's effective field of view; therefore, a shorter temporary storage duration should be allocated to it. Conversely, the farther the target is from the field of view boundary, the more likely it is within the field of view and was lost due to occlusion rather than leaving the field of view; therefore, a longer temporary storage duration should be allocated to it. Calculations using Formula 2 show that when the lost package is located near the geometric center of the detection area, the boundary distance is approximately half the area size (i.e., 0.5). The boundary influence factor is approximately 0.5, indicating that boundary factors have a certain reducing effect on the temporary storage time. When the lost package is located near the boundary of the detection area, such as when the boundary distance is 0.25... The boundary impact factor is set to a lower limit of 0.3, meaning that the temporary storage time will be significantly shortened due to boundary factors, but 30% of the base storage time will still be reserved to deal with situations where the target may briefly disappear near the boundary and then reappear. When the location of the lost package exceeds the boundary of the detection area, the boundary distance is 0, and the boundary impact factor is also set to 0.3. In this case, the system will not immediately delete the lost package, but will give it a final chance to be retrieved with the shortest possible storage time. By setting the boundary impact factor... The lower limit value (e.g., 0.3) ensures that even if the lost package is lost at the edge of the field of view, the system still retains a certain waiting window, avoiding premature deletion of the target due to single-frame positioning errors. Through the above method, this embodiment of the application achieves adaptive adjustment of the storage duration to the target position, allowing targets near the edge of the field of view to receive a shorter storage duration, and targets located within the field of view to receive a longer storage duration.

[0049] In step S204 of some embodiments, the boundary influence factor is determined. Speed ​​Influence Factor Then, the target temporary storage duration can be calculated according to the following formula 3. .

[0050] (Formula 3) in, This indicates the preset base storage time, a pre-defined time parameter representing the maximum storage time without any reduction factors. For example, it can refer to the waiting time given by the system when the target is completely stationary and located in the center of the field of view. The value of the base storage time can be pre-configured according to the physical characteristics of the actual sorting scenario, taking into account factors such as the total length of the chute, the distance between adjacent cameras, the average movement speed within the chute, and the real-time requirements of the system.

[0051] Under normal circumstances, once the target temporary storage duration is calculated and determined, it remains unchanged during the temporary storage period. The system manages the temporary storage based on this fixed duration until the temporary storage duration is exhausted or the lost package is successfully re-associated. However, in a feasible implementation, the target temporary storage duration can be adaptively extended based on the intermediate results of the re-association operation. The specific implementation method of this extension mechanism will be introduced later in conjunction with the relevant content of the re-association operation.

[0052] The embodiments of this application realize adaptive and personalized determination of the temporary storage duration, which significantly improves the rationality and effectiveness of the temporary storage mechanism under different motion states and position conditions.

[0053] In step S103 of some embodiments, after determining the target temporary storage duration, within the time window of the target temporary storage duration, a re-association operation is sequentially performed on each frame image captured by the camera until the lost package is successfully associated with the corresponding package. The re-association operation is a series of processing procedures performed on package targets in a temporarily lost state. Its purpose is to identify the detection results corresponding to the same physical package as the lost package in subsequent frames and restore their tracking identity binding relationship. Specifically, for each newly captured frame image by the camera (i.e., the execution frame image for which the re-association operation is performed), the search range of the lost package in the execution frame image is first predicted based on the loss information of the lost package. The search range is a spatial region used to limit the range of candidate associated packages to be searched in the execution frame image. Then, a package detection operation can be performed on the execution frame image, that is, the pre-trained target detection model is used to identify and locate all package instances in the execution frame image and output several detection results. For each detected detection result, it is determined whether its position falls within the determined search range. If it does, the package corresponding to the detection result is determined as a candidate associated package of the lost package. Understandably, when multiple detection results exist within the search range, these detection results are all candidate associated packages. Further, a matching score is calculated between each candidate associated package and the lost package. The matching score can be calculated using the multi-feature fusion matching method described in step S101 above, that is, a weighted fusion of motion feature similarity and appearance feature similarity. Motion feature similarity is calculated based on the Mahalanobis distance or Euclidean distance between the predicted position (determined by Kalman filtering) and the actual detection position, while appearance feature similarity is calculated based on the cosine similarity or feature distance between the RGB feature vector and the point cloud feature vector. This is the matching score. .in, For motion feature similarity, For similarity in appearance features, and This corresponds to the weight. After obtaining the matching scores of all candidate associated packages, the highest matching score can be selected and compared with a preset matching threshold. Matching threshold It is a pre-set judgment benchmark value used to determine whether two targets belong to the same physical package, and the matching threshold. The value can be set to a value between 0.5 and 0.7, with no limitation on the specific value. If the highest matching score is greater than the matching threshold, it is determined that the candidate associated package corresponding to the highest matching score has successfully matched the lost package, and the candidate associated package is identified as the target associated package of the lost package (e.g., the TrackID of the lost package is assigned to the candidate associated package), thereby completing the re-association operation. Then, the state of the lost package can be transitioned from the temporarily lost state to the re-association state (i.e., Lost → Recovering). If the highest matching score is less than or equal to the matching threshold, it is determined that no valid associated target was found in the execution frame, and the re-association attempt in the execution frame fails (at this time, the state changes to Recovering → Lost). The process continues to wait for the next frame and repeat the above re-association operation until the target storage time is exhausted or the association is successful. Through the above method, the embodiments of this application realize the systematic search and frame-by-frame re-association attempt for lost packages, effectively improving the success rate of identity retrieval when packages reappear after occlusion or rapid movement.

[0054] Reference Figure 3 In some embodiments, the loss information includes the state information of the lost package in the last frame before it was lost, and the state information includes the location information and velocity information of the lost package. The reassociation operation's "predicting the search range of the lost package in the execution frame image where the reassociation operation is performed based on the loss information" may include, but is not limited to, steps S301 to S303.

[0055] Step S301: Predict the predicted position and predicted state data of the lost package in the execution frame image based on the state information, wherein the predicted state data is used to characterize the uncertainty of the predicted position. Step S302: Based on the predicted status data, determine the uncertain area of ​​the lost package's status, and determine the target search radius based on the size data of the uncertain area; Step S303: Using the predicted location as the center and the target search radius as the radius, determine the search range for the lost package in the execution frame image.

[0056] In step S301 of some embodiments, the state information may refer to the motion state data of the lost package at the last frame before it was lost, specifically including the position and velocity information of the lost package. This state information can be used as the initial state input for current state prediction (such as position prediction). Specifically, the predicted position of the lost package in the execution frame image can be predicted using Kalman filtering or other prediction methods to obtain the predicted position and its predicted state data. Taking Kalman filtering as an example, the state information is used as the initial state input for Kalman filtering. Kalman filtering is a recursive state estimation algorithm based on a linear Gaussian state-space model. It iteratively solves for the optimal state estimate through two steps: prediction and update. In this embodiment, it is used to predict the current state of the lost package. In this embodiment, the Kalman-filtered state estimate corresponding to the last frame before the lost package is used as the initial state vector, i.e., the state vector. The state vector is defined as containing position information (i.e., x, y) and velocity information (i.e., ...). Then, the state vector is forward-propagated using the state transition matrix of the Kalman filter to predict the target's state at each execution frame. The state transition matrix has the following form:

[0057] The mathematical expression for the prediction step is: the predicted state vector of the execution frame is equal to the state transition matrix multiplied by the state estimation vector of the previous frame, i.e. During the prediction process, the predicted state covariance matrix (i.e., the predicted state data) can also be calculated using the covariance propagation formula. The predicted state covariance matrix is ​​a 4x4 square matrix, where the diagonal elements represent the prediction variance of each state component, and the off-diagonal elements represent the covariance between different state components. The overall size of the predicted state covariance matrix reflects the degree of uncertainty in the prediction result. Specifically, the predicted state covariance matrix can be determined according to the following formula 4.

[0058] (Formula 4) Here, Q represents the process noise covariance matrix, which is used to describe the uncertainty of the motion model itself.

[0059] Furthermore, since no new observation data is available to correct the prediction during the period of package loss, the longer the loss time, the greater the cumulative increase in prediction uncertainty. Therefore, in this embodiment, the prediction state covariance matrix can also incorporate the impact of the loss time. Specifically, the prediction state covariance matrix can be further determined according to the following formula 5: (Formula 5) Where n represents the number of frames lost.

[0060] Through the Kalman filter prediction described above, the predicted position of the lost packet in the execution frame and its predicted state covariance matrix can be obtained (i.e., This provides a quantitative basis for the reasonable determination of the subsequent search range. The predicted position refers to the estimated position of the target at the execution frame time obtained after forward propagating the historical state of the lost package using the state transition matrix of the Kalman filter. The predicted state covariance matrix is ​​the covariance matrix corresponding to the predicted state vector in the Kalman filter prediction step, used to quantify the uncertainty of each component of the predicted state and their correlation.

[0061] In step S302 of some embodiments, the state uncertainty region of the lost package is determined based on the predicted state data (such as the predicted state covariance matrix), and the target search radius is determined based on the size data of the state uncertainty region. The state uncertainty region refers to a region (such as an elliptical region) defined by eigenvalue decomposition of the position submatrix in the predicted state data (such as the predicted state covariance matrix), used to geometrically represent the probability distribution range of the target's true location. The size data can be the region size of the state uncertainty region, such as the major axis radius of the elliptical region. Specifically, the predicted state covariance matrix can be represented in the following form:

[0062] As shown in Formulas 6 to 8 below, the major axis and direction of the uncertain region can be obtained from the predicted state covariance matrix: (Formula 6) (Formula 7) (Formula 8) in, It represents the square of the major axis radius of the elliptical region. The major axis radius is the distance from the center of the ellipse to the boundary along the direction of maximum variance in the elliptical region, representing the degree of deviation in the direction of the greatest uncertainty of the target position. It represents the square of the minor axis radius of the elliptical region. The minor axis radius is the distance from the center of the ellipse to the boundary along the direction of minimum variance in the elliptical region, representing the degree of deviation in the direction of minimum uncertainty of the target position. This represents the orientation angle of the elliptical region. The orientation angle is used to represent the rotation angle of the major axis of the ellipse relative to the first direction of the world coordinate system.

[0063] Based on this, we can determine the size data (such as...) Determine the target search radius Target search radius It can refer to the radial distance centered on the predicted location, used to define the search range.

[0064] In some embodiments, step S302, "determining the target search radius based on the size data of the uncertain region," may include, but is not limited to, the following steps: Determine the basic search radius based on the size data of the region of uncertainty. The number of lost frames is determined based on the last frame and the execution frame before the lost package was lost, and the speed compensation item is determined based on the number of lost frames and speed information; The target search radius is obtained by summing the velocity compensation term and the basic search radius.

[0065] In this embodiment of the application, the basic search radius can be equal to Here, k represents the magnification factor, which is used to appropriately magnify the size data of the uncertain region to ensure that the search range can cover areas where the target's actual position falls outside the uncertain region but the probability is still within an acceptable range. The magnification factor k can be set between 2 and 3, without limitation.

[0066] Furthermore, the number of lost frames can be determined based on the last frame before the lost package was lost and the execution frame. The last frame before the lost package was identified can be the image frame corresponding to the last successful detection and matching of the target; the time of this frame serves as the starting point for the target entering the lost state. (Number of lost frames) It can refer to the number of image frames that pass between the time the package is lost and the time it enters the lost state (i.e., the last frame before it was lost) and the time the execution frame is reached; the number of lost frames. This reflects the length of time the target has been lost; a larger value indicates that the target has been moving without being observed, and the greater the accumulated positional shift. Thus, it can be used to determine the number of lost frames. The speed compensation term is calculated from the speed information. The velocity compensation term is an additional term used to compensate for the positional shift caused by the target's continued movement during the period of loss. Its physical meaning is that although the target is not observed during the period of loss, it does not stop moving but continues to move along the chute at a certain speed. Therefore, an offset equal to the velocity multiplied by time accumulates between the predicted position and the target's true position over time, and this offset needs to be compensated within the search range. The velocity compensation coefficient is... This is a preset constant, its function is to appropriately scale the cumulative velocity displacement to avoid over-compensation or under-compensation; the velocity compensation coefficient. The value can be set between 0.5 and 1.0, without limitation. As shown in Formula 9 below, the target search radius can be obtained by summing the velocity compensation term and the basic search radius. .

[0067] (Formula 9) In step S303 of some embodiments, the search range for the lost package can be determined in the execution frame image using the predicted position determined in step S301 as the center and the target search radius determined in step S302 as the radius. The predicted position is the estimated position of the target at the execution frame time, predicted by Kalman filtering or other prediction methods. It serves as the center point of the search range and represents the most likely spatial location of the target. The target search radius is the radial distance from the center point to the boundary of the search range, used to define the boundary of the search range.

[0068] Thus, when the lost package duration is short (e.g., 1-2 frames) and the speed is low, the prediction uncertainty is small, the cumulative speed offset is small, and the target search radius is correspondingly small. The search range is precisely limited to a small area near the predicted location, effectively eliminating interference from irrelevant packages at a distance. When the lost time is short but the target speed is high, the speed compensation term increases the target search radius to a moderate level to cover the significant displacement of fast targets in a short period. When the lost time is long (e.g., greater than or equal to 5 frames) and the target speed is low, the amplification term increases the target search radius to a moderate level to reflect the accumulation of uncertainty caused by a long period without observation updates. When the lost time is long and the target speed is high, the amplification term and the speed compensation term work together, and the target search radius reaches its maximum value, significantly expanding the search range to ensure that no high-speed, long-distance moving targets are missed.

[0069] In the reassociation operation, only detection results falling within the search range are subjected to candidate association filtering and matching score calculation; detection results outside the search range are directly filtered out and not considered. In this way, by constraining spatial location, the number of candidate targets that need to be matched with the lost package is significantly reduced, thereby reducing computational complexity and improving the real-time performance of the reassociation operation. Simultaneously, the target search radius, adaptively determined based on the size data of the uncertain region and velocity compensation, allows the search range to be dynamically adjusted according to prediction uncertainty and the duration of loss. Through the above methods, the embodiments of this application achieve adaptive determination of the search range, providing accurate and efficient spatial constraints for the reassociation operation.

[0070] In some embodiments, the matching threshold can also be dynamically adjusted. Specifically, before "determining the target associated package of the lost package among the candidate associated packages based on the matching score", the method provided in this application embodiment may include adjusting the matching threshold, which may include, but is not limited to, the following steps: Obtain the first appearance feature corresponding to the lost package, and obtain the second appearance feature of the candidate associated package in the execution frame image; The similarity between the first appearance feature and the second appearance feature is calculated to determine the number of packages whose similarity is greater than a preset appearance similarity threshold. The similarity package density is calculated based on the number of packages and the area of ​​the search range, and the matching threshold is adjusted according to the similarity package density.

[0071] In this embodiment, the first appearance feature corresponding to the lost package is first obtained, and the second appearance features of all candidate associated packages falling within the search range in the execution frame image are then obtained. The first appearance feature is a pre-stored numerical feature vector characterizing the visual appearance of the lost package. It has been extracted from historical images and stored in the database by a feature extraction network during the lost package registration or active tracking phase. The second appearance feature can refer to a numerical feature vector extracted in real time for each candidate associated package in the execution frame image. It is obtained through the same feature extraction network and the same extraction parameters as the first appearance feature to ensure comparability between the two. For example, both the first and second appearance features are RGB appearance feature vectors extracted by a ReID network. This feature vector can effectively characterize the visual information of the package in terms of color, texture, pattern, etc. Furthermore, the appearance features may also include point cloud morphological features, or a fusion vector of RGB features and point cloud features.

[0072] The similarity between the first appearance feature and the second appearance feature corresponding to each candidate associated package is calculated, and the number of packages with a similarity greater than a preset appearance similarity threshold is counted. The similarity calculation is a numerical operation that quantifies the degree of similarity between two appearance feature vectors; cosine similarity, Euclidean distance, or other distance metrics can be used, without limitation. The appearance similarity threshold is a preset judgment benchmark value used to binarize the similarity calculation result into "appearance similar" or "appearance dissimilar". The appearance similarity threshold can be set to a relatively high value, such as 0.6 or 0.7, to ensure that only packages with truly similar appearances are included in the statistics. After calculating the appearance similarity between each candidate associated package and the lost package in the execution frame, the similarity value can be compared with the appearance similarity threshold one by one. If the similarity is greater than the appearance similarity threshold, the candidate associated package is judged as a package with a similar appearance to the lost package and counted. If the similarity is less than or equal to the appearance similarity threshold, it is not included in the statistics. The final counted number is taken as the package count (denoted as ). The number of packages refers to the total number of candidate associated packages within the search range of the lost package that have an appearance similarity exceeding the appearance similarity threshold.

[0073] As shown in Formula 10 below, when determining the number of packages... Then, based on the quantity of the package and the area of ​​the search range Calculate the similar package density .

[0074] (Formula 10) As can be seen from Formula 10, the density of similar packages It is a quantitative indicator that represents the density of similar-looking packages within the search range. Its physical meaning is to reflect the competitive environment of the lost package in the execution frame: the higher the density, the more packages with similar appearances exist around the lost package, the greater the selection ambiguity faced by the system during matching, and the higher the risk of incorrect identity binding due to misassociation.

[0075] Thus, in obtaining similar package density Then, based on the similar package density Matching threshold Adjustments can be made. The adjustment principles can be: the higher the density of similar packages, the greater the risk of false association, and the matching threshold should be increased accordingly to filter out candidate packages that look similar but have different identities; the lower the density of similar packages, the smaller the risk of false association, and the matching threshold can remain at its original level or only slightly increased. In a feasible implementation, adjustments can be made according to the following formula 11.

[0076] (Formula 11) in, This indicates the adjusted matching threshold. This represents the density sensitivity coefficient, used to control the sensitivity of the matching threshold to density changes. Its value can be set between 0.5 and 1.0, without limitation.

[0077] In another feasible implementation, the matching threshold can be adjusted using a piecewise function, i.e., multiple density intervals are pre-defined, each interval corresponding to a different threshold adjustment range. For example, when the density of similar packages is in the low interval ( When the risk of false association is low, the adjusted matching threshold remains the same as the original matching threshold (i.e., ...). No adjustment is made. When the density of similar packages is in the middle range ( When this occurs, there is a certain risk of false association. The adjusted matching threshold is increased to 1.25 times the original matching threshold (i.e., The matching criteria are slightly tightened. When the density of similar packages is in the high range ( When the threshold is adjusted, the risk of false association increases significantly, and the adjusted matching threshold is increased to 1.5 times the original matching threshold (i.e., ...). The matching criteria are significantly tightened when the density of similar packages is in the extremely high range ( When the threshold is set to 1, the risk of false association is extremely high, and the adjusted matching threshold is increased to twice the original matching threshold (i.e., ...). The matching conditions are the most stringent to suppress erroneous associations to the greatest extent.

[0078] By adaptively adjusting the matching threshold based on the density of similar packages as described above, the embodiments of this application can effectively suppress the risk of false association in high-density similar package scenarios and maintain reasonable association sensitivity in low-density scenarios, thus achieving optimal adaptation between the matching judgment criteria and the complexity of the scenario.

[0079] In some embodiments, the reassociation operation, which involves "determining the target associated package of the lost package from among the candidate associated packages based on the matching score," may include, but is not limited to, the following steps: Based on the matching score, the initial associated package of the lost package is determined from each candidate associated package. The matching score of the initial associated package is higher than the preset matching threshold and not lower than the matching score of other candidate associated packages. Perform continuous multi-frame matching verification between the initially associated packages and the lost packages. The matching verification includes appearance consistency verification and / or motion consistency verification. Determine the number of frames that successfully match and verify. If the number of successful frames is greater than a preset frame threshold, initially identify the associated package as the target associated package for the lost package.

[0080] In this embodiment of the application, during the re-association operation, the relationship between the highest matching score among the candidate associated packages and the preset matching threshold (which can be the original matching threshold or the matching threshold dynamically adjusted according to the aforementioned embodiment) is first determined. If the highest matching score is greater than the matching threshold, the candidate associated package corresponding to the highest matching score is determined as the initial associated package of the lost package (at this time, the state changes to Lost → Recovering).

[0081] The system performs continuous multi-frame matching verification between the initially associated package and the lost package. This verification can include appearance consistency verification and / or motion consistency verification. "Continuous multi-frames" refers to several image frames following the initial identification of the associated package. These frames are sequentially arranged in time, and the system continuously verifies the stability of the matching relationship between the initially associated package and the lost package across these frames. The purpose of the matching verification is to eliminate randomness and noise interference that may exist in single-frame matching through multi-frame observation. Appearance consistency verification verifies whether the appearance features of the initially associated package remain consistent with those of the lost package in subsequent frames. As shown in Formula 12, the specific implementation of appearance consistency verification is as follows: In each subsequent frame (e.g., K frames in total), appearance features are extracted from the package target corresponding to the same tracking identity as the initially associated package in that frame. The similarity between these appearance features and the first appearance feature of the lost package is calculated to obtain the RGB similarity of each frame. Indicates the first (The RGB similarity of the frames), and then calculate the average similarity of these frames. .

[0082] (Formula 12) If the average value is greater than the preset appearance inspection threshold (i.e.) ,right If the value of is not limited, then the appearance consistency check is considered passed. The basis for the appearance consistency check is that although the appearance features of the same physical package may fluctuate between adjacent frames due to changes in viewing angle or lighting, the relative position of its feature vector in the feature space should be continuous and stable, without sudden changes. Therefore, the authenticity of the matching relationship can be effectively verified by observing the appearance feature consistency of multiple consecutive frames.

[0083] Motion consistency verification is used to confirm whether the motion trajectory of the initially associated package in subsequent frames conforms to kinematic laws. Specifically, the motion consistency verification is implemented by tracking the position of the initially associated package in subsequent frames, calculating the displacement between adjacent frames, and then calculating the acceleration. If the acceleration between frames does not exceed a preset acceleration threshold, meaning the target does not exhibit physically impossible motion patterns such as jumping or backwards, the motion consistency verification is considered successful. The basis for motion consistency verification is that the movement of the package in the chute is constrained by physical laws such as gravity and friction. Its acceleration is limited by the chute slope and the package's mass, and cannot experience large sudden changes in a short period. By checking the consistency of acceleration, erroneous associations caused by detection noise can be effectively eliminated.

[0084] The system counts the number of successfully matched frames. If the number of successful frames exceeds a preset frame threshold, the initially associated package is identified as the target associated package for the lost package. The number of successfully matched frames refers to the cumulative number of frames that meet the verification conditions (i.e., passing appearance consistency verification and / or motion consistency verification) during multiple consecutive frame matching verifications. This number reflects the sustained stability of the matching relationship over time. The preset frame threshold is a pre-defined positive integer that specifies how many frames of successful verification are required for a matching relationship to be considered a reliable association; the value of the frame threshold is not limited. When the number of successful frames exceeds the preset frame threshold, it is determined that the matching relationship between the lost package and the initially associated package has undergone sufficient verification time and has a high degree of confidence. At this point, the initially associated package is officially identified as the target associated package for the lost package (the state changes to Recovering → Confirmed). After identification, the target associated package inherits the globally unique tracking identifier of the lost package and resumes active tracking (i.e., the state changes to Confirmed → Active). When the target associated package leaves the field of view (e.g., moves out of the chute), the state changes to Active → Deleted.

[0085] Through the above method, the embodiments of this application realize multi-frame reliability verification of single-frame matching results. The matching relationship only takes effect after continuous confirmation of multiple consecutive frames, effectively avoiding erroneous associations caused by single-frame noise, brief occlusion, or accidental appearance similarity.

[0086] Understandably, if the target is lost again after the aforementioned multi-frame verification, the state changes to Confirmed → Lost. If the target leaves the camera's field of view after multi-frame verification, the state changes to Confirmed → Deleted. Furthermore, after multiple consecutive successful verifications, the target's appearance features can be selectively updated. This involves fusing the new appearance features corresponding to the detection results that successfully match the target in the execution frame with the temporarily stored historical appearance features according to preset weights. This ensures that the updated appearance features reflect the target's latest appearance changes while maintaining the continuity of historical information, thus adapting to the slow drift in appearance caused by changes in lighting or viewing angle during movement. Additionally, after the verification process is complete, three target lists can be output: targets that have passed continuous K-frame verification and whose state changes from Recovering to Confirmed are included in the Confirmed target list; targets still in the verification process and not yet meeting the K-frame verification pass conditions are retained in the Continue Recovering target list; and targets that failed verification are returned to the Lost target list to await subsequent re-association attempts.

[0087] In some embodiments, the reassociation operation may also include, but is not limited to, the following steps: If a candidate associated package that meets the preset extension conditions is detected within the target temporary storage period, the target temporary storage period of the lost package is extended. The extension conditions include: the matching score between the candidate associated package and the lost package is less than or equal to the matching threshold, and the matching score between the candidate associated package and the lost package is greater than the preset extension trigger threshold, wherein the extension trigger threshold is less than the matching threshold.

[0088] As described in the foregoing embodiments, the target temporary storage time is the maximum time allowed for each lost package to remain in the temporary storage list and wait for re-association, allocated independently. This timeframe becomes the effective waiting window for the lost package once initially determined. However, in actual tracking scenarios, a situation may arise where the system detects a candidate associated package whose matching score with the lost package fails to reach the matching threshold, preventing successful re-association. However, this score is already quite close to the matching threshold, indicating a high similarity between the candidate associated package and the lost package. In this case, if the original target temporary storage time is strictly followed, the lost package may be deleted just before successful retrieval due to the expiration of the temporary storage time, resulting in the permanent loss of a potentially recoverable target. To address this technical problem, this application introduces a temporary storage time extension mechanism, allowing for a moderate extension of the target temporary storage time under certain conditions. Specifically, the results of the re-association operation are continuously monitored within the target temporary storage time. When a candidate associated package that meets the preset extension conditions is detected, the temporary storage time extension operation is triggered. The extension conditions can include: the matching score between the candidate associated package and the lost package is less than or equal to the matching threshold, and the matching score between the candidate associated package and the lost package is greater than a preset extension trigger threshold. The extension trigger threshold is another pre-set judgment benchmark value, which is less than the matching threshold. It is used to define which candidate associated packages that "failed to match but have a high similarity" can trigger an extension of the temporary storage time. In other words, the purpose of setting the extension trigger threshold is to filter out candidate associated packages whose matching scores, although not reaching the matching threshold, are quite close to it. These candidate packages indicate that the lost package may be in the process of being gradually recovered, and the system should allow more waiting time. For example, if the matching threshold is set to 0.7, the extension trigger threshold can be set to 0.5. In this case, when the matching score between a candidate associated package and the lost package is 0.65, since 0.65 is less than 0.7 (not reaching the matching threshold) and 0.65 is greater than 0.5 (exceeding the extension trigger threshold), the candidate associated package meets the preset extension conditions, and the system will trigger the temporary storage time extension operation. When the matching score is 0.45, since 0.45 is less than 0.5, the similarity of the candidate associated package is too low to indicate that the target is about to be found, and the system does not trigger the extension.

[0089] Regarding the determination of the extension range, in one feasible implementation, the extension range of the target temporary storage time can be a fixed value, such as increasing by 1 second or increasing the base temporary storage time by 10% each time an extension is triggered. In another feasible implementation, the extension range can also be related to the matching score. For example, the closer the matching score is to the matching threshold, the larger the extension range, because a higher matching score indicates that the target is closer to being retrieved, and it is worth giving more waiting time. It is understood that, in order to avoid the lost package remaining in the temporary storage list indefinitely due to frequent extensions, a total upper limit can be set for the extension of the temporary storage time, that is, the total length of the target temporary storage time shall not exceed a certain preset maximum temporary storage time. When this upper limit is reached, even if the extension conditions are met, no further extensions will be made, thereby achieving a balance between retrieval opportunities and resource consumption. And / or, a limit can be set for the number of extension operations, that is, each lost package can be extended a maximum of a predetermined number of times, such as a maximum of 3 times. After exceeding this number, even if the extension conditions are met, no further extensions will be made. Through the above methods, the embodiments of this application can provide additional retrieval opportunities for lost packages in boundary scenarios where the matching score is close to but has not yet reached the matching threshold, significantly improving the re-association success rate of the target in the critical state.

[0090] Understandably, when the target storage time (or the extended target storage time) is exhausted, if no candidate associated package with a matching score that meets the matching threshold is found, that is, the lost package fails to be successfully reassigned within the allowed waiting window, it is determined that the package target corresponding to the lost package cannot be retrieved or has left the current camera's field of view. At this time, a state transition will be triggered, changing the state of the lost package from temporarily lost to permanently lost (i.e., Lost → Deleted), and releasing the tracking resources occupied by the lost package and clearing the corresponding temporary entries to save system storage and computing resources.

[0091] Understandably, in the re-association operation, after calculating the matching scores between all lost packages and candidate associated packages in each frame, a greedy matching strategy can be executed. This involves confirming matching pairs sequentially according to their matching scores from highest to lowest. After each pair is confirmed, the lost package and its corresponding candidate associated package are removed from the list of packages to be matched. This ensures that each lost package is associated with at most one candidate associated package, and each candidate associated package is assigned to at most one lost package, thus achieving the globally optimal one-to-one matching result. After matching is complete, three target lists can be output: targets that successfully matched and entered the verification process are listed in the "Recovering" target list; new detection results that failed to associate with any existing lost packages are listed in the "New" target list; and targets that have not successfully matched after the storage period expires are listed in the "Deleted" target list.

[0092] The package association method provided in this application has the following beneficial effects: 1. By constructing a package tracking state machine that includes six states: new registration, active tracking, temporary loss, re-association, verification passed, and permanent loss, and defining clear transition conditions between each state, a closed-loop management of the entire lifecycle of a package is achieved from its first entry into the field of view to its final departure from the field of view. This effectively overcomes the shortcomings of calculating matching scores only at the single-frame level and lacking a systematic processing mechanism after target loss. This enables the tracking system to have clear behavioral logic and state transition paths when facing abnormal situations such as occlusion and detection failure.

[0093] 2. By adaptively calculating the target's temporary storage time based on the speed and location information of the lost package, the storage time is matched with the target's actual movement state and spatial location: targets moving at high speed or near the edge of the field of view receive a shorter storage time to save resources, while targets moving at low speed or within the field of view receive a longer storage time to ensure a chance of retrieval. This achieves precise allocation of temporary storage resources, avoiding resource waste or retrieval failure caused by fixed storage times.

[0094] 3. By using an adaptive search range determination mechanism based on Kalman filter prediction and covariance ellipse, the search range can be automatically scaled according to the changes in prediction uncertainty. Combined with the velocity compensation term, the cumulative displacement of the target during the loss period is compensated, realizing fine-grained hierarchical adjustment of the search range under different loss duration and velocity conditions, which significantly improves the search hit rate and computational efficiency of reassociation operation.

[0095] 4. By adaptively adjusting the matching threshold based on the density of similar packages, the matching judgment criteria can be dynamically tightened or relaxed as the number of similar packages in the search range changes. This effectively suppresses the risk of false association in high-density similar package scenarios and maintains reasonable association sensitivity in low-density scenarios, significantly improving the association reliability of the system in high-density, high-similarity sorting environments.

[0096] 5. By establishing a progressive matching and confirmation mechanism of "single-frame initial screening - multi-frame verification - threshold decision", appearance consistency verification and motion consistency verification of consecutive multi-frames are introduced after a single-frame match is successful. The association relationship is finally confirmed only when the number of successful frames reaches the preset threshold. This effectively avoids erroneous associations caused by single-frame detection noise, brief occlusion or accidental appearance similarity, and greatly improves the accuracy and robustness of identity binding.

[0097] 6. By introducing a temporary storage time extension mechanism, when the matching score of the candidate associated package is between the extension trigger threshold and the matching threshold, an additional waiting time is added for the lost package, so that the target in the "close to retrieval" critical state has more chances of retrieval. This effectively avoids the problem of the retrieveable target being accidentally deleted due to the temporary storage time just running out, and further improves the success rate of re-association.

[0098] Reference Figure 4 This application also provides a package association device for package tracking within a single camera. The device includes: The lost package acquisition unit 401 is used to acquire lost packages, wherein the lost package is a target package that has not been successfully associated, determined based on the current frame image and / or previous multiple frame images captured by the camera. The temporary storage duration determination unit 402 is used to obtain the loss information of each lost package and determine the target temporary storage duration of the lost package based on the loss information; wherein, the loss information includes the status information of the lost package in the last frame before it was lost; The reassociation unit 403 is used to perform a reassociation operation on each frame of image captured by the camera during the target temporary storage period until the lost package is successfully associated with the package. The reassociation operations include: Based on the loss information, predict the search range of the lost package in the execution frame image when performing the re-association operation. Based on the search range, perform package detection on the execution frame image and determine the candidate associated packages of the lost package. Calculate the matching score between each candidate associated package and the lost package. Based on the matching score, the target associated package of the lost package is determined from each candidate associated package. The matching score of the target associated package is higher than the preset matching threshold and not lower than the matching scores of other candidate associated packages.

[0099] It should be noted that the package association device provided in this application embodiment is used to implement the package association method provided in the above embodiment, and the specific implementation process corresponds to the package association method in the above embodiment. It can be referred to the aforementioned package association method, and will not be repeated here.

[0100] This application also provides an electronic device (i.e., a computer device), which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the package association methods described in the above embodiments. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0101] Reference Figure 5 , Figure 5 This illustration shows the hardware structure of an electronic device according to another embodiment, the electronic device comprising: The processor 501 can be implemented using a general-purpose central processing unit (CPU), graphics processing unit (GPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 502 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 502 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 502 and is called and executed by the processor 501 using the package association method of the embodiments of this application. The input / output interface 503 is used to implement information input and output; The communication interface 504 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 505 transmits information between various components of the device (e.g., processor 501, memory 502, input / output interface 503, and communication interface 504); The processor 501, memory 502, input / output interface 503, and communication interface 504 are connected to each other within the device via bus 505.

[0102] This application also provides a computer-readable storage medium storing a computer program for causing a computer to execute the package association method described in the above embodiments.

[0103] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0104] This invention also provides a computer program product that stores program instructions, which, when executed by a computer, cause the computer to implement the package association method described in any of the above embodiments.

[0105] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0106] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0109] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0110] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0112] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A package association method, characterized in that, The method is applied to package tracking within a single camera, and the method includes: Retrieve lost packages, wherein the lost packages are target packages that have not been successfully associated, as determined based on the current frame image and / or previous multiple frame images captured by the camera; For each lost package, the loss information of the lost package is obtained, and the target temporary storage time of the lost package is determined based on the loss information; wherein, the loss information includes the status information of the lost package in the last frame before it was lost; During the target storage period, a reassociation operation is performed sequentially on each frame of image captured by the camera until the lost package is successfully associated with the package. The reassociation operation includes: Based on the loss information, predict the search range of the lost package in the execution frame image where the reassociation operation is performed. Based on the search range, perform package detection on the execution frame image and determine the candidate associated packages of the lost package. Calculate the matching score between each candidate associated package and the lost package. Based on the matching score, the target associated package of the lost package is determined among the candidate associated packages, wherein the matching score of the target associated package is higher than a preset matching threshold and not lower than the matching score of other candidate associated packages.

2. The method according to claim 1, characterized in that, The status information includes the location and speed information of the lost package; The step of predicting the search range of the lost package in the execution frame image during the reassociation operation based on the loss information includes: Based on the state information, predict the predicted position and predicted state data of the lost package in the execution frame image of the reassociation operation, wherein the predicted state data is used to characterize the uncertainty of the predicted position; Based on the predicted state data, determine the uncertain state area of ​​the lost package, and determine the target search radius based on the size data of the uncertain state area; Using the predicted location as the center and the target search radius as the radius, the search range for the lost package is determined in the execution frame image.

3. The method according to claim 2, characterized in that, Determining the target search radius based on the size data of the uncertain region includes: The basic search radius is determined based on the size data of the uncertain region. The number of lost frames is determined based on the last frame and the execution frame before the lost package is lost, and a speed compensation term is determined based on the number of lost frames and the speed information; The target search radius is obtained by summing the velocity compensation term and the basic search radius.

4. The method according to claim 1, characterized in that, Before determining the target associated package of the lost package among the candidate associated packages based on the matching score, the method further includes adjusting the matching threshold, including: Obtain the first appearance feature corresponding to the lost package, and obtain the second appearance feature of the candidate associated package in the execution frame image; The similarity between the first appearance feature and the second appearance feature is calculated to determine the number of packages whose similarity is greater than a preset appearance similarity threshold. The similar package density is calculated based on the number of packages and the area of ​​the search range, and the matching threshold is adjusted based on the similar package density.

5. The method according to claim 1, characterized in that, The status information includes the location and speed information of the lost package; Determining the target temporary storage duration for the lost package based on the loss information includes: The speed influence factor is calculated based on the speed information and the preset speed threshold. The location of the lost package in the last frame before it was lost is determined based on the location information, and the boundary distance between the package and the nearest boundary in the camera detection area is determined. The boundary influence factor is calculated based on the boundary distance and the area size of the detection region. The target temporary storage time is calculated based on the boundary influence factor, the velocity influence factor, and the preset basic temporary storage time.

6. The method according to claim 5, characterized in that, The reassociation operation also includes: If a candidate associated package that meets the preset extension conditions is detected within the target temporary storage period, the target temporary storage period of the lost package is extended. The extension conditions include: the matching score between the candidate associated package and the lost package is less than or equal to the matching threshold, and the matching score between the candidate associated package and the lost package is greater than a preset extension trigger threshold, wherein the extension trigger threshold is less than the matching threshold.

7. The method according to claim 1, characterized in that, The step of determining the target associated package of the lost package among the candidate associated packages based on the matching score includes: Based on the matching score, the initial associated package of the lost package is determined from each of the candidate associated packages, wherein the matching score of the initial associated package is higher than a preset matching threshold and not lower than the matching score of other candidate associated packages; The preliminary associated package and the lost package are matched and verified for multiple consecutive frames, wherein the matching verification includes appearance consistency verification and / or motion consistency verification. Determine the number of frames that successfully match and verify. If the number of successful frames is greater than a preset frame threshold, determine the preliminary associated package as the target associated package of the lost package.

8. The method according to claim 1, characterized in that, Prior to retrieving the lost package, the method further includes: Package detection is performed on the current frame image captured by the camera to identify the packages to be registered in the current frame image; If the position of the package to be registered in the current frame image is within the boundary of the camera detection area, and the package to be registered does not match the already registered package target, the package to be registered will be registered as a new package target.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images