Parcel association method and device, electronic device, and storage medium

Abstract

Embodiments of the present application provide a parcel association method and device, electronic equipment and storage medium, belonging to the technical field of logistics sorting. The method comprises: determining a to-be-associated parcel in a to-be-detected image; determining a candidate associated parcel corresponding to the to-be-associated parcel from a previous image; determining current transmission data of the to-be-associated parcel; determining a motion feature weight and an appearance feature weight according to the current transmission data; determining a motion feature similarity according to an actual detection position and a predicted position of the to-be-associated parcel in the to-be-detected image, and determining an appearance feature similarity according to an appearance feature of the to-be-associated parcel and an appearance feature of the candidate associated parcel; weighting and fusing the motion feature similarity and the appearance feature similarity according to the motion feature weight and the appearance feature weight to obtain an association score; and if the association score is not less than a threshold, associating the to-be-associated parcel with the candidate associated parcel. The embodiments of the present application can improve the accuracy of single-camera parcel tracking.

Description

Technical Field

[0001] This application relates to the field of logistics sorting technology, and in particular to a parcel association method and apparatus, electronic equipment and storage medium. Background Technology

[0002] In automated logistics sorting systems, parcels are scanned by mainline scanning equipment to collect label information before falling into the corresponding sorting chute. To achieve fully automated and intelligent management, the parcels must maintain an accurate association with the label information after entering the chute. This allows backend personnel or automated equipment to identify the identity information of stacked parcels without repeated scanning, effectively improving sorting efficiency and reducing logistics error rates caused by misidentification or mispacking.

[0003] Currently, related technologies typically employ the deployment of multiple 2D or 3D vision cameras along the chute to achieve end-to-end identity binding of packages from the main line and chute to the final cage loading stage through real-time multi-view tracking. This process mainly includes two core components: continuous tracking within a single camera's field of view and identity relay across camera fields of view. In the single-camera continuous tracking stage, when packages change their posture or appearance due to stacking or occlusion, the package tracking in related technologies is prone to mismatches. Summary of the Invention

[0004] The main objective of this application is to propose a package association method, apparatus, electronic device, and storage medium that can improve the accuracy of package tracking within a single camera.

[0005] To achieve the above objectives, a first aspect of this application proposes a package association method, the method comprising: Acquire the image to be detected captured by the camera, and detect the image to be detected to determine at least one package to be associated contained in the image to be detected; For each package to be associated, candidate associated packages corresponding to the package to be associated are determined from the tracked packages contained in the previous frame image captured by the camera corresponding to the image to be detected. Based on the location data of the package to be associated in the image to be detected, and the location data of the candidate associated package in the previous frame image, the current transmission data of the package to be associated is determined, and the motion feature weight and appearance feature weight are dynamically determined based on the current transmission data. The predicted position is obtained by predicting the position of the candidate associated wrapping in the image to be detected, and the motion feature similarity is determined based on the position prediction deviation between the actual detected position of the candidate associated wrapping in the image to be detected and the predicted position. The appearance features of the package to be associated are obtained from the image to be detected, the appearance features of the candidate associated package are obtained from the previous frame image, and the appearance feature similarity is determined between the appearance features of the package to be associated and the appearance features of the candidate associated package. Based on the motion feature weights and appearance feature weights, the motion feature similarity and appearance feature similarity are weighted and fused to obtain the association score between the package to be associated and the corresponding candidate associated package; If the association score is not less than a preset threshold, the package to be associated is associated with the candidate associated package.

[0006] To achieve the above objectives, a second aspect of this application provides a package association device, the device comprising: A package detection unit is used to acquire an image to be detected captured by a camera, and to detect the image to be detected to determine at least one package to be associated contained in the image to be detected. The preliminary association unit is used to determine, for each package to be associated, a candidate associated package corresponding to the package to be associated from the tracked packages contained in the previous frame image captured by the camera corresponding to the image to be detected; The weight determination unit is used to determine the current transmission data of the package to be associated based on the positioning data of the package to be associated in the image to be detected and the positioning data of the candidate associated package in the previous frame image, and to dynamically determine the motion feature weight and appearance feature weight based on the current transmission data. The motion similarity determination unit is used to predict the position of the candidate associated wrapping in the image to be detected to obtain the predicted position, and to determine the motion feature similarity based on the position prediction deviation between the actual detected position of the candidate associated wrapping in the image to be detected and the predicted position. The appearance similarity determination unit is used to obtain the appearance features of the package to be associated based on the image to be detected, obtain the appearance features of the candidate associated package based on the previous frame image, and determine the appearance feature similarity between the appearance features of the package to be associated and the appearance features of the candidate associated package. The feature fusion unit is used to perform weighted fusion of the motion feature similarity and the appearance feature similarity according to the motion feature weight and the appearance feature weight, so as to obtain the association score between the package to be associated and the corresponding candidate associated package. The target association unit is used to associate the package to be associated with the candidate associated package if the association score is not less than a preset threshold.

[0007] To achieve the above objectives, a third aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in any one of the embodiments of the first aspect.

[0008] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the embodiments of the first aspect.

[0009] The package association method, apparatus, electronic device, and storage medium proposed in this application first detect the package to be sorted from the image to be detected, and preliminarily determine candidate associated packages from the previous frame image. Then, the current transmission data of the package to be sorted is acquired, and motion feature weights and appearance feature weights matching the current transmission conditions are dynamically determined based on the current transmission data. The motion feature similarity and appearance feature similarity between the package to be sorted and the candidate associated packages are also determined. Finally, the motion similarity and appearance feature similarity are weighted and fused according to the dynamically determined motion feature weights and appearance feature weights to obtain an association score. If the association score is greater than a preset association threshold, the package to be sorted and the candidate associated packages are determined to be the same package, and thus the package to be sorted and the candidate associated packages are associated. Therefore, this application embodiment achieves adaptive weight fusion for condition-aware multi-feature fusion. Thus, compared to methods using fixed weights for multi-feature fusion, this application embodiment can improve the accuracy of package tracking within a single camera. Attached Figure Description

[0010] Figure 1 This is a flowchart of one embodiment of the package association method provided in this application; Figure 2 This is a schematic diagram of the package association device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] In automated logistics sorting systems, parcels are scanned by mainline scanning equipment to collect label information before falling into the corresponding sorting chute. To achieve fully automated and intelligent management, the parcels must maintain an accurate association with the label information after entering the chute. This allows backend personnel or automated equipment to identify the identity information of stacked parcels without repeated scanning, effectively improving sorting efficiency and reducing logistics error rates caused by misidentification or mispacking.

[0013] Currently, related technologies typically employ the deployment of multiple 2D or 3D vision cameras along the chute to achieve end-to-end identity binding of packages from the main line and chute to the final cage loading stage through real-time multi-view tracking. This process mainly includes two core components: continuous tracking within a single camera's field of view and identity relay across camera fields of view. In the single-camera continuous tracking stage, when packages change their posture or appearance due to stacking or occlusion, the package tracking in related technologies is prone to mismatches.

[0014] Specifically, single-camera tracking methods can employ a multi-feature fusion strategy with fixed weights. However, this method has the following drawbacks that may lead to mismatches during tracking: when the package moves rapidly, image blurring causes instability in appearance features, and deterioration in 3D point cloud quality leads to instability in morphological features. However, with a fixed weight approach, appearance and morphological features still account for a relatively large weight during feature fusion.

[0015] Based on this, embodiments of this application provide a package association method and apparatus, an electronic device and a storage medium, which can perform adaptive weight fusion of multiple features based on working condition perception.

[0016] The package association method provided in this application relates to the field of logistics sorting technology. The package association method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, barcode scanner, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application implementing the package association method, but is not limited to the above forms.

[0017] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0018] Please see Figure 1 , Figure 1 This is an optional flowchart of the package association method provided in the embodiments of this application. In some embodiments of this application, the package association method is applied to package tracking within a single camera. Figure 1 The specific methods may include, but are not limited to, steps S101 to S107.

[0019] Step S101: Acquire the image to be detected captured by the camera, and detect the image to be detected to determine at least one package to be associated contained in the image to be detected. Step S102: For each package to be associated, determine the candidate associated packages corresponding to the package to be associated from the tracked packages contained in the previous frame image corresponding to the image to be detected captured by the camera. Step S103: Based on the location data of the package to be associated in the image to be detected and the location data of the candidate associated package in the previous frame image, determine the current transmission data of the package to be associated, and dynamically determine the motion feature weight and appearance feature weight based on the current transmission data. Step S104: Predict the position of the candidate associated package in the image to be detected to obtain the predicted position, and determine the motion feature similarity based on the position prediction deviation between the actual detected position and the predicted position of the package in the image to be detected. Step S105: Obtain the appearance features of the package to be associated based on the image to be detected, obtain the appearance features of the candidate associated packages based on the previous frame image, and determine the appearance feature similarity based on the appearance features of the package to be associated and the appearance features of the candidate associated packages. Step S106: Based on the motion feature weight and appearance feature weight, perform weighted fusion of motion feature similarity and appearance feature similarity to obtain the association score between the package to be associated and the corresponding candidate associated package. Step S107: If the association score is not less than a preset threshold, associate the package to be associated with the candidate associated package.

[0020] In step S101 of some embodiments, the end-to-end identity binding of the package from the main line, chute to the final cage loading stage is realized. This mainly includes two core steps: continuous tracking within a single camera's field of view and identity relay across camera fields of view. Continuous tracking within a single camera's field of view refers to maintaining the stable continuation of the package's tracking identity (Track ID) through cross-frame target matching in a continuous frame sequence of the same camera, ensuring that the binding relationship between the package and the waybill information does not break when the package moves within the coverage area of ​​a single camera. Identity relay across camera fields of view refers to transferring the package's identity identifier from the previous camera to the next camera when the package moves along the chute, leaves the current camera's field of view, and enters the detection area of ​​the next adjacent camera. This achieves seamless identity inheritance during camera switching. It should be noted that the method provided in this application embodiment is mainly used to solve the problem of mismatches that currently used fixed-weight multi-feature fusion methods are prone to when the appearance and other features of the package are unstable due to rapid movement within a single camera's field of view during continuous tracking. The method provided in this application embodiment can run independently in any single 3D camera deployed along the sorting chute, serving as the core matching engine of the camera's internal tracking module. When a package enters the chute from the main sorting line via the swing wheel or chute sorting mechanism and first appears within the detection field of view of the first camera along the chute, the camera identifies and registers the package using a target detection model, assigns it an initial tracking identity, and binds this identity to the waybill number reported by the main line six-sided scanning system, thereby triggering continuous tracking of the package. Subsequently, throughout the entire process of the package moving along the chute, each time it enters the coverage area of ​​a subsequent camera, the dynamic weighted matching strategy defined in this application embodiment can maintain cross-frame identity consistency within that camera until the package moves to the end of the chute and enters the cage-loading or temporary storage stage. For ease of explanation, the following description uses a camera deployed at any position along the chute as an example.

[0021] Specifically, firstly, the image to be detected, captured by a 3D camera, can be read in real time. This image can contain two-dimensional RGB color information, or corresponding depth information or three-dimensional point cloud data. Then, a deep learning-based object detection model (such as the YOLO series) can perform forward inference computation on the image. By regressing bounding boxes and class confidence scores, the detection region of each individual package is segmented and determined from the image background. Newly appearing packages in the image that have not yet established an identity relationship with the previous frame are defined as packages to be associated.

[0022] In step S102 of some embodiments, the historical state information of all tracked packages that have been successfully assigned a unique tracking identifier (Track ID) in the previous frame image corresponding to the image to be detected can first be retrieved from memory or cache. This historical state information may at least include the detection box position coordinates, velocity estimates, and corresponding feature vector caches of the package in the previous frame image. To avoid the O(n²) computational cost caused by exhaustively pairing each package to be associated with the image to be detected with all tracked packages in the previous frame image, this embodiment introduces a tracking algorithm for preliminary association and candidate screening. Specifically, the tracking algorithm may employ a simple threshold filter based on a motion model or a lightweight online tracker (e.g., the Kalman filter prediction stage in Sort or DeepSort): for each tracked package in the previous frame image, its predicted position in the current frame (i.e., the frame corresponding to the image to be detected) is derived based on its state vector and motion model in the previous frame, and the Mahalanobis distance or intersection-over-union (IoU) overlap between the predicted position and the detection positions of each package to be associated in the current frame is calculated. Then, packets falling within the prediction threshold (i.e., Mahalanobis distance less than a certain lenient threshold, or IoU greater than a preset low threshold) are considered candidate matches for the tracked packets in the current frame. In other words, the tracked packets are used as candidate associated packets for the packet to be associated, thus establishing a preliminary mapping relationship from the packet to be associated to the candidate associated packets. For example, if the previous frame contains three tracked packets (A, B, and C), and the image to be detected contains three packets to be associated (D, E, and F), the tracking algorithm can initially determine that packets A are associated with packet D, B with packet E, and C with packet F. That is, packet A is considered a candidate associated packet of packet D, packet B with packet E, and packet C with packet F. It is understood that although the preliminary association results determined by the tracking algorithm may have deviations, these results are only used to establish preliminary correspondences. Subsequent steps will further verify and correct these associations through fine-grained feature fusion. This method of preliminary association based on tracking algorithms effectively filters out a large number of invalid combinations that are spatially far apart and obviously impossible to pair by introducing the prior of motion continuity. This reduces the computational scale of subsequent fine feature comparison to the constant level or the linear matching range, thereby significantly reducing the real-time processing pressure of the system.

[0023] In step S103 of some embodiments, for each package to be associated (e.g., package D), the positioning data of the package to be associated in the image to be detected, and the positioning data of its candidate associated packages (e.g., package A) in the previous frame image are extracted, and the current transmission data of the package to be associated is determined based on these two positioning data. The positioning data may include the location of the package's detection box, the package's coordinates in the frame image, etc. The current transmission data may refer to data that describes the current transmission status of the package to be associated, which may be a comprehensive description of the package's physical motion state and visibility state in the current imaging environment. For example, when the current transmission data includes current motion speed and occlusion data, the current transmission status may include combinations of conditions such as fast motion or normal motion, occlusion or no occlusion.

[0024] In some embodiments, the positioning data includes location data, and the method for determining the current speed of movement may include, but is not limited to, the following steps: Based on the position data of the package to be associated in the image to be detected, and the position data of the candidate package to be associated in the previous frame image, calculate the position displacement between the package to be associated and the candidate package to be associated. The current motion speed is calculated based on the position displacement and the frame interval between the image to be detected and the previous frame.

[0025] In this embodiment, the location data can refer to the position coordinates wrapped within the corresponding frame image, such as the spatial coordinates of the center point of the detection box (or other feature points of the detection box) in a unified world coordinate system. In specific implementation, firstly, the location data wrapped within the image to be detected (denoted as...) can be used... ), and the positional data of the candidate associated wrapper in the previous frame image (denoted as ), ), calculate the positional displacement between the package to be associated and the candidate associated packages. The positional displacement represents the actual distance moved in space by the same physical package (such as packages A and D initially identified as the same package) between two consecutive observation frames. Its magnitude directly reflects the intensity of the package's movement. Understandably, to eliminate the displacement calculation error introduced by fluctuations in the detection box positioning, the position estimate smoothed by Kalman filtering can be used instead of the center point of the detection box of the package to be associated for displacement calculation. The representation method of the positional data is not limited in this regard.

[0026] After obtaining the position displacement, as shown in Formula 1 below, the position displacement can be used as a basis for calculation. The current motion speed is calculated by considering the frame interval between the image to be detected and the previous frame. .

[0027] (Formula 1) Among them, frame interval time It can refer to the time interval between two adjacent frames captured by the camera, which is determined by the camera's image capture frame rate. For example, when the camera captures images at a fixed frame rate of 30 frames per second, the time interval between adjacent frames is 1 / 30 of a second, which is approximately 33.33 milliseconds.

[0028] Understandably, considering that single-time speed calculations are easily affected by detection noise and inter-frame jitter, resulting in significant fluctuations, a multi-frame smoothing strategy can be used to determine the current motion speed to address this issue. As shown in Formula 2 below, the instantaneous speed calculated in the current frame is averaged with the speed estimates of the most recent N frames (which can be calculated using the method shown in Formula 1). (The specific value of N is set according to the actual frame rate and motion change frequency; for example, N=5 is not limited in this respect.) The averaged speed value is then used as the final current motion speed output, thereby effectively suppressing high-frequency noise, making the speed determination result smoother and more reliable, and avoiding frequent weight oscillations caused by single-frame speed jumps that affect matching stability.

[0029] (Formula 2) Determine the current velocity (i.e., instantaneous velocity) or average speed After that, the current movement speed can be compared with the preset speed threshold. A comparison is performed. If the current speed exceeds this speed threshold... If the speed is less than or equal to the speed threshold, then the corresponding package to be associated (e.g., package D) is determined to be in a fast-moving state. Conversely, if the current speed is less than or equal to the speed threshold, then the corresponding package to be associated (e.g., package D) is determined to be in a fast-moving state. If the corresponding package to be associated (e.g., package D) is determined to be in a normal and stable motion state, then the camera imaging quality is good and the reliability of the appearance features is high. Among these, the velocity threshold... The speed threshold can be flexibly set according to the physical parameters of the actual sorting scenario. For example, it can be set based on the slope of the chute (a steeper slope results in greater acceleration as the package slides down) and the initial speed imparted to the package by the sorting machine's wheel movement. This ensures that the speed determination result matches the actual working conditions, providing an accurate scenario classification basis for subsequent adaptive weight adjustments. The speed is set to 2.0 m / s, and no limit is imposed on it.

[0030] In some embodiments, the positioning data further includes a detection bounding box, and the method for determining occlusion data may include, but is not limited to, the following steps: Determine the target detection area of ​​the corresponding detection box in the image to be detected for the package to be associated; The process involves identifying adjacent packages from the packages contained in the image to be detected, determining the corresponding detection boxes of the adjacent packages in the image to be detected, and the overlap area between the detection boxes of the adjacent packages and the corresponding detection boxes of the packages to be detected. Occlusion data is calculated based on the overlapping area and the target detection area.

[0031] In this embodiment, the detection box can refer to the rectangular bounding box output by the target detection model (e.g., YOLO series) after inference on the image to be detected, used to identify the two-dimensional spatial range of each package in the image. This detection box is typically mathematically represented by the pixel coordinates of the top-left and bottom-right corners, or by the center point coordinates combined with width and height values. The target detection area can refer to the size of the pixel region covered by the rectangular detection box corresponding to the package to be associated (e.g., package D). Specifically, it can be obtained by calculating the product of the width and height values ​​of the detection box. The target detection area represents the projection size of the package to be associated onto the image plane.

[0032] Further, neighboring packages of the package to be associated are identified from the packages contained in the image to be detected, and the overlap area between the neighboring packages and the package to be associated is determined. A neighboring package can refer to other detected package instances in the image to be detected, excluding the package to be associated, whose detection boxes are spatially adjacent to the detection box of the package to be associated. In one feasible implementation, the criterion for determining whether a package is a neighboring package is to calculate the Euclidean distance between the center points of the detection boxes of other packages and the detection box of the package to be associated, or the distance between the detection box boundaries. When this distance is less than a preset spatial proximity threshold, it is considered a neighboring package (e.g., package E). It should be noted that in the same frame of the image, there may be multiple neighboring packages for the package to be associated, and the detection box of each neighboring package may overlap with the package to be associated to varying degrees. The overlap area can refer to the pixel area of ​​the area where the detection boxes of neighboring packages and the detection box of the package to be associated intersect on the image plane. When there are multiple neighboring packages, the overlap area of ​​the detection box of the package to be associated and all neighboring package detection boxes can be calculated separately, and the maximum value or the sum of these values ​​can be taken as the final overlap area. The specific method used can be set according to the actual scenario requirements. For example, in densely stacked scenarios, summing the values ​​can more comprehensively reflect the overall degree of occlusion of the packages to be associated, while in sparse scenarios, taking the maximum value can better reflect the impact of the main occlusion source.

[0033] Finally, in determining the overlapping area and target detection area Then, the occlusion data can be calculated according to the following formula 3. .

[0034] (Formula 3) As can be seen from Formula 3, the occlusion data This refers to the occlusion ratio, used to quantitatively describe the severity to which a package being associated is obscured by adjacent packages. Occlusion data. The value ranges from 0 to 1. When the value is 0, it means that the package to be associated has no overlap with any of the adjacent packages, that is, it is in a completely unobstructed state. When the value is close to 1, it means that the detection box of the package to be associated is almost completely covered by the detection boxes of the adjacent packages, that is, it is in a severely occluded or even completely occluded state.

[0035] Understandably, since the overlap of detection boxes on a 2D image plane is not necessarily equivalent to the actual physical occlusion of a wrapped entity in 3D space, occlusion data can be corrected by incorporating depth image information as needed. For example, for pixels within an overlapping area, the depth values ​​of the wrapped entity to be associated and its adjacent entities at the corresponding pixel location are compared. If the depth value of the adjacent entity is less than that of the wrapped entity to be associated (i.e., the adjacent entity is in front of the wrapped entity to be associated), the pixel is included in the effective occlusion area; otherwise, it is not. This allows for a more realistic calculation of the physical occlusion ratio based on depth sorting, effectively reducing occlusion misjudgments caused by visual projection.

[0036] Furthermore, in determining the occlusion data Then, the occlusion data can be... With the preset occlusion threshold Compare the data; if the data is obscured... Greater than the occlusion threshold If the data is obscured, it is determined that the package to be associated (e.g., package D) is currently in an obscured state; otherwise, if the data is obscured... Less than or equal to the occlusion threshold If the condition is met, it is determined to be in an unoccluded state. An occluded state indicates that the package to be associated is significantly obscured by adjacent packages, and its appearance features are incomplete due to the obstruction. The reliability of appearance features as a basis for determining whether they belong to the same package is significantly reduced; therefore, the contribution of appearance features needs to be weakened in subsequent adaptive weight calculations. An unoccluded state indicates that although there may be slight overlap, it has little impact on the integrity and discriminative power of the package's appearance features. Occlusion threshold This is a pre-defined percentage threshold used to distinguish between slight and significant occlusion. For example, it can be set to 0.2, meaning that an occlusion percentage exceeding 20% ​​is considered occlusion. In practical applications, the occlusion threshold... The value can be flexibly adjusted and calibrated based on factors such as the distribution of package size, camera installation height and viewing angle in the sorting scenario, to ensure the accuracy of occlusion judgment and scene adaptability, thereby providing a reliable scene classification basis for subsequent adaptive weight adjustment.

[0037] Based on the current transmission data (i.e., current movement speed and occlusion data) The system dynamically determines the weights used for fusing multiple features in the current frame. Specifically, when the multiple features include motion features and appearance features, the fusion weights can include motion feature weights and appearance feature weights. The core of dynamically determining the weights is that when the current motion speed indicates that the package to be associated is in a fast-moving state, the appearance imaging quality deteriorates due to motion blur. In this case, the continuity of the motion trajectory is more reliable, and the motion feature weights should be increased. When occluded data... When the package to be associated is in an occluded state, its appearance features are incomplete due to occlusion. In this case, the weight of motion features should be increased. That is, for various working conditions composed of combinations of fast or normal motion, occlusion or no occlusion, the weights of motion features and appearance features need to be adaptively adjusted under different working conditions to improve the accuracy of subsequent association matching judgments.

[0038] In some embodiments, step S103, "dynamically determining motion feature weights and appearance feature weights based on the current transmitted data," may include, but is not limited to, the following steps: Obtain reference appearance weight and reference motion weight, wherein the reference appearance weight is greater than the reference motion weight. The reference appearance weight and reference motion weight correspond to the baseline weights under the condition that the current motion speed is not greater than the preset speed threshold and the occlusion data is not greater than the preset occlusion threshold. Determine the occlusion level based on occlusion data; If the current motion speed is greater than the preset speed threshold, the reference appearance weight is reduced according to the occlusion level, and the reference motion weight is increased accordingly to obtain the motion feature weight and appearance feature weight; wherein, the motion feature weight is greater than or equal to the appearance feature weight.

[0039] If the current motion speed is not greater than the preset speed threshold, the reference appearance weight is reduced according to the occlusion level, and the reference motion weight is increased accordingly to obtain the motion feature weight and appearance feature weight; among them, the motion feature weight is less than the appearance feature weight.

[0040] In this embodiment, reference appearance weights and reference motion weights are first obtained. These weights are a set of preset baseline weights, corresponding to the optimal weight configuration under ideal conditions where the package is in normal, stable motion and unobstructed. Specifically, the baseline weights are set based on the following physical principle: when the package's movement speed does not exceed a preset speed threshold... (i.e., in a normal and stable motion state) and the occlusion data is not greater than the occlusion threshold. When the camera image quality is good, and the appearance features (such as RGB texture features and point cloud morphology features) are complete, clear, and reliable, the appearance features have the strongest discriminative power as a matching basis, thus giving them a higher fusion weight. Meanwhile, although motion features also have some reference value under normal operating conditions, their discriminative power is not as rich as that of appearance features, so motion features are given a relatively lower weight; that is, the reference appearance weight is greater than the reference motion weight. For example, the reference appearance weight can be set to 0.7, and the reference motion weight to 0.3, with a sum of 1. It is understandable that this set of baseline weights is not a fixed absolute value, but rather a set of hyperparameters that can be calibrated and adjusted offline based on factors such as camera resolution, lighting conditions, and the richness of the wrapping texture in the actual scene. Its purpose is to provide a standardized starting point for subsequent scene adaptive adjustments.

[0041] After obtaining the reference appearance weights and reference motion weights, the occlusion level can be determined based on the occlusion data. (Occlusion data) The occlusion ratio, calculated based on the aforementioned embodiments, ranges from 0 to 1, representing the proportion of the area of ​​the package to be associated that is obscured by adjacent packages to the total area of ​​its own detection frame. The occlusion level is a discretized level identifier obtained by quantizing the occlusion ratio into intervals, used to characterize the severity of occlusion. In one feasible implementation, multiple consecutive occlusion ratio intervals can be pre-defined, each interval corresponding to an occlusion level. For example: when the occlusion ratio is less than or equal to a first threshold (e.g., 0.2), it is classified as no occlusion or mild occlusion; when the occlusion ratio is between the first and second thresholds (e.g., 0.2 to 0.5), it is classified as moderate occlusion; when the occlusion ratio is greater than the second threshold (e.g., 0.5), it is classified as severe occlusion or even complete occlusion. The number of occlusion levels and the corresponding ratio intervals for each level can be flexibly configured according to the density of package stacking in the actual sorting scenario. The finer the interval division, the finer the granularity of subsequent weight adjustments. Thus, the occlusion data of the package to be associated can be determined based on the occlusion ratio intervals. The corresponding occlusion level. The higher the occlusion level (i.e., the more severe the occlusion), the more of the appearance features of the package to be associated are obscured, and the more significant the decrease in the reliability of the appearance features. Therefore, the weight of the reference appearance needs to be reduced to a greater extent.

[0042] After determining the occlusion level of the package to be associated, a specific weight adjustment strategy can be determined based on the current movement speed. Specifically, if the current movement speed is greater than a speed threshold... This indicates that the package to be associated is in rapid motion. Due to camera frame rate limitations, motion blur occurs, resulting in loss of RGB image texture details. Simultaneously, the 3D point cloud quality deteriorates due to scanning distortion, severely compromising the overall reliability of the appearance features. In this situation, the reference appearance weight is reduced based on the previously determined occlusion level, and the reference motion weight is correspondingly increased. This ensures that the final determined motion feature weight (derived from the adjusted reference motion feature weight) is greater than or equal to the appearance feature weight (derived from the adjusted appearance feature weight), meaning that motion features dominate the subsequent fusion calculation. It is understandable that the specific reduction in the reference appearance weight is positively correlated with the occlusion level. For example, when the occlusion level is mild, the reference appearance weight can be reduced slightly from the baseline (e.g., reduced by (0.05, 0.2)); when the occlusion level is moderate, the reduction is increased (e.g., reduced by (0.2, 0.4)); when the occlusion level is severe, the reference appearance weight can be significantly reduced (e.g., reduced by (0.4, 0.7)), and in extreme cases (i.e., complete occlusion), it can be reduced to 0, making the matching completely dependent on motion features. Correspondingly, the motion feature weight is increased by the same amount to ensure that the sum of the two is always equal to 1. There is no specific limitation on the magnitude of the weight reduction. For example, assuming the reference appearance weight is 0.7 and the reference motion weight is 0.3, if the package to be associated is currently in rapid motion and mildly occluded, then the appearance weight will be... When the appearance feature weight is reduced by 0.2, the motion feature weight increases by 0.2 accordingly. At this point, the motion feature weight equals the appearance feature weight, and the two are balanced. If the package to be associated is in rapid motion and heavily occluded, the appearance weight is reduced by 0.4, and the motion weight increases by 0.4 accordingly. In this case, the motion feature weight is significantly greater than the appearance feature weight and dominates. The physical basis for this adjustment logic is that during rapid motion, the appearance feature itself is already unreliable due to blurring, and severe occlusion further weakens the discriminative power of the appearance feature. Therefore, the contribution of the motion feature should be significantly increased. When the occlusion is slight, although the appearance feature is affected by motion blur, it still retains some usable information. Therefore, it is not necessary to reduce the appearance weight too low, and the motion feature weight should not be excessively increased to maintain a certain balance between the two features.

[0043] Furthermore, if the current speed does not exceed a preset speed threshold This indicates that the package to be associated is in a normal, stable motion state. At this point, the imaging quality is good, and the appearance features themselves have high reliability. However, some appearance information will still be lost due to occlusion. In this case, the reference appearance weight is reduced and the reference motion weight is increased accordingly based on the occlusion level. However, the adjusted motion feature weight is still less than the appearance feature weight, meaning that appearance features still dominate in subsequent fusion calculations. It is understandable that the adjustment magnitude in this case is more gradual compared to the rapid motion scenario, because when the speed is normal, appearance features are only damaged in the occluded area, and the appearance information in the unoccluded area remains complete and reliable, requiring no significant weight transfer. The specific reduction magnitude is also positively correlated with the occlusion level, but is generally controlled within a small range. That is, different mapping relationships between the reduction magnitude and the occlusion level can be set for the two states: rapid motion and normal, stable motion. For example, assuming the reference appearance weight is 0.7 and the reference motion weight is 0.3, if the package to be associated is currently in a normal, stable motion state with slight occlusion, the appearance weight is reduced to 0.65, and the motion weight is increased to 0.35. The appearance feature weight is still significantly higher than the motion feature weight. If the package to be associated is in normal, stable motion and moderately occluded, the appearance weight is reduced to 0.6, and the motion weight is increased to 0.4, with appearance features still taking precedence. Through this method, while ensuring the priority of appearance features, the compensation effect of motion features is appropriately introduced, thereby maintaining high matching robustness even in occluded scenarios. Overall, regardless of the motion speed, the embodiments of this application follow the common adjustment principle of "the more severe the occlusion, the greater the reduction in appearance weight and the greater the increase in motion weight." The difference lies in the overall magnitude of the adjustment and whether the final motion feature weight exceeds the appearance feature weight, thus accurately adapting the reliability of features under different working conditions.

[0044] The weight determination method based on dynamic adjustment of benchmark weight provided in this application embodiment achieves intelligent adjustment of feature fusion weight from fixed optimal to dynamic adaptation by pre-setting the optimal benchmark weight under normal stable motion and unobstructed conditions, and then adjusting the benchmark weight according to the real-time perceived speed conditions and occlusion level.

[0045] In other embodiments, step S103, "dynamically determining motion feature weights and appearance feature weights based on the current transmitted data," may include, but is not limited to, the following steps: The speed weight is determined based on the ratio between the current speed and the preset speed threshold. The occlusion weights are determined based on the occlusion data, and the velocity weights and occlusion weights are summed and normalized to obtain the motion feature weights. The appearance feature weights are determined based on the motion feature weights, where the sum of the motion feature weights and the appearance feature weights is 1.

[0046] In this embodiment of the application, firstly based on the current speed of movement (or and speed threshold The velocity weights are determined using the method shown in Formula 4 below. .

[0047] (Formula 4) Speed ​​weight This is used to quantify the contribution of speed to the final motion feature weights. Its physical meaning is: the faster the speed, the higher the credibility of the motion feature in the association matching; therefore, the greater the contribution of speed to the motion feature weights should be. When the current motion speed... When =0, velocity weight =0 indicates that the package to be associated is completely stationary, and the speed factor does not contribute to the motion feature weights. In this case, the reliability of the motion features is determined solely by the occlusion factor. When the motion speed... Reaching or exceeding the speed threshold At that time, speed weight =1 indicates that the contribution of the velocity factor to the motion feature weight has reached its maximum value. Further increases in velocity will no longer additionally increase the motion feature weight, because at this point, the appearance feature has been reduced to its minimum confidence level due to motion blur. Through the above calculation method, the velocity weight... Constrained within a closed interval of 0 to 1, it has the same value range as the subsequent occlusion weights, which facilitates weighted summation operations.

[0048] Determining the speed weight Then, based on the occlusion data Determine occlusion weight Occlusion weight This is used to quantify the independent contribution of occlusion factors to the final motion feature weights. Its physical meaning is: the greater the degree of occlusion, the more information is lost from the appearance features due to occlusion, and the lower the reliability of the appearance features. Therefore, the relative importance of motion features in matching should be higher, and the contribution of occlusion factors to the motion feature weights should be greater. In one feasible implementation, the occlusion weights... The value can be directly taken as the occlusion data itself, that is = At this point, the occlusion weight and the occlusion data are proportionally and linearly positively correlated: when When =0 (no obstruction), = Occlusion factors do not contribute to the weights of motion features; when When =1 (complete occlusion), = Occlusion factors contribute the most to the weight of motion features. It's understandable that this positive correlation only holds when the occlusion data represents the occlusion area; depending on the actual needs, occlusion data can also be represented by location, coordinates, etc. In another more flexible implementation, a piecewise mapping function or a nonlinear mapping relationship can be used to determine the occlusion weight. For example, when the occlusion ratio is below a certain threshold, the occlusion is considered negligible, and the occlusion weight is zero; when the occlusion ratio exceeds another threshold, the weight rises rapidly to reflect the physical characteristic that the destructive impact on appearance features increases sharply after the degree of occlusion reaches a certain level. It's understandable that regardless of the mapping method used, it is necessary to ensure a monotonically positive correlation between the occlusion weight and the occlusion data; that is, the more severe the occlusion, the greater the occlusion weight.

[0049] After obtaining the velocity weights respectively and occlusion weight Then, the two can be summed to obtain the unnormalized weighted contribution value of the motion feature. Due to speed weight and occlusion weight Their respective value ranges are both 0 to 1, therefore The value range is 0 to 2. To normalize it to the standard weight interval of 0 to 1, the following is performed: After normalization, the specific calculation method is shown in Formula 5 below: (Formula 5) The result obtained from Formula 5 is the final motion feature weight. In determining the weights of motion features Then, the appearance feature weights can be calculated. =1- .

[0050] It is understood that the method for determining motion feature weights and appearance feature weights based on direct calculation in this application differs from the baseline weight adjustment method in the aforementioned embodiments in that this application does not rely on a preset baseline weight as a starting point. Instead, the weight values ​​are independently determined by formulaic calculation based on the current motion speed and occlusion data perceived in real time in the current frame. This method has the advantages of a direct calculation path and no need for offline calibration of baseline weights, making it suitable for real-time systems with ample computing resources and a need for rapid response to extreme changes in operating conditions. In this application embodiment, the data in the two dimensions of speed and occlusion are independently quantified to reduce the credibility of appearance features through speed weights and occlusion weights, respectively. Then, they are fused into the final motion feature weights and appearance feature weights through summation and normalization. The faster the speed and the more severe the occlusion, the higher the motion feature weight and the lower the appearance feature weight, thereby accurately achieving adaptive matching between the feature fusion weights and real-time operating conditions.

[0051] In step S104 of some embodiments, motion feature similarity is determined based on the position prediction deviation between the package to be associated and the candidate associated package. The position prediction deviation can refer to the deviation between the predicted position of the candidate associated package (e.g., package A) in the current frame and the actual detection position of the package to be associated (e.g., package D) in the current frame. The actual detection position can be determined based on a target detection model, such as the detection bounding box of the package to be associated. The predicted position can be determined based on Kalman filtering; that is, motion feature similarity can be determined based on a kinematic model established using Kalman filtering. This kinematic model measures the consistency of the motion trajectory by the degree of matching between the predicted position and the actual detection position. Specifically, a Kalman filter state model is pre-constructed for the candidate associated packages in the previous frame image, defining the target state vector as a four-dimensional vector containing the target center position coordinates and velocity components. Where x and y represent the center coordinates of the candidate association wrapper on the image plane, x y This represents the corresponding velocity component. Kalman filtering uses a state transition equation to describe the motion evolution of candidate associated packets from the previous frame to the current frame. Its state transition matrix F is constructed as a 4×4 matrix based on the inter-frame time interval Δt, specifically in the form:

[0052] Simultaneously, process noise is introduced. The covariance matrix is This characterizes the inherent uncertainty of the motion model. At the observation level, observation equations are established. The observation matrix It is a 2×4 matrix, specifically in the form of:

[0053] Observation matrix Its function is to extract two-dimensional position observations from a four-dimensional state vector. The observation noise has a covariance matrix of R, which characterizes the uncertainty of the target detection location itself. In the prediction step, based on the optimal state estimate of the candidate associated package (e.g., package A) in the previous frame, a one-step prediction is performed using the state transition matrix F to obtain the predicted state vector for the current frame. and the predicted state covariance matrix The two-dimensional position coordinates extracted from the predicted state vector are the predicted position. After obtaining the predicted position, its matching degree is calculated by comparing it with the actual detection position of the package to be associated (e.g., package D) in the current frame (e.g., determined by the center coordinates of the detection box output by the object detection model). For example, Mahalanobis distance can be used as a metric to measure the degree of matching between the predicted and actual detection positions of the target. The specific calculation method is shown in Formula 6 below: (Formula 6) in, This is the actual detection location vector of the current frame. Predict the location of the target. The uncertainty covariance matrix for the predicted location is given. This matrix combines the predicted state covariance and the observation noise covariance, reflecting the confidence interval of the predicted location. Compared to Euclidean distance, Mahalanobis distance introduces... As a weighted matrix, it can adaptively adjust the distance metric scale according to the uncertainty of the predicted position in different directions. When the prediction uncertainty is large, the tolerance for positional deviation increases accordingly, thus more robustly measuring the matching degree between the detected and predicted positions. After calculating the Mahalanobis distance, it can be converted into a motion similarity score with a value range between 0 and 1. This means that a higher score indicates stronger motor consistency. Specifically, this can be converted according to the following formula 7: (Formula 7) Alternatively, as shown in Formula 8 below, a simpler linear truncation method can be used: (Formula 8) in, To predict the Euclidean distance between the location and the detection location, Indicates the detection location. The preset maximum allowable displacement threshold, when Similarity exceeding this threshold The threshold is 0. The motion similarity calculated in the above manner has the physical meaning that the closer the predicted position is to the detected position, the higher the motion similarity, indicating that the target's motion trajectory is continuous and conforms to kinematic laws, meaning that the motion wrapped between adjacent frames has good coherence. Mahalanobis distance takes into account the uncertainty of the predicted position and can tolerate large distance deviations when the uncertainty is high, thus avoiding mismatches caused by inaccurate state estimation.

[0054] In step S105 of some embodiments, appearance features of the package to be associated can be obtained from the image to be detected, and appearance features of the candidate package to be associated can be obtained from the previous frame image. These two appearance features can be used to determine the similarity of appearance features between the package to be associated and the candidate package to be associated. The appearance features may include RGB features and point cloud features, and correspondingly, the appearance feature similarity may include RGB feature similarity and point cloud feature similarity.

[0055] In some embodiments, the method for determining the similarity of appearance features may include, but is not limited to, the following steps: The texture feature similarity is determined based on the texture features of the package to be associated and the texture features of the candidate associated packages. The point cloud feature similarity is determined based on the point cloud features of the package to be associated and the point cloud features of the candidate associated packages. Texture feature weights are determined based on velocity weights, and point cloud feature weights are determined based on texture feature weights. Texture feature weights are positively correlated with velocity weights, and the sum of texture feature weights and point cloud feature weights is 1. Based on the weights of texture features and point cloud features, the similarity of texture features and the similarity of point cloud features are weighted and fused to obtain the similarity of appearance features.

[0056] In this embodiment, texture features are RGB features, and texture feature similarity is RGB feature similarity. For RGB feature similarity, RGB features are first extracted from the packets to be associated. Specifically, this can be done based on the detection bounding boxes of the packets to be associated. The corresponding target RGB image region is cropped from the image to be detected. Then, the cropped image is scaled to a preset fixed input size (e.g., 256 pixels × 128 pixels) to obtain a scaled image. The scaled image is then fed into a Re-identification (ReID) convolutional neural network (such as OSNet, MGN, or ResNet architectures) for forward propagation to extract highly discriminative appearance feature vectors. (i.e., RGB feature vectors). Wherein, This represents the overall mapping function of the ReID feature extraction network, where D represents the dimension of the output feature vector (which can be set to 256 or 512 dimensions). After extracting the feature vector, to eliminate the influence of differences in feature magnitude on similarity calculation, the feature vector is normalized using the following L2 norm normalization process, so that the normalized feature vector has a magnitude of 1:

[0057] The same cropping, scaling, ReID network forward inference, and L2 normalization operations are performed on the candidate associated packages to obtain their corresponding RGB features. Then, as shown in Formula 9, the cosine similarity between the RGB features of the package to be associated and the candidate associated packages is calculated as the RGB similarity. (Formula 9) in, This represents the RGB feature vector of the package to be associated. This represents the RGB feature vector of the candidate associated wrapper. RGB similarity. The closer to 1, the more similar the package to be associated and the candidate associated package are in terms of appearance and semantics such as texture and color; the closer to -1, the greater the difference.

[0058] For point cloud feature similarity, the first step is to acquire 3D point cloud data corresponding to the package to be associated from multiple camera perspectives. Specifically, for each camera perspective, a depth image is acquired, and the point clouds from each perspective are transformed to a unified world coordinate system using camera intrinsic and extrinsic parameters to form a complete target scene point cloud. Then, a subset of the target point cloud is segmented from the scene point cloud based on the target detection box or 3D bounding box. Each point It includes 3D spatial coordinates (x, y, z) and optional additional feature information such as RGB color or normal vector. The segmented raw target point cloud undergoes preprocessing operations, including outlier removal (using statistical filtering algorithms to remove noise points deviating from the mean by more than a preset standard deviation by calculating the distance distribution between each point and its neighbors) and voxel downsampling (dividing the point cloud space into a fixed-size voxel grid, retaining only one representative point in each voxel grid, thus significantly reducing the amount of point cloud data while maintaining the geometric structure), to improve the efficiency and robustness of subsequent feature extraction. The preprocessed target point cloud is then input into a point cloud feature extraction network (such as PointNet++ or DGCNN, a deep learning-based point cloud processing architecture), and the network's forward computation extracts feature vectors representing the 3D geometric shape of the target. (i.e., point cloud feature vectors). Wherein, The function represents the overall mapping function of the point cloud feature extraction network, and M represents the dimension of the output morphological feature vector. As optional supplementary features, explicit geometric features can also be extracted from the point cloud, including the bounding box dimensions (L, W, H, i.e., the length, width, and height of the point cloud bounding box), the bounding volume V = L × W × H, and the flatness. These are used as auxiliary discriminative information in addition to deep features. The same multi-view point cloud acquisition, target point cloud segmentation, preprocessing, and point cloud network feature extraction operations are performed on candidate associated packages to obtain their point cloud feature vectors. Then, the Euclidean distance between the feature vectors of the packages to be associated and the candidate associated packages is calculated. This distance serves as a measure of difference in the feature space. The smaller the value, the more similar the package to be associated and the candidate associated packages are in terms of 3D morphology. Furthermore, as shown in Equation 10 below, the point cloud feature distance can be converted into a point cloud similarity score. : (Formula 10) in This is a preset scale parameter used to control the sensitivity of the distance-to-similarity mapping.

[0059] Alternatively, the point cloud similarity score can be determined using the linear truncation normalization method shown in Formula 11. : (Formula 11) in, The maximum feature distance threshold is used to map the point cloud similarity to a value range of 0 to 1. The closer the similarity is to 1, the more similar the geometric shapes of the two packages are.

[0060] In obtaining texture feature similarity Similarity to point cloud features Then, based on the speed weights Determine texture feature weights (i.e., RGB feature weights), and based on texture feature weights Determine point cloud feature weights Specifically, embodiments of this application further incorporate speed weighting. The physical basis for the sub-feature weight allocation applied within appearance features is as follows: Under rapid motion conditions, the imaging quality of 3D point clouds is more severely affected by motion distortion than that of RGB images. This is because point cloud acquisition relies on structured light or time-of-flight principles, and motion can cause distortion phenomena such as stretching, deformation, or uneven density in the point cloud. While RGB images also suffer from motion blur, they still retain certain texture and contour information. Therefore, the faster the speed, the more significantly the reliability of point cloud features decreases compared to RGB features, and the weight of point cloud features should be reduced accordingly, while the weight of RGB features should be increased. Based on the above logic, texture feature weights... The value can be directly taken as the speed weight. ,Right now Since the speed weight is positively correlated with speed, the texture feature weight is also positively correlated with the speed weight; the faster the speed, the higher the texture feature weight. Correspondingly, the point cloud feature weight... The value is then taken as ,Right now The sum of the texture feature weights and the point cloud feature weights is always equal to 1. In a more flexible implementation, the texture feature weights can also be determined using a nonlinear mapping function of the velocity weights to more accurately fit the physical characteristic that the point cloud quality decays faster with velocity. Regardless of whether a linear or nonlinear mapping is used, the basic constraints must be met: the texture feature weights are positively correlated with the velocity weights, and the sum of the texture feature weights and the point cloud feature weights is 1.

[0061] After determining the texture feature weights and point cloud feature weights, the texture feature similarity and point cloud feature similarity are weighted and fused based on these weights to obtain the appearance feature similarity. That is, the appearance feature similarity. It is a comprehensive appearance matching score that integrates RGB texture semantic information and point cloud geometric morphology information. Appearance feature similarity is used in subsequent steps to perform a secondary fusion with motion feature similarity to generate the final matching score. The specific fusion calculation method is as follows: .

[0062] Through the adaptive weight allocation within the aforementioned appearance features, RGB features and point cloud features are dynamically optimized and combined according to motion speed within the overall framework of appearance features, further improving the accuracy of appearance feature similarity expression and matching robustness under various motion conditions.

[0063] In step S106 of some embodiments, after determining the appearance feature weights and motion feature weights, the motion feature similarity and appearance feature similarity can be weighted and fused based on the motion feature weights and appearance feature weights to obtain an association score. Related scores This comprehensively reflects the overall matching degree between the packages to be associated and the candidate associated packages in terms of motion consistency and appearance consistency.

[0064] In step S107 of some embodiments, the association score is compared with a preset association threshold (e.g., 0.5, which is not limited). If the association score is greater than or equal to the association threshold, it is determined that the package to be associated (e.g., package D) and the candidate associated package (e.g., package A) have the same identity in the spatiotemporal domain and feature domain (i.e., they belong to the same package), thus satisfying the association condition. The association operation is then performed, that is, the tracking identity identifier (Track ID) already possessed by the candidate associated package is assigned to the current package to be associated. Conversely, if the association score is less than the preset association threshold, the current association is rejected, and the package to be associated is temporarily stored in a pending queue. After other packages in the current frame complete the association determination, a second review determination is performed based on the global matching results; or, the package to be associated is initialized as a potential newly emerging target and assigned a temporary tracking identifier, waiting for continuous observation and confirmation in subsequent consecutive frames; or, when the confidence level is low or ambiguous matching occurs, the package to be associated is pushed to the manual review interface, where operators remotely confirm or intervene based on the actual image. Through the above-mentioned association mechanism, continuous and robust identity binding of packages across frames within the field of view of a single camera is achieved, effectively ensuring that the one-to-one correspondence between the label information and the physical package remains unbroken during the logistics sorting process.

[0065] The package association method provided in this application addresses the problem of mismatches that easily occur in single-camera package tracking by proposing an adaptive feature fusion tracking strategy. Specifically, by calculating the package's movement speed and occlusion ratio in real time, the current working condition is accurately identified, providing a reliable basis for subsequent adaptive weight adjustment. At the feature extraction level, this application constructs a dual-channel metric framework for collaborative matching of motion features and appearance features. Motion features are measured based on Kalman filter prediction and Mahalanobis distance to measure trajectory continuity, while appearance features simultaneously extract RGB features based on the ReID network and 3D point cloud morphology features based on PointNet++, ensuring comprehensiveness and discriminative power in appearance discrimination. At the adaptive fusion level, this application establishes an inner and outer dual-layer dynamic weight allocation mechanism: the outer layer dynamically adjusts the fusion ratio of motion features and appearance features according to speed and occlusion, while the inner layer further performs secondary weight allocation between RGB features and point cloud features based on speed, fully considering the physical characteristics that point cloud distortion is more severe than RGB image distortion under rapid movement conditions. Ultimately, a matching decision is made by comparing the comprehensive correlation score with a preset threshold, supplemented by multi-level backup mechanisms such as temporary storage and review, new target initialization, or manual confirmation, thus constructing a robust decision-making system that combines automation and human intervention. Compared to traditional fixed-weight schemes, the embodiments of this application improve the accuracy of correlation matching in single-camera tracking scenarios.

[0066] The following examples illustrate various combinations of operating conditions.

[0067] Example 1: Normal movement + no obstruction.

[0068] Operating conditions: Packages to be sorted are moving at a normal speed with no obstructions.

[0069] Input parameter: Current speed m / s, velocity threshold m / s, occlusion data .

[0070] Weight Calculation: Speed ​​Weight Obscuring weight Weighted contribution: , After normalization, the motion feature weights Appearance feature weight Among them, RGB feature weights Point cloud feature weights .

[0071] Results: Appearance features dominated (87.5%), with point cloud features being the primary feature (75%) and RGB features being secondary (25%).

[0072] Example 2: Rapid movement + unobstructed view.

[0073] Operating conditions: Packages to be sorted are moving at a high speed, the image is blurry and unobstructed.

[0074] Input parameter: Current speed m / s, velocity threshold m / s, occlusion data .

[0075] Weight Calculation: Speed ​​Weight Obscuring weight Weighted contribution: , After normalization, the motion feature weights Appearance feature weight Among them, RGB feature weights Point cloud feature weights .

[0076] Results: Motion features and appearance features are balanced (50% each), with appearance features completely dependent on RGB features.

[0077] Example 3: Normal movement + severe occlusion.

[0078] Operating condition description: The packages to be sorted are moving at a normal speed, but are severely obstructed.

[0079] Input parameter: Current speed m / s, velocity threshold m / s, occlusion data .

[0080] Weight Calculation: Speed ​​Weight Obscuring weight Weighted contribution: , After normalization, the motion feature weights Appearance feature weight Among them, RGB feature weights Point cloud feature weights .

[0081] Results: Appearance features were slightly higher (57.5%), with point cloud features being the main feature (75%) and RGB features being secondary (25%).

[0082] Example 4: Rapid movement + severe occlusion.

[0083] Working condition description: The packages to be sorted are moving rapidly and are severely obscured.

[0084] Input parameter: Current speed m / s, velocity threshold m / s, occlusion data .

[0085] Weight Calculation: Speed ​​Weight Obscuring weight Weighted contribution: , After normalization, the motion feature weights Appearance feature weight Among them, RGB feature weights Point cloud feature weights .

[0086] Results: Motion features dominated (80%), appearance features were secondary (20%), and appearance features were completely dependent on RGB features.

[0087] Example 5: Rapid movement + complete occlusion.

[0088] Working condition description: The packages to be sorted are moving rapidly and are completely obscured.

[0089] Input parameter: Current speed m / s, velocity threshold m / s, occlusion data .

[0090] Weight Calculation: Speed ​​Weight Obscuring weight Weighted contribution: , After normalization, the motion feature weights Appearance feature weight Among them, RGB feature weights Point cloud feature weights .

[0091] Results: Motion features completely dominated (100%), while appearance features had a weight of 0.

[0092] Reference Figure 2 This application also provides a package association device for package tracking within a single camera. The device includes: The package detection unit 201 is used to acquire the image to be detected captured by the camera, and to detect the image to be detected to determine at least one package to be associated contained in the image to be detected. The preliminary association unit 202 is used to determine, for each package to be associated, a candidate associated package corresponding to the package to be associated from the tracked packages contained in the previous frame image corresponding to the image to be detected, captured by the camera. The weight determination unit 203 is used to determine the current transmission data of the package to be associated based on the positioning data of the package to be associated in the image to be detected and the positioning data of the candidate associated package in the previous frame image, and to dynamically determine the motion feature weight and appearance feature weight based on the current transmission data. The motion similarity determination unit 204 is used to predict the position of the candidate associated package in the image to be detected and obtain the predicted position, and to determine the motion feature similarity based on the position prediction deviation between the actual detected position and the predicted position of the package in the image to be detected. The appearance similarity determination unit 205 is used to obtain the appearance features of the package to be associated based on the image to be detected, obtain the appearance features of the candidate associated packages based on the previous frame image, and determine the appearance feature similarity based on the appearance features of the package to be associated and the appearance features of the candidate associated packages. The feature fusion unit 206 is used to perform weighted fusion of motion feature similarity and appearance feature similarity according to motion feature weight and appearance feature weight, so as to obtain the association score between the package to be associated and the corresponding candidate associated package. The target association unit 207 is used to associate the package to be associated with the candidate associated package if the association score is not less than a preset threshold.

[0093] It should be noted that the package association device provided in this application embodiment is used to implement the package association method provided in the above embodiment, and the specific implementation process corresponds to the package association method in the above embodiment. It can be referred to the aforementioned package association method, and will not be repeated here.

[0094] This application also provides an electronic device (i.e., a computer device), which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the package association methods described in the above embodiments. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0095] Reference Figure 3 , Figure 3 This illustration shows the hardware structure of an electronic device according to another embodiment, the electronic device comprising: The processor 301 can be implemented using a general-purpose central processing unit (CPU), graphics processing unit (GPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 302 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called and executed by the processor 301 using the package association method of the embodiments of this application. Input / output interface 303 is used to implement information input and output; The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304); The processor 301, memory 302, input / output interface 303 and communication interface 304 are connected to each other within the device via bus 305.

[0096] This application also provides a computer-readable storage medium storing a computer program for causing a computer to execute the package association method described in the above embodiments.

[0097] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] This invention also provides a computer program product that stores program instructions, which, when executed by a computer, cause the computer to implement the package association method described in any of the above embodiments.

[0099] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0100] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0103] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0104] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0106] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A package association method, characterized in that, The method is applied to package tracking within a single camera, and the method includes: Acquire the image to be detected captured by the camera, and detect the image to be detected to determine at least one package to be associated contained in the image to be detected; For each package to be associated, candidate associated packages corresponding to the package to be associated are determined from the tracked packages contained in the previous frame image captured by the camera corresponding to the image to be detected. Based on the location data of the package to be associated in the image to be detected, and the location data of the candidate associated package in the previous frame image, the current transmission data of the package to be associated is determined, and the motion feature weight and appearance feature weight are dynamically determined based on the current transmission data. The predicted position is obtained by predicting the position of the candidate associated wrapping in the image to be detected, and the motion feature similarity is determined based on the position prediction deviation between the actual detected position of the candidate associated wrapping in the image to be detected and the predicted position. The appearance features of the package to be associated are obtained from the image to be detected, the appearance features of the candidate associated package are obtained from the previous frame image, and the appearance feature similarity is determined between the appearance features of the package to be associated and the appearance features of the candidate associated package. Based on the motion feature weights and appearance feature weights, the motion feature similarity and appearance feature similarity are weighted and fused to obtain the association score between the package to be associated and the corresponding candidate associated package; If the association score is not less than a preset threshold, the package to be associated is associated with the candidate associated package.

2. The method according to claim 1, characterized in that, The currently transmitted data includes the current movement speed and occlusion data. The step of dynamically determining motion feature weights and appearance feature weights based on the current transmitted data includes: The speed weight is determined based on the ratio between the current speed and the preset speed threshold. The occlusion weights are determined based on the occlusion data, and the velocity weights and occlusion weights are summed and normalized to obtain the motion feature weights. The appearance feature weights are determined based on the motion feature weights, wherein the sum of the motion feature weights and the appearance feature weights is 1.

3. The method according to claim 2, characterized in that, The appearance features include texture features and point cloud features. Determining the appearance feature similarity based on the appearance features of the package to be associated and the appearance features of the candidate associated packages includes: The texture feature similarity is determined based on the texture features of the package to be associated and the texture features of the candidate associated packages; The point cloud feature similarity is determined based on the point cloud features of the package to be associated and the point cloud features of the candidate associated packages. Texture feature weights are determined based on the velocity weights, and point cloud feature weights are determined based on the texture feature weights, wherein the texture feature weights are positively correlated with the velocity weights, and the sum of the texture feature weights and the point cloud feature weights is 1; Based on the texture feature weights and the point cloud feature weights, the texture feature similarity and the point cloud feature similarity are weighted and fused to obtain the appearance feature similarity.

4. The method according to claim 2, characterized in that, The positioning data includes location data, and the method for determining the current movement speed includes: Based on the position data of the package to be associated in the image to be detected, and the position data of the candidate package to be associated in the previous frame image, calculate the position displacement between the package to be associated and the candidate package to be associated. The current motion speed is calculated based on the position displacement and the frame interval between the image to be detected and the previous frame image.

5. The method according to claim 2, characterized in that, The positioning data includes a detection box, and the method for determining the occlusion data includes: Determine the target detection area of ​​the package to be associated in the corresponding detection box in the image to be detected; The adjacent packages of the package to be associated are determined from the packages contained in the image to be detected, and the detection boxes corresponding to the adjacent packages in the image to be detected are determined, and the overlap area between the detection boxes corresponding to the package to be associated is determined. The occlusion data is calculated based on the overlapping area and the target detection area.

6. The method according to claim 1, characterized in that, The currently transmitted data includes the current movement speed and occlusion data. The step of dynamically determining the motion feature weights and appearance feature weights based on the currently transmitted data includes: Obtain reference appearance weight and reference motion weight, wherein the reference appearance weight is greater than the reference motion weight, and the reference appearance weight and the reference motion weight correspond to the baseline weights under the condition that the current motion speed is not greater than a preset speed threshold and the occlusion data is not greater than a preset occlusion threshold; The occlusion level is determined based on the occlusion data; If the current motion speed is greater than a preset speed threshold, the reference appearance weight is reduced according to the occlusion level, and the reference motion weight is increased accordingly to obtain motion feature weight and appearance feature weight; wherein, the motion feature weight is greater than or equal to the appearance feature weight.

7. The method according to claim 6, characterized in that, The step of dynamically determining motion feature weights and appearance feature weights based on the current transmitted data further includes: If the current motion speed is not greater than a preset speed threshold, the reference appearance weight is reduced according to the occlusion level, and the reference motion weight is increased accordingly to obtain motion feature weight and appearance feature weight; wherein, the motion feature weight is less than the appearance feature weight.

8. A package association device, characterized in that, The device is used for package tracking within a single camera, and the device includes: A package detection unit is used to acquire an image to be detected captured by a camera, and to detect the image to be detected to determine at least one package to be associated contained in the image to be detected. The preliminary association unit is used to determine, for each package to be associated, a candidate associated package corresponding to the package to be associated from the tracked packages contained in the previous frame image captured by the camera corresponding to the image to be detected; The weight determination unit is used to determine the current transmission data of the package to be associated based on the positioning data of the package to be associated in the image to be detected and the positioning data of the candidate associated package in the previous frame image, and to dynamically determine the motion feature weight and appearance feature weight based on the current transmission data. The motion similarity determination unit is used to predict the position of the candidate associated wrapping in the image to be detected to obtain the predicted position, and to determine the motion feature similarity based on the position prediction deviation between the actual detected position of the candidate associated wrapping in the image to be detected and the predicted position. The appearance similarity determination unit is used to obtain the appearance features of the package to be associated based on the image to be detected, obtain the appearance features of the candidate associated package based on the previous frame image, and determine the appearance feature similarity between the appearance features of the package to be associated and the appearance features of the candidate associated package. The feature fusion unit is used to perform weighted fusion of the motion feature similarity and the appearance feature similarity according to the motion feature weight and the appearance feature weight, so as to obtain the association score between the package to be associated and the corresponding candidate associated package. The target association unit is used to associate the package to be associated with the candidate associated package if the association score is not less than a preset threshold.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images