Motion blur barcode recognition method and system based on multi-frame image fusion

Abstract

The application provides a motion blur barcode recognition method and system based on multi-frame image fusion, which comprises the following steps: acquiring a continuous image sequence containing a motion blur barcode, extracting motion trajectory features of the continuous image sequence, obtaining a motion vector field and a pixel displacement trajectory set of the barcode region in each frame image unit; performing multi-frame image fusion on the continuous image sequence based on the motion vector field and the pixel displacement trajectory set, generating a candidate barcode image set; performing deblurring enhancement processing on the candidate barcode image set, obtaining a clear barcode image unit after deblurring processing; performing barcode region positioning and distortion correction processing on the clear barcode image unit, generating a standardized barcode image, and obtaining a recognition result. The application can significantly improve the accuracy and reliability of barcode recognition in a motion blur scene.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method and system for recognizing motion-blurred barcodes based on multi-frame image fusion. Background Technology

[0002] Barcode recognition is a technology that involves capturing and analyzing barcode images to decipher the encoded information they contain. Currently, barcode recognition typically processes single-frame barcode images, extracting the barcode region using algorithms such as edge detection and threshold segmentation, enhancing the blurred image with deblurring algorithms, and finally decoding and recognizing the code. However, in real-world applications, when the barcode is in motion, the captured image becomes motion-blurred. In such cases, deblurring based solely on a single frame is insufficient to accurately restore the barcode's sharp features, easily leading to barcode region positioning errors and inaccurate bar / space width measurements, resulting in decoding and recognition errors and a low barcode recognition rate in motion-blurred scenarios. Summary of the Invention

[0003] This invention provides a method and system for recognizing motion-blurred barcodes based on multi-frame image fusion.

[0004] In a first aspect, embodiments of the present invention provide a method for recognizing motion-blurred barcodes based on multi-frame image fusion. The method includes: acquiring a continuous image sequence containing a motion-blurred barcode, the continuous image sequence comprising multiple image units with time-series correlation; extracting motion trajectory features from the continuous image sequence to obtain a set of motion vector fields and pixel displacement trajectories for the barcode region in each frame image unit; performing multi-frame image fusion on the continuous image sequence based on the set of motion vector fields and pixel displacement trajectories to generate a set of candidate barcode images containing different degrees of motion compensation; performing deblurring enhancement processing on the set of candidate barcode images to obtain a clear barcode image unit after deblurring; performing barcode region localization and distortion correction processing on the clear barcode image unit to generate a standardized barcode image; and performing barcode information decoding and recognition based on the standardized barcode image to obtain a recognition result.

[0005] In a second aspect, embodiments of the present invention provide a computer system, comprising: a memory storing a computer program; and a processor for loading the computer program to implement the motion-blurred barcode recognition method based on multi-frame image fusion as described in the first aspect.

[0006] This invention provides a motion-blurred barcode recognition method based on multi-frame image fusion. By acquiring a continuous image sequence containing motion-blurred barcodes and constructing a time-series correlation, it can fully utilize the dynamic motion information between multiple frames, laying a data foundation for subsequent accurate capture of barcode motion features. By extracting motion trajectory features from the continuous image sequence to obtain a motion vector field and a set of pixel displacement trajectories, it achieves dual capture of the macroscopic trend and microscopic pixel-level trajectory of barcode motion, avoiding the limitations of describing motion with a single motion parameter and improving the comprehensiveness and accuracy of motion feature expression. Multi-frame image fusion based on the motion vector field and pixel displacement trajectory set generates a candidate barcode image set containing different degrees of motion compensation, effectively addressing the uncertainty of motion blur. The generation of multiple candidate results improves the robustness of subsequent processing and avoids the bias that may exist in a single fusion result. Deblurring and enhancement processing of the candidate barcode image set can fully utilize the advantageous features of each candidate image, improving the reliability of the deblurring effect and providing high-quality image input for subsequent decoding and recognition. By performing region localization and distortion correction on clear barcode image units and generating standardized barcode images, a complete processing flow from motion-blurred images to accurate decoding results was finally realized. The various links are closely related and work together, which significantly improves the accuracy and reliability of barcode recognition in motion-blurred scenes. Attached Figure Description

[0007] Figure 1 This is a flowchart of a motion-blurred barcode recognition method based on multi-frame image fusion provided by an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of the composition of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] Please see Figure 1 The flowchart below illustrates a motion-blurred barcode recognition method based on multi-frame image fusion, provided by an embodiment of the present invention. This method can be executed by a computer system and includes:

[0011] Step S100: Obtain a continuous image sequence containing a motion-blurred barcode, the continuous image sequence comprising multiple frame image units with time-series association.

[0012] A continuous image sequence is a series of images acquired sequentially over a period of time. These images are temporally related; each subsequent frame is acquired at a point in time after the previous frame. Multi-frame image units are the basic elements constituting a continuous image sequence, with each unit representing an image acquired at a specific moment. Acquiring a continuous image sequence containing motion-blurred barcodes can be done using equipment such as high-speed cameras, capturing continuous images as the barcode moves. For example, in a logistics warehouse, when goods with barcodes move on a conveyor belt, a high-speed camera can capture images at a certain frame rate, resulting in a continuous image sequence containing the motion-blurred barcode. The frame rate can be determined based on the barcode's movement speed and the required image resolution to ensure a sufficient number of image frames are captured.

[0013] Step S200: Extract motion trajectory features from the continuous image sequence to obtain the motion vector field and pixel displacement trajectory set of the barcode region in each frame image unit.

[0014] Motion trajectory feature extraction extracts feature information reflecting the motion of barcode regions from a continuous image sequence. The motion vector field is a two-dimensional vector field, where each vector represents the direction and distance of motion of the corresponding pixel in the barcode region. The pixel displacement trajectory set records the positional change trajectory of each pixel in the barcode region across consecutive image frames. Motion trajectory feature extraction from a continuous image sequence provides crucial foundational information for image fusion and deblurring. Specifically, by analyzing the motion vector field and pixel displacement trajectory set, the motion of the barcode region in different image frames can be understood, thereby enabling motion compensation and deblurring of the image.

[0015] As one implementation method, step S200 involves extracting motion trajectory features from the continuous image sequence to obtain the motion vector field and pixel displacement trajectory set of the barcode region in each frame image unit. Specifically, this may include the following steps S210~S250:

[0016] Step S210: Perform grayscale conversion processing on the continuous image sequence. Convert the color image unit into a grayscale image sequence by weighted averaging. The grayscale value of each pixel in the grayscale image sequence is generated by calculating the three-channel pixel values ​​of the original image unit according to preset weights, and maintains the same spatial resolution as the original image unit.

[0017] Grayscale conversion is the process of converting a color image to a grayscale image. A color image consists of pixel values ​​from three channels: red (R), green (G), and blue (B), while a grayscale image contains only the pixel values ​​from one channel, namely the grayscale value. The weighted average method calculates the grayscale value of each pixel by averaging the pixel values ​​of the three channels of the original image unit according to preset weights. The preset weights can be determined based on the sensitivity of the human eye to different colors; for example, the green channel has the highest weight, followed by the red channel, and the blue channel has the lowest. For example, the weights are set as follows: red channel weight 0.299, green channel weight 0.587, and blue channel weight 0.114. During grayscale conversion, for each pixel in the original image unit, its red, green, and blue channel pixel values ​​are multiplied by their respective weights, and the results are summed to obtain the grayscale value of that pixel. A grayscale image sequence is a series of grayscale images obtained after grayscale conversion, where the grayscale value of each pixel reflects the brightness information of the corresponding position in the original image unit. During grayscale conversion, the same spatial resolution as the original image unit is maintained to ensure that subsequent processing can accurately locate and analyze the barcode area.

[0018] Step S220: Corner detection processing is performed by calculating the eigenvalue ratio of the gray-level covariance matrix within the local window of the image, and corner feature sets in the barcode area of ​​each frame image unit with adjacent pixel gray-level differences exceeding a preset threshold are identified. The corner feature set includes a corner coordinate sequence with spatial distribution characteristics and the corresponding response intensity value.

[0019] Corner detection is the process of finding corners with significant features in an image. Corners are points in an image with large gray-level variations, located at the edges, corners, etc., of objects. Corners can be effectively detected by calculating the eigenvalue ratio of the gray-level covariance matrix within a local window of the image. The gray-level covariance matrix is ​​a 2x2 matrix that describes the variation of pixel gray-level values ​​within a local window of the image. The eigenvalue ratio is the ratio between two eigenvalues ​​of the gray-level covariance matrix. By comparing the eigenvalue ratio with a preset threshold, it can be determined whether a corner exists within that local window. The preset threshold is a pre-set value used to filter out corners with sufficiently significant gray-level variations. The corner feature set is a collection of corners obtained after corner detection processing, where each corner contains its coordinate information and corresponding response intensity value. The corner coordinate sequence records the position of the corner in the image, while the response intensity value reflects the salience of the corner.

[0020] As one implementation method, step S220 involves corner detection processing by calculating the eigenvalue ratio of the gray-level covariance matrix within a local window of the image, identifying a set of corner features in each frame image unit where the gray-level difference between adjacent pixels in the barcode region exceeds a preset threshold. Specifically, this may include the following steps S221~S225:

[0021] Step S221: Perform multi-scale Gaussian filtering on each frame image unit in the grayscale image sequence, and generate Gaussian blurred image sequences of different scale spaces by constructing a Gaussian pyramid. The Gaussian blurred image sequence contains image units with different blur levels from low to high, and the image size of each layer decreases according to a preset ratio.

[0022] Multi-scale Gaussian filtering is a process of smoothing an image at different scales. Gaussian filtering is a linear smoothing filter that smooths noise and details in an image by performing a convolution operation, resulting in a smoother image. Multi-scale Gaussian filtering performs Gaussian filtering at different scales to obtain feature information of the image at different scales. A Gaussian pyramid is a pyramid structure used to represent an image at different scales. By continuously downsampling and Gaussian filtering the image, a sequence of Gaussian blurred images at different scales is generated. In constructing a Gaussian pyramid, the original image is first Gaussian filtered to obtain the first layer of Gaussian blurred images. Then, the first layer of Gaussian blurred images is downsampled to obtain the second layer of Gaussian blurred images, which is half the size of the first layer. This process is repeated until the preset number of pyramid layers is reached. The Gaussian blurred image sequence is a series of Gaussian blurred images obtained after multi-scale Gaussian filtering, where the blur level of each image unit varies, gradually increasing from low to high. The size of each layer of images decreases by a preset ratio, for example, by a factor of 2, meaning that the size of each layer of images is half the size of the layer above.

[0023] Step S222: Calculate the difference images of adjacent scale images in the Gaussian blurred image sequence, generate a Laplacian pyramid in multi-scale space, and determine the scale space characteristics of the barcode region by analyzing the distribution of zero cross points in the difference images.

[0024] A difference image is an image showing the difference between Gaussian blurred images at adjacent scales. By calculating the difference image, regions in an image that exhibit significant changes at different scales can be highlighted. A Laplacian pyramid is a pyramid structure used to represent the detail information of an image at different scales, obtained by calculating the difference images of Gaussian blurred images at adjacent scales. When generating the Laplacian pyramid, for each layer of Gaussian blurred image, the difference image between it and the next layer is calculated to obtain the Laplacian image for that layer. This process is repeated until the top layer of the pyramid is reached. Zero-crossing points are points in the difference image where the gray value changes from positive to negative or from negative to positive. By analyzing the distribution of zero-crossing points in the difference image, the scale-space characteristics of the barcode region can be determined. Scale-space characteristics are the feature information of the barcode region at different scales, including shape, size, and position. By analyzing scale-space characteristics, barcode regions can be better located and identified.

[0025] Step S223: Calculate the gradient magnitude matrix and gradient direction matrix of the image at each scale layer of the Laplacian pyramid, and perform convolution operations in the horizontal and vertical directions using the direction operator to generate a gradient magnitude image reflecting the intensity of pixel grayscale changes and a gradient direction image reflecting the direction of change.

[0026] The gradient magnitude matrix is ​​a matrix of the same size as the image, where each element represents the intensity of the grayscale change at the corresponding pixel. The gradient direction matrix is ​​a matrix of the same size as the image, where each element represents the direction of the grayscale change at the corresponding pixel. Directional operators are operators used to calculate the image gradient; exemplary directional operators include the Sobel operator and the Prewitt operator. When performing convolution operations in the horizontal and vertical directions, the directional operators are convolved with the image to obtain the gradient components in the horizontal and vertical directions. Then, the gradient magnitude matrix is ​​obtained by calculating the square root of the sum of the squares of the horizontal and vertical gradient components. The gradient direction matrix is ​​obtained by calculating the arctangent values ​​of the horizontal and vertical gradient components. The gradient magnitude image is an image generated based on the gradient magnitude matrix, where the grayscale value of each pixel represents the intensity of the grayscale change at that point. The gradient direction image is an image generated based on the gradient direction matrix, where the grayscale value of each pixel represents the direction of the grayscale change at that point.

[0027] Step S224: Local extremum point detection is performed on the gradient magnitude image using an adaptive threshold non-maximum suppression algorithm. The suppression window size is dynamically adjusted to adapt to the feature density of different regions, and a set of candidate corner points with the largest local gray-level change is extracted.

[0028] The adaptive thresholding non-maximum suppression algorithm is used to detect local extrema in an image. Non-maximum suppression retains only the pixels with the largest local gray-level changes in the gradient magnitude image, while setting the gray-level values ​​of other pixels to zero. Adaptive thresholding dynamically adjusts the size of the suppression window based on the feature density of local regions to adapt to the feature distribution of different areas. During local extrema detection, a suppression window is first determined on the gradient magnitude image. Then, within this window, the gradient magnitude of each pixel is compared with the gradient magnitudes of surrounding pixels. If the gradient magnitude of a pixel is the largest within the window, it is retained; otherwise, its gray-level value is set to zero. By dynamically adjusting the size of the suppression window, a smaller suppression window can be used in regions with high feature density to improve corner detection accuracy, while a larger suppression window can be used in regions with low feature density to avoid missing corners. The candidate corner set is a collection of pixels with the largest local gray-level changes obtained after local extrema detection; these pixels may be corners in the image.

[0029] Step S225: Perform texture feature verification processing based on gray-level co-occurrence matrix on the candidate corner point set, calculate the texture energy and entropy parameters of the neighborhood of the candidate corner point, retain the corner points whose texture features conform to the barcode bar spacing distribution characteristics, and generate the final corner point feature set.

[0030] The gray-level co-occurrence matrix (GLCM) is a matrix used to describe the texture features of an image, reflecting the gray-level relationships between adjacent pixels. Texture energy and entropy parameters are two texture feature parameters calculated based on the GLCM. Texture energy represents the uniformity of the image texture, while entropy represents the complexity of the texture. During texture feature verification, for each corner point in the candidate corner point set, the GLCM of its neighborhood is calculated. Then, the texture energy and entropy parameters are calculated based on the GLCM. The bar-space distribution characteristics of a barcode refer to the distribution patterns of the width and spacing of the bars and spaces. By comparing the matching degree of the texture energy and entropy parameters of the candidate corner point's neighborhood with the bar-space distribution characteristics of the barcode, corner points whose texture features conform to the bar-space distribution characteristics are retained, generating the final corner feature set. The final corner feature set contains corner points with significant features in the barcode region and whose texture features conform to the bar-space distribution characteristics of the barcode.

[0031] Step S230: Perform feature matching processing based on fast nearest neighbor search on the corner feature set of adjacent frame image units. Accelerate the similarity search of feature descriptors by constructing a hierarchical index structure, establish the corner correspondence relationship across frames, and generate a set of corner matching pairs containing matching confidence scores. Each matching pair in the set of corner matching pairs contains the coordinate information and matching confidence of the corresponding corner points in the previous and next frames.

[0032] Feature matching is the process of finding matching corner points within the corner feature sets of adjacent frame image units. Fast nearest neighbor search (FRPS) is an algorithm used to accelerate feature descriptor similarity search by constructing a hierarchical index structure, reducing the search space and improving search efficiency. A feature descriptor is a vector used to describe corner features; exemplary feature descriptors include SIFT and SURF descriptors. During feature matching, a feature descriptor is first constructed for each corner point in the corner feature sets of adjacent frame image units. Then, a hierarchical index structure, such as a KD-tree or ball tree, is constructed to index the feature descriptors of the previous frame image unit. Next, the feature descriptor of the current frame image unit is input into the index structure for fast nearest neighbor search to find the most similar feature descriptor. Finally, a matching confidence score is calculated based on the similarity of the feature descriptors, establishing a cross-frame corner point correspondence. The corner point matching pair set is a collection of corner point matching pairs obtained after feature matching, where each matching pair contains the coordinate information and matching confidence of the corresponding corner points in the preceding and following frames. The match confidence score indicates the credibility of the match; the higher the score, the more credible the match.

[0033] As one implementation method, step S230 involves performing feature matching processing based on fast nearest neighbor search on the corner feature sets of adjacent frame image units. This is achieved by constructing a hierarchical index structure to accelerate the similarity search of feature descriptors, establishing cross-frame corner correspondences, and generating a set of corner matching pairs containing matching confidence scores. Specifically, this may include the following steps S231-S235:

[0034] Step S231: Construct a high-dimensional feature descriptor based on the gradient orientation histogram for the corner feature set of the current frame image unit, and generate a rotation-invariant feature vector by calculating the statistical features of the gradient orientation histogram of the corner neighborhood.

[0035] A high-dimensional feature descriptor based on histogram of gradient orientations (HOC) is a feature vector used to describe corner features. It is generated by calculating the statistical features of the histogram of gradient orientations in the corner neighborhood. The histogram of gradient orientations describes the distribution of gradient directions in a local region of an image. It divides the gradient directions of pixels in the corner neighborhood into multiple intervals, and the sum of the gradient magnitudes in each interval is calculated to obtain the histogram of gradient orientations. When constructing the feature descriptor, the size of the corner neighborhood is first determined, for example, as a square region. Then, the gradient magnitude and gradient direction of each pixel are calculated within this neighborhood. Next, the gradient direction is divided into multiple intervals, and the sum of the gradient magnitudes in each interval is calculated to obtain the histogram of gradient orientations. Finally, the histogram of gradient orientations is normalized to obtain a rotation-invariant feature vector. Rotation invariance is the property of a feature vector to remain unchanged when the image is rotated. By constructing rotation-invariant feature vectors, the accuracy of feature matching can be improved.

[0036] Step S232: Perform the same feature descriptor construction process on the previous frame image unit to generate the previous frame feature descriptor set, ensuring that it has the same dimension and scale as the current frame feature descriptor.

[0037] To enable effective feature matching between corner feature sets of adjacent frame image units, it is necessary to ensure that the feature descriptor set of the previous frame has the same dimension and scale as the feature descriptor set of the current frame. Therefore, the same feature descriptor construction process is performed on the previous frame image unit as on the current frame image unit. Specifically, first, the corner feature set of the previous frame image unit is determined, and then a high-dimensional feature descriptor based on the gradient direction histogram is constructed for each corner. During the construction process, the same parameters as the current frame image unit, such as the corner neighborhood size and gradient direction interval partitioning, are used to ensure that the generated previous frame feature descriptor set has the same dimension and scale as the current frame feature descriptor set.

[0038] Step S233: Construct a hierarchical nearest neighbor search index structure. Use a random space partitioning algorithm to perform multi-dimensional space partitioning on the feature descriptor subset of the previous frame, and generate a hierarchical data structure that supports fast nearest neighbor search.

[0039] Randomized space partitioning algorithms are used to partition multidimensional spaces. Exemplary randomized space partitioning algorithms include KD-tree algorithms and ball tree algorithms. When constructing a hierarchical nearest neighbor search index structure, each feature descriptor in the previous frame's feature descriptor set is first considered as a point in a multidimensional space. Then, a randomized space partitioning algorithm is used to partition the multidimensional space into multiple subspaces. Within each subspace, the randomized space partitioning algorithm is continued until a preset partitioning condition is met. Finally, a hierarchical data structure, such as a KD-tree or ball tree, is generated, which supports fast nearest neighbor search. During fast nearest neighbor search, this hierarchical data structure allows for rapid location of the previous frame feature descriptor most similar to the current frame's feature descriptor, thereby improving search efficiency.

[0040] Step S234: Input the current frame feature descriptor into the nearest neighbor search structure, perform multi-nearest neighbor search to obtain multiple nearest neighbor matching results for each descriptor, and evaluate the matching confidence by calculating the nearest neighbor distance ratio.

[0041] Multiple nearest neighbor search (MNN) is the process of finding multiple previous frame feature descriptors that are most similar to the current frame's feature descriptor within a nearest neighbor search structure. During MNN, the current frame's feature descriptor is input into a hierarchical nearest neighbor search index structure, which quickly locates multiple previous frame feature descriptors that are most similar to the current frame's feature descriptor. The nearest neighbor distance ratio is the ratio of the distance between the current frame's feature descriptor and its nearest previous frame feature descriptor to the distance between its second nearest previous frame feature descriptor. By calculating the nearest neighbor distance ratio, the matching confidence between the current frame's feature descriptor and its nearest previous frame feature descriptor can be evaluated. A smaller nearest neighbor distance ratio indicates a higher matching confidence; conversely, a larger ratio indicates a lower matching confidence. Calculating the nearest neighbor distance ratio allows for a matching confidence assessment of each matching result, providing a basis for matching refinement.

[0042] Step S235: Perform matching purification processing through distance ratio threshold filtering and random sampling consistency algorithm to eliminate incorrect matching pairs, retain correct matching pairs that satisfy the geometric constraints of the basic matrix, and generate a set of corner matching pairs containing matching confidence scores.

[0043] Distance ratio thresholding filters set a threshold based on the nearest neighbor distance ratio, discarding matching pairs with a ratio greater than the threshold and retaining only those with a ratio less than the threshold. Random Sample Consensus (RANSAC) is an iterative algorithm for estimating mathematical model parameters. It selects a subset of matching pairs from the set through random sampling, calculates the fundamental matrix, and validates the remaining matching pairs against the fundamental matrix. The fundamental matrix describes the geometric relationship between two images and can be calculated from corresponding points in the matching pair set. In the match purification process, the matching pair set is first filtered based on the distance ratio threshold, eliminating incorrect matching pairs. Then, RANSAC is used to process the filtered set of matching pairs, iterating multiple times to calculate the fundamental matrix and validate the matching pairs against it. Finally, correct matching pairs that satisfy the geometric constraints of the fundamental matrix are retained, generating a set of corner matching pairs containing matching confidence scores.

[0044] Step S240: Based on the set of corner matching pairs, calculate the dense motion vector field between adjacent frame image units using the optical flow estimation algorithm. By performing bilinear interpolation on the sparse corner motion vectors, generate a dense motion vector field covering the entire barcode area. The direction of each vector in the motion vector field represents the pixel movement direction, and the vector magnitude represents the pixel movement distance.

[0045] Optical flow estimation algorithms are used to calculate the motion velocity of pixels in an image. Exemplary optical flow estimation algorithms include the Lucas-Kanade algorithm and the Horn-Schunck algorithm. When calculating the dense motion vector field between adjacent frame image units based on a set of corner matching pairs, the displacement vectors of corresponding corner points in the preceding and following frames are first calculated for each matching pair, resulting in sparse corner motion vectors. Then, the optical flow estimation algorithm is used to process the sparse corner motion vectors to calculate the dense motion vector field between adjacent frame image units. Bilinear interpolation is a method for interpolating sparse data. It calculates the value of unknown points by weighted averaging the values ​​of four surrounding known points. When generating a dense motion vector field covering the entire barcode area, bilinear interpolation is performed on the sparse corner motion vectors to extend them to the entire barcode area. The motion vector field is a two-dimensional vector field, where the direction of each vector represents the pixel movement direction, and the vector magnitude represents the pixel movement distance. By calculating the dense motion vector field between adjacent frame image units, we can understand the motion of the barcode region between adjacent frames, providing basic information for image fusion and deblurring.

[0046] Step S250: Perform trajectory tracking processing on the pixels of each frame image unit based on filtering prediction according to the dense motion vector field. Optimize the pixel position estimation through the prediction-update iteration process to generate a set of pixel displacement trajectories containing timestamps. The position change record of a pixel in the barcode area of ​​each trajectory in the set of pixel displacement trajectories and the trajectory confidence parameter in consecutive frames are recorded.

[0047] Filter-based prediction-based trajectory tracking is a method for tracking the motion trajectories of pixels in an image. Exemplary filtering prediction methods include Kalman filtering and extended Kalman filtering. During trajectory tracking, the displacement vector of each pixel between adjacent frames is first determined based on a dense motion vector field. Then, a filtering prediction method is used to predict the pixel's position, obtaining an estimated pixel position for the next time step. Next, the predicted value is updated based on the actual acquired image data to obtain a more accurate pixel position estimate. This prediction-update iterative process continuously optimizes the pixel position estimate. A timestamp is added to each pixel's position change record to indicate the corresponding time. The pixel displacement trajectory set is a collection of pixel displacement trajectories obtained after trajectory tracking, where each trajectory corresponds to the position change record of a pixel in the barcode area across consecutive frames, along with a trajectory confidence parameter. The trajectory confidence parameter represents the reliability of the trajectory; a higher confidence level indicates a more reliable trajectory.

[0048] Step S300: Perform multi-frame image fusion on the continuous image sequence based on the motion vector field and pixel displacement trajectory set to generate a candidate barcode image set containing different degrees of motion compensation.

[0049] Multi-frame image fusion is the process of fusing multiple frames from a continuous image sequence to obtain a clearer and more accurate barcode image. Motion vector fields and pixel displacement trajectory sets provide motion information about the barcode region in different frames. Based on this information, motion compensation processing can be performed on the continuous image sequence to reduce the impact of motion blur. In multi-frame image fusion, motion compensation processing is first performed on each frame's image units according to the motion vector field and pixel displacement trajectory set, mapping the barcode region pixels of different frames to a reference coordinate system. Then, different fusion methods are used to fuse the motion-compensated image units, resulting in a set of candidate barcode images containing different degrees of motion compensation. Different fusion methods can be selected according to specific needs; exemplary fusion methods include weighted averaging and wavelet transform.

[0050] As one implementation method, step S300 involves performing multi-frame image fusion on a continuous image sequence based on the motion vector field and pixel displacement trajectory set to generate a candidate barcode image set containing different degrees of motion compensation. Specifically, this may include the following steps S310~S350:

[0051] Step S310: Perform density-based trajectory clustering on the set of pixel displacement trajectories. Measure the trajectory similarity by calculating the dynamic time warping distance between trajectories, and divide trajectories with similar directional and velocity features into multiple trajectory clusters.

[0052] Density-based trajectory clustering is a method for clustering trajectory data. Exemplary clustering algorithms include DBSCAN and OPTICS. Dynamic time warping distance (DTDM) is a distance metric used to measure the similarity between two trajectories. It uses dynamic programming to find the optimal matching path between two trajectories, thereby calculating their distance. When performing density-based trajectory clustering on a set of pixel displacement trajectories, the direction and velocity features of each trajectory are first calculated to generate a trajectory feature vector. Then, DTDM is used to measure the similarity between trajectories, dividing trajectories with similar direction and velocity features into multiple trajectory clusters. A trajectory cluster is a set of trajectories with similar features; the trajectories in each cluster have similar direction and velocity features.

[0053] As one implementation method, step S310 involves performing density-based trajectory clustering on the set of pixel displacement trajectories. By calculating the dynamic time warp distance between trajectories to measure trajectory similarity, trajectories with similar directional and velocity characteristics are divided into multiple trajectory clusters. Specifically, this may include the following steps S311~S315:

[0054] Step S311: Extract the direction and velocity features of each trajectory in the pixel displacement trajectory set, and generate a trajectory feature vector. The trajectory feature vector contains the average direction angle and average velocity value of the trajectory.

[0055] Directional features represent the direction of motion of a trajectory, such as by the average direction angle. Velocity features represent the speed of motion of a trajectory, such as by the average velocity value. When extracting the directional and velocity features of each trajectory in a set of pixel displacement trajectories, the starting and ending points of the trajectory are first determined, and then the displacement vector of the trajectory is calculated. The average direction angle of the trajectory is obtained by calculating the direction angle of the displacement vector. The average velocity value of the trajectory is obtained by calculating the ratio of the magnitude of the displacement vector to the trajectory time. The average direction angle and average velocity value of the trajectory are combined into a trajectory feature vector for trajectory clustering processing.

[0056] Step S312: Perform cluster analysis on the trajectory feature vectors using density clustering algorithm, determine the reachability relationship between trajectories by setting the neighborhood radius parameter, divide density-connected trajectories into the same trajectory cluster, and generate an initial trajectory cluster set.

[0057] Density clustering algorithms are clustering algorithms based on the density of data points. Exemplary density clustering algorithms include DBSCAN and OPTICS. When performing clustering analysis on trajectory feature vectors using density clustering algorithms, a neighborhood radius parameter is first set. This parameter is used to determine the reachability relationship between trajectories. If the distance between two trajectories is less than the neighborhood radius, these two trajectories are considered reachable. Then, based on the reachability relationship, density-connected trajectories are grouped into the same trajectory cluster. Density connection means that there is a path consisting of reachable trajectories between two trajectories. By continuously merging density-connected trajectories, an initial set of trajectory clusters is finally generated.

[0058] Step S313: Calculate the intra-cluster dispersion index of each initial trajectory cluster. By calculating the average distance between all trajectory feature vectors within the cluster and the cluster center, the degree of concentration of the trajectory feature vectors within the cluster is reflected.

[0059] Intra-cluster dispersion is an index used to measure the degree of concentration of data points within a cluster, reflecting the dispersion of data points from the cluster center. When calculating the intra-cluster dispersion index for each initial trajectory cluster, the cluster center of each initial trajectory cluster is first determined, for example, by using the average value of all trajectory feature vectors within the cluster as the cluster center. Then, the distances between all trajectory feature vectors within the cluster and the cluster center are calculated. These distances are summed and divided by the number of trajectories to obtain the average distance between all trajectory feature vectors within the cluster and the cluster center. This average distance is the intra-cluster dispersion index; the smaller the index value, the more concentrated the distribution of trajectory feature vectors within the cluster.

[0060] Step S314: Merge adjacent trajectory clusters with intra-cluster dispersion indices below a preset threshold, determine merging priority by calculating the inter-cluster distance matrix, generate an intermediate trajectory cluster set, and reduce the number of trajectory clusters.

[0061] The preset threshold is a pre-defined value used to filter out trajectory clusters with lower intra-cluster dispersion indices. The inter-cluster distance matrix is ​​a matrix where each element represents the distance between two trajectory clusters. Exemplary inter-cluster distance calculation methods include the single-link method, the full-link method, and the average-link method. When merging adjacent trajectory clusters with intra-cluster dispersion indices below the preset threshold, the intra-cluster dispersion indices of each initial trajectory cluster are first calculated, and trajectory clusters with intra-cluster dispersion indices below the preset threshold are filtered out. Then, the inter-cluster distance matrix between these trajectory clusters is calculated, and the merging priority is determined based on the inter-cluster distance matrix. Trajectory clusters that are closer in distance are merged first until the preset merging conditions are met. Finally, an intermediate trajectory cluster set is generated, which contains fewer trajectory clusters than the initial trajectory cluster set.

[0062] Step S315: Calculate the representative trajectory of each trajectory cluster in the intermediate trajectory cluster set. Generate the representative trajectory by performing time-aligned averaging on all trajectories in the cluster. This representative trajectory is used to describe the overall motion characteristics of the cluster.

[0063] Time-aligned averaging aligns all trajectories within a cluster in time, then averages the trajectory positions at each time point to obtain a representative trajectory. When calculating the representative trajectory for each trajectory cluster in the intermediate trajectory cluster set, all trajectories within the cluster are first time-aligned to ensure they have corresponding position records at the same time point. Then, the trajectory positions at each time point are averaged to obtain the representative trajectory. The representative trajectory is a trajectory that describes the overall motion characteristics of the cluster, reflecting the average motion of all trajectories within the cluster.

[0064] Step S320: Construct a motion compensation model for multi-frame images based on trajectory clusters and motion vector fields. By performing curve fitting on the representative trajectory of each trajectory cluster, establish a mathematical model of the position change of the trajectory in the time dimension.

[0065] Curve fitting is a process used to find mathematical relationships between data points. Exemplary curve fitting methods include polynomial fitting, exponential fitting, and logarithmic fitting. When constructing a motion compensation model for multi-frame images based on trajectory clusters and motion vector fields, the representative trajectory of each trajectory cluster is first determined. Then, curve fitting is performed on the representative trajectory to establish a mathematical model of the trajectory's positional changes over time. This mathematical model can describe the positional changes of the trajectory at different points in time, providing a basis for motion compensation processing.

[0066] As one implementation method, step S320, which involves constructing a motion compensation model for multiple frames of images based on trajectory clusters and motion vector fields, may specifically include the following steps S321~S325:

[0067] Step S321: Construct the basic framework of the motion compensation model. Perform polynomial curve fitting on the representative trajectory of each trajectory cluster, solve the fitting coefficients by the least squares method, and establish the mathematical expression of the trajectory. The mathematical expression reflects the position change law of the trajectory in the time dimension and serves as the core parameter of the motion compensation model.

[0068] Polynomial curve fitting uses polynomial functions to fit data points. The least squares method is used to solve for the polynomial fitting coefficients. By minimizing the sum of squared errors between the data points and the fitted curve, the optimal fitting coefficients are obtained. In constructing the basic framework of the motion compensation model, polynomial curve fitting is first performed on the representative trajectory of each trajectory family, selecting an appropriate polynomial order, such as a first-order polynomial or a second-order polynomial. Then, the least squares method is used to solve for the fitting coefficients, obtaining the mathematical expression of the trajectory. This mathematical expression reflects the positional change pattern of the trajectory in the time dimension and serves as the core parameter of the motion compensation model.

[0069] Step S322: In the motion compensation model, calculate the predicted position values ​​of each pixel in the trajectory cluster at different time points based on mathematical expressions, and generate a position prediction matrix. The rows of the position prediction matrix represent pixel indices, and the columns represent time points, thus forming the position prediction layer of the motion compensation model.

[0070] The position prediction matrix is ​​a two-dimensional matrix where each row represents the predicted position value of a pixel, and each column represents a time point. In the motion compensation model, when calculating the predicted position values ​​of each pixel in a trajectory cluster at different time points based on mathematical expressions, the initial position of each pixel in the trajectory cluster is first determined. Then, the predicted position value of that pixel at different time points is calculated according to the mathematical expression. The predicted position values ​​of all pixels at different time points are combined into the position prediction matrix, which constitutes the position prediction layer of the motion compensation model.

[0071] Step S323: Calculate the inverse motion compensation parameters in the motion compensation model based on the position prediction matrix and motion vector field. Map the future frame pixels to the reference frame coordinates through coordinate transformation. The inverse motion compensation parameters include displacement compensation amount and rotation compensation angle, forming the parameter calculation layer of the motion compensation model.

[0072] Coordinate transformation is a method used to transform a point in one coordinate system to another. Exemplary coordinate transformation methods include translation, rotation, and scaling. In the motion compensation model, when calculating inverse motion compensation parameters based on the position prediction matrix and motion vector field, the predicted position value of the pixel in the future frame is first determined based on the position prediction matrix, and then the motion vector of that pixel is determined based on the motion vector field. The future frame pixel is mapped to the reference frame coordinates through coordinate transformation, and the displacement compensation amount and rotation compensation angle are calculated. The displacement compensation amount represents the displacement distance of the pixel in the reference frame, and the rotation compensation angle represents the rotation angle of the pixel in the reference frame. The inverse motion compensation parameters, including the displacement compensation amount and rotation compensation angle, form the parameter calculation layer of the motion compensation model.

[0073] Step S324: Construct a motion compensation lookup table in the motion compensation model that contains inverse motion compensation parameters for all trajectory clusters. Each entry in the motion compensation lookup table corresponds to a set of compensation parameters for a trajectory cluster, serving as the parameter storage layer of the motion compensation model.

[0074] The motion compensation lookup table is a data structure used to store the inverse motion compensation parameters for all trajectory clusters. When constructing the motion compensation lookup table containing the inverse motion compensation parameters for all trajectory clusters in the motion compensation model, the inverse motion compensation parameters for each trajectory cluster are first calculated, and then these parameters are stored in the motion compensation lookup table. Each entry in the motion compensation lookup table corresponds to a set of compensation parameters for a trajectory cluster, which contains the displacement compensation amount and rotation compensation angle for all pixels in that trajectory cluster. The motion compensation lookup table serves as the parameter storage layer for the motion compensation model, providing a tool for quickly looking up inverse motion compensation parameters during motion compensation processing.

[0075] Step S325: Perform bicubic interpolation on the motion compensation lookup table of the motion compensation model to generate a continuous motion compensation parameter space, so that all pixels in the image can obtain the corresponding compensation parameters, thus completing the construction of the motion compensation model.

[0076] Bicubic interpolation is a process used to interpolate discrete data. It calculates the value of an unknown point by weighted averaging the values ​​of 16 surrounding known points. When performing bicubic interpolation on the motion compensation lookup table of a motion compensation model, the discrete compensation parameters in the lookup table are first treated as discrete data points. Then, bicubic interpolation is used to interpolate these discrete data points, generating a continuous space of motion compensation parameters. In this space, all pixels in the image can obtain their corresponding compensation parameters. Through bicubic interpolation, the motion compensation model is constructed, providing accurate compensation parameters for motion compensation processing.

[0077] Step S330: Perform inverse motion compensation processing on each frame image unit according to the motion compensation model, and map the barcode region pixels of different frames to the reference coordinate system through coordinate transformation to generate a motion-compensated image unit sequence.

[0078] Inverse motion compensation (IMC) is a process that maps the pixels of barcode regions in different frames to a reference coordinate system based on the IMC parameters in the motion compensation model, thereby eliminating the effects of motion blur. When performing IMC on each frame image unit according to the motion compensation model, the motion compensation lookup table in the model is first read to obtain the IMC parameters corresponding to each trajectory cluster. Then, the trajectory cluster affiliation of each pixel in the current frame image unit is determined, and the corresponding IMC parameters are obtained. Next, the pixel coordinates are transformed based on the IMC parameters, mapping them to the reference coordinate system. Finally, the pixel values ​​at the corresponding coordinates in the reference frame image unit are sampled using a bilinear interpolation algorithm to generate motion-compensated pixel values. The motion-compensated pixel values ​​of all pixels are rearranged to generate motion-compensated image units. This process is repeated for all frame image units to obtain a sequence of motion-compensated image units.

[0079] As one implementation method, step S330 involves performing inverse motion compensation processing on each frame's image unit according to the motion compensation model, mapping the barcode region pixels of different frames to the reference coordinate system through coordinate transformation, and generating a motion-compensated image unit sequence. Specifically, this may include the following steps S331~S335:

[0080] Step S331: Read the motion compensation lookup table in the motion compensation model and obtain the inverse motion compensation parameters corresponding to each trajectory cluster.

[0081] The motion compensation lookup table is a data structure used in the motion compensation model to store inverse motion compensation parameters. When reading the motion compensation lookup table in the motion compensation model, the storage location and data format of the lookup table are first determined, and then the data in the lookup table is read using the appropriate reading method. Each entry in the motion compensation lookup table corresponds to a set of compensation parameters for a trajectory cluster, which contains the displacement compensation amount and rotation compensation angle of all pixels in that trajectory cluster. By reading the motion compensation lookup table, the inverse motion compensation parameters corresponding to each trajectory cluster are obtained, providing a basis for motion compensation processing.

[0082] Step S332: For each pixel in the current frame image unit, determine the trajectory cluster affiliation. By calculating the distance between the pixel coordinates and the trajectory represented by each trajectory cluster, determine the trajectory cluster to which the pixel belongs and obtain the corresponding inverse motion compensation parameters.

[0083] Trajectory cluster attribution determination is the process of identifying the trajectory cluster to which each pixel in the current frame image unit belongs. When determining the trajectory cluster attribution for each pixel in the current frame image unit, the distance between the pixel's coordinates and the trajectories represented by each trajectory cluster is first calculated. Exemplary distance calculation methods include Euclidean distance and Manhattan distance. Then, the closest trajectory cluster is selected as the trajectory cluster to which the pixel belongs, and the corresponding inverse motion compensation parameters are obtained. Through trajectory cluster attribution determination, corresponding inverse motion compensation parameters are assigned to each pixel to ensure the accuracy of motion compensation processing.

[0084] Step S333: Perform an affine transformation on the pixel based on the inverse motion compensation parameters, and calculate the corresponding position coordinates of the pixel in the reference frame image unit through a combination of rotation, translation and scaling transformations.

[0085] Affine transformation is a method for performing geometric transformations on images. It transforms a point in one coordinate system to another through a combination of rotation, translation, and scaling. When performing an affine transformation on a pixel based on inverse motion compensation parameters, the pixel is first rotated according to the rotation compensation angle in the inverse motion compensation parameters, then translated according to the displacement compensation amount, and finally scaled as needed. Through this combination of rotation, translation, and scaling, the corresponding coordinates of the pixel in the reference frame image unit are calculated. Affine transformation can effectively map barcode region pixels from different frames to a reference coordinate system, eliminating the effects of motion blur.

[0086] Step S334: Sample the pixel values ​​at corresponding position coordinates in the reference frame image unit using a bilinear interpolation algorithm to generate motion-compensated pixel values.

[0087] Bilinear interpolation is a method for image interpolation. It calculates the value of an unknown point by weighted averaging the values ​​of four surrounding known points. When sampling pixel values ​​at corresponding coordinates in a reference frame image unit using bilinear interpolation, four known pixels surrounding the corresponding coordinate are first identified. Then, weighting coefficients are calculated based on the distances between the corresponding coordinate and these four known pixels. Finally, the pixel values ​​of these four known pixels are weighted and averaged according to the weighting coefficients to generate the motion-compensated pixel value. Bilinear interpolation can effectively improve the quality of motion-compensated images and reduce interpolation errors.

[0088] Step S335: Rearrange the motion-compensated pixel values ​​of all pixels to generate motion-compensated image units. Repeat the above process to process all frame image units to obtain a sequence of motion-compensated image units.

[0089] When rearranging the pixel values ​​after motion compensation, the pixels are arranged according to their original position order in the image to ensure that the generated motion-compensated image units have the correct spatial structure. This process is repeated for all frame image units, including trajectory cluster assignment, affine transformation, and bilinear interpolation, ultimately yielding a sequence of motion-compensated image units. These image units undergo inverse motion compensation, eliminating motion blur and providing clearer image data for image fusion and deblurring.

[0090] Step S340: Perform multi-scale fusion processing on the motion-compensated image unit sequence, and perform weighted averaging on the pixel values ​​at the same spatial location using different weight coefficients to generate a multi-scale fused image set.

[0091] Multi-scale fusion processing is a method for fusing multiple images at different scales to preserve the image's hierarchical features. When performing multi-scale fusion processing on a motion-compensated image unit sequence, weighting coefficients for different scales are first determined; these coefficients can be set according to specific needs. Then, a weighted average is performed on the pixel values ​​at the same spatial location in the motion-compensated image unit sequence. That is, at each scale, the pixel value at the corresponding location is multiplied by the corresponding weighting coefficient, and the results are summed to obtain the fused pixel value at that location. By performing weighted averaging on the pixel values ​​at all spatial locations, a multi-scale fused image set is generated. Each image in the multi-scale fused image set retains the feature information of the motion-compensated image unit sequence at different scales, providing richer image data for sharpness assessment.

[0092] Step S350: Perform sharpness assessment processing on the multi-scale fused image set, evaluate the image quality by calculating the image gradient energy and entropy parameters, retain the fused images whose sharpness index exceeds the preset threshold, and generate a candidate barcode image set containing different degrees of motion compensation.

[0093] Sharpness assessment is a method used to evaluate image sharpness by calculating the image's gradient energy and entropy parameters. Image gradient energy represents the intensity of pixel grayscale changes in an image; a larger gradient energy indicates sharper edges. Image entropy represents the image's information entropy; a larger entropy value indicates richer image information. When performing sharpness assessment on a multi-scale fused image set, the gradient energy and entropy parameters of each image in the set are first calculated. Then, fused images with sharpness indicators exceeding a preset threshold are selected, generating a candidate barcode image set containing different degrees of motion compensation. Images in the candidate barcode image set have high sharpness, providing more suitable image data for deblurring and enhancement processing.

[0094] Step S400: Perform deblurring enhancement processing on the candidate barcode image set to obtain clear barcode image units after deblurring.

[0095] Deblurring and enhancement processing is a method used to eliminate image blur and enhance image sharpness. It improves image quality by performing deblurring and enhancement processing on a candidate barcode image set. When performing deblurring and enhancement processing on a candidate barcode image set, a blur kernel estimation model is first constructed. A blur kernel estimation process is then performed on each candidate image in the set, generating a set of blur kernel functions. Next, blind deconvolution processing is performed on each candidate barcode image based on the set of blur kernel functions, generating a preliminary deblurred image set. Then, edge enhancement processing is applied to the preliminary deblurred image set to highlight the bar and space boundaries of the barcode. Next, unsharpening mask processing is applied to the preliminary deblurred images based on the edge feature images to enhance the high-frequency components of the image. Finally, the sharpness of the enhanced deblurred image set is evaluated, and the image with the highest overall evaluation is selected as the sharp barcode image unit after deblurring.

[0096] In one implementation, step S400 involves performing deblurring and enhancement processing on the candidate barcode image set to obtain a clear barcode image unit after deblurring. Specifically, this may include the following steps S410~S450:

[0097] Step S410: Construct a fuzzy kernel estimation model. By training a neural network, perform fuzzy kernel estimation processing on each candidate image in the candidate barcode image set to generate a fuzzy kernel function set. The fuzzy kernel function set contains the point spread function corresponding to different candidate images.

[0098] A blur kernel estimation model is used to estimate the blur kernel of an image. It learns the features of the blur kernel by training a neural network. To construct the blur kernel estimation model, a training dataset is first constructed by obtaining a sample set of barcode images containing different degrees of blur. Then, a deep convolutional neural network (CNN) structure is built as the basic framework of the blur kernel estimation model. This neural network structure includes a feature extraction layer, a multi-scale fusion layer, and a kernel parameter prediction layer, with each layer connected by a non-linear activation function. Next, the CNN structure is trained using the training dataset, and the network weight parameters are optimized using the backpropagation algorithm to minimize the loss function of the blur kernel prediction error. Finally, a set of candidate barcode images is input into the trained blur kernel estimation model to obtain the blur kernel parameter set corresponding to each candidate image. Based on these parameters, a corresponding point spread function is constructed to generate a set of blur kernel functions.

[0099] As one implementation method, step S410 involves constructing a fuzzy kernel estimation model by training a neural network to perform fuzzy kernel estimation processing on each candidate image in the candidate barcode image set, generating a set of fuzzy kernel functions. Specifically, this may include the following steps S411~S415:

[0100] Step S411: Obtain a set of barcode image samples containing different degrees of blur, and construct a training dataset for the blur kernel estimation model. The training dataset contains blurred image samples and corresponding clear image samples.

[0101] When acquiring a sample set of barcode images containing varying degrees of blur, clear barcode images can be processed to generate blurred image samples by simulating different blur conditions. For example, different blur kernel functions can be used to perform convolution operations on clear barcode images to simulate different degrees of motion blur, Gaussian blur, etc. Simultaneously, the corresponding clear image samples are retained. The blurred image samples and their corresponding clear image samples are combined into a training dataset, which is used to train the blur kernel estimation model. The quality and diversity of the training dataset have a significant impact on the performance of the blur kernel estimation model; therefore, it is necessary to ensure that the sample set contains a sufficient number of barcode image samples with different degrees of blur.

[0102] Step S412: Construct a deep convolutional neural network structure as the basic framework of the fuzzy kernel estimation model. The neural network structure includes a feature extraction layer, a multi-scale fusion layer, and a kernel parameter prediction layer. Each layer is connected by a non-linear activation function.

[0103] The feature extraction layer, the first layer of a deep convolutional neural network, extracts image features through convolution operations. The multi-scale fusion layer fuses features from different scales to improve the model's feature representation ability. The kernel parameter prediction layer predicts the parameters of the blur kernel. Nonlinear activation functions, such as ReLU and Sigmoid, introduce nonlinearity to enhance the model's learning ability. When constructing the deep convolutional neural network structure, the kernel size, number of kernels, stride, and other parameters for each layer are first determined. Then, nonlinear activation functions are used to connect the layers, forming a complete neural network structure. This structure serves as the basic framework for the blur kernel estimation model, used to learn the features of the image blur kernel.

[0104] Step S413: Train the convolutional neural network structure using the training dataset, optimize the network weight parameters using the backpropagation algorithm, and minimize the loss function of the fuzzy kernel prediction error.

[0105] The backpropagation algorithm minimizes the loss function by calculating the gradient of the loss function with respect to the network weight parameters and then updating the network weight parameters based on this gradient. When training a convolutional neural network (CNN) using a training dataset, blurred image samples from the training dataset are first input into the CNN to obtain the blur kernel prediction results. Then, the error between the predicted blur kernel and the true blur kernel, i.e., the loss function, is calculated. Next, the backpropagation algorithm is used to calculate the gradient of the loss function with respect to the network weight parameters, and the network weight parameters are updated based on this gradient. This process is repeated iteratively until the loss function converges or the preset number of training iterations is reached. Through model training, the network weight parameters are optimized, enabling the blur kernel estimation model to estimate the blur kernel more accurately.

[0106] Step S414: Input the candidate barcode image set into the trained fuzzy kernel estimation model to obtain the fuzzy kernel parameter set corresponding to each candidate image. The fuzzy kernel parameter set defines the shape and size of the fuzzy kernel function.

[0107] When inputting a set of candidate barcode images into a trained fuzzy kernel estimation model, each candidate image in the set is first preprocessed to meet the input requirements of the fuzzy kernel estimation model. Then, the preprocessed candidate images are input into the trained fuzzy kernel estimation model. Based on the learned feature information, the model performs fuzzy kernel estimation on each candidate image and outputs the corresponding set of fuzzy kernel parameters. The fuzzy kernel parameter set defines the shape and size of the fuzzy kernel function; these parameters are used in subsequent blind deconvolution processing.

[0108] Step S415: Construct the corresponding point spread function based on the fuzzy kernel parameter set, and convert the spatial fuzzy kernel into a frequency domain representation through Fourier transform to generate a fuzzy kernel function set.

[0109] The point spread function (PSF) is a function used to describe the blurring process of an image, representing the diffusion of a point in the image during the blurring process. When constructing the corresponding PSF based on the set of blur kernel parameters, the shape and size of the PSF are determined according to the parameter values ​​in the set. Then, the spatial domain blur kernel is converted to a frequency domain representation using a Fourier transform, resulting in a frequency domain blur kernel. The frequency domain blur kernel allows for easier convolution operations, which can be used for blind deconvolution processing. The frequency domain blur kernels corresponding to all candidate images are combined into a blur kernel function set, which contains the PSF functions corresponding to different candidate images.

[0110] Step S420: Perform blind deconvolution processing on each candidate barcode image based on the set of fuzzy kernel functions, and solve the estimated value of the deblurred image through an iterative optimization algorithm to generate a preliminary set of deblurred images.

[0111] Blind deconvolution is a method for eliminating image blur. Without knowing the specific form of the blur kernel, it uses an iterative optimization algorithm to solve for the deblurred image estimate. When performing blind deconvolution on candidate barcode images based on a set of blur kernel functions, the deblurred image estimate is first initialized by inputting the Gaussian-filtered candidate barcode images as initial estimates into the blind deconvolution iteration process. Then, in each iteration, a blurred image estimate is calculated based on the current deblurred image estimate and the blur kernel function, simulating the blurring process through convolution. Next, the difference image between the blurred image estimate and the original candidate barcode image is calculated, and the spatial distribution of the estimation error is measured using a mean squared error function. The deblurred image estimate is then updated using a gradient descent algorithm based on the difference image, with the gradient direction of the error function guiding the parameter update. Finally, prior image constraints are introduced to perform sparsity regularization on the updated deblurred image estimate, suppressing noise amplification. The blur image estimation, difference calculation, and update steps are repeated until the iteration converges, generating a preliminary set of deblurred images.

[0112] As one implementation method, step S420 involves performing blind deconvolution processing on each candidate barcode image based on the set of fuzzy kernel functions, and solving for the deblurred image estimate through an iterative optimization algorithm to generate a preliminary deblurred image set. Specifically, this may include the following steps S421~S425:

[0113] Step S421: Initialize the deblurred image estimate. The candidate barcode image is processed by Gaussian filtering and used as the initial estimate for the blind deconvolution iteration process.

[0114] Gaussian filtering is a process of smoothing an image. By performing a convolution operation on the image, it smooths out noise and details, resulting in a smoother image. When initializing the deblurred image estimate, the candidate barcode image, after Gaussian filtering, is used as the initial estimate input to the blind deconvolution iteration process. Gaussian filtering reduces noise in the image, improves the quality of the initial estimate, and provides a better starting point for the iterative optimization process.

[0115] Step S422: In each iteration, calculate the blurred image estimate based on the current deblurred image estimate and the blur kernel function, and simulate the blurring process through convolution operation.

[0116] Convolution is a method used for image filtering. It involves convolving a filter with the image to obtain the filtered image. In each iteration, when calculating the blurred image estimate based on the current deblurred image estimate and the blur kernel function, the current deblurred image estimate is convolved with the blur kernel function to simulate the blurring process. The blurred image estimate obtained through convolution represents the result of blurring the image under the given deblurred image estimate and blur kernel function.

[0117] Step S423: Calculate the difference image between the estimated value of the blurred image and the original candidate barcode image, and measure the spatial distribution of the estimation error using the mean square error function.

[0118] The mean squared error (MSE) function is used to measure the difference between two images. It is calculated by averaging the sum of the squared differences between corresponding pixels in the two images. When calculating the difference image between the estimated value of the blurred image and the original candidate barcode image, the estimated value is first subtracted pixel-by-pixel from the original candidate barcode image to obtain the difference image. Then, the MSE function is used to process the difference image, measuring the spatial distribution of the estimation error. The MSE function effectively reflects the degree of difference between the estimated value of the blurred image and the original candidate barcode image, providing a basis for parameter updates.

[0119] Step S424: Update the deblurred image estimate based on the difference image using the gradient descent algorithm, and guide the parameter update by calculating the gradient direction of the error function.

[0120] Gradient descent is an algorithm used to optimize functions. It calculates the gradient direction of the function and then updates the parameters in the opposite direction of the gradient to minimize the function value. When updating the deblurred image estimate using gradient descent based on a difference image, the mean squared error function is first used as the error function, and its gradient with respect to the deblurred image estimate is calculated. Then, the deblurred image estimate is updated according to the gradient direction, gradually reducing the value of the error function. Through continuous iterative updates, the deblurred image estimate is progressively optimized, making it closer to the true deblurred image.

[0121] Step S425: Introduce prior image constraints, perform sparsity regularization on the updated deblurred image estimate to suppress noise amplification, and repeat the blurry image estimation, difference calculation and update steps until iterative convergence.

[0122] Image prior constraints are used in the image deblurring process to constrain the estimated values ​​of the deblurred image using prior knowledge of the image, thereby improving the deblurring effect. Sparsity regularization is a method used to regularize images, suppressing noise amplification by constraining the image's sparsity. When introducing image prior constraints and performing sparsity regularization on the updated deblurred image estimates, the prior constraints, such as image sparsity and smoothness, are first determined. Then, sparsity regularization is applied to the updated deblurred image estimates by adding a regularization term to the error function to constrain the sparsity of the estimates. The steps of blurring image estimation, difference calculation, and updating are repeated until iterative convergence, generating a preliminary set of deblurred images.

[0123] Step S430: Perform edge enhancement processing on the preliminary deblurred image set, extract image edge features through multi-directional edge detection operators, generate edge feature images, and highlight the bar and space boundaries of the barcode in the edge feature images.

[0124] Multidirectional edge detection operators are used to detect image edges. Examples of multidirectional edge detection operators include the Sobel operator, Prewitt operator, and Canny operator. When performing edge enhancement processing on a preliminary deblurred image set, a suitable multidirectional edge detection operator, such as the Canny operator, is first selected. Then, each image in the preliminary deblurred image set is input into the multidirectional edge detection operator to extract the image's edge features. Through multidirectional edge detection operators, edges in different directions in the image can be detected, generating an edge feature image. The edge feature image highlights the bar and space boundaries of the barcode, providing clearer edge information for unsharpened mask processing.

[0125] Step S440: Perform unsharpened mask processing on the preliminary deblurred image based on the edge feature image, and enhance the high-frequency components of the image through Gaussian blur difference to generate an enhanced deblurred image set.

[0126] Unsharpening masking is a method used to enhance image sharpness. It enhances the high-frequency components of an image by applying Gaussian blur difference processing. When performing unsharpening masking on a preliminary deblurred image based on edge feature images, the preliminary deblurred image is first subjected to Gaussian blur processing to obtain a blurred image. Then, the preliminary deblurred image is subtracted from the blurred image to obtain the high-frequency component image. Finally, the high-frequency component image is added to the preliminary deblurred image to generate an enhanced deblurred image set. Unsharpening masking can effectively enhance the edge and detail information of an image, improving image sharpness.

[0127] Step S450: Evaluate the sharpness of the enhanced deblurred image set. By calculating the weighted combination index of the sum of the image gradient magnitude and the entropy value, select the image with the highest comprehensive evaluation as the sharp barcode image unit after deblurring.

[0128] Sharpness assessment is a method used to evaluate image sharpness. It assesses image quality by calculating a weighted combination of the sum of the image's gradient magnitudes and its entropy value. The sum of the image's gradient magnitudes represents the intensity of pixel grayscale changes in the image; a larger sum of gradient magnitudes indicates sharper image edges. The image entropy value represents the image's information entropy; a larger entropy value indicates richer image information. When assessing the sharpness of an enhanced deblurred image set, the sum of the gradient magnitudes and entropy value of each image in the set are first calculated. Then, based on preset weighting coefficients, a weighted combination of the gradient magnitudes and entropy values ​​is calculated. Finally, the image with the highest overall evaluation is selected as the sharp barcode image unit after deblurring.

[0129] Step S500: Perform barcode region localization and distortion correction processing on the clear barcode image unit to generate a standardized barcode image, and perform barcode information decoding and recognition based on the standardized barcode image to obtain the recognition result.

[0130] Barcode region localization and distortion correction is a method for locating barcode regions and correcting barcode distortion. It involves processing clear barcode image units to generate standardized barcode images. Barcode information decoding and recognition is a method for identifying barcode information. It involves decoding the standardized barcode image to obtain the corresponding number or letter sequence. In the barcode region localization and distortion correction process for clear barcode image units, firstly, a multi-directional edge detection algorithm is used to extract edge features from the clear barcode image units, generating a binary edge image. Then, morphological operations are performed on the binary edge image to connect broken edge segments, generating a connected edge image. Next, the boundary contour of the barcode region is detected in the connected edge image, and a set of contour feature points is extracted. Then, a quadrilateral boundary of the barcode region is fitted based on the set of contour feature points to generate a barcode region localization box. Finally, perspective transformation is performed on the original clear barcode image units based on the barcode region localization box to generate a standardized barcode image. When decoding and recognizing barcode information based on standardized barcode images, the standardized barcode image is first subjected to local adaptive binarization to generate a black and white binary image. Then, the horizontal pixel sequence of the binary image is scanned to record the bar and space width information, generating a width sequence. Next, the width sequence is encoded and parsed according to the barcode encoding rules to generate a corresponding character encoding set. The character encoding set is then validated, and codes that do not conform to the validation rules are removed. Finally, the validated character encoding set is converted into readable text information to generate the barcode recognition result.

[0131] In one implementation, step S500 involves performing barcode region localization and distortion correction on the clear barcode image unit to generate a standardized barcode image. Specifically, this may include the following steps S510-S550:

[0132] Step S510: Extract edge features from clear barcode image units using a multi-directional edge detection algorithm, and generate a binary edge image by combining the edge responses in the horizontal, vertical, and diagonal directions.

[0133] Multi-directional edge detection algorithms are an effective means of extracting edge features from images. Their core lies in using operators in different directions to perform convolution operations on the image, thereby capturing grayscale changes in various directions. When extracting edge features from clear barcode image units, classic edge detection operators such as the Sobel operator and the Prewitt operator can be used. Taking the Sobel operator as an example, it contains convolution kernels in both horizontal and vertical directions. By performing convolution operations on the image in the horizontal and vertical directions respectively, the gradient magnitudes of the image in these two directions can be obtained. For diagonal edge responses, this can be achieved by appropriately combining the gradient magnitudes in the horizontal and vertical directions. For example, the square root of the sum of the squares of the gradient magnitudes in the two directions can be calculated to represent the overall edge strength of the image in each direction.

[0134] In the process of generating a binary edge image, a threshold is preset. The edge intensity image obtained through multi-directional edge detection is compared with this threshold. If the edge intensity of a pixel is greater than the threshold, it is assigned a value of 255 (representing white, i.e., an edge point); if it is less than the threshold, it is assigned a value of 0 (representing black, i.e., a non-edge point). This converts a continuous edge intensity image into a binary image with only black and white colors, facilitating subsequent processing. For example, when processing a barcode image in a logistics warehouse, multi-directional edge detection is performed using the Sobel operator, with a threshold set to 128. Binarizing the edge intensity image yields a clear binary edge image, where the edges of the barcode are clearly displayed as white lines.

[0135] Step S520: Perform morphological operations on the binary edge image, including dilation and erosion operations with multiple structural elements, connect broken edge segments, and generate a connected edge image.

[0136] Morphological operations are a series of methods that manipulate the shape of an image, primarily including two basic operations: dilation and erosion. Dilation enlarges objects in an image by sliding a structuring element (such as a circle or square) across the image; if the structuring element overlaps with an object, the pixels corresponding to the overlapping portion are assigned white. Erosion, on the other hand, shrinks objects in an image; only when the structuring element is completely contained within the image object are the pixels corresponding to the center of the structuring element assigned white.

[0137] When processing binary images with edges, dilation and erosion operations using multiple structuring elements can better connect broken edge segments. First, select appropriate structuring elements, such as 3×3 square structuring elements and 5×5 circular structuring elements. First, perform a dilation operation using the 3×3 square structuring element, which can connect some smaller broken edges to a certain extent. Then, perform an erosion operation using the 5×5 circular structuring element to remove some unnecessary noise and small connected parts generated by the dilation operation, making the edges clearer and more accurate. By repeatedly alternating between dilation and erosion operations, the broken edge segments can eventually be connected, generating a connected edge image. For example, when processing a binary image of a barcode edge where the edges are broken due to uneven lighting, by first using a 3×3 square structuring element for dilation and then using a 5×5 circular structuring element for erosion, the broken edges can be effectively connected, making the barcode edges form a continuous whole.

[0138] Step S530: Detect the boundary contour of the barcode region in the connected edge image, and extract the contour feature point set by the chain code-based contour tracking algorithm. The contour feature point set contains the ordered coordinate sequence of the boundary contour.

[0139] Chain code-based contour tracking algorithms are methods used to track the boundary contours of objects in images. Chain code is an encoding method that uses numbers to represent contour directions; examples include 4-chain code and 8-chain code. 4-chain code uses four numbers, 0, 1, 2, and 3, to represent the four directions: right, down, left, and up, respectively; while 8-chain code uses eight numbers, 0-7, to represent eight different directions.

[0140] When detecting the boundary contour of a barcode region in a connected edge image, a starting point needs to be found first. For example, the white pixel in the top left corner of the image can be chosen as the starting point. Then, starting from the starting point, contour tracking is performed using chain code rules. For example, using 8-chain code, starting from the starting point, the eight neighboring pixels are checked sequentially in a clockwise direction. If a white pixel is found, the chain code value corresponding to that direction is recorded, and this white pixel is used as the new current point to continue tracking. This process is repeated until the starting point is reached, completing the tracking of the entire contour.

[0141] During the tracking process, the coordinates of each contour point are recorded, forming an ordered coordinate sequence, i.e., a set of contour feature points. This set accurately describes the boundary contour of the barcode region. For example, in a connected edge image after morphological processing, the 8-chain contour tracking algorithm starts tracking from the top-left white pixel, recording the coordinates of each contour point in sequence, and finally obtaining an ordered coordinate sequence containing the boundary contour of the barcode region, providing accurate data for barcode region localization.

[0142] Step S540: Fit the quadrilateral boundary of the barcode region based on the set of contour feature points, determine the coordinates of the four vertices of the barcode region using the least squares fitting algorithm, and generate the barcode region positioning box.

[0143] Least squares fitting is a method used to find the best-fit curve or line between data points. Its core idea is to determine the fitting parameters by minimizing the sum of squared errors between the data points and the fitted curve or line. When fitting the quadrilateral boundary of a barcode region based on a set of contour feature points, since the barcode region is, for example, approximately quadrilateral, the problem can be transformed into finding a quadrilateral to fit these contour feature points.

[0144] In practice, we first assume that the four sides of the quadrilateral are straight lines, each containing two parameters: slope and intercept. We substitute points from the contour feature point set into these line equations, calculate the distance from each point to the corresponding line, and use the sum of the squares of these distances as an error function. Through a least-squares fitting algorithm, we continuously adjust the parameters of the line equations to minimize the error function. When the error function reaches its minimum value, the four lines obtained constitute the quadrilateral boundary of the barcode area.

[0145] After determining the boundary of the quadrilateral, the coordinates of its four vertices need to be found. These coordinates can be obtained by solving for the intersection of two adjacent lines. For example, for two line equations y = k1x + b1 and y = k2x + b2, solving for x and y using the simultaneous system of equations yields the coordinates of their intersection. Solving the system of equations pairwise for each of the four lines gives the coordinates of the quadrilateral's four vertices. Connecting these four vertex coordinates generates the barcode region bounding box. For instance, when processing a connected edge image containing a tilted barcode, a least-squares fitting algorithm can be used to fit the set of contour feature points into a quadrilateral boundary, allowing the determination of the four vertex coordinates and thus accurately locating the barcode region. The generated bounding box tightly surrounds the barcode.

[0146] Step S550: Perform perspective transformation processing on the original clear barcode image units according to the barcode area positioning box, and convert the tilted or distorted barcode area into a regular rectangular area through the projection matrix to generate a standardized barcode image.

[0147] Perspective transformation is a geometric transformation method used to convert an image from one viewpoint to another. Its key lies in finding a suitable projection matrix to map points in the original image to corresponding points in the target image. When performing perspective transformation on original clear barcode image units based on the barcode region positioning box, it is first necessary to determine the coordinates of the four vertices of the barcode region in the original image (i.e., the coordinates of the four vertices of the barcode region positioning box), as well as the coordinates of the four vertices of the target rectangular region.

[0148] To calculate the projection matrix, at least four pairs of corresponding points (i.e., points in the original image and corresponding points in the target image) can be used. Using these four pairs of corresponding points, a system of linear equations can be established to solve for the elements of the projection matrix. In practical applications, the `getPerspectiveTransform` function from the OpenCV library can be used to calculate the projection matrix.

[0149] After obtaining the projection matrix, the `warpPerspective` function is used to perform perspective transformation on the original clear barcode image units. This function maps each pixel in the original image to its corresponding position in the target rectangular region based on the projection matrix, thereby converting tilted or distorted barcode regions into rectangular regions. For example, when processing an image where the barcode is tilted due to the shooting angle, by calculating the projection matrix and performing perspective transformation, the tilted barcode region is converted into a rectangular region, and the resulting standardized barcode image facilitates subsequent decoding and recognition processing.

[0150] Step S560: Perform local adaptive binarization processing on the standardized barcode image, determine the dynamic threshold by calculating the average gray level of the neighborhood, and generate a black and white binary image.

[0151] Local adaptive binarization is a method that determines the binarization threshold based on the gray-level characteristics of local regions in an image, making it better suited to conditions such as uneven lighting in images. When performing local adaptive binarization on standardized barcode images, the first step is to determine an appropriate neighborhood size.

[0152] For each pixel in the image, calculate the average gray value of its neighborhood. Taking a 5×5 neighborhood window as an example, sum the gray values ​​of all pixels within the window, then divide by the total number of pixels in the window (i.e., 25) to obtain the neighborhood average gray value. Use this neighborhood average gray value as the dynamic threshold for that pixel. Next, compare the actual gray value of the pixel with the dynamic threshold. If the actual gray value is greater than the dynamic threshold, assign the pixel a value of 255 (white); if it is less than the dynamic threshold, assign it a value of 0 (black).

[0153] By performing this process on each pixel in the image, a standardized barcode image can be converted into a black-and-white binary image. This method effectively handles potential uneven lighting issues in barcode images, making the bars and spaces of the barcode more clearly displayed in the binary image. For example, when processing a standardized barcode image taken under different lighting conditions, a 5×5 neighborhood window is used for local adaptive binarization, and the average gray level of the neighborhood of each pixel is calculated as a dynamic threshold. In the generated black-and-white binary image, the barcode's bar and space boundaries are clearly defined, providing a good foundation for extracting bar and space width information.

[0154] Step S570: Scan the horizontal pixel sequence of the binary image, record the width information of the gaps, and generate a width sequence containing the width values ​​of consecutive gaps.

[0155] Scanning the horizontal pixel sequence of a binary image is crucial for accurately obtaining the width information of the bars and spaces in a barcode. During scanning, starting from the top of the binary image, each row of pixels is scanned from left to right. When encountering white pixels (representing spaces in the barcode) and black pixels (representing bars), their consecutive lengths are recorded.

[0156] In practice, a counter can be used to record the number of consecutive pixels with the same color. When a pixel is scanned, if the color of the pixel is the same as the color of the previous pixel, the counter is incremented by 1; if the colors are different, the value of the counter is recorded as the width of the current bar or empty space, and then the counter is reset to 1.

[0157] By scanning the entire horizontal pixel sequence of a binary image, a series of width values ​​for consecutive empty bars can be obtained. Arranging these width values ​​in the scanning order generates a width sequence. For example, when scanning a barcode binary image that has undergone local adaptive binarization, starting from the first row, scanning is performed from left to right. When a white pixel is encountered, a counter is started and counted until a black pixel is encountered. The counter value at this point is recorded as the width of the first empty bar. Then, the black pixels are counted, and when a white pixel is encountered, its width is recorded as the width of the first bar, and so on, until a width sequence containing the width values ​​of all consecutive empty bars is generated.

[0158] Step S580: The width sequence is encoded and parsed according to the barcode encoding rules, and the width sequence is converted into the corresponding character encoding set by looking up the encoding rule table.

[0159] Barcode encoding rules are the rules that define the correspondence between the width combinations of bars and spaces in a barcode and the character encoding. Different types of barcodes (such as EAN-13, Code 39, etc.) have different encoding rules. When encoding and parsing a width sequence according to barcode encoding rules, it is first necessary to determine the type of barcode being processed, and then look up the corresponding encoding rule table.

[0160] The encoding rule table contains, for example, the character codes corresponding to various bar-space width combinations. When parsing the width sequence, the width sequence is grouped according to the barcode encoding rules, with each group of bar-space width combinations corresponding to a character code. For example, for an EAN-13 barcode, each group contains 7 modules (bars or spaces). By looking up the EAN-13 encoding rule table, the width combination of each group of 7 modules is converted into the corresponding character code.

[0161] In practice, the width sequence can be segmented according to encoding rules. Each segment is then compared to the bar / space width combinations in the encoding rule table to find matching character codes. Combining all matching character codes yields the corresponding character code set. For example, when processing the width sequence of an EAN-13 barcode, the width sequence is segmented into groups of seven. The EAN-13 encoding rule table is then consulted sequentially, and each width combination is converted into its corresponding character code, ultimately resulting in a set containing 13 character codes.

[0162] Step S590: Perform verification processing on the character encoding set, verify the integrity and correctness of the encoding by calculating the check bits, and remove encodings that do not conform to the verification rules.

[0163] Verification is a crucial step in ensuring the accuracy of barcode encoding, and different types of barcodes have different verification rules. Taking the EAN-13 barcode as an example, its verification rule involves calculating a check digit from the first 12 digits of the barcode (i.e., the numbers corresponding to the first 12 character codes in the character encoding set). This check digit is then compared with the 13th digit of the barcode. If they match, the encoding is complete and correct; if they differ, an error exists in the encoding.

[0164] To calculate the check digit of an EAN-13 barcode, first multiply the sum of the odd-numbered digits (digits 1, 3, 5, 7, 9, and 11) by 3, then add the sum of the even-numbered digits (digits 2, 4, 6, 8, 10, and 12). Next, take the modulus of this sum with 10, and subtract this modulus from 10. The result is the check digit.

[0165] For other types of barcodes, such as Code 39 barcodes, there are also corresponding verification rules. When verifying a set of character codes, a check digit is calculated according to the corresponding verification rules based on the type of barcode being processed, and then compared with the actual check digit. If it does not conform to the verification rules, the character code set is considered invalid and discarded. For example, when processing a set of EAN-13 barcode character codes, if the calculated check digit does not match the actual 13th digit, it indicates that the code has an error and is discarded to ensure the accuracy of the final recognition result.

[0166] Step S5100: Convert the verified character encoding set into readable text information to generate a barcode recognition result, which includes the number or letter sequence corresponding to the barcode.

[0167] When converting a verified set of character codes into readable text information, it is necessary to map the character codes to corresponding numbers or letters according to the barcode's encoding rules. Different types of barcodes have different correspondences between character codes and numbers or letters. For example, for EAN-13 barcodes, the character codes directly correspond to the numbers 0-9; for Code 39 barcodes, the character codes can correspond to the numbers 0-9, the letters A, Z, and some special characters.

[0168] During the conversion, each character in the character encoding set is simply replaced with its corresponding number or letter according to the encoding rule table. For example, for a valid EAN-13 barcode character encoding set, each character is converted into its corresponding number according to the EAN-13 encoding rules. These numbers are then arranged sequentially to obtain the corresponding number sequence of the barcode, i.e., the barcode recognition result. For Code 39 barcodes, each character in the character encoding set is converted into its corresponding number, letter, or special character, combined into a readable string as the recognition result. The final generated barcode recognition result can be used for further information retrieval and processing, such as querying product information in a product management system.

[0169] It is understood that the various algorithms involved in the above descriptions of the embodiments of the present invention can all be obtained from relevant content in the prior art. To save space, they will not be elaborated on in the embodiments of the present invention. In addition, those skilled in the art can supplement the details based on common knowledge in the art when implementing the solutions of the present invention. For example, they can use normalization to eliminate dimensional conflicts before feature fusion, use interpolation to eliminate dimensional differences, reasonably set thresholds based on historical data, experience or business scenario requirements, train the model based on a general model training method, set the number of layers in the model structure based on actual needs, select activation functions, etc. The present invention will not provide redundant descriptions of overly detailed implementation processes here.

[0170] For example, in step S350, a sharpness assessment is performed on the multi-scale fused image set. Image quality is evaluated by calculating image gradient energy and entropy parameters. Fused images with sharpness indices exceeding a preset threshold are retained, generating a candidate barcode image set containing different degrees of motion compensation. Alternatively, image quality can be evaluated by separately calculating and normalizing the image gradient energy and entropy parameters to generate a comprehensive sharpness index. Fused images with comprehensive sharpness indices exceeding a preset threshold are retained, generating a candidate barcode image set containing different degrees of motion compensation. A similar combined evaluation (sum of gradient magnitudes and entropy value) follows the same principle.

[0171] Alternatively, in step S311, the direction and velocity features of each trajectory in the set of pixel displacement trajectories are extracted to generate trajectory feature vectors. These feature vectors contain the average direction angle and average velocity value of the trajectory. A density clustering algorithm is used to perform cluster analysis on the trajectory feature vectors. By setting a neighborhood radius parameter, the reachability relationship between trajectories is determined, and density-connected trajectories are divided into the same trajectory cluster, generating an initial trajectory cluster set. The intra-cluster dispersion index of each initial trajectory cluster is calculated. By calculating the average distance between all trajectory feature vectors within a cluster and the cluster center, the degree of concentration of the trajectory feature vectors within the cluster is reflected. This can be achieved by extracting the direction and velocity features of each trajectory in the set of pixel displacement trajectories, normalizing the direction and velocity features respectively, generating normalized trajectory feature vectors; performing cluster analysis on the normalized trajectory feature vectors using a density clustering algorithm, determining the reachability relationship between trajectories by setting a neighborhood radius parameter, dividing density-connected trajectories into the same trajectory cluster, generating an initial trajectory cluster set; calculating the intra-cluster dispersion index of each initial trajectory cluster, and reflecting the degree of concentration of the intra-cluster trajectory feature vectors by calculating the average Euclidean distance between all normalized trajectory feature vectors in the cluster and the cluster center.

[0172] Furthermore, in step S325, bicubic interpolation is performed on the motion compensation lookup table of the motion compensation model to generate a continuous motion compensation parameter space, ensuring that all pixels in the image can obtain the corresponding compensation parameters, thus completing the construction of the motion compensation model. Specifically, bicubic interpolation can be performed on the displacement compensation amount and rotation compensation angle in the motion compensation lookup table of the motion compensation model to generate continuous displacement compensation parameter spaces and rotation compensation parameter spaces, ensuring that all pixels in the image can obtain the corresponding displacement compensation amount and rotation compensation angle, thus completing the construction of the motion compensation model.

[0173] Alternatively, in step S450, the sharpness of the enhanced deblurred image set is evaluated. This is achieved by calculating a weighted combination index of the sum of image gradient magnitudes and entropy values, selecting the image with the highest overall evaluation as the sharp barcode image unit after deblurring. Specifically, the sharpness of the enhanced deblurred image set is evaluated by calculating and normalizing the sum of image gradient magnitudes and entropy values ​​to generate a weighted combination index, and then selecting the image with the highest overall evaluation as the sharp barcode image unit after deblurring.

[0174] Similar examples will not be given here. Those skilled in the art, based on their general knowledge in the field, are capable of basic data preprocessing.

[0175] Please see Figure 2 , Figure 2 This is a schematic diagram of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 can be connected via a bus or other means. The processor 101 (or Central Processing Unit, CPU) is the computing and control core of the computer system, capable of parsing various instructions and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 101; the communication interface 102 can also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system used to store programs and data. It is understood that the memory 103 here can include the computer system's built-in memory, or it can include extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system; this invention does not limit this storage space.

[0176] In one embodiment, the processor 101 executes the motion-blurred barcode recognition method based on multi-frame image fusion provided in the above embodiments of the present invention by running a computer program in the memory 103.

Claims

1. A method for recognizing motion-blurred barcodes based on multi-frame image fusion, characterized in that, The method includes: Obtain a continuous image sequence containing a motion-blurred barcode, the continuous image sequence comprising multiple frame image units with time-series association; Motion trajectory feature extraction is performed on the continuous image sequence to obtain the motion vector field and pixel displacement trajectory set of the barcode region in each frame image unit. Specifically, this includes: performing grayscale conversion processing on the continuous image sequence, converting color image units into grayscale image sequences, wherein the grayscale value of each pixel in the grayscale image sequence is generated by calculating the three-channel pixel values ​​of the original image unit according to a preset weight, while maintaining the same spatial resolution as the original image unit; corner detection processing is performed by calculating the eigenvalue ratio of the grayscale covariance matrix within the local window of the image to identify corner feature sets in the barcode region of each frame image unit where the grayscale difference between adjacent pixels exceeds a preset threshold, wherein the corner feature set includes a corner coordinate sequence with spatial distribution characteristics and the corresponding response intensity value; and feature matching processing based on fast nearest neighbor search is performed on the corner feature sets of adjacent frame image units, and the feature descriptor is accelerated by constructing a hierarchical index structure. Similarity search is used to establish cross-frame corner point correspondences, generating a set of corner point matching pairs containing matching confidence scores. Each matching pair in the set contains the coordinate information and matching confidence of corresponding corner points in the preceding and following frames. Based on the set of corner point matching pairs, a dense motion vector field between adjacent frame image units is calculated. By performing bilinear interpolation on the sparse corner point motion vectors, a dense motion vector field covering the entire barcode area is generated. The direction of each vector in the motion vector field represents the pixel movement direction, and the vector magnitude represents the pixel movement distance. Based on the dense motion vector field, the pixels of each frame image unit are subjected to trajectory tracking processing based on filtering prediction. The pixel position estimation is optimized through a prediction-update iteration process, generating a set of pixel displacement trajectories containing timestamps. Each trajectory in the set of pixel displacement trajectories corresponds to the position change record and trajectory confidence parameter of a pixel in the barcode area in consecutive frames. Based on the motion vector field and pixel displacement trajectory set, multi-frame image fusion is performed on the continuous image sequence to generate a candidate barcode image set containing different degrees of motion compensation; specifically, this includes: performing density-based trajectory clustering on the pixel displacement trajectory set, measuring trajectory similarity by calculating the dynamic time warp distance between trajectories, and dividing trajectories with similar direction and velocity characteristics into multiple trajectory clusters; constructing a motion compensation model for multi-frame images based on the trajectory clusters and motion vector field, and establishing a mathematical model of trajectory position change in the time dimension by curve fitting processing on the representative trajectory of each trajectory cluster; according to the motion... The compensation model performs inverse motion compensation processing on each frame's image units, mapping the barcode region pixels of different frames to a reference coordinate system through coordinate transformation to generate a motion-compensated image unit sequence. The motion-compensated image unit sequence undergoes multi-scale fusion processing, using different weighting coefficients to perform weighted averaging of pixel values ​​at the same spatial location, generating a multi-scale fused image set. The multi-scale fused image set undergoes sharpness evaluation processing, assessing image quality by calculating image gradient energy and entropy parameters, retaining fused images with sharpness indicators exceeding a preset threshold, and generating a candidate barcode image set containing different degrees of motion compensation. The candidate barcode image set is subjected to deblurring and enhancement processing to obtain clear barcode image units after deblurring; The clear barcode image unit is subjected to barcode region localization and distortion correction processing to generate a standardized barcode image. Based on the standardized barcode image, the barcode information is decoded and recognized to obtain the recognition result.

2. The method according to claim 1, characterized in that, The corner detection process, which involves calculating the eigenvalue ratio of the gray-level covariance matrix within a local window of the image, identifies a set of corner features in each frame's image unit where the gray-level difference between adjacent pixels in the barcode region exceeds a preset threshold. This includes: Multi-scale Gaussian filtering is performed on each frame image unit in the grayscale image sequence. Gaussian blurred image sequences of different scale spaces are generated by constructing a Gaussian pyramid. The Gaussian blurred image sequence contains image units with different blur levels from low to high, and the image size of each layer decreases according to a preset ratio. Calculate the difference images of adjacent scale images in the Gaussian blurred image sequence, generate a Laplacian pyramid in multi-scale space, and determine the scale space characteristics of the barcode region by analyzing the distribution of zero cross points in the difference images. The gradient magnitude matrix and gradient direction matrix of the image are calculated at each scale layer of the Laplacian pyramid. Convolution operations are performed in the horizontal and vertical directions using a direction operator to generate a gradient magnitude image that reflects the intensity of pixel grayscale changes and a gradient direction image that reflects the direction of change. Local extrema are detected on the gradient magnitude image using an adaptive threshold non-maximum suppression algorithm. The size of the suppression window is dynamically adjusted to adapt to the feature density of different regions, and a set of candidate corner points with the largest local gray-level changes is extracted. The candidate corner point set is subjected to texture feature verification processing based on gray-level co-occurrence matrix. The texture energy and entropy parameters of the neighborhood of the candidate corner point are calculated. Corner points whose texture features conform to the barcode bar-space distribution characteristics are retained to generate the final corner point feature set.

3. The method according to claim 2, characterized in that, The corner feature set of adjacent frame image units is subjected to feature matching processing based on fast nearest neighbor search. A hierarchical index structure is constructed to accelerate the similarity search of feature descriptors, establish cross-frame corner correspondences, and generate a set of corner matching pairs containing matching confidence scores, including: A high-dimensional feature descriptor based on the gradient orientation histogram is constructed for the corner feature set of the current frame image unit. A rotation-invariant feature vector is generated by calculating the statistical features of the gradient orientation histogram of the corner neighborhood. The same feature descriptor construction process is performed on the image unit of the previous frame to generate the feature descriptor set of the previous frame, ensuring that it has the same dimension and scale as the feature descriptor of the current frame. A hierarchical nearest neighbor search index structure is constructed. The previous frame feature descriptor subset is partitioned into multiple dimensions using a random space partitioning algorithm to generate a hierarchical data structure that supports fast nearest neighbor search. Input the current frame feature descriptor into the nearest neighbor search structure, perform multi-nearest neighbor search to obtain multiple nearest neighbor matching results for each descriptor, and evaluate the matching confidence by calculating the nearest neighbor distance ratio. Match purification is performed by filtering based on distance ratio thresholds and random sampling consistency algorithm to remove incorrect matching pairs and retain correct matching pairs that satisfy the geometric constraints of the underlying matrix, generating a set of corner matching pairs containing matching confidence scores.

4. The method according to claim 1, characterized in that, The process involves density-based trajectory clustering of the pixel displacement trajectory set. By calculating the dynamic time-warped distance between trajectories to measure trajectory similarity, trajectories with similar directional and velocity characteristics are divided into multiple trajectory clusters, including: Extract the direction and velocity features of each trajectory in the set of pixel displacement trajectories to generate a trajectory feature vector, which includes the average direction angle and average velocity value of the trajectory. The trajectory feature vectors are clustered using a density clustering algorithm. The reachability relationship between trajectories is determined by setting a neighborhood radius parameter. Densely connected trajectories are divided into the same trajectory cluster to generate an initial set of trajectory clusters. Calculate the intra-cluster dispersion index of each initial trajectory cluster. By calculating the average distance between all trajectory feature vectors within the cluster and the cluster center, the degree of concentration of the trajectory feature vectors within the cluster is reflected. Merge adjacent trajectory clusters whose intra-cluster dispersion index is below a preset threshold, determine the merging priority by calculating the inter-cluster distance matrix, generate an intermediate trajectory cluster set, and reduce the number of trajectory clusters; Calculate the representative trajectory of each trajectory cluster in the intermediate trajectory cluster set. Generate the representative trajectory by performing time-aligned averaging on all trajectories in the cluster. This representative trajectory is used to describe the overall motion characteristics of the cluster.

5. The method according to claim 4, characterized in that, The motion compensation model for constructing multi-frame images based on the trajectory clusters and motion vector fields includes: The basic framework for constructing the motion compensation model is established by performing polynomial curve fitting on the representative trajectory of each trajectory cluster, solving for the fitting coefficients using the least squares method, and establishing the mathematical expression of the trajectory. In the motion compensation model, the predicted position values ​​of each pixel in the trajectory cluster at different time points are calculated based on the mathematical expression, and a position prediction matrix is ​​generated. The rows of the position prediction matrix represent pixel indices, and the columns represent time points, which constitute the position prediction layer of the motion compensation model. In the motion compensation model, inverse motion compensation parameters are calculated based on the position prediction matrix and motion vector field. Future frame pixels are mapped to reference frame coordinates through coordinate transformation. The inverse motion compensation parameters include displacement compensation amount and rotation compensation angle, forming the parameter calculation layer of the motion compensation model. In the motion compensation model, a motion compensation lookup table containing inverse motion compensation parameters of all trajectory clusters is constructed. Each entry in the motion compensation lookup table corresponds to a set of compensation parameters for a trajectory cluster, which serves as the parameter storage layer of the motion compensation model. Bicubic interpolation is performed on the motion compensation lookup table of the motion compensation model to generate a continuous motion compensation parameter space, so that all pixels in the image can obtain the corresponding compensation parameters, thus completing the construction of the motion compensation model.

6. The method according to claim 5, characterized in that, The step of performing inverse motion compensation processing on each frame image unit according to the motion compensation model, mapping the barcode region pixels of different frames to the reference coordinate system through coordinate transformation, and generating a motion-compensated image unit sequence includes: Read the motion compensation lookup table in the motion compensation model to obtain the inverse motion compensation parameters corresponding to each trajectory cluster; For each pixel in the current frame image unit, determine the trajectory cluster affiliation. By calculating the distance between the pixel coordinates and the trajectory represented by each trajectory cluster, determine the trajectory cluster to which the pixel belongs and obtain the corresponding inverse motion compensation parameters. Based on the inverse motion compensation parameters, an affine transformation is performed on the pixel to calculate the corresponding position coordinates of the pixel in the reference frame image unit. The pixel values ​​at corresponding position coordinates in the reference frame image unit are sampled using a bilinear interpolation algorithm to generate motion-compensated pixel values. After motion compensation, the pixel values ​​of all pixels are rearranged to generate motion-compensated image units. The above process is repeated to process all frame image units to obtain a sequence of motion-compensated image units.

7. The method according to claim 1, characterized in that, The step of performing deblurring and enhancement processing on the candidate barcode image set to obtain clear barcode image units after deblurring includes: A fuzzy kernel estimation model is constructed. By training a neural network, fuzzy kernel estimation processing is performed on each candidate image in the candidate barcode image set to generate a fuzzy kernel function set, which contains point spread functions corresponding to different candidate images. Based on the set of fuzzy kernel functions, blind deconvolution processing is performed on each candidate barcode image, and the estimated value of the deblurred image is solved by an iterative optimization algorithm to generate a preliminary set of deblurred images. Edge enhancement processing is performed on the preliminary deblurred image set. Image edge features are extracted by a multi-directional edge detection operator to generate an edge feature image. The edge feature image highlights the bar and space boundaries of the barcode. The initial deblurred image is processed by unsharpening masking based on the edge feature image, and the high-frequency components of the image are enhanced by Gaussian blur difference to generate an enhanced deblurred image set. The sharpness of the enhanced deblurred image set is evaluated by calculating a weighted combination index of the sum of the image gradient magnitudes and the entropy value, and the image with the highest comprehensive evaluation is selected as the clear barcode image unit after deblurring.

8. A computer system, characterized in that, include: A memory, wherein a computer program is stored; A processor is configured to load the computer program to implement the motion-blurred barcode recognition method based on multi-frame image fusion as described in any one of claims 1-7.

Images