A sorting video recognition processing method and system based on YOLO

Abstract

The application relates to the technical field of image processing, and discloses a sorting video recognition processing method and system based on YOLO. The method comprises the following steps: processing mask data by adopting a YOLO-Inpaint algorithm, embedding image repair loss calculation in a YOLO network, learning the mapping relationship of inferring complete article features from incomplete features through adversarial training, and obtaining a repair feature map. Based on the repair feature map, the article completeness score is calculated, and the ratio of the visible pixel area to the predicted total area and the similarity between the repair area and the real area are comprehensively considered. The completeness score and the detection confidence are fused according to the adaptive weight of the occlusion degree to generate a compensated recognition result. A time sequence labeled recognition data stream is constructed, which contains article categories, position coordinates, occlusion states and confidence, and a processing report is output. The application solves the accuracy and robustness problems of article occlusion recognition in sorting video, and improves the recognition success rate and processing efficiency under the occlusion condition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a sorting video recognition and processing method and system based on YOLO. Background Technology

[0002] Existing sorting video recognition and processing technologies are mainly based on traditional object detection algorithms, such as YOLO, SSD, and Faster R-CNN deep learning methods. These algorithms learn the visual features of items by training on a large amount of labeled data, and can achieve high recognition accuracy under ideal conditions. The traditional YOLO algorithm adopts a single-stage detection architecture, transforming the object detection task into a regression problem. It uses a convolutional neural network to simultaneously predict the position coordinates and category probability of the item, which has the advantages of fast detection speed and good real-time performance, and is widely used in the field of industrial sorting.

[0003] However, existing technologies have significant limitations when handling occlusion scenarios in sorting videos. When sorted items are partially or completely obscured by operator hands, machinery, or other objects, traditional YOLO algorithms can only identify them based on the incomplete features of the visible portion, leading to a sharp drop in recognition accuracy. This is especially true when the occlusion exceeds 50%, where the failure rate increases significantly. Existing algorithms lack feature restoration and compensation mechanisms for occluded areas, making it impossible to recover the complete feature representation of items from incomplete visual information. Furthermore, they lack adaptive processing strategies for different degrees of occlusion. Summary of the Invention

[0004] This application provides a sorting video recognition and processing method and system based on YOLO, which is used to solve the problems of accuracy and robustness of item occlusion recognition in sorting videos, and improves the recognition success rate and processing efficiency under occlusion conditions.

[0005] In a first aspect, this application provides a sorting video recognition and processing method based on YOLO, the sorting video recognition and processing method based on YOLO comprising:

[0006] Step S1: Mark the occlusion boundaries by edge detection and connected component analysis, and segment each frame of the sorting operation video into visible and occluded areas to construct occlusion mask data;

[0007] Step S2: The occlusion mask data is processed using the YOLO-Inpaint algorithm. The YOLO-Inpaint algorithm embeds image restoration loss calculation into the YOLO detection network and learns the mapping relationship between inferring object features from incomplete features through adversarial training to obtain a restored feature map. This includes: performing pixel-level fusion processing on the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers; constructing an adversarial training architecture of a generator network and a discriminator network based on the input feature matrix. The generator network is responsible for inferring object features within the occluded area from visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features to obtain an adversarial loss feedback signal; calculating the image restoration loss value based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between generated features and real features, and the perceptual error measures the semantic similarity between generated features and real features to obtain a comprehensive loss gradient; backpropagating the comprehensive loss gradient to the feature extraction layer of the YOLO detection network for weight update, and the updated network performs feature restoration processing on the occluded area to obtain a restored feature map.

[0008] Step S3: Calculate the item integrity score based on the repair feature map output by the YOLO-Inpaint algorithm. The score is calculated by the ratio of the visible pixel area to the predicted total area and the similarity between the repaired area and the real area.

[0009] Step S4: The item integrity score and the detection confidence are weighted and fused together. The weight ratio is adaptively adjusted according to the degree of occlusion to generate the sorted item recognition result after occlusion compensation.

[0010] Step S5: Based on the sorted item recognition results, construct a time-annotated sorting video recognition data stream, including the item category, location coordinates, occlusion status, and recognition confidence of each frame, and output a sorting video recognition processing report.

[0011] Secondly, this application provides a YOLO-based sorting video recognition and processing system, the YOLO-based sorting video recognition and processing system comprising:

[0012] The segmentation module is used to mark occlusion boundaries through edge detection and connected component analysis, segment each frame of the sorting operation video into visible and occluded regions, and construct occlusion mask data.

[0013] An embedding module is used to process the occlusion mask data using the YOLO-Inpaint algorithm. The YOLO-Inpaint algorithm embeds image restoration loss calculation into the YOLO detection network and learns the mapping relationship between incomplete features and inference of object features through adversarial training to obtain a restored feature map. This includes: performing pixel-level fusion processing on the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers; constructing an adversarial training architecture of a generator network and a discriminator network based on the input feature matrix. The generator network is responsible for inferring object features within the occluded area from visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features, obtaining an adversarial loss feedback signal; calculating the image restoration loss value based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between generated features and real features, and the perceptual error measures the semantic similarity between generated features and real features, obtaining a comprehensive loss gradient; backpropagating the comprehensive loss gradient to the feature extraction layer of the YOLO detection network for weight update, and the updated network performs feature restoration processing on the occluded area to obtain a restored feature map.

[0014] The calculation module is used to calculate the item integrity score based on the repair feature map output by the YOLO-Inpaint algorithm. The score is calculated by the ratio of the visible pixel area to the predicted total area and the similarity between the repaired area and the real area.

[0015] The fusion module is used to weight and fuse the item integrity score and the detection confidence score. The weight ratio is adaptively adjusted according to the degree of occlusion to generate the sorted item recognition result after occlusion compensation.

[0016] The output module is used to construct a time-annotated sorting video recognition data stream based on the sorted item recognition results, including the item category, location coordinates, occlusion status and recognition confidence of each frame, and output a sorting video recognition processing report.

[0017] Thirdly, a YOLO-based sorting video recognition and processing device is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the YOLO-based sorting video recognition and processing device to execute the aforementioned YOLO-based sorting video recognition and processing method.

[0018] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the above-described YOLO-based sorting video recognition processing method.

[0019] The technical solution provided in this application solves the problem that traditional methods cannot accurately identify and locate occlusion areas by constructing occlusion mask data through edge detection and connected component analysis to mark occlusion boundaries. The YOLO-Inpaint algorithm embeds image restoration loss calculation and employs adversarial training to learn the mapping relationship between incomplete and complete features, overcoming the limitation of existing YOLO algorithms that can only recognize based on visible parts. Through the adversarial mechanism between the generator and discriminator networks, it can intelligently infer the object features of the occluded part from the visible features around the occluded area, significantly improving the recognition accuracy and robustness under occlusion conditions. The object integrity score establishes a scientific method for quantitatively evaluating the quality of occlusion restoration by using the ratio of visible pixel area to predicted total area and the similarity calculation between the restored area and the real area, providing an objective basis for judging the credibility of the recognition results. The technical feature of adaptively adjusting the weight ratio according to the degree of occlusion and performing weighted fusion achieves an intelligent balance between detection confidence and integrity score. It makes full use of the high reliability of the original detection results when there is slight occlusion, and relies more on the integrity score after repair when there is heavy occlusion, ensuring that the best recognition effect can be obtained under various occlusion conditions.

[0020] In specific applications of sorting video recognition processing, the adversarial training mechanism and image restoration loss calculation features of the YOLO-Inpaint algorithm have brought revolutionary improvements to item recognition in occluded scenes. The algorithm learns the mapping relationship between incomplete and complete features, enabling the sorting system to maintain stable recognition performance in complex occluded environments in industrial settings. The construction technology of time-series labeled sorting video recognition data streams organizes discrete frame recognition results into continuous time-series data. This not only facilitates full-process tracking and quality monitoring of sorting but also provides complete data support for performance optimization and fault diagnosis of sorting equipment. Especially in practical applications of automated sorting lines, this time-series data stream can effectively support advanced functions such as sorting efficiency analysis, anomaly detection, and predictive maintenance, thereby significantly improving the intelligence level and operational reliability of the entire sorting system. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an embodiment of the sorting video recognition and processing method based on YOLO in this application.

[0023] Figure 2This is a schematic diagram of the adaptive weight allocation mechanism in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of an embodiment of the YOLO-based sorting video recognition and processing system in this application.

[0025] Figure 4 This is a schematic block diagram of the YOLO-based sorting video recognition and processing device in an embodiment of the present invention. Detailed Implementation

[0026] This application provides a sorting video recognition processing method and system based on YOLO. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0027] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the sorting video recognition and processing method based on YOLO in this application includes:

[0028] Step S1: Mark the occlusion boundaries by edge detection and connected component analysis, and segment each frame of the sorting operation video into visible and occluded areas to construct occlusion mask data;

[0029] Step S2: The occlusion mask data is processed using the YOLO-Inpaint algorithm. The YOLO-Inpaint algorithm embeds image restoration loss calculation into the YOLO detection network and learns the mapping relationship between inferring object features from incomplete features through adversarial training to obtain a restored feature map. This includes: performing pixel-level fusion processing on the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers; constructing an adversarial training architecture of a generator network and a discriminator network based on the input feature matrix. The generator network is responsible for inferring the object features in the occluded area from the visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features to obtain an adversarial loss feedback signal; calculating the image restoration loss value based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between generated features and real features, and the perceptual error measures the semantic similarity between generated features and real features to obtain a comprehensive loss gradient; backpropagating the comprehensive loss gradient to the feature extraction layer of the YOLO detection network for weight update, and the updated network performs feature restoration processing on the occluded area to obtain a restored feature map.

[0030] Step S3: Calculate the item integrity score based on the repair feature map output by the YOLO-Inpaint algorithm. The score is calculated by the ratio of the visible pixel area to the predicted total area and the similarity between the repaired area and the real area.

[0031] Step S4: The item integrity score and the detection confidence are weighted and fused, and the weight ratio is adaptively adjusted according to the degree of occlusion to generate the sorted item recognition result after occlusion compensation.

[0032] Step S5: Construct a time-annotated sorting video recognition data stream based on the sorting item recognition results, including the item category, location coordinates, occlusion status and recognition confidence of each frame, and output a sorting video recognition processing report.

[0033] It is understood that the executing entity of this application can be a YOLO-based sorting video recognition and processing system, or it can be a terminal or a server; no specific limitation is made here. This application's embodiments use a server as an example for illustration.

[0034] Specifically, each frame of the sorting operation video first undergoes grayscale conversion, transforming the RGB three-channel color image into a single-channel grayscale image, followed by Gaussian filtering preprocessing to remove noise interference. The Canny edge detection algorithm calculates the gradient of the denoised grayscale image, using the Sobel operator to calculate the gradient intensity in the horizontal and vertical directions, and then performs non-maximum suppression processing, retaining the local maximum points of the gradient magnitude as edge candidate points. The connected component analysis algorithm uses an eight-connected region detection method to perform pixel connectivity analysis on the edge contour image, classifying adjacent edge pixels into the same connected component and assigning them a unique label. Occlusion discrimination analysis is based on the geometric features of connected components, calculating the area, perimeter, aspect ratio, and other shape descriptors of each connected component. When the area of ​​a connected component is less than a set threshold and the shape irregularity exceeds a preset parameter, the region is marked as an occluded region, and the remaining regions are marked as visible regions, ultimately generating binary mask data containing occlusion information.

[0035] The core of the YOLO-Inpaint algorithm lies in integrating image inpainting technology into the YOLO detection framework. When fusing the occlusion mask data with the original frame image at the pixel level, the occlusion mask is first binarized, with pixel values ​​in the occluded area set to 1 and those in the visible area set to 0. The RGB three-channel data of the original frame image is then channel-expanded with the binary occlusion marker map to form a four-channel fused data structure. The first three channels store color information, and the fourth channel records the occlusion state. The generator network employs an encoder-decoder architecture. The encoder extracts high-dimensional representations of visible features around the occluded area, and the decoder infers the object features within the occluded area based on these features. The discriminator network uses a convolutional neural network structure to distinguish between generated and real features. The adversarial loss feedback signal is generated through a game between the generator and the discriminator. The generator attempts to generate realistic inpainted features to deceive the discriminator, while the discriminator tries to identify the falsity of the generated features. The image inpainting loss value includes two parts: reconstruction error and perceptual error. The reconstruction error is calculated by comparing the differences between generated and real features pixel by pixel, while the perceptual error is obtained through semantic similarity measurement of the deep feature layer. The comprehensive loss gradient is passed to the convolutional layer of the YOLO detection network through the backpropagation algorithm to update the network weight parameters, enabling the network to learn to recover complete item features from incomplete information.

[0036] The calculation of the item integrity score involves multiple geometric and feature analyses. After pixel segmentation, the restored feature map is divided into foreground and background pixels using a threshold segmentation method. Foreground pixels represent the item area, and background pixels represent the non-item area. The visible pixel area is obtained by counting the total number of foreground pixels, with each pixel representing a minimum unit area in the image. The bounding rectangle fitting calculation is based on the minimum bounding rectangle algorithm of the item outline boundary, finding the smallest rectangle that can completely enclose the item outline; the product of its length and width is the predicted total area. The area integrity coefficient is calculated as the ratio of the visible pixel area to the predicted total area, reflecting the visibility of the item. The structural similarity index algorithm is used to calculate the feature similarity between the restored area and the real area. This algorithm comprehensively considers the similarity of brightness, contrast, and structure, and obtains the similarity coefficient through statistical feature comparison within a local window. The item integrity score is obtained by weighted averaging of the area integrity coefficient and the similarity coefficient; the weight allocation is determined according to the specific application scenario and data characteristics.

[0037] The weighted fusion mechanism dynamically adjusts weight allocation based on the degree of occlusion. The occlusion degree coefficient is calculated as the ratio of the total number of pixels in the occluded area to the total number of pixels in the item area, directly reflecting the severity of occlusion. The adaptive weight allocation algorithm sets two key thresholds: when the occlusion degree coefficient is less than the lower threshold, it indicates slight occlusion and high detection confidence, thus allocating a larger weight to the detection confidence; when the occlusion degree coefficient is greater than the higher threshold, it indicates severe occlusion, increasing the importance of the item integrity score and correspondingly increasing its weight. The dynamic weight allocation scheme is implemented using a piecewise linear function, smoothly transitioning between the two thresholds using linear interpolation. The comprehensive score is calculated by weighting and summing the item integrity score and the detection confidence, comprehensively reflecting the recognition reliability under occlusion conditions. The threshold judgment process determines the optimal recognition threshold based on historical statistical data; when the comprehensive score exceeds this threshold, it is marked as a valid recognition; otherwise, it is marked as an invalid recognition.

[0038] Structured data recording organizes the recognition results of each frame into a standardized data format, containing four core fields: item category identifier, bounding box coordinates, occlusion status marker, and recognition confidence score. A temporal index mapping table establishes a one-to-one correspondence between video frame numbers and recognition results; the frame number serves as the timeline index, and the recognition result data is the corresponding numerical record. The sorting video recognition data stream is formed by sequentially combining data over time, with each time node containing complete recognition information for the current frame and recording its temporal relationship with adjacent frames. Statistical analysis calculates the performance metrics of the entire video sequence, including key parameters such as recognition accuracy, occlusion handling success rate, and average confidence score. These metrics are derived from the statistical summarization of all valid recognition results.

[0039] In one specific embodiment, step S1 includes:

[0040] The single-frame images of the sorting operation video are preprocessed with grayscale conversion and Gaussian filtering to obtain the noise-reduced grayscale images;

[0041] Gradient calculation and non-maximum suppression processing are performed on the denoised grayscale image based on the Canny edge detection algorithm to obtain the edge contour image;

[0042] The edge contour image is input into a connected component analysis algorithm to perform pixel connectivity detection and region labeling, resulting in a connected region label map.

[0043] Based on the area size and shape characteristics of each connected region in the connected region label map, occlusion discrimination analysis is performed. Regions with an area smaller than a preset threshold and irregular shape are marked as occluded regions, while the remaining regions are marked as visible regions, thus obtaining occlusion mask data.

[0044] Specifically, the single-frame grayscale conversion process of the sorting operation video converts the original RGB three-channel color image into a single-channel grayscale representation. The conversion method uses a weighted average algorithm, multiplying the red channel by 0.299, the green channel by 0.587, and the blue channel by 0.114, and then adding these three values ​​to obtain the grayscale pixel value. This weighting allocation is based on the physiological characteristics of human eyes' sensitivity to different colors. Gaussian filtering preprocessing uses a two-dimensional Gaussian kernel to perform convolution operations on the grayscale image. The size of the Gaussian kernel is usually set to 5x5 or 7x7 pixels. The standard deviation parameter controls the filtering strength; a smaller standard deviation retains more detail information, while a larger standard deviation produces a stronger smoothing effect. During the filtering process, the new value of each pixel is calculated by the weighted average of its neighboring pixels according to Gaussian weights. Boundary pixels are processed using mirror expansion or zero-padding. The filtered image has significantly reduced noise while preserving edge information. The Canny edge detection algorithm first calculates the gradient of the denoised grayscale image. It uses the Sobel operator to calculate pixel gradients in both the horizontal and vertical directions. The horizontal Sobel operator uses a 3x3 convolution kernel, and the vertical operator is the transpose of the horizontal operator. The gradient magnitude at each pixel is calculated by taking the square root of the sum of the squared horizontal and squared vertical gradients. The gradient direction is obtained by calculating the ratio of the horizontal to the vertical gradient using the arctangent function. Non-maximum suppression checks the gradient magnitude of each pixel along the gradient direction. If the gradient magnitude is not a local maximum in the gradient direction, it is set to zero. This process refines broad edge regions into single-pixel-width edge lines. A dual-thresholding process sets two parameters: a high threshold and a low threshold. Pixels with gradient magnitudes higher than the high threshold are directly marked as strong edges, pixels with gradient magnitudes between the two thresholds are marked as weak edges, and pixels with gradient magnitudes lower than the low threshold are marked as non-edges. The edge connection process uses 8-neighborhood connectivity analysis to connect weak edge pixels with strong edge pixels to form a complete edge contour.

[0045] The connected component analysis algorithm performs pixel connectivity detection on edge contour images using a two-pass scanning method. The first pass scans the image line by line, starting from the top left corner. When a foreground pixel is encountered, its left and top neighbors are checked for labels. If no labeled pixel exists in the neighborhood, a new label is assigned. If a labeled pixel exists, its label is inherited. If multiple pixels with different labels exist in the neighborhood, the smallest label is selected, and the label equivalence is recorded. The second pass resolves the label equivalence problem, merging equivalent labels into the same connected component. Each connected component ultimately receives a unique label. Region labeling processes calculate the geometric properties of each connected component, including the number of pixels, centroid coordinates, circumscribed rectangle, and perimeter. The number of pixels is obtained by counting the number of pixels with the same label. The centroid coordinates are calculated by averaging the coordinates of all pixels. The circumscribed rectangle is determined by the minimum and maximum coordinate values ​​of the pixels in the connected component. The perimeter is calculated using the chain code representation of the boundary pixels. Occlusion discrimination analysis makes decisions based on the area and shape characteristics of connected components. The area threshold is set according to the typical size of the sorted items and the image resolution. Shape irregularity is measured by various shape descriptors, including parameters such as roundness, rectangularity, and aspect ratio. Roundness is defined as the ratio of area to the square of perimeter multiplied by 4π. A value close to 1 indicates that the shape is close to a circle, and a value deviating from 1 indicates that the shape is irregular. Rectangularity is defined as the ratio of the area of ​​the connected component to the area of ​​its smallest bounding rectangle. The aspect ratio is calculated by the ratio of the long side to the short side of the bounding rectangle. When the area of ​​the connected component is less than the preset threshold and any shape descriptor is outside the normal range, the region is marked as an occluded region, and the remaining regions that meet the conditions are marked as visible regions.

[0046] The process of generating occlusion mask data converts the occlusion discrimination result into a binary image format. The pixel value of the occluded area is set to 255 (white), and the pixel value of the visible area is set to 0 (black). This binary representation facilitates subsequent pixel-level fusion processing and algorithm calculations. The mask data has the same spatial resolution and pixel correspondence as the original image. Each mask pixel accurately identifies the occlusion state of the corresponding pixel in the original image, and the boundaries of connected regions in the mask maintain a high degree of consistency with the boundaries of the actual occluded object.

[0047] In one specific embodiment, step S2 includes:

[0048] The occlusion mask data is fused with the original frame image at the pixel level to generate an input feature matrix with occlusion markers.

[0049] An adversarial training architecture is constructed based on the input feature matrix to build a generator network and a discriminator network. The generator network is responsible for inferring the features of objects in the occluded area from the visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features to obtain the adversarial loss feedback signal.

[0050] The image restoration loss value is calculated based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between the generated features and the real features, and the perceptual error measures the semantic similarity between the generated features and the real features. The comprehensive loss gradient is obtained.

[0051] The comprehensive loss gradient is backpropagated to the feature extraction layer of the YOLO detection network for weight update. The updated network then performs feature restoration processing on the occluded areas to obtain the restored feature map.

[0052] Specifically, the pixel-level fusion processing of the occlusion mask data and the original frame image converts the two-dimensional occlusion mask into a four-dimensional feature representation. The occlusion mask data is then binarized, assigning a value of 1 to pixels marked as occluded areas and a value of 0 to pixels in visible areas, forming a single-channel binary mask image. The original frame image maintains the RGB three-channel format, with each pixel containing red, green, and blue color components, with values ​​ranging from 0 to 255. The channel expansion fusion process concatenates the RGB three-channel data with the binary mask image along the channel dimension, expanding the original three-channel image into four-channel fused data. The fourth channel specifically records the occlusion state information at each pixel location. The pixel feature vector construction process organizes the four channel values ​​at each spatial location into a feature vector. The first three elements of the vector represent the color information of the pixel, and the fourth element represents the occlusion state. All pixel feature vectors are normalized to map the numerical range to between 0 and 1. The normalization method uses maximum-minimum scaling. The maximum and minimum values ​​of each channel are calculated separately, and then the pixel value is subtracted from the minimum value and divided by the difference between the maximum and minimum values. The input feature matrix with occlusion markings is organized according to the spatial structure of the image. The number of rows in the matrix equals the image height, and the number of columns equals the image width. Each matrix element contains complete information of the four feature components.

[0053] The adversarial training architecture of the generator and discriminator networks is built on the principles of generative adversarial networks (GANs). The generator network employs an encoder-decoder structure. The encoder progressively compresses the spatial resolution of the input feature matrix and increases the number of feature channels through multiple convolutional operations. Each convolutional layer is followed by batch normalization and activation functions. The downsampling process progressively compresses the input image from its original resolution to the minimum resolution of the feature map. The decoder progressively restores the spatial resolution through transposed convolutional operations while reducing the number of feature channels. During upsampling, skip connections are used to concatenate the features of the corresponding layers of the encoder with the features of the decoder, preserving the transmission of detailed information. The generator network specifically handles feature inference for occluded regions. By analyzing the texture, color, and geometric features of visible pixels around the occluded region, it learns the possible appearance of an object within the occluded region. The inference process is based on contextual information and global consistency constraints of the object. The discriminator network uses a convolutional neural network structure, receiving both real and generated feature maps as input. It extracts discriminative features through multiple convolutional and pooling operations, and finally outputs a scalar value representing the probability of the input feature being real. The training objective of the discriminator is to maximize its ability to distinguish between real and generated features. The adversarial loss feedback signal is calculated by the discriminator's judgment result on the generated features. A high loss value is generated when the discriminator judges the generated features as false, and a low loss value is generated when the discriminator is fooled by the generated features. This feedback signal guides the update direction of the generator network parameters.

[0054] The calculation of image inpainting loss comprises two components: reconstruction error and perceptual error. Reconstruction error is calculated by comparing the numerical differences between generated and real features pixel-by-pixel, using either L1 or L2 loss functions. L1 loss calculates the absolute value of the difference between the generated and real pixel values, while L2 loss calculates the square of the difference. Reconstruction error measures the pixel-level accuracy of the generated result. Perceptual error calculates the semantic similarity between generated and real features based on deep feature representations. A pre-trained deep network extracts high-level semantic features, and perceptual differences are calculated by comparing the cosine similarity or Euclidean distance of feature vectors. Perceptual error focuses on the semantic consistency of the generated result rather than pixel-level precise matching. The comprehensive loss gradient is calculated by weightedly combining reconstruction error, perceptual error, and adversarial loss. The weights of each loss are adjusted according to the training stage and specific task requirements. Gradient calculation uses automatic differentiation, updating the weights along the negative gradient direction of the network parameters.

[0055] The backpropagation process transmits the combined loss gradient from the network's output layer to the input layer layer by layer. When passing through the feature extraction layer of the YOLO detection network, the gradient information guides the adjustment of the convolutional kernel weights and bias parameters. Weight updates employ stochastic gradient descent or Adam optimization algorithms, while the learning rate parameter controls the update step size. During feature inpainting, the updated YOLO network reprocesses the input feature matrix with occlusion markers. The network applies the learned inpainting knowledge to the occluded region, inferring the object features within the occluded region based on the feature information of the surrounding visible region. The inpainted feature map presents a reasonable object appearance and texture information in the occluded region.

[0056] In one specific embodiment, the process of performing pixel-level fusion processing of the occlusion mask data and the original frame image can specifically include the following steps:

[0057] Binarize the occlusion mask data by setting the pixel values ​​of the occluded area to 1 and the pixel values ​​of the visible area to 0, and obtain a binary occlusion marker image.

[0058] The RGB three-channel data of the original frame image is fused with the binary occlusion marker map through channel expansion, and an occlusion marker channel is added to the original three channels to obtain four-channel fused image data;

[0059] Pixel feature vectors are constructed based on four-channel fused image data. Each pixel location contains three color components (red, green, and blue) and one occlusion state component. The feature vectors are then normalized to obtain standardized pixel feature vectors.

[0060] The standardized pixel feature vectors are reorganized into a matrix form according to their spatial location. The matrix dimension is the image height multiplied by the image width multiplied by the number of feature channels, resulting in an input feature matrix with occlusion markers.

[0061] Specifically, the binarization encoding of the occlusion mask data converts continuous value masks into standard binary representations. The encoding rules strictly classify pixels according to their attributes: pixel values ​​in occluded areas are uniformly set to 1, while pixel values ​​in visible areas are uniformly set to 0. This binarization ensures that each pixel location has only two distinct state identifiers. The binary occlusion marker image is a single-channel image with the same size as the original sorting video frame. The value of each pixel location directly corresponds to the occlusion state at that location, and the boundaries of connected regions in the marker image maintain a precise correspondence with the boundaries of the actual occluded objects. The RGB three-channel data of the original frame image maintains a standard color space format: the red channel records the intensity of the red component of the pixel, the green channel records the intensity of the green component, and the blue channel records the intensity of the blue component. The value range of each channel is between 0 and 255. The channel expansion and fusion operation is performed at the data structure level, concatenating the RGB three-channel data with the single-channel binary occlusion marker image in the third dimension. The concatenation process maintains the spatial correspondence of the pixels. The original three-channel data is retained as the first three channels, and the binary occlusion marker image is added to the data structure as the fourth channel. The dimensional structure of the four-channel fused image data is height × width × 4, where the first three channels store color information, the fourth channel stores occlusion status information, and each pixel position contains four independent numerical components.

[0062] The pixel feature vector construction process reorganizes the four-channel fused image data into a vector representation. The values ​​of the four channels at each spatial location are combined into a feature vector. The first element of the vector corresponds to the red component, the second to the green component, the third to the blue component, and the fourth to the occlusion state component. The color component values ​​range from 0 to 255, while the occlusion state component values ​​range from 0 to 1. The difference in value ranges between these components needs to be normalized. Normalization uses a maximum-minimum scaling method, calculating the normalization parameter for each channel. The minimum value for the color channel is 0, and the maximum value is 255. The normalization formula is that the new value equals the original value divided by 255. The occlusion state channel values ​​are already between 0 and 1, so no additional normalization is needed. All elements of the standardized pixel feature vector have a value range between 0 and 1, and the vector length is fixed at 4 elements. Each vector fully describes the color and occlusion attributes of a pixel location.

[0063] The matrix recombination process arranges standardized pixel feature vectors according to the spatial structure of the image. The recombined matrix maintains the two-dimensional spatial layout of the original image. The number of rows corresponds to the image height, and the number of columns corresponds to the image width. Each matrix element stores the four-dimensional feature vector of the corresponding pixel. The complete dimension of the input feature matrix with occlusion markings is height × width × 4. The third dimension of 4 indicates that each pixel position contains 4 feature components. Each element in the matrix is ​​a standardized value ranging from 0 to 1. The data access mode of the feature matrix follows row-major order. The first dimension index corresponds to the row coordinates of the image, the second dimension index corresponds to the column coordinates of the image, and the third dimension index corresponds to the type of feature component. Index values ​​0, 1, and 2 correspond to the red, green, and blue color components, respectively, and index value 3 corresponds to the occlusion state component.

[0064] In one specific embodiment, step S3 includes:

[0065] The repaired feature map is segmented into pixels. The pixels in the repaired object area are marked as foreground and the pixels in the background area are marked as background. The number of foreground pixels is counted to obtain the area value of visible pixels.

[0066] Based on the object outline boundary in the repaired feature map, the bounding rectangle is fitted and calculated. The product of the length and width of the bounding rectangle is calculated to obtain the predicted total area value.

[0067] The area integrity coefficient is obtained by calculating the ratio of the visible pixel area value to the predicted total area value.

[0068] The similarity between the repaired region and the corresponding real region in the repair feature map is calculated. The structural similarity index algorithm is used to calculate the similarity of texture features and obtain the similarity coefficient. The area integrity coefficient and the similarity coefficient are weighted and averaged to obtain the item integrity score.

[0069] Specifically, the pixel segmentation process for the restored feature map employs a threshold segmentation method to classify image pixels into foreground and background categories. The segmentation process first performs grayscale conversion on the restored feature map, transforming multi-channel feature data into single-channel grayscale values. The conversion method calculates the comprehensive feature value of each pixel by weighted averaging of the feature intensity of each channel. The threshold segmentation algorithm sets a segmentation threshold, which is automatically calculated using Otsu's method based on the histogram distribution of the restored feature map. Otsu's method finds the segmentation point with the largest inter-class variance as the optimal threshold by traversing all possible thresholds. In the pixel classification process, pixels with feature values ​​higher than the threshold are marked as foreground, representing the restored object area, while pixels with feature values ​​lower than the threshold are marked as background, representing non-object areas. Foreground pixel statistics are calculated by traversing the segmented image to determine the total number of pixels marked as foreground. Each foreground pixel represents a minimum area unit in the image. The total number of pixels is multiplied by the actual area represented by each pixel to obtain the visible pixel area value. The pixel area unit is determined based on the image resolution and the actual shooting distance. The bounding rectangle fitting calculation is based on the boundary points of the object's outline in the repaired feature map. Outline extraction uses an edge detection algorithm to find the boundary line between the object and the background. The set of boundary points contains the coordinate information of all pixels on the outline. The minimum bounding rectangle algorithm traverses all boundary points to find the smallest rectangle enclosing the outline. The coordinates of the four vertices of the rectangle are determined by the minimum and maximum coordinate values ​​of the boundary points. The coordinates of the top-left vertex are the minimum x and y coordinates among the boundary points, and the coordinates of the bottom-right vertex are the maximum x and y coordinates among the boundary points. The length of the bounding rectangle is calculated by subtracting the x-coordinate of the top-left vertex from the x-coordinate of the bottom-right vertex, and the width is calculated by subtracting the y-coordinate of the top-left vertex from the y-coordinate of the bottom-right vertex. The product of the length and width is the predicted total area.

[0070] The area integrity coefficient is calculated by dividing the visible pixel area by the predicted total area. The result ranges from 0 to 1, with a value closer to 1 indicating higher visibility and a value closer to 0 indicating more severe occlusion. The structural similarity index algorithm calculates the similarity of texture features between the restored and real regions. This algorithm considers the similarity of brightness, contrast, and structure. Brightness similarity is calculated by comparing the average pixel values ​​of the two regions; contrast similarity is calculated by comparing the standard deviations of the pixel values; and structural similarity is calculated by comparing the normalized pixel values ​​after removing the effects of brightness and contrast. The structural similarity index calculation process first divides the restored and real regions into local windows of the same size, typically 8×8 or 11×11 pixels. Then, three similarity components are calculated for each window. The final similarity index is calculated by the weighted product of these three components. The similarity coefficient ranges from -1 to 1. The closer the value is to 1, the more similar the repaired area is to the real area. The closer the value is to -1, the greater the difference.

[0071] The item integrity score is calculated by weighted averaging of the area integrity coefficient and the similarity coefficient. In the weighted averaging formula, the area integrity coefficient has a weight of 0.6, and the similarity coefficient has a weight of 0.4. This weighting is determined based on the importance of area information in occlusion recognition. The weighted averaging process involves multiplying the area integrity coefficient by its weight and the similarity coefficient by its weight, then summing the two weighted values ​​to obtain the final item integrity score. The score ranges from 0 to 1.

[0072] In one specific embodiment, step S4 includes:

[0073] The occlusion degree coefficient is calculated based on the occlusion mask data. The ratio of the total number of pixels in the occluded area to the total number of pixels in the object area is calculated to obtain the occlusion degree value.

[0074] An adaptive weight allocation algorithm is designed based on the occlusion degree value. When the occlusion degree value is less than the preset low threshold, the detection confidence weight is set as the dominant weight. When the occlusion degree value is greater than the preset high threshold, the item integrity score weight is set as the dominant weight, thus obtaining a dynamic weight allocation scheme.

[0075] The item integrity score and the detection confidence score are weighted and summed according to a dynamic weighting scheme to calculate the integrated score value.

[0076] The comprehensive score value is subjected to threshold judgment processing. When the comprehensive score value is higher than the recognition threshold, it is marked as a valid recognition result. When the comprehensive score value is lower than the recognition threshold, it is marked as an invalid recognition result, thus obtaining the sorted item recognition result after occlusion compensation.

[0077] Specifically, the occlusion severity coefficient is calculated based on pixel statistical analysis of the occlusion mask data. The calculation process first involves pixel classification and statistics of the occlusion mask data. The total number of pixels in the occluded area is obtained by traversing the mask image and counting the number of pixels with a value of 1. The total number of pixels in the item area is obtained by counting the number of pixels within the entire item outline. The item area includes the sum of pixels in both the visible and occluded areas. The occlusion severity value is calculated by dividing the total number of pixels in the occluded area by the total number of pixels in the item area. This ratio directly reflects the severity of the occlusion, ranging from 0 to 1. A value close to 0 indicates slight occlusion, while a value close to 1 indicates severe occlusion. The adaptive weight allocation algorithm designs a segmented weight strategy based on the occlusion severity value. The algorithm presets two key threshold parameters: a low threshold typically set to 0.3 and a high threshold typically set to 0.7. These threshold parameters are determined based on the actual needs of the sorting scenario and historical data statistics. The weight allocation logic employs a conditional judgment approach. When the occlusion level is less than a preset low threshold, it indicates a mild occlusion, and the original detection result from the YOLO detection network has high reliability. Therefore, the detection confidence weight is set as the primary weight of 0.8, and the item integrity score weight is set as the secondary weight of 0.2. When the occlusion level is greater than a preset high threshold, it indicates a severe occlusion, and the repair result and integrity score from the YOLO-Inpaint algorithm have higher reference value. Therefore, the item integrity score weight is set as the primary weight of 0.8, and the detection confidence weight is set as the secondary weight of 0.2. When the occlusion level falls between the two thresholds, a linear interpolation method is used to calculate the weight allocation. The interpolation formula maps the occlusion level to the weight range, achieving a smooth transition in weights.

[0078] The dynamic weight allocation scheme generates a standardized data structure from the weight calculation results. The scheme includes two values: a detection confidence weight and an item integrity score weight. The sum of these two weights is always equal to 1, ensuring the mathematical correctness of the weighted fusion. The weighted summation operation fuses the item integrity score and detection confidence score according to the dynamic weight allocation scheme. The formula is: item integrity score multiplied by its corresponding weight plus detection confidence score multiplied by its corresponding weight. The resulting comprehensive score considers both occlusion repair quality and original detection reliability. The comprehensive score ranges from 0 to 1; a higher value indicates higher overall reliability, while a lower value indicates greater uncertainty. Threshold judgment processing performs a binary classification decision based on the comprehensive score, using a preset recognition threshold. The recognition threshold is determined according to the accuracy requirements and misidentification tolerance of the sorting task, typically set between 0.6 and 0.8. The judgment logic is implemented using simple numerical comparison. When the comprehensive score value is higher than the recognition threshold, the judgment algorithm marks the recognition result as a valid recognition, indicating that the recognition result has reached an acceptable confidence level. When the comprehensive score value is lower than the recognition threshold, the judgment algorithm marks the recognition result as an invalid recognition, indicating that the recognition result has a large degree of uncertainty and needs further processing or manual confirmation.

[0079] The sorted item recognition results after occlusion compensation include recognition status markers and relevant scoring information. Valid recognition results are accompanied by complete information such as item category, location coordinates, occlusion status, and final confidence level. Invalid recognition results only retain basic detection information and are marked as pending confirmation. The data format of the recognition results adopts a structured representation, which facilitates subsequent data stream processing and statistical analysis. The confidence level information in the results establishes a data foundation for sorting quality assessment and equipment performance monitoring.

[0080] Figure 2 This diagram illustrates an adaptive weight allocation mechanism based on occlusion severity values. The horizontal axis represents the occlusion severity value range from 0 to 1, and the vertical axis represents the weight allocation ratio. The light gray area in the diagram represents the allocation ratio of detection confidence weights, and the dark gray area represents the allocation ratio of item integrity score weights. When the occlusion severity value is less than 0.3 (low threshold), the detection confidence weight dominates at 0.8; when the occlusion severity value is greater than 0.7 (high threshold), the item integrity score weight dominates at 0.8; linear interpolation is used between 0.3 and 0.7 to achieve a smooth transition in weights. This mechanism dynamically adjusts the weight ratio of the two scores according to the actual occlusion situation. In cases of mild occlusion, it relies more on the original detection result, while in cases of severe occlusion, it relies more on the repaired integrity score, thus achieving optimal fusion of the sorted item recognition results after occlusion compensation.

[0081] In one specific embodiment, step S5 includes:

[0082] The sorted item recognition results after occlusion compensation are processed into structured data, and the item category, bounding box position coordinates, occlusion status identifier and recognition confidence value of each frame are organized into structured data records.

[0083] A time-series index mapping table is constructed based on structured data records. The video frame number is used as the time index, and the recognition result data of each frame is used as the corresponding data value. A one-to-one correspondence between the frame number and the recognition result is established to obtain the time-series labeled dataset.

[0084] The time-series labeled datasets are concatenated and combined in chronological order to form a continuous sorting video recognition data stream. Each time node in the data stream contains complete recognition information of the current frame and its association with the previous and next frames.

[0085] The sorting video recognition data stream is statistically analyzed and formatted for output. The recognition accuracy, occlusion handling success rate and average confidence are calculated. The analysis results are integrated with the original data stream into a standard format document to obtain the sorting video recognition processing report.

[0086] Specifically, the structuring processing of the sorted item recognition results after occlusion compensation converts the original recognition output into a standardized data format. The process first defines the field specifications for the structured data records. Each record includes an item category field storing the name of the recognized item type, a bounding box position coordinate field storing the rectangular positioning information of the item in the image, an occlusion status identifier field recording the occlusion level of the item, and a recognition confidence score field storing the reliability score of the algorithm output. The data records are organized using a key-value pair structure, with each field name as the key and the corresponding recognition result as the value. All fields are combined to form a complete recognition record. The record format uses standardized data exchange formats such as JSON or XML to ensure cross-system compatibility. The bounding box position coordinates include the coordinate information of the top-left and bottom-right corners, calculated based on the image pixel coordinate system. The occlusion status identifier uses enumerated values, including classification levels such as no occlusion, slight occlusion, moderate occlusion, and severe occlusion. The recognition confidence score ranges from 0 to 1, indicating the reliability of the recognition result. The temporal index mapping table is constructed based on the association between video frame numbers and recognition result data. The mapping table uses a hash table data structure for fast index access. The keys in the table are video frame numbers, and the values ​​are the structured recognition result data for the corresponding frame. Video frame numbers are sequentially incremented from 1, uniquely identifying a point in time within the video. The recognition result data contains complete information about all detected items in that frame. The mapping process iterates through all processed video frames, binding the recognition result of each frame with its corresponding frame number, forming a one-to-one correspondence between frame numbers and recognition data. The mapping table supports quick lookup of the corresponding recognition result based on the frame number and also supports batch extraction of recognition data based on a time range.

[0087] The generation of the time-series labeled dataset involves integrating the mapping relationships of all frames to form a complete time-series data structure. The dataset is organized chronologically, with each time node corresponding to a video frame. The node data includes the recognition result and timestamp information for that frame. Serial combination processing connects the discrete frame recognition results into a continuous data stream. The processing establishes relationships between adjacent frames, judging the continuity of items by comparing their position coordinates and feature information. When the positional changes of items in adjacent frames are within a reasonable range and the feature similarity is high, a tracking association is established, forming the motion trajectory of the items in the time series. The structure of the sorting video recognition data stream includes two dimensions: a time axis and an item trajectory. The time axis records the video's progress, while the item trajectory records the state changes of each item at different time points. Each time node in the data stream contains complete recognition information for the current frame, and also records the relationships with the previous and next frames. These relationships include the correspondence of item IDs, position change vectors, state transition records, and other temporal features.

[0088] Statistical analysis is performed to evaluate the performance and quality of the sorting video recognition data stream. The analysis process calculates several key indicators reflecting the overall performance of the algorithm. Recognition accuracy is calculated as the ratio of correctly recognized items to the total number of recognized items. Correct recognition is based on manually labeled standard answers or pre-defined verification rules. The statistical process verifies the accuracy of all recognition results throughout the data stream. Occlusion handling success rate is specifically calculated for items with occlusion. The proportion of successfully handled occluded items is calculated by comparing the recognition results with the standard answer. This indicator directly reflects the repair effect of the YOLO-Inpaint algorithm and the effectiveness of the occlusion compensation mechanism. Average confidence is calculated as the arithmetic mean of the confidence scores of all valid recognition results. This indicator reflects the overall reliability and stability of the algorithm's output. Formatted output processing integrates the statistical analysis results with the original data stream into a standard format document. The document structure includes a data stream overview, a summary of statistical indicators, detailed recognition records, and explanations of anomalies. The document format uses standard formats such as PDF or HTML for easy reading and sharing.

[0089] The YOLO-based sorting video recognition and processing method in the embodiments of this application has been described above. The YOLO-based sorting video recognition and processing system in the embodiments of this application is described below. Please refer to [link / reference]. Figure 3 One embodiment of the YOLO-based sorting video recognition and processing system in this application includes:

[0090] The segmentation module is used to mark occlusion boundaries through edge detection and connected component analysis, segment each frame of the sorting operation video into visible and occluded regions, and construct occlusion mask data.

[0091] An embedding module is used to process the occlusion mask data using the YOLO-Inpaint algorithm. The YOLO-Inpaint algorithm embeds image restoration loss calculation into the YOLO detection network and learns the mapping relationship between incomplete features and inference of object features through adversarial training to obtain a restored feature map. This includes: performing pixel-level fusion processing on the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers; constructing an adversarial training architecture of a generator network and a discriminator network based on the input feature matrix. The generator network is responsible for inferring object features within the occluded area from visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features, obtaining an adversarial loss feedback signal; calculating the image restoration loss value based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between generated features and real features, and the perceptual error measures the semantic similarity between generated features and real features, obtaining a comprehensive loss gradient; backpropagating the comprehensive loss gradient to the feature extraction layer of the YOLO detection network for weight update, and the updated network performs feature restoration processing on the occluded area to obtain a restored feature map.

[0092] The calculation module is used to calculate the item integrity score based on the repair feature map output by the YOLO-Inpaint algorithm. The score is calculated by the ratio of the visible pixel area to the predicted total area and the similarity between the repaired area and the real area.

[0093] The fusion module is used to weight and fuse the item integrity score and the detection confidence score. The weight ratio is adaptively adjusted according to the degree of occlusion to generate the sorted item recognition result after occlusion compensation.

[0094] The output module is used to construct a time-annotated sorting video recognition data stream based on the sorted item recognition results, including the item category, location coordinates, occlusion status and recognition confidence of each frame, and output a sorting video recognition processing report.

[0095] above Figure 3 The YOLO-based sorting video recognition and processing system in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The YOLO-based sorting video recognition and processing device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0096] Reference Figure 4 This invention also provides a YOLO-based sorting video recognition and processing device, which can be a server, and its internal structure can be as follows: Figure 4As shown, the YOLO-based sorting video recognition processing device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor, designed as a computer, provides computing and control capabilities. The memory of the YOLO-based sorting video recognition processing device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the YOLO-based sorting video recognition processing device stores the data corresponding to this embodiment. The network interface of the YOLO-based sorting video recognition processing device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0097] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the YOLO-based sorting video recognition and processing device to which the present invention is applied.

[0098] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the YOLO-based sorting video recognition processing method.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a YOLO-based sorting video recognition and processing device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sorting video recognition and processing method based on YOLO, characterized in that, The method includes: Step S1: Mark the occlusion boundaries by edge detection and connected component analysis, and segment each frame of the sorting operation video into visible and occluded areas to construct occlusion mask data; Step S2: The occlusion mask data is processed using the YOLO-Inpaint algorithm. The YOLO-Inpaint algorithm embeds image restoration loss calculation into the YOLO detection network and learns the mapping relationship between inferring object features from incomplete features through adversarial training to obtain a restored feature map. This includes: performing pixel-level fusion processing on the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers; constructing an adversarial training architecture of a generator network and a discriminator network based on the input feature matrix. The generator network is responsible for inferring object features within the occluded area from visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features to obtain an adversarial loss feedback signal; calculating the image restoration loss value based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between generated features and real features, and the perceptual error measures the semantic similarity between generated features and real features to obtain a comprehensive loss gradient; backpropagating the comprehensive loss gradient to the feature extraction layer of the YOLO detection network for weight update, and the updated network performs feature restoration processing on the occluded area to obtain a restored feature map. Step S3: Calculate the item integrity score based on the repair feature map output by the YOLO-Inpaint algorithm. The score is calculated by the ratio of the visible pixel area to the predicted total area and the similarity between the repaired area and the real area. Step S4: The item integrity score and the detection confidence are weighted and fused together. The weight ratio is adaptively adjusted according to the degree of occlusion to generate the sorted item recognition result after occlusion compensation. Step S5: Based on the sorted item recognition results, construct a time-annotated sorting video recognition data stream, including the item category, location coordinates, occlusion status, and recognition confidence of each frame, and output a sorting video recognition processing report.

2. The sorting video recognition and processing method based on YOLO according to claim 1, characterized in that, Step S1 includes: The single-frame images of the sorting operation video are preprocessed with grayscale conversion and Gaussian filtering to obtain the noise-reduced grayscale images; Gradient calculation and non-maximum suppression processing are performed on the denoised grayscale image based on the Canny edge detection algorithm to obtain the edge contour image; The edge contour image is input into a connected component analysis algorithm for pixel connectivity detection and region labeling to obtain a connected region label map. Based on the area size and shape characteristics of each connected region in the connected region label map, occlusion discrimination analysis is performed. Regions with an area smaller than a preset threshold and irregular shape are marked as occluded regions, and the remaining regions are marked as visible regions, thus obtaining occlusion mask data.

3. The sorting video recognition and processing method based on YOLO according to claim 1, characterized in that, The step of performing pixel-level fusion processing between the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers includes: The occlusion mask data is binarized and encoded, with the pixel values ​​of the occluded area set to 1 and the pixel values ​​of the visible area set to 0, to obtain a binary occlusion marker image. The RGB three-channel data of the original frame image is fused with the binary occlusion marker image through channel expansion, and an occlusion marker channel is added to the original three channels to obtain four-channel fused image data; Pixel feature vectors are constructed based on the four-channel fused image data. Each pixel location contains three color components (red, green, and blue) and one occlusion state component. The feature vectors are then normalized to obtain standardized pixel feature vectors. The standardized pixel feature vectors are reorganized into a matrix form according to their spatial location. The matrix dimension is the image height multiplied by the image width multiplied by the number of feature channels, resulting in an input feature matrix with occlusion markers.

4. The sorting video recognition and processing method based on YOLO according to claim 1, characterized in that, Step S3 includes: The repaired feature map is segmented into pixels. The pixels in the repaired object area are marked as foreground and the pixels in the background area are marked as background. The number of foreground pixels is counted to obtain the visible pixel area value. Based on the object outline boundary in the repaired feature map, the bounding rectangle is fitted and calculated. The product of the length and width of the bounding rectangle is calculated to obtain the predicted total area value. The area integrity coefficient is obtained by calculating the ratio of the visible pixel area value to the predicted total area value. The similarity of the repaired region and the corresponding real region in the repair feature map is calculated. The similarity of the texture features is calculated using the structural similarity index algorithm to obtain the similarity coefficient. The area integrity coefficient and the similarity coefficient are weighted and averaged to obtain the item integrity score.

5. The sorting video recognition and processing method based on YOLO according to claim 1, characterized in that, Step S4 includes: The occlusion degree coefficient is calculated based on the occlusion mask data. The ratio of the total number of pixels in the occluded area to the total number of pixels in the item area is calculated to obtain the occlusion degree value. An adaptive weight allocation algorithm is designed based on the occlusion degree value. When the occlusion degree value is less than the preset low threshold, the detection confidence weight is set as the dominant weight. When the occlusion degree value is greater than the preset high threshold, the item integrity score weight is set as the dominant weight, thus obtaining a dynamic weight allocation scheme. The item integrity score and the detection confidence score are weighted and summed according to the dynamic weight allocation scheme to calculate the integrated score value. The comprehensive score value is subjected to threshold judgment processing. When the comprehensive score value is higher than the recognition threshold, it is marked as a valid recognition result. When the comprehensive score value is lower than the recognition threshold, it is marked as an invalid recognition result, thus obtaining the sorted item recognition result after occlusion compensation.

6. The sorting video recognition and processing method based on YOLO according to claim 1, characterized in that, Step S5 includes: The sorted item recognition results after occlusion compensation are processed into structured data, and the item category, bounding box position coordinates, occlusion status identifier and recognition confidence value of each frame are organized into structured data records. Based on the structured data records, a time-series index mapping table is constructed. The video frame number is used as the time index, and the recognition result data of each frame is used as the corresponding data value. A one-to-one correspondence between the frame number and the recognition result is established to obtain the time-series labeled dataset. The time-series labeled dataset is concatenated and combined in chronological order to form a continuous sorting video recognition data stream. Each time node in the data stream contains complete recognition information of the current frame and its association with the previous and next frames. The sorting video recognition data stream is statistically analyzed and formatted for output. The recognition accuracy, occlusion handling success rate and average confidence are calculated. The analysis results are integrated with the original data stream into a standard format document to obtain a sorting video recognition processing report.

7. A sorting video recognition and processing system based on YOLO, characterized in that, For implementing the YOLO-based sorting video recognition and processing method as described in any one of claims 1-6, the YOLO-based sorting video recognition and processing system comprises: The segmentation module is used to mark occlusion boundaries through edge detection and connected component analysis, segment each frame of the sorting operation video into visible and occluded regions, and construct occlusion mask data. An embedding module is used to process the occlusion mask data using the YOLO-Inpaint algorithm. The YOLO-Inpaint algorithm embeds image restoration loss calculation into the YOLO detection network and learns the mapping relationship between incomplete features and inference of object features through adversarial training to obtain a restored feature map. This includes: performing pixel-level fusion processing on the occlusion mask data and the original frame image to generate an input feature matrix with occlusion markers; constructing an adversarial training architecture of a generator network and a discriminator network based on the input feature matrix. The generator network is responsible for inferring object features within the occluded area from visible features around the occluded area, and the discriminator network is responsible for distinguishing between real features and generated features, obtaining an adversarial loss feedback signal; calculating the image restoration loss value based on the adversarial loss feedback signal. The loss value consists of two parts: reconstruction error and perceptual error. The reconstruction error measures the pixel difference between generated features and real features, and the perceptual error measures the semantic similarity between generated features and real features, obtaining a comprehensive loss gradient; backpropagating the comprehensive loss gradient to the feature extraction layer of the YOLO detection network for weight update, and the updated network performs feature restoration processing on the occluded area to obtain a restored feature map. The calculation module is used to calculate the item integrity score based on the repair feature map output by the YOLO-Inpaint algorithm. The score is calculated by the ratio of the visible pixel area to the predicted total area and the similarity between the repaired area and the real area. The fusion module is used to weight and fuse the item integrity score and the detection confidence score. The weight ratio is adaptively adjusted according to the degree of occlusion to generate the sorted item recognition result after occlusion compensation. The output module is used to construct a time-annotated sorting video recognition data stream based on the sorted item recognition results, including the item category, location coordinates, occlusion status and recognition confidence of each frame, and output a sorting video recognition processing report.

8. A sorting video recognition and processing device based on YOLO, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the YOLO-based sorting video recognition processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it causes the processor to perform the YOLO-based sorting video recognition processing method as described in any one of claims 1 to 6.

Images