Method and system for monitoring grid congestion based on machine vision analysis

Abstract

The present invention discloses a grid congestion monitoring method and system based on machine vision analysis, and relates to the technical field of logistics congestion monitoring. The present invention comprises the following steps: video stream is acquired in real time by a camera device and preprocessed; feature map is extracted from an input image by using a convolution operation; the fused feature map predicts the position of an object by regressing the object; a trained model generates multiple bounding boxes and category predictions; a frame and confidence score obtained from a target detection algorithm are received; high confidence results are matched with a target by an algorithm; the position information and confidence score of the detection frame are updated to a tracking target; additional matching is performed between unmatched tracking targets and low confidence detection frames; the life cycle of tracking trajectories is managed; the present invention detects grid congestion by machine vision, and can automatically identify whether the number of packages in the current grid reaches two or more, and when the number of packages exceeds two or more, it is determined to be a congested grid area, and an alarm system is automatically triggered.

Description

Technical Field

[0001] The present invention belongs to the technical field of logistics congestion monitoring, and in particular relates to a grid congestion monitoring method and system based on machine vision analysis. Background Art

[0002] The cross-belt sorter is an indispensable and important equipment in the modern logistics system. It can quickly and accurately sort a large number of parcels according to their destinations. When the cross-belt sorter sorts the parcels to the designated grid, the parcels slide down quickly along the chute, and after being detected by the photoelectric sensor, they are accurately delivered to the corresponding grid. At this time, the staff can carry out subsequent processing according to the grid mark, such as packaging, loading, etc. Due to the high speed of the cross-belt sorter and the large parcels, congestion often occurs at the chute, which seriously affects the subsequent parcel sorting. In order to solve such problems, staff are usually required to conduct timely inspections to ensure that the cross-belt sorter can operate normally. Sorting; usually the cross-belt sorting equipment occupies a large area. In addition, there may be multiple sets of cross-belt sorting machines on the site, and more staff need to be arranged for inspection. When congestion occurs, it cannot be discovered in the first time, which is not only inefficient, but also affects the sorting efficiency of the equipment, thereby reducing the production capacity of the operation center. In view of this, we urgently need to explore and introduce a more efficient and intelligent congestion detection technology. This technology can use advanced machine vision to achieve automatic detection. When congestion is detected at the grid, it will automatically issue an accurate voice alarm through the alarm light installed on the grid and the audio equipment in the corresponding area, and promptly notify the staff to handle the congestion. Summary of the invention

[0003] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0004] The present invention is a grid congestion monitoring method based on machine vision analysis, comprising the following steps:

[0005] Step S1: Real-time shooting is performed in the slot area by using a detection head, and the video stream is obtained in real time by a camera device and the input data is pre-processed;

[0006] Step S2: The preprocessed image data is input into the backbone network of the deep neural network, and a multi-level feature map is extracted from the input image using a convolution operation;

[0007] Step S3: Based on the fused feature map, the detection head of the deep neural network adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box of the object;

[0008] Step S4: Apply the trained deep neural network model to the new video frame data to generate multiple bounding boxes and category predictions;

[0009] Step S5: receiving the bounding box and confidence score obtained from the target detection algorithm, and classifying the detection results;

[0010] Step S6: For high-confidence detection results, target matching is performed based on the Kalman filter and the Hungarian algorithm;

[0011] Step S7: Update the position information and confidence score of the detection box to the tracking target and enter the tracking of the next time step;

[0012] Step S8: performing additional matching between unmatched tracking targets and low-confidence detection boxes;

[0013] Step S9: managing the life cycle of the tracking trajectory. If the tracking target fails to find a matching detection result in multiple time steps, the tracking of the target is terminated and its trajectory is deleted;

[0014] The step S6 includes the following sub-steps:

[0015] Step S61, matching high-confidence targets: using high-confidence detection results, target matching is performed based on the Kalman filter and the Hungarian algorithm, and the position and speed of the target in the next frame are predicted by inputting the position and speed information of the current target, and the predicted covariance is given; the formula for predicting the target state is as follows:

[0016]

[0017] in, represents the prior state estimate at time K, represents the posterior state estimate at time K-1, A K represents the state transfer matrix, B K represents the control variable matrix, U K Indicates control input.

[0018] The covariance prediction formula is as follows:

[0019] in, represents the prior estimated covariance at time K, P K-1 represents the posterior estimated covariance at time K-1, Q represents the system noise covariance matrix;

[0020] Step S62, congestion judgment: two ROI areas are marked in the slot chute area. When the cross belt sorter sorts the parcel to the designated slot, the parcel slides down the chute and passes through the two marked ROI areas. When two or more frames appear in ROI area 1 or ROI area 2, it is judged that the slot area is congested, and the congestion signal is sent out through the alarm light installed on the slot and the audio equipment in the corresponding area.

[0021] Step S63, obtaining a matching solution: by calculating the distance between the predicted bounding box and the current frame detection box, evaluating the matching degree, and obtaining a matching solution.

[0022] Furthermore, in step S1, the video stream is acquired and input in real time by the detection head, the input image data is scaled, the input image or video frame is uniformly adjusted to a predetermined size, and the pixel value of the image is normalized to the [0,1] interval to meet the input requirements of the deep learning network.

[0023] Furthermore, in step S2, the preprocessed image data is input into the backbone network of the deep neural network, and the network uses convolution operations to extract multi-level feature maps from the input image. The feature maps are used to describe the texture, edge, shape and spatial position visual features of different areas in the image. Through the feature pyramid network and the path aggregation network, features of different scales are fused to ensure that the target detection model can effectively process multi-scale objects.

[0024] Further, the step S3 includes the following sub-steps:

[0025] Step S31, target prediction: Based on the fused feature map, target detection prediction is performed in the detection head. The detection head adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box size of the object. At each feature map position, the detection head will simultaneously predict the object category, object position, and the confidence of whether the object exists;

[0026] Step S32, verify correctness: optimize the prediction by calculating the loss function of the network. The loss function includes classification loss, bounding box regression loss and confidence loss. Adjust the weight of the model through back propagation and gradient descent algorithm.

[0027] Furthermore, the step S4 includes the following sub-steps:

[0028] Step S41, data inference: In the inference stage, the trained deep neural network model is applied to the new video frame data to generate multiple bounding boxes and category predictions.

[0029] Step S42, avoiding repeated detection: performing non-maximum suppression on the detected target results. This step suppresses overlapping detection boxes and retains the bounding box with the highest confidence, thereby avoiding multiple detections of the same target.

[0030] Furthermore, in step S5, the bounding box obtained from the target detection algorithm and the confidence score of each detection box are received, and the detection results are preliminarily classified, and the detection boxes are divided into high-confidence and low-confidence categories according to the confidence. The high-confidence detection boxes are used for the main tracking process, and the low-confidence detection boxes are used as supplementary information.

[0031] Furthermore, in step S7, the position information and confidence score of the detection box are updated to the tracking target, and the state of the Kalman filter is updated at the same time. The updated target enters the tracking of the next time step. For high-confidence targets that are not successfully matched, new tracking trajectories are started. These new targets will be initialized as new tracking objects and predicted by the Kalman filter.

[0032] Furthermore, in step S8, supplementary matching is performed using the unmatched tracking targets and low-confidence detection results in step S7, and additional matching is performed between the unmatched tracking targets and the low-confidence detection frames. The matching strategy is similar to that of the high-confidence, that is, matching is performed based on distance through the Hungarian algorithm. When the low-confidence detection results are successfully matched, the status of these targets is updated; if the match is still unsuccessful, the tracking target is considered to have temporarily disappeared.

[0033] Furthermore, in step S9, the life cycle of the tracking trajectory is managed. If no matching detection results are found for the tracking target in multiple time steps, the target is considered to have disappeared, and the tracking of the target is terminated and its trajectory is deleted. A newly detected target will start a new tracking trajectory if it does not match the existing trajectory. In this way, the algorithm continues to process new targets in the scene.

[0034] Furthermore, in step S10, the tracking information of each target is output, including the target ID, location information, and category label in each time step;

[0035] A grid congestion monitoring system based on machine vision analysis, including a congestion analysis unit, an image acquisition unit and a terminal interaction unit;

[0036] Furthermore, the congestion analysis unit includes an image processing module, a target detection and recognition module, a congestion analysis module, an alarm prompt module and a data storage management module; the image processing module is wirelessly connected to the target detection and recognition module, the congestion analysis module and the alarm prompt module in sequence; the alarm prompt module is wirelessly connected to the data storage management module;

[0037] The image processing module is used to receive the image uploaded by the image acquisition unit, extract and optimize the image information from the original image data, and remove the noise in the image to form an image that can be displayed, stored or further processed; the target detection and recognition module is applied to the video frame data to be processed through the deep neural network model, generates multiple bounding boxes and category predictions, and predicts the position of the object by regressing the center coordinates and bounding box size of the object; the congestion analysis module is used to calculate the number of frames in the current uploaded image, and mark two RO I areas in the grid chute area. When the cross-belt sorter sorts the package to the designated grid, the package slides down along the chute. When the package passes through the two marked RO I areas and multiple frames appear, it is determined that the grid area is congested; the alarm prompt module is used to send a congestion signal through the alarm light installed on the grid and the audio equipment in the corresponding area when multiple frames appear in the current grid area; the data storage management module is used to store the alarm information uploaded when congestion occurs;

[0038] Furthermore, the image acquisition unit includes a detection head, a speaker, an alarm light and a wireless communication module; the detection head is electrically connected to the wireless communication module; the detection head is located on the grid, used to collect the parcels in the grid, and transmit the video data to the congestion analysis unit through the wireless communication module;

[0039] Furthermore, the terminal interaction unit includes a display terminal and a human-computer interaction interface; the display terminal is electrically connected to the human-computer interaction interface; the display terminal is used to receive the alarm signal issued by the alarm prompt module; the human-computer interaction interface is used to present the real-time image of the alarm grid, and can also simultaneously view the real-time images, congestion status, and historical data of other grids, and perform parameter setting operations.

[0040] The present invention has the following beneficial effects:

[0041] 1. The present invention detects grid congestion through machine vision and can automatically identify whether the number of packages in the current grid has reached two or more. When the number of packages exceeds two or more, it is determined to be a congested grid area and the alarm system is automatically triggered. No manual inspection is required, thereby reducing labor and labor costs.

[0042] 2. The present invention can improve the sorting speed of the cross-belt sorting equipment and improve the sorting efficiency by automatically identifying grid congestion.

[0043] 3. Through intelligent algorithms and system optimization, the system of the present invention can realize intelligent analysis of equipment problems such as on-site congestion rate and reflux rate, which not only realizes the unified integration of various application data, but also greatly improves the efficiency of operation site management.

[0044] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 It is a flow chart of the grid congestion monitoring method based on machine vision analysis of the present invention;

[0047] Figure 2 It is a flow chart of the grid congestion monitoring system based on machine vision analysis of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1-2 As shown, the present invention is a grid congestion monitoring method based on machine vision analysis, comprising the following steps:

[0050] Step S1: Real-time shooting is performed in the slot area by using a detection head, and the video stream is obtained in real time by a camera device and the input data is pre-processed;

[0051] Step S2: The preprocessed image data is input into the YOLOv8 backbone network, and a multi-level feature map is extracted from the input image using a convolution operation;

[0052] Step S3: Based on the fused feature map, the detection head of YOLOv8 adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box of the object;

[0053] Step S4: Apply the trained YOLOv8 model to the new video frame data to generate multiple bounding boxes and category predictions;

[0054] Step S5: receiving the bounding box and confidence score obtained from the target detection algorithm, and classifying the detection results;

[0055] Step S6: For high-confidence detection results, target matching is performed based on the Kalman filter and the Hungarian algorithm;

[0056] Step S7: Update the position information and confidence score of the detection box to the tracking target and enter the tracking of the next time step;

[0057] Step S8: performing additional matching between unmatched tracking targets and low-confidence detection boxes;

[0058] Step S9: managing the life cycle of the tracking trajectory. If the tracking target fails to find a matching detection result in multiple time steps, the tracking of the target is terminated and its trajectory is deleted;

[0059] Step S6 includes the following sub-steps:

[0060] Step S61, matching high-confidence targets: using high-confidence detection results, target matching is performed based on the Kalman filter and the Hungarian algorithm, and the position and speed of the target in the next frame are predicted by inputting the position and speed information of the current target, and the predicted covariance is given; the formula for predicting the target state is as follows:

[0061]

[0062] in, represents the prior state estimate at time K, represents the posterior state estimate at time K-1, A K represents the state transfer matrix, B K represents the control variable matrix, U K Indicates control input.

[0063] The covariance prediction formula is as follows:

[0064] in, represents the prior estimated covariance at time K, P K-1 represents the posterior estimated covariance at time K-1, Q represents the system noise covariance matrix;

[0065] Step S62, congestion judgment: two ROI areas are marked in the grid opening chute area. When the cross-belt sorter sorts the parcel to the designated grid opening, the parcel slides down the chute and passes through the two marked ROI areas. When two or more frames appear in ROI area 1 or ROI area 2, it is judged that the grid opening area is congested, and the congestion signal is sent through the alarm light installed on the grid opening and the voice alarm of the audio equipment in the corresponding area. Under normal circumstances, the parcels should slide down the chute in order, and the number of parcels appearing in each ROI area at the same time is relatively limited. Once two or more frames appear, it means that the parcels are piled up in this area and the flow rate is slowed down, which will cause the subsequent grid openings to fail to receive and process the parcels in time, thus causing more serious congestion problems.

[0066] Step S63, obtaining a matching solution: by calculating the distance between the predicted bounding box and the current frame detection box, evaluating the matching degree, and obtaining a matching solution.

[0067] In step S1, the video stream is acquired and input in real time by the detection head. The input image data is scaled to uniformly adjust the input image or video frame to a predetermined size. During the scaling process, the appropriate proportional relationship must be followed to avoid deformation caused by excessive stretching or compression of the image, thereby affecting the accuracy of subsequent feature extraction. The pixel value of the image is normalized to the interval [0,1]. For the original pixel value of one pixel in the image, X, its normalized pixel value is X1. Assuming that the original pixel value range is [Xp,Xq], the calculation formula of X1 is:

[0068]

[0069] The purpose is to make the pixel data of different images at the same level, in line with the deep learning network's requirements for input data standardization, so that the subsequent network can perform feature learning and analysis more stably and efficiently, thereby improving the accuracy and reliability of the entire monitoring method.

[0070] In step S2, the preprocessed image data is input into the backbone network of YOLOv8. The network uses convolution operation to extract multi-level feature maps from the input image. The feature maps are used to describe the texture, edge, shape and spatial position visual features of different areas in the image. Features of different scales are fused through feature pyramid network and path aggregation network to ensure that the target detection model can effectively process multi-scale objects. Whether it is a large package close to the detection head or a relatively small package far away from the grid, features can be accurately extracted.

[0071] Step S3 includes the following steps:

[0072] Step S31, target prediction: Based on the fused feature map, target detection prediction is performed in the detection head. The detection head adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box size of the object. At each feature map position, the detection head will simultaneously predict the object category, object position, and the confidence of whether the object exists. A probability value is obtained by comprehensively evaluating the feature performance of the corresponding area of ​​the feature map to measure the possibility of the actual package existing here. This anchor-free method avoids the limitations of the traditional anchor method that requires the pre-setting of anchor frames of fixed size and proportion, and can more flexibly adapt to the detection needs of packages of different shapes and sizes, thereby improving the detection accuracy and adaptability to complex scenes.

[0073] Step S32, verify correctness: optimize the prediction by calculating the loss function of the network. The loss function includes classification loss, bounding box regression loss and confidence loss. Adjust the weight of the model through back propagation and gradient descent algorithm.

[0074] Step S4 includes the following sub-steps:

[0075] Step S41, data inference: In the inference stage, the trained YOLOv8 model is applied to the new video frame data to generate multiple bounding boxes and category predictions. For each area in the image, based on the information extracted from the feature map and the trained weight parameters, it is determined whether there is a package and its category, and then the corresponding bounding box is generated to frame the location of the package and give the corresponding category prediction result.

[0076] Step S42, avoiding repeated detection: performing non-maximum suppression on the detected target results. This step suppresses overlapping detection boxes and retains the bounding box with the highest confidence, thereby avoiding multiple detections of the same target.

[0077] In step S5, the bounding box obtained from the target detection algorithm and the confidence score of each detection box are received, and the detection results are preliminarily classified. The detection boxes are divided into high-confidence and low-confidence categories according to the confidence. The high-confidence detection boxes are used for the main tracking process, and the low-confidence detection boxes are used as supplementary information. By setting a confidence threshold, when the confidence score corresponding to the detection box is greater than or equal to this threshold, it is classified as a high-confidence detection box. This type of detection box is considered to have high reliability, so it is used in the main tracking process. Subsequently, in the key link of target tracking, these high-confidence detection results are mainly relied on to perform accurate target matching and status update operations.

[0078] In step S7, the position information and confidence score of the detection box are updated to the tracking target, and the state of the Kalman filter is updated at the same time. The updated target enters the tracking of the next time step. For high-confidence targets that are not successfully matched, new tracking trajectories are started. These new targets will be initialized as new tracking objects and predicted by the Kalman filter. This is in consideration of the fact that in actual scenarios, new packages may enter the grid chute area or packages that were previously lost in tracking may reappear. By starting a new trajectory and using the Kalman filter to predict based on the location of their first appearance and possible initial speed and other information, these newly appeared targets can be promptly included in the tracking scope of the entire monitoring system to ensure full coverage and continuous tracking of all packages in the grid area.

[0079] In step S8, the tracking targets that were not successfully matched in step S7 and the low-confidence detection results are used for supplementary matching. In the actual parcel sorting scenario, due to complex situations such as occlusion and light changes between packages, some tracking targets may not be successfully matched with the high-confidence detection frame for a while. Additional matching is performed between the unmatched tracking targets and the low-confidence detection frame. The matching strategy is similar to that of the high-confidence, that is, matching is performed based on distance through the Hungarian algorithm. When the low-confidence detection result is successfully matched, the status of these targets is updated; if it is still not matched successfully, the tracking target is regarded as temporarily disappeared.

[0080] In step S9, the life cycle of the tracking trajectory is managed. If the tracking target fails to find a matching detection result in multiple time steps, it is considered that the target has disappeared, and the tracking of the target is terminated and its trajectory is deleted. In the actual monitoring process, since the package may have successfully passed through the grid chute and entered the grid to be processed or left the monitoring area due to an accident, when a tracking target cannot find a matching detection result for multiple consecutive time steps, it can be reasonably inferred that the package is no longer within the grid area that needs to be paid attention to. At this time, the tracking of it is terminated and the corresponding trajectory is deleted to avoid invalid tracking calculations and data storage, and release computing resources for accurate tracking of other packages that are active in the grid area. The newly detected target will start a new tracking trajectory if it does not match the existing trajectory. In this way, the algorithm continues to process new targets in the scene.

[0081] In step S10, the tracking information of each target is output, including the target ID, location information, and category label in each time step;

[0082] A grid congestion monitoring system based on machine vision analysis, including a congestion analysis unit, an image acquisition unit and a terminal interaction unit;

[0083] The congestion analysis unit includes an image processing module, a target detection and recognition module, a congestion analysis module, an alarm prompt module and a data storage management module; the image processing module is wirelessly connected to the target detection and recognition module, the congestion analysis module and the alarm prompt module in sequence; the alarm prompt module is wirelessly connected to the data storage management module;

[0084] The image processing module is used to receive the image uploaded by the image acquisition unit, extract and optimize the image information from the original image data, and remove the noise in the image to form an image that can be displayed, stored or further processed; the target detection and recognition module is applied to the video frame data to be processed through the YOLOv8 model, generates multiple bounding boxes and category predictions, and predicts the position of the object by regressing the center coordinates and bounding box size of the object; the congestion analysis module is used to calculate the number of frames in the current uploaded image, and mark two RO I areas in the grid chute area. When the cross-belt sorter sorts the package to the designated grid, the package slides down the chute. When the package passes through the two marked RO I areas and multiple frames appear, it is judged that the grid area is congested; the alarm prompt module is used to send a congestion signal through the alarm light installed on the grid and the audio equipment in the corresponding area when multiple frames appear in the current grid area; the data storage management module is used to store the alarm information uploaded when congestion occurs;

[0085] The image acquisition unit includes a detection head, a speaker, an alarm light and a wireless communication module; the detection head is electrically connected to the wireless communication module; the detection head is located on the grid opening, used to collect the parcels at the grid opening, and transmit the video data to the congestion analysis unit through the wireless communication module;

[0086] The terminal interaction unit includes a display terminal and a human-computer interaction interface; the display terminal is electrically connected to the human-computer interaction interface; the display terminal is used to receive the alarm signal issued by the alarm prompt module; the human-computer interaction interface is used to present the real-time image of the alarm grid, and can also view the real-time images, congestion status, and historical data of other grids at the same time, and perform parameter setting operations.

[0087] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0088] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for monitoring grid congestion based on machine vision analysis, characterized in that: The steps include: Step S1: Real-time shooting is performed in the slot area by using a detection head, and the video stream is obtained in real time by a camera device and the input data is pre-processed; Step S2: The preprocessed image data is input into the backbone network of the deep neural network, and a multi-level feature map is extracted from the input image using a convolution operation; Step S3: Based on the fused feature map, the detection head of the deep neural network adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box of the object; Step S4: Apply the trained deep neural network model to the new video frame data to generate multiple bounding boxes and category predictions; Step S5: receiving the bounding box and confidence score obtained from the target detection algorithm, and classifying the detection results; Step S6: For high-confidence detection results, target matching is performed based on the Kalman filter and the Hungarian algorithm; Step S7: Update the position information and confidence score of the detection box to the tracking target and enter the tracking of the next time step; Step S8: performing additional matching between unmatched tracking targets and low-confidence detection boxes; Step S9: managing the life cycle of the tracking trajectory. If the tracking target fails to find a matching detection result in multiple time steps, the tracking of the target is terminated and its trajectory is deleted; The step S6 includes the following sub-steps: Step S61, matching high-confidence targets: using high-confidence detection results, target matching is performed based on the Kalman filter and the Hungarian algorithm, and the position and speed of the target in the next frame are predicted by inputting the position and speed information of the current target, and the predicted covariance is given; the formula for predicting the target state is as follows: in, represents the prior state estimate at time K, represents the posterior state estimate at time K-1, A K represents the state transfer matrix, B K represents the control variable matrix, U K Indicates control input. The covariance prediction formula is as follows: in, express K The prior estimate covariance of the moment, P K-1 express K The posterior estimated covariance at time -1, Q represents the system noise covariance matrix; Step S62, congestion judgment: mark two ROI areas in the compartment chute area. When the cross-belt sorter sorts the package to the designated compartment, the package slides down the chute and passes through the two marked ROI areas. When two or more frames appear in ROI area 1 or ROI area 2, it is judged that the compartment area is congested, and the congestion signal is sent through the alarm light installed on the compartment and the voice alarm of the audio equipment in the corresponding area. Step S63, obtaining a matching solution: by calculating the distance between the predicted bounding box and the current frame detection box, evaluating the matching degree, and obtaining a matching solution.

2. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: In step S1, the video stream is acquired and input in real time by the detection head, the input image data is scaled, the input image or video frame is uniformly adjusted to a predetermined size, and the pixel value of the image is normalized to the interval [0,1].

3. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: In step S2, the preprocessed image data is input into the backbone network of the deep neural network. The network uses convolution operations to extract multi-level feature maps from the input image. The feature maps are used to describe the texture, edge, shape and spatial position visual features of different regions in the image. Features of different scales are fused through a feature pyramid network and a path aggregation network.

4. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: Described step S3 comprises the following steps: Step S31, target prediction: Based on the fused feature map, target detection prediction is performed in the detection head. The detection head adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box size of the object. At each feature map position, the detection head will simultaneously predict the object category, object position, and the confidence of whether the object exists; Step S32, verify correctness: optimize the prediction by calculating the loss function of the network. The loss function includes classification loss, bounding box regression loss and confidence loss. Adjust the weight of the model through back propagation and gradient descent algorithm.

5. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: The step S4 includes the following sub-steps: Step S41, data inference: In the inference stage, the trained deep neural network model is applied to the new video frame data to generate multiple bounding boxes and category predictions. Step S42, avoiding repeated detection: performing non-maximum suppression on the detected target results. This step suppresses overlapping detection boxes and retains the bounding box with the highest confidence, thereby avoiding multiple detections of the same target.

6. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: In step S5: receiving the bounding box obtained from the target detection algorithm and the confidence score of each detection box, preliminarily classifying the detection results, and dividing the detection boxes into high-confidence and low-confidence categories according to the confidence. The high-confidence detection boxes are used for the main tracking process, and the low-confidence detection boxes are used as supplementary information.

7. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: In step S7, the position information and confidence score of the detection box are updated to the tracking target, and the state of the Kalman filter is updated at the same time. The updated target enters the tracking of the next time step. For high-confidence targets that are not successfully matched, new tracking trajectories are started. These new targets will be initialized as new tracking objects and predicted by the Kalman filter.

8. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: In the step S8: supplementary matching is performed using the unmatched tracking targets and low-confidence detection results in step S7, and additional matching is performed between the unmatched tracking targets and the low-confidence detection frames. The matching strategy is similar to that of the high-confidence, that is, matching is performed based on distance using the Hungarian algorithm. When the low-confidence detection results are successfully matched, the status of these targets is updated; If the match is still not successful, the tracking target is considered to have temporarily disappeared.

9. The method for monitoring grid congestion based on machine vision analysis according to claim 1, characterized in that: In step S9: managing the life cycle of the tracking trajectory, if the tracking target fails to find a matching detection result in multiple time steps, it is considered that the target has disappeared, the tracking of the target is terminated and its trajectory is deleted; Newly detected targets will initiate new tracking trajectories if they are not matched to existing trajectories. In this way, the algorithm continuously processes new objects in the scene.

10. A grid congestion monitoring system based on machine vision analysis, comprising a congestion analysis unit, an image acquisition unit and a terminal interaction unit, characterized in that: The congestion analysis unit includes an image processing module, a target detection and recognition module, a congestion analysis module, an alarm prompt module and a data storage management module; the image processing module is wirelessly connected to the target detection and recognition module, the congestion analysis module and the alarm prompt module in sequence; the alarm prompt module is wirelessly connected to the data storage management module; The image processing module is used to receive the image uploaded by the image acquisition unit, extract and optimize the image information from the original image data, and remove the noise in the image to form an image that can be displayed, stored or further processed; the target detection and recognition module is applied to the video frame data to be processed through the deep neural network model, generates multiple bounding boxes and category predictions, and predicts the position of the object by regressing the center coordinates and bounding box size of the object; the congestion analysis module is used to calculate the number of frames in the current uploaded image, and mark two ROI areas in the grid chute area. When the cross-belt sorting machine sorts the package to the designated grid, the package slides down the chute. When the package passes through the two marked ROI areas and multiple frames appear, it is determined that the grid area is congested; the alarm prompt module is used to send a congestion signal through the alarm light installed on the grid and the audio equipment in the corresponding area when multiple frames appear in the current grid area; the data storage management module is used to store the alarm information uploaded when congestion occurs; The image acquisition unit includes a detection head, a speaker, an alarm light and a wireless communication module; the detection head is electrically connected to the wireless communication module; the detection head is located on the grid, used to collect parcels at the grid, and transmit video data to the congestion analysis unit through the wireless communication module; The terminal interaction unit includes a display terminal and a human-computer interaction interface; the display terminal is electrically connected to the human-computer interaction interface; the display terminal is used to receive the alarm signal issued by the alarm prompt module; the human-computer interaction interface is used to present the real-time image of the alarm grid, and can also simultaneously view the real-time images, congestion status, and historical data of other grids, and perform parameter setting operations.

Images