A package misclassification monitoring method and system based on machine vision analysis

Abstract

The present application provides a method and system for monitoring package misclassification based on machine vision analysis, a computer device, and a readable storage medium. The method includes: obtaining video stream input data of multiple detection targets, processing the input data based on a target detection model, and outputting detection results, wherein the detection results include the category label, bounding box, and confidence score of each detection box for each detection target; classifying the detection results according to the confidence score, and performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting tracking information, wherein the tracking information includes the category label, location information, and identity information of the detection target in each time step; determining the target drop-grid area based on the tracking information, and determining whether to alarm based on the target drop-grid area and the misclassification recognition algorithm. The present application can automatically identify misclassified packages through the above method, reduce manual recognition, and improve the efficiency of misclassification recognition.

Description

Technical Field

[0001] The present application relates to the field of logistics technology, and in particular to a package misclassification monitoring method and system based on machine vision analysis, a computer device and a readable storage medium. Background Art

[0002] In the logistics and express delivery industry, the sorting loop is widely used as an important automated equipment. It successfully solves the problems of low efficiency and high error rate of traditional manual sorting. However, under the high-load operation of the equipment, express parcels often have problems such as abnormal grid drop and grid drift, which leads to the phenomenon of express parcels being misplaced and delayed. The reason may be that the parcel is light in weight or there are magnets in the parcel. Usually, in order to solve such problems, it is necessary to manually identify on-site and then pick out the wrongly sorted parcels.

[0003] In automated sorting equipment, operators need to manage multiple sorting grids, and missorted package detection requires on-site manual identification. This is not only inefficient, but also prone to missed detections and false detections. Summary of the invention

[0004] Based on this, it is necessary to provide a package misclassification monitoring method and system, computer equipment and readable storage medium based on machine vision analysis to address the above technical problems.

[0005] A method for monitoring package misclassification based on machine vision analysis, comprising:

[0006] Obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target;

[0007] The detection results are classified according to the confidence scores, and the target is matched based on the Kalman filter and the Hungarian algorithm, and the tracking information is output, wherein the tracking information includes the category label, location information and identity information of the detected target in each time step;

[0008] A target grid area is determined according to the tracking information, and whether to issue an alarm is determined based on the target grid area and the misclassification recognition algorithm.

[0009] In one embodiment, the package misclassification monitoring method based on machine vision analysis further includes:

[0010] Acquire historical video stream input data, and preprocess the historical video stream input data, wherein the preprocessing includes format conversion, feature extraction and feature fusion;

[0011] The neural network model is trained based on the preprocessed historical video stream input data, and the target detection model is determined.

[0012] In one embodiment, the steps of classifying the detection results according to the confidence scores, performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting the tracking information include:

[0013] The detection results are classified according to the confidence scores of the detection frames to determine the detection frames with high confidence and the detection frames with low confidence;

[0014] Use the Kalman filter and Hungarian algorithm to match the high-confidence detection box to the target;

[0015] If the target is matched successfully, the position information and confidence score of the high-confidence detection box are updated to the detection result of the corresponding detection target, and the tracking information is output.

[0016] In one embodiment, the package misclassification monitoring method based on machine vision analysis further includes:

[0017] If the target is not matched successfully, the unmatched target and the low-confidence detection frame are matched based on the Kalman filter and Hungarian algorithm;

[0018] If the unmatched target is successfully matched with the low-confidence detection frame, the position information and confidence score of the low-confidence detection frame are updated to the detection result of the corresponding detection target, and the tracking information is output.

[0019] In one embodiment, the step of determining a target grid area according to the tracking information, and determining whether to issue an alarm based on the target grid area and a misclassification recognition algorithm includes:

[0020] Determine a target grid area according to the tracking information, and determine whether the target grid is valid based on the target grid area and a pre-configured ROI area;

[0021] If the target grid is valid, the actual grid number of the target grid area is obtained, and whether to issue an alarm is determined based on the actual grid number and the misclassification recognition algorithm.

[0022] In one embodiment, if the target grid is valid, the steps of obtaining the actual grid number of the target grid area, and determining whether to issue an alarm based on the actual grid number and the misclassification recognition algorithm include:

[0023] If the target grid is valid, then the actual grid number of the target grid area is obtained;

[0024] Determine target grid drop action information and transport vehicle rotation information based on the actual grid number of the target grid drop area;

[0025] Whether to issue an alarm is determined based on the target grid-falling action information and the transport vehicle rotation information through the misclassification recognition algorithm.

[0026] In one embodiment, the step of determining whether to issue an alarm based on the target grid-dropping action information and the transport vehicle rotation information by the misclassification recognition algorithm includes:

[0027] Determine whether the target grid-falling action information and the transport vehicle rotation information are consistent within an error range by using the error classification recognition algorithm;

[0028] If the target grid drop action information and the transport vehicle rotation information are inconsistent within the error range, an alarm is issued, wherein the alarm is in the form of sound and light alarm;

[0029] If the target grid-dropping action information and the transport vehicle rotation information are consistent within the error range, no alarm is issued.

[0030] A parcel misclassification monitoring system based on machine vision analysis, comprising:

[0031] A detection module, configured to obtain video stream input data of multiple detection targets, process the input data based on a target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target;

[0032] A tracking module connected to the detection module, the tracking module is used to classify the detection results according to the confidence score, and perform target matching based on the Kalman filter and the Hungarian algorithm, and output tracking information, wherein the tracking information includes the category label, location information and identity information of the detected target in each time step;

[0033] The processing module is connected to the tracking module, and is used to determine the target grid area according to the tracking information, and determine whether to issue an alarm based on the target grid area and the misclassification recognition algorithm.

[0034] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the above embodiments when executing the computer program.

[0035] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of the above embodiments.

[0036] Compared with the prior art, the above-mentioned method and system for monitoring package misclassification based on machine vision analysis, computer equipment and readable storage medium. The method includes: obtaining video stream input data of multiple detection targets, processing the input data based on the target detection model, and outputting the detection results, wherein the detection results include the category label, bounding box and confidence score of each detection box of each detection target; classifying the detection results according to the confidence score, and performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting tracking information, wherein the tracking information includes the category label, location information and identity information of the detection target in each time step; determining the target grid area according to the tracking information, and determining whether to alarm based on the target grid area and the misclassification recognition algorithm. The present application can automatically identify misclassified packages through the above method, reduce manual recognition, reduce labor, and improve the efficiency of misclassification recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A flowchart of a method for monitoring package misclassification based on machine vision analysis provided in an embodiment of the present application;

[0039] Figure 2 A structural block diagram of a parcel misclassification monitoring system based on machine vision analysis provided in one embodiment of the present application;

[0040] Figure 3 This is a diagram of the internal structure of a computer device provided in one embodiment of the present application.

[0041] Description of reference numerals:

[0042] 10. Package misclassification monitoring system based on machine vision analysis; 100. Detection module; 200. Tracking module; 300. Processing module. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0044] The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings). In the description of this application, it should be understood that the orientation or position relationship indicated by the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. are based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to this application.

[0045] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0046] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element at the same time.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0048] See also Figure 1 , an embodiment of the present application provides a package misclassification monitoring method based on machine vision analysis. The package misclassification monitoring method based on machine vision analysis includes:

[0049] S102: Obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target.

[0050] In some implementations, the system can obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output the detection results. Specifically, the video stream data can be obtained in real time by a camera device and input into the system. The system can process the input data, and the processing steps include but are not limited to: image scaling, feature extraction, etc. Among them, image scaling can be to uniformly adjust the input image or video frame to a predetermined size (for example, 640×640 pixels), and normalize the pixel value of the image to the [0,1] interval to meet the input requirements of the deep learning network.

[0051] The processed image data is input into the target detection model (which can be a specific neural network model, such as the backbone network model of YOLOv8). The target detection model uses a series of convolution operations to extract multi-level feature maps from the input image, and then fuses the feature maps of different scales through the feature pyramid network (FPN) and the path aggregation network (PANet). Target detection is performed based on the fused feature maps, and non-maximum suppression (NMS) is performed on the detected target results. This step suppresses overlapping detection boxes and retains the bounding box with the highest confidence to avoid multiple detections of the same target. The final detection result is then output.

[0052] S104: Classify the detection results according to the confidence scores, perform target matching based on the Kalman filter and the Hungarian algorithm, and output tracking information, wherein the tracking information includes the category label, location information, and identity information of the detected target in each time step.

[0053] In some embodiments, the detection results can be preliminarily classified into two categories: high confidence detections and low confidence detections according to the confidence score of the detection frame. The high confidence detection frame is then matched based on the Kalman filter and the Hungarian algorithm. Among them, the Kalman filter is used to predict the target position, and the Hungarian algorithm is used to solve the best match between the object detected in the current frame and the previously tracked object. In some embodiments, the Kalman filter predicts the state of each tracked target, including the speed and position of the target. The Hungarian algorithm calculates the distance between the predicted bounding box and the current frame detection frame (usually the Euclidean distance or IOU, Intersection Over Union), and finds an optimal matching solution.

[0054] If the target is successfully matched (i.e. for the successfully matched target), the position information and confidence score of the high-confidence detection box are updated to the detection result of the corresponding detection target, and the tracking information is output. At the same time, the state of the Kalman filter is updated. The updated target enters the tracking of the next time step. For the high-confidence target that is not successfully matched, a new tracking trajectory is started. These new targets will be initialized as new tracking objects and predicted by the Kalman filter.

[0055] In some embodiments, if the target is not matched successfully, the unmatched target and the low-confidence detection frame are matched based on the Kalman filter and the Hungarian algorithm; if the unmatched target is matched successfully with the low-confidence detection frame, the position information and confidence score of the low-confidence detection frame are updated to the detection result of the corresponding detection target, and the tracking information is output. This ensures that the tracked target will not be lost due to temporary occlusion or reduction in target confidence.

[0056] In some embodiments, if the target still fails to match, the tracked target is considered to have temporarily disappeared (the target's life cycle will enter a countdown). If the target still fails to find a matching detection result in multiple time steps (such as 30 frames), the target is considered to have disappeared, and the tracking of the target is terminated and its track is deleted. In some embodiments, a newly detected target (whether high confidence or low confidence) will start a new tracking track if it does not match an existing track. In this way, the system can continue to process new targets in the scene.

[0057] In some embodiments, the tracking information includes the category label, location information, and identity information of the detected target in each time step. The identity information may be the ID information of the detected target. This information may be used for subsequent grid drop judgment.

[0058] S106: Determine a target grid area according to the tracking information, and determine whether to issue an alarm based on the target grid area and the misclassification recognition algorithm.

[0059] In some embodiments, the system can determine the target drop-off area according to the tracking information, and determine whether to issue an alarm based on the target drop-off area and the misclassification recognition algorithm. Specifically, when the detection target in the tracking information falls on the pre-assigned grid, the target drop-off area of ​​the detection target can be determined. If the detection target is determined to be valid, the pre-set actual grid number is obtained, and it is determined whether to issue an alarm based on the misclassification recognition algorithm. In this way, the misclassified parcels can be automatically identified, reducing manual identification, reducing labor, and improving the efficiency of misclassification recognition.

[0060] In some embodiments, the step of determining the target grid area according to the tracking information, and determining whether to issue an alarm based on the target grid area and the misclassification recognition algorithm includes: determining the target grid area according to the tracking information, and determining whether the target grid is valid based on the target grid area and a pre-configured ROI area; if the target grid is valid, obtaining the actual grid number of the target grid area, and determining whether to issue an alarm based on the actual grid number and the misclassification recognition algorithm.

[0061] In some embodiments, two ROI areas can be marked in the grid area corresponding to each grid. When the transport vehicle rotates, the package (i.e., the target) will slide onto the pre-assigned grid. If the package passes through or falls on two pre-configured ROI areas, the parcel is judged to be valid. At this time, it is necessary to obtain the actual pre-set grid number corresponding to the target grid area, and determine whether to issue an alarm based on the actual grid number and the misclassification recognition algorithm.

[0062] Specifically, the target grid drop action information and the transport vehicle rotation information can be determined based on the actual grid number of the target grid drop area; and whether to issue an alarm can be determined based on the target grid drop action information and the transport vehicle rotation information by the misclassification recognition algorithm. In some embodiments, the misclassification recognition algorithm can be used to determine whether the target grid drop action information and the transport vehicle rotation information are consistent within an error range. If the absolute value of the difference between the transport vehicle rotation time and the time when the machine vision detects the parcel grid drop action, no alarm will be issued. If the target grid drop action information and the transport vehicle rotation information are inconsistent within the error range, or only the machine vision detects the parcel grid drop action signal but does not detect the transport vehicle rotation grid drop signal, the grid will issue an alarm. In some embodiments, the alarm can be in the form of an audible and visual alarm.

[0063] In some embodiments, the package misclassification monitoring method based on machine vision analysis also includes: obtaining historical video stream input data and preprocessing the historical video stream input data, wherein the preprocessing includes format conversion, feature extraction and feature fusion; training a neural network model based on the preprocessed historical video stream input data, and determining a target detection model.

[0064] In some embodiments, historical video stream input data can be obtained from a server or a memory. The obtained historical video stream input data is preprocessed, and the preprocessing steps include but are not limited to: image scaling, uniformly adjusting the input image or video frame to a predetermined size (for example, 640×640 pixels), and normalizing the pixel value of the image to the interval [0,1] to meet the input requirements of the deep learning network. The preprocessed image data is input into a neural network model (such as the backbone network model of YOLOv8), and a series of convolution operations are used by the model to extract multi-level feature maps (Feature Maps) from the input image data. These feature maps are used to describe the texture, edges and other visual features of different areas in the image. Then, through the feature pyramid network (FPN) and the path aggregation network (PANet), the feature maps of different scales are fused to ensure that the target detection model can effectively process multi-scale objects.

[0065] Based on the fused feature map, the detection head predicts the target detection. Among them, the detection head of YOLOv8 adopts an anchor-free method to predict the position of the object by regressing the center coordinates and bounding box size of the object. At the position of each feature map, the detection head will simultaneously predict the object category (classification score), object location (coordinates and size of the bounding box), and the confidence of whether the object exists. Then the loss function of the network model is calculated and the above model is optimized. The loss function includes classification loss, bounding box regression loss, and confidence loss. Through back propagation and gradient descent algorithms, the weights of the model are adjusted to improve the accuracy of detection. In this way, the final target detection model can be determined.

[0066] In summary, through intelligent algorithms and system optimization, the system can realize intelligent analysis of equipment problems such as standardization of on-site package supply operations, accuracy of trolley drop-off, and rate of grid floating at the grid opening. It not only realizes the unified integration of various application data, but also greatly improves the on-site management efficiency of sorting operations.

[0067] See also Figure 2Another embodiment of the present application provides a parcel misclassification monitoring system 10 based on machine vision analysis. The parcel misclassification monitoring system 10 based on machine vision analysis includes: a detection module 100, a tracking module 200, and a processing module 300. The detection module 100 is used to obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output the detection result. Among them, the detection result includes the category label, bounding box and confidence score of each detection box of each detection target. The tracking module 200 is connected to the detection module 100. The tracking module is used to classify the detection results according to the confidence score, and perform target matching based on the Kalman filter and the Hungarian algorithm, and output tracking information. Among them, the tracking information includes the category label, location information and identity information of the detection target in each time step. The processing module 300 is connected to the tracking module 200. The processing module is used to determine the target drop-grid area according to the tracking information, and determine whether to alarm based on the target drop-grid area and the misclassification recognition algorithm.

[0068] In some embodiments, the specific structure of the detection module 100 is not limited, as long as it has the function of obtaining video stream input data of multiple detection targets, processing the input data based on the target detection model, and outputting the detection results. Specifically, the detection module 100 can be a microprocessor or an integrated controller. The specific structure of the detection module 100 can be selected according to actual needs and is not specifically limited here.

[0069] In some embodiments, the specific structure of the tracking module 200 is not limited, as long as it has the function of classifying the detection results according to the confidence score, performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting tracking information. Specifically, the tracking module 200 can be an image processor or an integrated controller. The specific structure of the tracking module 200 can be selected according to actual needs and is not specifically limited here.

[0070] In some embodiments, the specific structure of the processing module 300 is not limited, as long as it has the function of determining the target drop-grid area according to the tracking information, and determining whether to alarm based on the target drop-grid area and the misclassification recognition algorithm. Specifically, the processing module 300 can be a microprocessor or an integrated controller. The specific structure of the processing module 300 can be selected according to actual needs and is not specifically limited here. This embodiment adopts the above method to automatically identify misclassified packages, reduce manual recognition, reduce labor, and improve the efficiency of misclassification recognition.

[0071] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for monitoring parcel misclassification based on machine vision analysis is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0072] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0073] See also Figure 3 Another embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the package misclassification monitoring method based on machine vision analysis described in any one of the above embodiments are implemented.

[0074] In one embodiment, the processor implements the following steps when executing the computer program:

[0075] S102: Obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target;

[0076] S104: classifying the detection results according to the confidence scores, performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting tracking information, wherein the tracking information includes the category label, location information, and identity information of the detected target in each time step;

[0077] S106: Determine a target grid area according to the tracking information, and determine whether to issue an alarm based on the target grid area and the misclassification recognition algorithm.

[0078] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for monitoring package misclassification based on machine vision analysis described in any one of the above embodiments are implemented.

[0079] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0080] S102: Obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target;

[0081] S104: classifying the detection results according to the confidence scores, performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting tracking information, wherein the tracking information includes the category label, location information, and identity information of the detected target in each time step;

[0082] S106: Determine a target grid area according to the tracking information, and determine whether to issue an alarm based on the target grid area and the misclassification recognition algorithm.

[0083] The above-mentioned computer device and computer-readable storage medium can automatically identify misclassified packages using the above-mentioned method, reduce manual identification, reduce labor, and improve the efficiency of misclassification identification.

[0084] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0085] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0086] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for monitoring package misclassification based on machine vision analysis, characterized in that: include: Obtain video stream input data of multiple detection targets, process the input data based on the target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target; The detection results are classified according to the confidence scores, and the target is matched based on the Kalman filter and the Hungarian algorithm, and the tracking information is output, wherein the tracking information includes the category label, location information and identity information of the detected target in each time step; A target grid area is determined according to the tracking information, and whether to issue an alarm is determined based on the target grid area and the misclassification recognition algorithm.

2. The package misclassification monitoring method based on machine vision analysis according to claim 1, characterized in that: The method further comprises: Acquire historical video stream input data, and preprocess the historical video stream input data, wherein the preprocessing includes format conversion, feature extraction and feature fusion; The neural network model is trained based on the preprocessed historical video stream input data, and the target detection model is determined.

3. The package misclassification monitoring method based on machine vision analysis according to claim 1, characterized in that: The steps of classifying the detection results according to the confidence scores, performing target matching based on the Kalman filter and the Hungarian algorithm, and outputting tracking information include: The detection results are classified according to the confidence scores of the detection frames to determine the detection frames with high confidence and the detection frames with low confidence; Use the Kalman filter and Hungarian algorithm to match the high-confidence detection box to the target; If the target is matched successfully, the position information and confidence score of the high-confidence detection box are updated to the detection result of the corresponding detection target, and the tracking information is output.

4. The method for monitoring package misclassification based on machine vision analysis according to claim 3, characterized in that: The method further comprises: If the target is not matched successfully, the unmatched target and the low-confidence detection frame are matched based on the Kalman filter and Hungarian algorithm; If the unmatched target is successfully matched with the low-confidence detection frame, the position information and confidence score of the low-confidence detection frame are updated to the detection result of the corresponding detection target, and the tracking information is output.

5. The package misclassification monitoring method based on machine vision analysis according to claim 1, characterized in that: The step of determining the target grid area according to the tracking information, and determining whether to issue an alarm based on the target grid area and the misclassification recognition algorithm includes: Determine a target grid area according to the tracking information, and determine whether the target grid is valid based on the target grid area and a pre-configured ROI area; If the target grid is valid, the actual grid number of the target grid area is obtained, and whether to issue an alarm is determined based on the actual grid number and the misclassification recognition algorithm.

6. The method for monitoring package misclassification based on machine vision analysis according to claim 5, characterized in that: If the target grid is valid, the steps of obtaining the actual grid number of the target grid area, and determining whether to issue an alarm based on the actual grid number and the misclassification recognition algorithm include: If the target grid is valid, then the actual grid number of the target grid area is obtained; Determine target grid drop action information and transport vehicle rotation information based on the actual grid number of the target grid drop area; Whether to issue an alarm is determined based on the target grid-falling action information and the transport vehicle rotation information through the misclassification recognition algorithm.

7. The package misclassification monitoring method based on machine vision analysis according to claim 6, characterized in that: The step of determining whether to issue an alarm based on the target grid-falling action information and the transport vehicle rotation information by the misclassification recognition algorithm includes: Determine whether the target grid-falling action information and the transport vehicle rotation information are consistent within an error range by using the error classification recognition algorithm; If the target grid drop action information and the transport vehicle rotation information are inconsistent within the error range, an alarm is issued, wherein the alarm is in the form of sound and light alarm; If the target grid-dropping action information and the transport vehicle rotation information are consistent within the error range, no alarm is issued.

8. A parcel misclassification monitoring system based on machine vision analysis, characterized in that: include: A detection module, configured to obtain video stream input data of multiple detection targets, process the input data based on a target detection model, and output detection results, wherein the detection results include a category label, a bounding box, and a confidence score of each detection box for each detection target; A tracking module connected to the detection module, the tracking module is used to classify the detection results according to the confidence score, and perform target matching based on the Kalman filter and the Hungarian algorithm, and output tracking information, wherein the tracking information includes the category label, location information and identity information of the detected target in each time step; The processing module is connected to the tracking module, and is used to determine the target grid area according to the tracking information, and determine whether to issue an alarm based on the target grid area and the misclassification recognition algorithm.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images