An object state detection method and system

Abstract

The application provides an object state detection method, comprising the following steps: S101, target detection: inputting a to-be-detected image into an object detection model to obtain a target object and a position thereof; S102, target tracking: comparing target objects obtained in sequence to assign the same number to the same target; S103, position comparison: judging a position change of the target through position comparison logic; and S104, event detection: inputting the position change into a state machine, and outputting an alarm event by the state machine. The target tracking technology is adopted to track the to-be-detected target, the same number is assigned to the same target, and the state change of each target can be determined and recorded according to the number comparison. The state machine is adopted to describe the target state, and the state of the target in the target detection process is fully considered and logically judged, so that the problem that the existing object detection model cannot provide effective event alarm can be solved, and the false alarm problem that the object is regarded as moving before the object meets the stable state can be reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electronic monitoring, and particularly relates to an object state detection method and system. BACKGROUND

[0002] Package loss belongs to an event that needs to be detected in a home entrance scene. If there is enough data and training by a deep learning model, a package detection model can be obtained to detect package objects in images. However, in the prior art, only the ability to detect packages is available, and there are many false positives and missed reports for the technology of implementing package loss events and generating alarms. SUMMARY

[0003] To solve the above problems, the application provides an object state detection method, which comprises the following steps

[0004] S101, target detection: inputting a to-be-detected image into an object detection model to obtain a target object and a position thereof;

[0005] S102, target tracking: comparing the obtained target objects to assign the same number to the same target;

[0006] S103, position comparison: judging the position change of the target by position comparison logic;

[0007] and

[0008] S104, event detection: inputting the position change into a state machine, and outputting an alarm event by the state machine.

[0009] The above scheme uses target tracking technology to track the to-be-detected target, assigns the same number to the same target, and can clearly compare and record the state change of each target according to the number; the state machine describes the target state, fully considers and logically judges the state of the target in the target detection process, which not only solves the problem that the existing object detection model cannot provide effective event alarms, but also reduces the false positive problem that the object is considered as moving before meeting the stable state.

[0010] Further, the to-be-detected image input into the object detection model in S101 is a plurality of continuous pictures converted from video or RTSP streaming images, and the output of the object detection model further includes the size and confidence score of the target object.

[0011] Preferably, the object detection model in S101 uses YOLOv4, YOLOv5 or YOLOR.

[0012] Further, the object detection model in S101 is a model trained with a package as a target object.

[0013] Preferably, S101 further comprises setting a region of interest (ROI) which is excluded from the analysis of the object detection model. The user can set the ROI to represent the area in which the user believes the package should be placed in the scene in which the camera is installed. The area can be set as a quadrilateral or polygon. The package position inferred by the model is first filtered by the ROI filter to remove the packages outside the ROI. This can reduce the computational load of subsequent processing and reduce false positives, achieving better detection effect in application experience.

[0014] Preferably, the position change result in S103 includes target disappearance, target movement, and target maintaining original position. Further, the position change of the target is determined by calculating the IoU (Intersection over Union). When the IoU is greater than a set threshold, the target is determined to maintain the original position, otherwise the target is determined to move. The IoU is the ratio of the overlapping part to the set part of the positions of the target obtained in succession.

[0015] Preferably, the states of the state machine in S104 include no target detected, target present, and target present and position stable. The alarm events include no alarm, target movement alarm, and target loss alarm. Further, the state transition and output of the state machine include:

[0016] When the current state is no target detected, if the input position change is target movement, the state is transferred to target present;

[0017] When the current state is target present, if the input position change is target disappearance, the state is transferred to no target detected; if the input position change is target movement, the state is transferred to target present; if the input position change is target maintaining original position, the state is transferred to target present and position stable;

[0018] and,

[0019] When the current state is target present and position stable, if the input position change is target disappearance, the state is transferred to no target detected, and the target loss alarm is output; if the input position change is target movement, the state is transferred to target present, and the target movement alarm is output.

[0020] For the case of target present, the above scheme designs two states of target present and target present and position stable in the state machine. For the detection of the package placement process, the stable package is distinguished by the state of target present and position stable, which can greatly avoid false alarms.

[0021] In a second aspect, the present application provides an object state detection system, comprising:

[0022] The target detection module is configured to input the image to be detected into an object detection model to obtain a target object and a position of the target object.

[0023] The target tracking module is configured to compare the obtained target objects to assign the same number to the same target object.

[0024] The position comparison module is configured to determine a position change of the target object by using a position comparison logic.

[0025] and

[0026] The event detection module is configured to input the position change into a state machine, and the state machine outputs an alarm event.

[0027] The above scheme proposes an object state detection method and system, which has a reliable continuous image processing and judgment mechanism, especially for the scene of package state monitoring and package loss event detection, can effectively reduce false positives and false negatives, and enables users to obtain meaningful event alarm information. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings are included to provide a further understanding of the present application. The elements in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. For purposes of clarity, not every component is called out in the figures.

[0029] Figure 1 The object state detection method flowchart in an embodiment;

[0030] Figure 2 The object state detection method architecture flowchart for package loss event detection in an embodiment

[0031] Figure 3 The IoU calculation method schematic diagram of the object state detection method in an embodiment;

[0032] Figure 4 The state change logic schematic diagram of the state machine in an embodiment;

[0033] Figure 5 The object detection system structure schematic diagram in an embodiment. DETAILED DESCRIPTION

[0034] Figure 1 The object state detection method flowchart in an embodiment of the present application specifically includes:

[0035] S101, target detection: inputting the image to be detected into an object detection model to obtain a target object and a position of the target object;

[0036] S102, target tracking: comparing the obtained target objects to assign the same number to the same target object.

[0037] S103, position comparison: determine the position change of the target through position comparison logic;

[0038] S104, event detection: input the position change into the state machine, and the state machine outputs the alarm event.

[0039] Figure 2 For an embodiment of the architecture flow chart for package loss event detection, the flow includes a package detector, a ROI object filter, an object tracker, position comparison logic, and FSM logic. Specifically:

[0040] S101, target detection: input the image to be detected into the object detection model to obtain the target object and its position.

[0041] The image to be detected can be a video or an RTSP stream image, which is converted into multiple consecutive pictures and enters the flow.

[0042] The picture is sent to the deep learning object detection model (Package Detector). This model is trained with packages as target objects. The model can use the current mainstream object detection model architecture such as YOLOv4, YOLOv5, and YOLOR. The model trained with packages as target objects will make inferences based on the input picture and output the inference results, including the position, size information of the detected package objects in the picture, and the confidence score (confidence) of each object.

[0043] In a preferred embodiment, in order to achieve better detection effect in application experience, a ROI (Region Of Interest) can be set by the user to represent the area where the user thinks the package should be placed in the scene where the camera is installed. This area can be set as a quadrilateral or a polygon. The package position obtained by the model inference will first pass through the ROI filter (ROI Filter) to remove packages outside the ROI, thereby reducing the computational load of subsequent processing and reducing false positives.

[0044] S102, target tracking: compare the target objects obtained in sequence to assign the same number to the same target.

[0045] The screened package objects will be sent to an object tracker (Object Tracker) again. The object tracker will compare the package objects detected in this picture with the package objects detected in the previous picture, and assign each object an independent tracking number. With the tracking number, the state change of each object can be clearly compared and recorded according to the number.

[0046] S103, Position Comparison: Determine the position change of the target by position comparison logic.

[0047] The numbered package objects are determined by the package position comparison logic, and the position change of each package object can be known. The position change is divided into three types, namely "package disappearance", "package movement", and "package maintaining original position".

[0048] The package position comparison logic first determines whether the package in the previous frame still exists through the results of the object tracker. If the tracking ID of the previous frame does not exist in this frame, it means that the "package has disappeared". Then, the positions of the package objects with the same tracking ID in the previous frame and this frame are compared. The comparison is based on the calculation of IoU (Intersection over Union), Figure 3 The IoU calculation method in an embodiment is shown in the figure. As can be seen from the figure, IoU is the ratio of the overlapping part of the target position obtained in sequence to the set part. In the package position comparison logic, a threshold is set to determine the position change of the target in combination with the threshold. If the IoU is greater than the threshold, it means that the "package maintains the original position", otherwise it means that the "package has moved".

[0049] S104, Event Detection: Input the position change into the state machine, and the state machine outputs the alarm event.

[0050] After determining the position change of each package object, a state machine (FSM Logic) is designed to input the position change as a way to determine whether to issue a package movement or loss alarm.

[0051] Figure 4 The state change logic of the state machine in an embodiment. This finite state machine is a Mealy FSM, and the output is determined by the input and the current state. Among them,

[0052] The states of the state machine are divided into three types: S0, S1, and S2, which represent "no package detected", "package", and "package and stable position", respectively.

[0053] The input of the state machine is the position change output by the "package position comparison logic";

[0054] The output of the state machine represents the alarm issued. The alarm is divided into three types: "no alarm", "package movement alarm", and "package loss alarm".

[0055] The following table is a truth table representing the state transition and output of the state machine in an embodiment. As shown in the following table, the FSM Logic is:

[0056] If the current state is S0 (no target detected) and the input position change is that the target has moved (moved), the state is transferred to S1 (target detected).

[0057] If the current state is S1 (target detected) and the input position change is that the target has disappeared (-), the state is transferred to S0 (no target detected); if the input position change is that the target has moved (moved), the state is transferred to S1 (target detected); if the input position change is that the target maintains the original position (hold), the state is transferred to S2 (target detected and position stable).

[0058] If the current state is S2 (target detected and position stable) and the input position change is that the target has disappeared (-), the state is transferred to S0 (no target detected) and the GONE alarm is output; if the input position change is that the target has moved (moved), the state is transferred to S1 (target detected) and the MOVED alarm is output.

[0059]

[0060] Table 1 Truth Table Corresponding to the State Machine

[0061]

[0062] S1 and S2 of the state machine are both states with a package, the difference is that the package of S2 is a position stable state, which represents a package that has been placed in position. It can be understood that if the stable package is not distinguished by the S2 state, many false alarms will occur.

[0063] Figure 5 Fig. 1 is a schematic diagram of an object detection system structure 500 in an embodiment, the system comprising:

[0064] A target detection module 501 is configured to input an image to be detected into an object detection model to obtain a target object and a position thereof.

[0065] A target tracking module 502 is configured to compare target objects obtained in sequence to assign the same number to the same target object.

[0066] A position comparison module 503 is configured to judge a position change of a target object by position comparison logic.

[0067] An event detection module 504 is configured to input the position change into a state machine, and the state machine outputs an alarm event.

[0068] An event detection module 504 is configured to input the position change into a state machine, and the state machine outputs an alarm event.

[0069] ​The package loss event detection method based on object detection provided by the application reduces the operation amount through ROI screening, and further refers to the package state described by the object tracking technology and the state machine, so as to not only solve the problem that the object detection model cannot provide effective event alarm, but also reduce the false alarm problem that the object is considered as moving before meeting the stable state.

[0070] Although the content of the application is specifically shown and introduced in combination with the preferred embodiments, it should be understood by those skilled in the art that various changes can be made to the application in form and details without departing from the spirit and scope of the application defined by the appended claims, without creative labor.

Claims

1. A method of detecting the state of a wrapped object, characterized by, The method comprises the following steps: S101, target detection: inputting a to-be-detected image into an object detection model to obtain a package object and a position thereof, and pre-setting an ROI to make the object detection model pre-remove a position outside the ROI in the to-be-detected image; S102, target tracking: comparing the package objects obtained in sequence to assign the same number to the same target; S103, position comparison: judging the position change of the target through position comparison logic; The position comparison is to judge the position change of the target by calculating the IoU, and when the IoU is greater than a set threshold, it is judged that the target maintains the original position, otherwise it is judged that the target has moved; The IoU is the ratio of the overlapping part to the set part of the package object positions in the last frame and the current frame; The position change result includes: target disappearance, target movement and target maintaining the original position; and S104, event detection: inputting the position change into a state machine, and the state machine outputs an alarm event; The states of the state machine include: no target detected, target present and target present and position stable, and the alarm events include: no alarm, target movement alarm and target loss alarm; The state transition and output of the state machine include: when the current state is no target detected, if the input position change is target movement, the state is transferred to target present; When the current state is target present, if the input position change is target disappearance, the state is transferred to no target detected; if the input position change is target movement, the state is transferred to target present; if the input position change is target maintaining the original position, the state is transferred to target present and position stable; and When the current state is target present and position stable, if the input position change is target disappearance, the state is transferred to no target detected, and the target loss alarm is output; if the input position change is target movement, the state is transferred to target present, and the target movement alarm is output.

2. The method of claim 1, wherein, The to-be-detected image input into the object detection model in S101 is a plurality of continuous pictures converted from video or RTSP streaming images, and the output of the object detection model further includes the size and confidence score of the package object.

3. The method of claim 1, wherein, The object detection model in S101 adopts YOLOv4, YOLOv5 or YOLOR.

4. The method of claim 1, wherein, The object detection model in S101 is a model trained with packages as package objects.

5. A system for detecting the status of a wrapped object, the system comprising: It comprises: A target detection module configured to input a to-be-detected image into an object detection model to obtain a package object and a position thereof; Pre-set an ROI to make the object detection model pre-remove a position outside the ROI in the to-be-detected image; A target tracking module configured to compare the package objects obtained in sequence to assign the same number to the same target; A position comparison module configured to judge the position change of the target by calculating the IoU, and when the IoU is greater than a set threshold, it is judged that the target maintains the original position, otherwise it is judged that the target has moved; The IoU is the ratio of the overlapping part to the set part of the package object positions in the last frame and the current frame; The position change result includes: target disappearance, target movement and target maintaining the original position; and An event detection module is configured to input the position change into a state machine, and the state machine outputs an alarm event; The states of the state machine include: no target detected, target present, and target present and position stable, and the alarm events include: no alarm, target moving alarm, and target lost alarm; The state transition and output of the state machine include: when the current state is no target detected, if the input position change is target moving, the state transitions to target present; when the current state is target present, if the input position change is target disappearing, the state transitions to no target detected; if the input position change is target moving, the state transitions to target present; if the input position change is target maintaining the original position, the state transitions to target present and position stable; and, when the current state is target present and position stable, if the input position change is target disappearing, the state transitions to no target detected, and outputs target lost alarm; if the input position change is target moving, the state transitions to target present, and outputs target moving alarm.

Images