Suitcase action snapshot method and device, computer and storage medium

Abstract

The application discloses a luggage lifting action snapshot method and device, a computer and a storage medium, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring each frame of image in a video; detecting each frame of image by using a pedestrian luggage interaction detection algorithm to obtain an interactive basket in each frame of image, wherein the interactive basket is formed by a pedestrian and a luggage; matching the luggage basket and the interactive basket in a continuous frame image sequence to form a luggage tracking sequence and an interactive group tracking sequence; determining whether the pedestrian and the luggage interact according to the number of image frames in the interactive group tracking sequence; if the interaction occurs and the luggage in the luggage tracking sequence changes in motion trajectory, it is determined that the luggage lifting action occurs; and using a preset image collector to capture the current pedestrian and the luggage. The application can reduce the possibility of false judgment and capture caused by interference objects, improve the accuracy of capture, and ensure the binding relationship between the captured pedestrian and the luggage.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for capturing a luggage lifting action, a computer and a storage medium. BACKGROUND

[0002] When passengers leave the country, they need to pass through security checks. When passengers enter the country, they need to pass through customs and border inspection to check whether they carry valuable items and plant and animal quarantine. Once the luggage with problems is found, the passenger to whom the luggage belongs needs to be quickly found so that corresponding measures can be taken. Therefore, the technical problem of how to accurately associate the luggage with problems and the passenger and timely transmit the associated information to the staff is faced.

[0003] Thanks to the development of current artificial intelligence technology, a target detection algorithm based on a neural network can be relatively maturely transplanted to many terminal devices to realize the analysis of actions in real-time videos. Currently, a method for detecting a passenger and a luggage based on a monitoring video generally uses a target detection algorithm combined with logical relationship judgment. First, a target detection algorithm based on deep learning is used to detect the positions of a passenger and a luggage. Then, a tracking algorithm is used to track the passenger and the luggage. Finally, the occurrence of a luggage lifting action is judged according to the motion trajectory of the luggage to trigger the capture.

[0004] However, the mechanism for triggering the detection of a passenger and a luggage in the prior art completely relies on logical judgment. In some specific cases, such as the presence of an interfering object, misjudgment and miscapture may occur, and the capture accuracy is low. SUMMARY

[0005] The present application provides a method and device for capturing a luggage lifting action, a computer and a storage medium to solve the problem of inaccuracy in capturing a luggage in motion and the corresponding passenger in the prior art.

[0006] The technical solutions of the present application to solve the above technical problems are as follows:

[0007] The present application provides a method for capturing a luggage lifting action, which comprises:

[0008] obtaining each frame of image in a video;

[0009] detecting each frame of image by using a passenger and luggage interaction detection algorithm to obtain a luggage basket, a passenger and a luggage interaction group basket in each frame of image, the interaction group basket comprising a first interaction point and a second interaction point, the first interaction point being the center point of the passenger, and the second interaction point being the center point of the luggage;

[0010] matching the luggage basket and the interaction group basket in the image sequence of consecutive frames to form a luggage tracking sequence and an interaction group tracking sequence;

[0011] determining whether the pedestrian and the luggage case interact according to the number of image frames in the interaction group tracking sequence;

[0012] if the interaction occurs and the luggage case in the luggage case tracking sequence changes in motion trajectory, determining that the luggage lifting action occurs;

[0013] capturing the current pedestrian and the luggage case by using a preset image collector.

[0014] According to the luggage lifting action capturing method, the matching of the luggage case basket and the interaction group basket in the image sequence of the continuous frames to form the luggage case tracking sequence and the interaction group tracking sequence comprises:

[0015] The matching of the luggage case basket and the interaction group basket in the image sequence of the continuous frames to form the luggage case tracking sequence and the interaction group tracking sequence is performed by using an intersection-over-union function.

[0016] According to the luggage lifting action capturing method, before the matching of the luggage case basket and the interaction group basket in the image sequence of the continuous frames to form the luggage case tracking sequence and the interaction group tracking sequence by using the intersection-over-union function, the method comprises:

[0017] The position of the luggage case basket in the image sequence of the continuous frames is predicted by using a Kalman filter prediction model.

[0018] If the position of the luggage case basket in the adjacent frame is within the predicted position range, the matching of the luggage case basket and the interaction group basket in the image sequence of the continuous frames to form the luggage case tracking sequence and the interaction group tracking sequence is performed by using the intersection-over-union function.

[0019] According to the luggage lifting action capturing method, the determination of whether the pedestrian and the luggage case interact according to the number of image frames in the interaction group tracking sequence comprises:

[0020] If the distance between the first interaction point and the second interaction point in the image sequence of the continuous frames in the interaction group tracking sequence is less than a preset distance, and the number of image frames of the continuous frames is greater than or equal to 3 frames, it is determined that the pedestrian and the luggage case interact.

[0021] According to the luggage lifting action capturing method, there is only one pedestrian and the corresponding luggage case in each interaction group box.

[0022] According to the luggage lifting action capturing method, the method further comprises:

[0023] If each frame of image is detected by using a pedestrian-luggage case interaction detection algorithm, and the current frame of image detected contains two pedestrians and the same luggage case, and the distance between the interaction points of the two pedestrians and the same luggage case is less than the preset distance, the interaction group box comprises two interaction group boxes.

[0024] In a second aspect, the present application provides a suitcase action snapshot device, which comprises:

[0025] an acquisition module, configured to acquire each frame of image in the video;

[0026] a detection module, configured to detect each frame of image by using a pedestrian suitcase interaction detection algorithm to acquire a suitcase basket and an interaction group basket formed by the pedestrian and the suitcase in each frame of image, the interaction group basket comprising a first interaction point and a second interaction point, the first interaction point being a center point of the pedestrian, and the second interaction point being a center point of the suitcase;

[0027] a tracking module, configured to match the suitcase basket and the interaction group basket in a continuous frame image sequence to form a suitcase tracking sequence and an interaction group tracking sequence;

[0028] an interaction judgment module, configured to judge whether the pedestrian and the suitcase have interaction according to the number of image frames in the interaction group tracking sequence;

[0029] a suitcase judgment module, configured to judge that the suitcase action occurs when the suitcase has interaction and the suitcase in the suitcase tracking sequence has a change in motion trajectory;

[0030] a control output module, configured to use a preset image collector to snapshot the current pedestrian and the suitcase.

[0031] According to the above-mentioned suitcase action snapshot device, the tracking module uses an intersection-over-union function to match the suitcase basket and the interaction group basket in the continuous frame image sequence to form the suitcase tracking sequence and the interaction group tracking sequence.

[0032] In a third aspect, the present application provides a computer device, which comprises a processor configured to implement the steps of the above-mentioned suitcase action snapshot method when executing a computer program stored in a memory.

[0033] In a fourth aspect, the present application further provides a storage medium, which stores a computer program, and the computer program is configured to implement the steps of the above-mentioned suitcase action snapshot method when executed by a processor.

[0034] The technical scheme provided by the embodiments of the present application has the following beneficial effects:

[0035] The application utilizes a pedestrian luggage interaction detection algorithm to detect each frame of image in the captured video, matches and tracks the interactive group frame formed by the luggage basket corresponding to the luggage, the pedestrian and the luggage, to determine whether the pedestrian and the luggage interact, and determines that the lifting action occurs when it is determined that the interaction occurs and the luggage in the luggage tracking sequence changes the motion trajectory, the action of the pedestrian to the luggage occurs, and then the preset image collector is used to capture the current pedestrian and the luggage to establish the binding relationship of the pedestrian and the luggage. While tracking the luggage, the interactive group formed by the pedestrian and the luggage is also tracked to jointly determine whether the lifting action of the pedestrian to the luggage occurs, reduce the possibility of false judgment and capture caused by interference objects, improve the accuracy of capture, and ensure the binding relationship of the captured pedestrian and the luggage. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The flow chart of the lifting action capture method provided by the present application.

[0038] Figure 2 The functional module schematic diagram of the lifting action capture device provided by the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0040] Reference Figure 1 The flow chart of the lifting action capture method provided by the present application, which can be but is not limited to applied in personnel-intensive flow places such as stations, ports, airports or shopping malls, etc. The luggage is mostly in a dynamic motion process, such as being transported in the scenes of airport, high-speed rail security conveyor belt or luggage turntable, etc. The binding relationship of the pedestrian and the luggage is established by capturing the lifting action of the personnel, which is helpful to monitor the pedestrian who takes the corresponding luggage and identify the identity information of the pedestrian, and improves the safety monitoring effect of the luggage.

[0041] As Figure 1 shown, the lifting action capture method can include the following steps:

[0042] S101: Obtain each frame image in a video, which can be a video collected in real time by a camera or a video file stored in a preset directory, without limitation.

[0043] S102: Detect each frame image by using a pedestrian luggage interaction detection algorithm to obtain a luggage basket and an interaction group basket formed by a pedestrian and a luggage basket in each frame image, the interaction group basket comprising a first interaction point and a second interaction point, the first interaction point being a center point of the pedestrian, and the second interaction point being a center point of the luggage basket.

[0044] In this step, the pedestrian luggage interaction detection algorithm is a pre-trained detection algorithm, which detects pedestrians and luggage baskets in each frame image, and takes the center point of the pedestrian as the first interaction point and the center point of the luggage basket as the second interaction point, and determines whether the pedestrian and the luggage basket interact by judging whether the Euclidean distance between the first interaction point and the second interaction point is less than a preset distance.

[0045] It can be understood that, for the pedestrian detected by the pedestrian luggage interaction detection algorithm, a pedestrian basket can be generated to frame the position of the pedestrian in the image; for the luggage basket detected by the pedestrian luggage interaction detection algorithm, a luggage basket can be generated to frame the position of the luggage basket in the image; for the pedestrian and the luggage basket detected by the pedestrian luggage interaction detection algorithm to interact, that is, the Euclidean distance between the first interaction point and the second interaction point is less than the preset distance, an interaction group basket can be generated to frame the positions of the corresponding pedestrian and luggage basket in the image.

[0046] In this step, preferably, each interaction group basket has and only has one pedestrian and corresponding luggage basket.

[0047] S103: Match the luggage basket and the interaction group basket in the image sequence of the continuous frames to form a luggage tracking sequence and an interaction group tracking sequence.

[0048] In this step, the intersection over union (IOU) function can be used to match the luggage basket and the interaction group basket in the image sequence of the continuous frames to form the luggage tracking sequence and the interaction group tracking sequence.

[0049] Before this step, the position of the luggage basket in the image sequence of the continuous frames can also be predicted by using a Kalman filter prediction model, and only when the position of the luggage basket in the adjacent frame is within the predicted position range, the matching of the luggage basket and the interaction group basket in the image sequence of the continuous frames by using the intersection over union function is performed to form the luggage tracking sequence and the interaction group tracking sequence.

[0050] S104: Determine whether the pedestrian and the luggage case interact according to the number of image frames in the interaction group tracking sequence.

[0051] In this step, the interaction group tracking sequence generally includes multiple image frames. Specifically, if the distance between the first interaction point and the second interaction point in the image of the continuous frame in the interaction group tracking sequence is less than a preset distance, and the number of image frames of the continuous frame is greater than or equal to 3 frames, it is determined that the pedestrian and the luggage case interact.

[0052] S105: If the interaction occurs and the luggage case in the luggage case tracking sequence changes the motion trajectory, it is determined that the lifting action occurs.

[0053] In this step, the change of the motion trajectory of the luggage case in the luggage case tracking sequence is mainly completed through the interaction ratio function matching and the Kalman filter prediction model. In the normal motion state, the luggage case will travel according to the predetermined trajectory and remain stable. Therefore, by judging the change of the motion trajectory of the luggage case, it is known whether the luggage case is disturbed by the pedestrian during the motion. If there is no pedestrian disturbance, the luggage case will maintain a stable motion trajectory until it disappears in the image. If there is a pedestrian disturbance such as lifting action, the motion trajectory of the luggage case will change.

[0054] S106: Bind the pedestrian and the luggage case according to the distance between the first interaction point and the second interaction point, and use a preset image collector to capture the current pedestrian and the luggage case. Here, the preset image collector can use a pre-set dedicated capture instrument, or a common photographic device such as a camera for shooting the video.

[0055] The present application uses a pedestrian-luggage case interaction detection algorithm to detect each frame of image in the obtained shooting video. By matching and tracking the interaction group frame formed by the corresponding luggage case basket of the luggage case and the pedestrian and the luggage case, it is determined whether the pedestrian and the luggage case interact. When it is determined that the interaction occurs and the luggage case in the luggage case tracking sequence changes the motion trajectory, it is determined that the lifting action occurs. The action of the pedestrian on the luggage case occurs, and then the preset image collector is used to capture the current pedestrian and the luggage case to establish the binding relationship between the pedestrian and the luggage case. While tracking the luggage case, the interaction group formed by the pedestrian and the luggage case is also tracked to jointly determine whether the lifting action of the pedestrian on the luggage case occurs, thereby reducing the possibility of false judgment and capture caused by interference objects, improving the accuracy of capture, and ensuring the binding relationship between the captured pedestrian and the luggage case.

[0056] It can be understood that when it is detected that the distance between the interaction point of the multiple persons and the same suitcase in the image is less than the preset distance, for example, two pedestrians and the same suitcase, the two pedestrians can be regarded as suspicious personnel or target personnel, and therefore the two pedestrians and the suitcase can form two interaction group frames, and the current two pedestrians and the suitcase can be captured by using the preset image collector after tracking meets the interaction judgment and the suitcase moving action judgment.

[0057] In the present application, the pedestrian can also be tracked, and an interaction group frame can be obtained when the distance between the interaction point between the pedestrian and the suitcase is less than the preset distance. Of course, the center point of the corresponding pedestrian can also be regarded as the first interaction point only when it is judged that there is a pedestrian within the preset range of the suitcase, and whether the distance between the pedestrian and the suitcase meets the preset distance can be judged.

[0058] Referring to Figure 2 The suitcase moving action capturing device 100 provided by the present application can include an acquisition module 11, a detection module 12, a tracking module 13, an interaction judgment module 14, a suitcase moving action judgment module 15, and a control output module 16.

[0059] The acquisition module 11 is mainly used for acquiring each frame of image in the video.

[0060] The detection module 12 is mainly used for detecting each frame of image by using a pedestrian-suitcase interaction detection algorithm to obtain a suitcase frame and an interaction group frame formed by the pedestrian and the suitcase in each frame of image, the interaction group frame including a first interaction point and a second interaction point, the first interaction point being the center point of the pedestrian, and the second interaction point being the center point of the suitcase.

[0061] The tracking module 13 is mainly used for matching the suitcase frame and the interaction group frame in the image sequence of the continuous frames to form a suitcase tracking sequence and an interaction group tracking sequence. Specifically, the tracking module 13 can mainly use an intersection-over-union function to match the suitcase frame and the interaction group frame in the image sequence of the continuous frames to form the suitcase tracking sequence and the interaction group tracking sequence.

[0062] The interaction judgment module 14 is mainly used for judging whether the pedestrian and the suitcase interact according to the number of image frames in the interaction group tracking sequence.

[0063] The suitcase moving action judgment module 15 is mainly used for judging that the suitcase moving action occurs when the interaction occurs and the motion trajectory of the suitcase in the suitcase tracking sequence changes.

[0064] The control output module 16 is mainly used for capturing the current pedestrian and the suitcase by using a preset image collector.

[0065] It should be understood that after the corresponding modules perform the corresponding functions, the effects that can be achieved are the same as the aforementioned bag action shooting method, and thus will not be described here again.

[0066] In addition, the present application provides a computer device, comprising a processor for implementing the steps of the aforementioned bag action shooting method when executing a computer program stored in a memory.

[0067] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The processor is the control center of the computer device, and connects all parts of the computer device through various interfaces and lines.

[0068] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, etc.; and the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0069] In addition, the present application also provides a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the aforementioned bag action shooting method.

[0070] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of action shot of a suitcase, characterized by, The method comprises: acquiring each frame image in a video; detecting each frame image by using a pedestrian and luggage interaction detection algorithm to obtain a luggage basket and an interactive basket formed by the pedestrian and the luggage in each frame image, the interactive basket comprising a first interaction point and a second interaction point, the first interaction point being a center point of the pedestrian, and the second interaction point being a center point of the luggage; matching the luggage basket and the interactive basket in a continuous frame image sequence to form a luggage tracking sequence and an interactive basket tracking sequence, comprising: predicting the position of the luggage basket in the continuous frame image sequence by using a Kalman filter prediction model; if the position of the luggage basket in the adjacent frame is within the predicted position range, matching the luggage basket and the interactive basket in the continuous frame image sequence by using an intersection-over-union function to form the luggage tracking sequence and the interactive basket tracking sequence; judging whether the pedestrian and the luggage interact according to the number of image frames in the interactive basket tracking sequence; if the interaction occurs and the luggage in the luggage tracking sequence changes in motion trajectory, determining that the luggage handling action occurs; capturing the current pedestrian and the luggage by using a preset image collector.

2. The suitcase action snapshot method of claim 1, wherein, The judging whether the pedestrian and the luggage interact according to the number of image frames in the interactive basket tracking sequence comprises: if the distance between the first interaction point and the second interaction point in the continuous frame image in the interactive basket tracking sequence is less than a preset distance, and the number of image frames of the continuous frame image is greater than or equal to 3 frames, determining that the pedestrian and the luggage interact.

3. The suitcase action snapshot method of claim 2, wherein, There is only one pedestrian and corresponding luggage in each interactive basket.

4. The suitcase action snapshot method of claim 3, wherein, The method further comprises: if the detection of each frame image by using the pedestrian and luggage interaction detection algorithm, and the current frame image detected contains two pedestrians and the same luggage, and the distance between the interaction points of the two pedestrians and the same luggage is less than the preset distance, the interactive basket comprises two interactive baskets.

5. A suitcase action snapshot device, characterized by, The device comprises: an acquisition module, which acquires each frame image in a video; a detection module, which detects each frame image by using a pedestrian and luggage interaction detection algorithm to obtain a luggage basket and an interactive basket formed by the pedestrian and the luggage in each frame image, the interactive basket comprising a first interaction point and a second interaction point, the first interaction point being a center point of the pedestrian, and the second interaction point being a center point of the luggage; a tracking module, which matches the luggage basket and the interactive basket in a continuous frame image sequence to form a luggage tracking sequence and an interactive basket tracking sequence, comprising: predicting the position of the luggage basket in the continuous frame image sequence by using a Kalman filter prediction model; if the position of the luggage basket in the adjacent frame is within the predicted position range, matching the luggage basket and the interactive basket in the continuous frame image sequence by using an intersection-over-union function to form the luggage tracking sequence and the interactive basket tracking sequence; an interaction judgment module, which judges whether the pedestrian and the luggage interact according to the number of image frames in the interactive basket tracking sequence; A suitcase determining module is configured to determine the suitcase action when the interaction occurs and the motion trajectory of the suitcase in the suitcase tracking sequence changes; A control output module is configured to capture the current pedestrian and the suitcase by using a preset image collector.

6. A computer, characterized by The computer comprises a processor configured to implement the steps of the suitcase action capturing method according to any one of claims 1-4 when executing a computer program stored in a memory.

7. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is configured to implement the steps of the suitcase action capturing method according to any one of claims 1-4 when executed by the processor.

Images