Tracking Method and Tracking System for Articles Inspected by Machine in Advance

Abstract

The present application provides a method for tracking pre-screened items. Items are subjected to security inspection upstream of the baggage carousel. When an item is an item of interest, an image of the item of interest is acquired, and the characteristics of the item of interest are recorded. The images of the persons under surveillance corresponding to the items of interest are continuously updated on the path between the baggage carousel and the exit, and the characteristics of the persons under surveillance are recorded, where the persons under surveillance are all pedestrians who have come into contact with the items of interest. The items of interest and the persons under surveillance corresponding to the items of interest are tracked on the path between the baggage carousel and the exit. According to the item tracking method of the present application, through continuous tracking and data updating of the items of interest and the persons under surveillance who have come into contact with the items of interest, the behaviors of all the persons under surveillance during the period from when the items of interest are picked up to the exit can be comprehensively and effectively supervised, helping the staff to lock in the target in advance, thus improving the customs clearance efficiency and avoiding the situation of missed inspection.

Description

Technical Field

[0001] This application relates to the field of item security inspection. Specifically, this application relates to a method and a system for tracking pre-screened items. Background Art

[0002] Luggage inspection is an important link in places such as airports and customs. Checked luggage first passes through a pre-screening system, where the images are examined to mark a small number of problematic suitcases. When these suitcases pass through the detection gate, the alarm system is triggered, and the staff intercepts and inspects them. However, the current pre-screening system does not track pedestrians or suitcases throughout the process, lacking real-time tracking, accurate positioning, and abnormal behavior recognition after the pedestrian picks up the checked luggage. Relying only on the labels of the suitcases not only delays the inspection by the staff but also cannot monitor abnormal situations such as suitcase replacement and damaged labels, posing a risk of missed inspections during luggage inspection. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in the prior art.

[0004] For example, the first object of this application is to provide a method for tracking pre-screened items, which monitors from the moment a pedestrian picks up the checked luggage until the exit, effectively tracking and updating data throughout the process, and only performing special inspections when the checked luggage or the pedestrian is abnormal, thereby improving the customs clearance efficiency while avoiding the probability of missed inspections;

[0005] The second object of this application is to provide a system for tracking pre-screened items, which can be implemented in combination with the above item tracking method.

[0006] To achieve the above object, the first aspect of this application provides a method for tracking pre-screened items, including the following steps:

[0007] Conduct a security inspection on items upstream of the luggage carousel to determine whether the items are items of interest;

[0008] When the item is an item of interest, obtain an image of the item of interest and record the characteristics of the item of interest;

[0009] When the item is an item of interest, continuously update the image of the person under surveillance corresponding to the item of interest on the path between the luggage carousel and the exit, and record the characteristics of the person under surveillance, where the person under surveillance is all pedestrians who have come into contact with the item of interest; and

[0010] Track the item of interest and the person under surveillance corresponding to the item of interest on the path between the luggage carousel and the exit.

[0011] According to the article tracking method of the present application, through continuous tracking and data updating of the article of interest and the persons under surveillance who have come into contact with the article of interest, the behaviors of all persons under surveillance during the period from when the article of interest is picked up to the exit can be comprehensively and effectively supervised, helping the staff to lock in the target in advance, which not only improves the customs clearance efficiency but also avoids the situation of missed inspection, thus reducing the work difficulty of the staff while enhancing the supervision intensity.

[0012] Further, while updating the image of the person under surveillance corresponding to the article of interest, obtain the image of the article of interest and update the recorded features of the article of interest.

[0013] Further, performing a security inspection on the article to determine whether the article belongs to the article of interest includes:

[0014] Obtaining a perspective image of the article; and

[0015] Judging whether it belongs to the article of interest according to the perspective image.

[0016] Further, when the article belongs to the article of interest, continuously update the image of the person under surveillance corresponding to the article of interest on the path between the baggage carousel and the exit, and record the features of the person under surveillance, including:

[0017] When a pedestrian comes into contact with the article of interest, regard the pedestrian as the person under surveillance of the article of interest and capture the image of the person under surveillance.

[0018] Further, capturing the image of the person under surveillance specifically includes capturing the body image of the person under surveillance corresponding to the article of interest.

[0019] Further, when the article belongs to the article of interest, continuously update the image of the person under surveillance corresponding to the article of interest on the path between the baggage carousel and the exit, and record the features of the person under surveillance, further including:

[0020] Extracting and recording the features of the body image;

[0021] Associating the features of the body image with the features of the article of interest and generating a pairing group.

[0022] Further, tracking the article of interest and the person under surveillance corresponding to the article of interest includes:

[0023] Locking the article of interest according to the image of the article of interest and providing the real-time body image of the carrier;

[0024] Obtaining the data information of the pairing group associated with the features of the article of interest according to the features of the article of interest;

[0025] When the features of the carrier's body image are different from the data information of the pairing group, it is determined that the item of interest and the person under surveillance corresponding to the item of interest need special inspection.

[0026] Further, tracking the item of interest and the person under surveillance corresponding to the item of interest includes:

[0027] Lock the person under surveillance according to the features of the body image of the person under surveillance, and provide the image of the items carried by the person under surveillance in real time;

[0028] According to the features of the body image of the person under surveillance, obtain the data information of the pairing group associated with the features of the body image of the person under surveillance;

[0029] When the image of the item carried by the person under surveillance is different from the data information of the pairing group, it is determined that the item carried by the person under surveillance needs special inspection.

[0030] Further, tracking the item of interest and the person under surveillance corresponding to the item of interest includes:

[0031] When the item of interest is lost, it is determined that the item of interest and the person under surveillance corresponding to the item of interest need special inspection.

[0032] Further, tracking the item of interest and the person under surveillance corresponding to the item of interest includes:

[0033] When the person under surveillance or the item of interest approaches a concealed area, it is determined that the item of interest and the person under surveillance corresponding to the item of interest need special inspection.

[0034] Further, capturing the image of the person under surveillance also includes capturing the facial image of the person under surveillance corresponding to the item of interest.

[0035] Further, when the item belongs to the item of interest, continuously update the image of the person under surveillance corresponding to the item of interest on the path between the baggage carousel and the exit, and record the features of the person under surveillance, and further include:

[0036] Extract and record the features of the facial image;

[0037] Associate the features of the facial image, the features of the body image, and the features of the item of interest, and generate a pairing group.

[0038] Further, it is characterized in that the method further includes:

[0039] When special inspection is required for the item of interest and / or the person under surveillance corresponding to the item of interest, an alarm message is pushed.

[0040] Further, the alarm message includes: the image of the item of interest and the movement trajectory of the item of interest, the image of the person under surveillance and the movement trajectory of the person under surveillance, as well as the perspective image of the item of interest and the judgment result.

[0041] Further, when the item belongs to an item of interest, an image of the item of interest is acquired and the characteristics of the item of interest are recorded. Specifically, it includes:

[0042] When the item belongs to an item of interest, a first image of the item of interest is acquired and the first characteristics of the item of interest are recorded;

[0043] According to the first characteristics of the item of interest, a second image of the item of interest is acquired and the second characteristics of the item of interest are recorded.

[0044] The second aspect of the present application provides an early-stage machine inspection item tracking system, including: a non-destructive testing device, which is arranged upstream of the baggage carousel and is used to acquire the perspective image of the item and judge whether it belongs to an item of interest according to the perspective image; an item image marking and assigning device, which is arranged upstream of the baggage carousel and is used to acquire the image of the item of interest and record the characteristics of the item of interest; a video monitoring device, which is arranged on the path between the baggage carousel and the exit and is used to continuously track and update the record of the image of the item of interest and the image of the person under surveillance corresponding to the item of interest, where the person under surveillance is all pedestrians who have come into contact with the item of interest.

[0045] Further, the video monitoring device is configured to: when a pedestrian comes into contact with the item of interest, regard the pedestrian as the person under surveillance of the item of interest, capture the image of the person under surveillance; and extract the characteristics of the person under surveillance and update and record them in the database.

[0046] Further, the video monitoring device is also configured to: regularly capture the image of the item of interest; extract the characteristics of the item of interest and update and record them in the database.

[0047] Further, the video monitoring device includes: a body characteristic acquisition device, which is used to capture the body image of the person under surveillance corresponding to the item of interest and record the characteristics of the body image; and an association device, which is used to associate the characteristics of the body image and the characteristics of the item of interest and generate a pairing group.

[0048] Further, the video monitoring device further includes a facial feature acquisition device, which is configured to capture a facial image of the person under surveillance corresponding to the item of interest and record the features of the facial image.

[0049] Further, at least one of the video monitoring devices is disposed near the exit.

[0050] Further, the item tracking system further includes: an information display device, which is configured to display an image of the item of interest, the movement trajectory of the item of interest, an image of the person under surveillance, the movement trajectory of the person under surveillance, a perspective image of the item of interest, and a judgment result.

[0051] The information display device is configured to be activated when an abnormality occurs in the item of interest and / or the person under surveillance, where the abnormality includes at least one of a change in the data information of the pairing group, loss of the item of interest, the position of the item of interest approaching a concealed area, and the position of the person under surveillance approaching a concealed area.

[0052] Further, the item image marking and assigning device includes a first capture camera and a second capture camera.

[0053] The first capture camera is disposed between the non-destructive testing device and the second capture camera and adjacent to the non-destructive testing device, and is configured to acquire a first image of the item of interest and record the first features of the item of interest.

[0054] The second capture camera is disposed at the entrance of the baggage carousel, and is configured to acquire a second image of the item of interest and record the second features of the item of interest according to the first features of the item of interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Other objects and advantages of the present application will become apparent and can help to have a comprehensive understanding of the present application through the description of the present application with reference to the accompanying drawings below.

[0056] Figure 1 Schematically shows a scene diagram of an item tracking system based only on video monitoring according to an embodiment of the present application;

[0057] Figure 2 Schematically shows based on an embodiment of the present application Figure 1 of the flowchart of an item tracking method for a scene;

[0058] Figure 3 Schematically shows a flowchart of an item tracking method according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions of the embodiments of this application in conjunction with the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of this application without creative efforts fall within the scope of protection of this application.

[0060] Unless otherwise defined, the technical terms or scientific terms used in this application should have the ordinary meaning understood by those of ordinary skill in the art. The "first", "second", and similar terms used in this application do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or items appearing before this term cover the elements or items listed after this term and their equivalents, without excluding other elements or items.

[0061] In this document, unless otherwise specifically stated, directional terms such as "upper", "lower", "left", "right", "inner", "outer", etc. are used to represent the orientation or positional relationship based on the orientation shown in the drawings, and are only for the convenience of describing this application, rather than indicating or implying that the device, element, or component referred to must have a specific orientation, be constructed or operated in a specific orientation. It should be understood that when the absolute position of the object being described changes, the relative positional relationship they represent may also change accordingly. Therefore, these directional terms should not be construed as limitations on this application.

[0062] Baggage inspection is an important link in places such as airports and customs. Checked baggage first passes through the preliminary machine inspection system, where the images are examined and a small number of problematic suitcases are marked. When these baggages pass through the detection gate, the tags on the baggages are read to intercept or release them. To further strengthen supervision, when the baggage leaves the baggage carousel, the marked problematic baggage or the person in control of the marked problematic baggage triggers the alarm system when the marked problematic baggage or the person in control of the problematic baggage is identified at the exit, and the staff intercepts and inspects them.

[0063] However, in the prior art, there is no full - process tracking of problematic baggage or the person in control of the problematic baggage. Only when the tag reaches near the detector does it trigger an alarm. If a pedestrian tears up the tag after picking up the baggage but before leaving the customs, it will cause the problem items to be missed during inspection. This not only delays the inspection by the staff but also brings the risk of missed inspection. In addition, if a pedestrian deliberately changes the appearance of himself or the baggage before reaching the detection exit after picking up the baggage, it will cause the image - marked detection device to be unable to identify the problematic baggage and the pedestrian, bringing certain risks to the security inspection field.

[0064] In summary, in the current technology, there is a lack of real-time tracking, accurate positioning, and abnormal behavior recognition after a pedestrian picks up the luggage.

[0065] Based on this, the present application provides an item tracking system, method, and device based on video surveillance. After a pedestrian picks up the problematic luggage, the problematic luggage and the person in control of the problematic luggage are tracked and positioned in real time, and their behaviors and movement trajectories are monitored and predicted to predict the exit position of the problematic pedestrian, facilitating interception by the staff.

[0066] The following will be explained by taking the arriving luggage at the airport as an example. It should be noted that the item tracking system of the present application can be used in places such as ports, airports, and stations. Of course, the above-mentioned places cannot be used as a limitation to the present application. Within this place, after a pedestrian picks up an item and before the item is subject to special inspection, those with a certain trajectory displacement and time period are within the scope of the present application.

[0067] Figure 1 It is a schematic diagram of an item tracking system 100 based only on video surveillance and video analysis in an embodiment of the present application.

[0068] As Figure 1 shown, the item tracking system 100 includes: a non-destructive testing device 110, an item image marking and assigning device 120, and a video surveillance device 140.

[0069] The non-destructive testing device 110 is arranged upstream of the luggage carousel 150 and is used to obtain a perspective image of the item and determine whether it is an item of interest based on the perspective image.

[0070] In one embodiment, the item of interest can be a suspicious item, for example, a suspicious luggage. The non-destructive testing device 110 can perform a security inspection on the arriving luggage at the airport. The non-destructive testing device 110 can perform a perspective inspection on the luggage to obtain a perspective image of the luggage, and then analyze the perspective image to determine whether the luggage is an item of interest.

[0071] The item image marking and assigning device 120 is used to obtain an image of the item of interest and record the characteristics of the item of interest.

[0072] In customs inspection, the key is to intercept and inspect the item of interest. The item image marking and assigning device 120 can be understood as a control device for suspicious luggage. It can take a photo of the suspicious luggage to be controlled as an image mark, extract and record the characteristics of the luggage, and save the recorded content in the database. After recording, during later video surveillance, the item of interest can be locked based on the characteristics of the luggage, and the movement trajectory of the item of interest can be tracked across cameras. For example, the appearance of the luggage where the item of interest is stored can be recorded, and the luggage with this appearance characteristic can be tracked.

[0073] The video surveillance device 140 is arranged on the path between the baggage carousel 150 and the exit 170, and is used to continuously track and update the images of the items of interest and the images of the persons under surveillance corresponding to the items of interest. Among them, the persons under surveillance are all pedestrians who have come into contact with the items of interest, including but not limited to the owners of the items of interest, and also include suspicious pedestrians who are circled out in their entirety by means of action recognition or other classification methods that can be associated with the items of interest.

[0074] All persons under surveillance of the item of interest can also be regarded as key objects for interception and inspection, so as to prevent the item of interest from being transferred among multiple persons under surveillance or switched to other unmonitored baggage. It can be understood that the video surveillance device 140 can be used as a tracking device for suspects, taking pictures of the suspects to be placed under surveillance as image marks, extracting and recording the characteristics of all persons under surveillance, and saving the recorded content in the database corresponding to the relevant item of interest. After recording, in the later stage, all persons under surveillance related to the item of interest can be locked based on the characteristics of the persons under surveillance, and by tracking the movement trajectories of all persons under surveillance, they can be intercepted and subjected to special inspections when abnormalities occur. For example, the recorded characteristics can be the characteristics of pedestrians who pick up or store the baggage containing the item of interest, and can record the appearance characteristics, dressing styles, etc. of the pedestrians.

[0075] It can be understood that for video analysis only, all persons under surveillance who have come into contact with the item of interest need to be locked and tracked. Within the path from when the item of interest is extracted on the baggage carousel 150 to the exit 170, if there are other pedestrians who have come into contact with the item of interest, these pedestrians who have come into contact with the item of interest can also be regarded as persons under surveillance, and after extracting their characteristics, they are stored in the database for supervision and tracking.

[0076] According to an embodiment of the present application, for the tracking and data update of persons under surveillance, the video surveillance device 140 is configured to: when a pedestrian comes into contact with an item of interest, regard the pedestrian as a person under surveillance of the item of interest, capture an image of the person under surveillance; and extract the characteristics of the person under surveillance and record them in the database.

[0077] Timely data update of persons under surveillance and locking of all persons under surveillance related to the item of interest can reduce the probability of loss of the item of interest and improve the customs clearance speed and efficiency.

[0078] According to an embodiment of the present application, for the tracking and data update of the item of interest, the video surveillance device 140 is further configured to: regularly capture an image of the item of interest; extract the characteristics of the item of interest and update and record them in the database.

[0079] Considering that the appearance image of the item of interest captured by the item image marking device 120 on the conveyor belt is single and only from one perspective, which cannot fully reflect the characteristics of the item of interest, the appearance image of the item of interest is updated regularly in the video surveillance device 140 to obtain the appearance pictures of the item of interest from multiple angles, which is beneficial to locking the item of interest and avoiding the situation of tracking loss or omission.

[0080] The video surveillance device 140 is configured with a video tracking algorithm and computer vision technology. After the item of interest is picked up by its owner, the owner is recognized as a person under surveillance by the video surveillance device. The video surveillance device 140 can perform cross-camera video surveillance on the person under surveillance and the item of interest through multiple cameras to ensure that the item of interest is not taken out of the recorded luggage within the path from the time it is picked up to the security inspection exit.

[0081] Among them, in video surveillance, if an abnormality occurs to the item of interest or the person under surveillance (for example, the recorded luggage and the person under surveillance enter the video surveillance blind area, the recorded luggage and the person under surveillance leave the video surveillance blind area, the recorded luggage is opened, or the recorded luggage is lost), it can be directly determined that the luggage and the person under surveillance associated with the luggage need to be subject to special inspection. The movement trajectories of the person under surveillance and the recorded luggage can be clearly known through the camera, that is, the positions of the recorded luggage and all persons under surveillance can be obtained in real time through the video surveillance device 140. When necessary, movement trajectory prediction and interception measures can be taken for the recorded luggage and all persons under surveillance according to the information provided by the video surveillance device 140, which is beneficial to reducing the work difficulty of the staff.

[0082] In one embodiment, when the recorded luggage approaches a concealed area, the staff can timely obtain the current position information of the recorded luggage according to the position information provided by the camera, and then intercept the luggage in time, thereby reducing the work difficulty of the staff.

[0083] According to the item tracking system 100 in this scenario, through the use of cameras, video tracking algorithms, etc. and computer vision technology for cross-camera tracking and monitoring of the item of interest and the person under surveillance, only based on video analysis, it helps the staff lock the target in advance, which not only improves the customs clearance efficiency but also avoids the situation of missed inspection, thus reducing the work difficulty of the staff while enhancing the supervision intensity.

[0084] It should be noted that the regulatory ability can be enhanced through the cross-camera tracking method of video surveillance, but usually when there is no abnormality, the recorded luggage also needs to be subject to special inspection at the exit 170.

[0085] A virtual marking device is provided in the object image marking device 120. The virtual marking device can extract the features of the luggage through image recognition technology, such as the shape, size, color, texture and material of the luggage, and then record the extracted feature information to help lock the problematic luggage.

[0086] like Figure 1 As shown, the object image marking device 120 is also arranged upstream of the luggage carousel 150 and after the non-destructive testing device 110. The non-destructive testing device 110 and the object image marking device 120 can form a preliminary machine inspection, which is to inspect and mark the luggage before the control person picks up the luggage.

[0087] According to an embodiment of the present application, at least one video monitoring device 140 is arranged at intervals on the inner circumference of the luggage carousel 150 and is used to obtain an image of a person controlling an item of interest located on the outer circumference of the luggage carousel 150 .

[0088] Generally, in order to clearly capture the facial features and body features of the pedestrian when the luggage is being picked up by the pedestrian, the video monitoring devices 140 are arranged at intervals on the inner circumference of the luggage carousel 150. When the pedestrian bends down and touches the luggage, a snapshot is taken, the pedestrian is identified as the monitored person, and the facial features and body features are extracted. The facial features and body features are stored in the database and bound to the recorded luggage to achieve person association.

[0089] According to an embodiment of the present application, the video monitoring device 140 further includes a facial feature acquisition device, a body feature acquisition device, and an association device. The facial feature acquisition device is used to acquire a facial image of a person who controls an item of interest and record the features of the facial image; the body feature acquisition device is used to acquire a body image of a person who controls an item of interest and record the features of the body image; the association device is used to associate the features of the facial image, the features of the body image, and the features of the item of interest and generate a pairing group.

[0090] When capturing the person under surveillance, the features of the person under surveillance and the features of the luggage are bound to generate a pairing group. After the pairing group is generated, the pairing group can be monitored in real time, and the person association is achieved by converting the location tracking and positioning of the luggage from the physical tag to the tracking and positioning of the luggage and the person under surveillance.

[0091] In actual operation, since the hall in the airport is high, it is difficult to focus on and acquire facial features, so the practicability of the facial feature acquisition device is not high. The present application provides another embodiment, by associating the features of the body image and the features of the object of interest as a paired group for tracking.

[0092] According to another embodiment of the present application, the video monitoring device 140 includes a body feature acquisition device and an association device.

[0093] The body feature acquisition device is used to acquire the body image of the person in control of the item of interest and record the features of the body image; the association device is used to associate the features of the body image with the features of the item of interest and generate a pairing group.

[0094] According to an embodiment of the present application, the video surveillance device 140 is configured to: acquire the movement trajectory of the item of interest according to the image of the item of interest, and acquire the movement trajectory of the person in control of the item of interest according to the image of the person in control of the item of interest.

[0095] For the item of interest, the video surveillance device 140 can monitor the movement trajectory of the luggage in real time according to the feature information of the luggage recorded by the virtual marker assigning device; for the person in control, the video surveillance device 140 can, according to the feature information of the luggage recorded by the virtual marker assigning device, extract the feature information of the facial image and the body image of all the persons in control recorded in the pairing group, and monitor the movement trajectories of all the persons in control in real time.

[0096] According to an embodiment of the present application, at least one video surveillance device 140 is provided near the exit 170.

[0097] The exit 170 can be understood as an interception passage. By arranging the video surveillance device 140 near the interception passage, effective interception can be carried out in a timely manner when the pairing group changes.

[0098] According to an embodiment of the present application, the item tracking system 100 further includes an information display device (not shown). The information display device is used to display the image of the item of interest and the movement trajectory of the item of interest, the image of the person in control and the movement trajectory of the person in control, as well as the perspective image of the item of interest and the judgment result. The information display device is configured to be activated when an abnormality occurs in the item of interest and / or the person in control. While an alarm reminder can be issued, the above-mentioned image information and trajectory information are displayed, where the abnormality includes at least one of a change in the data information of the pairing group, the loss of the item of interest, the position of the item of interest approaching a concealed area, and the position of the person in control approaching a concealed area. Here, the concealed area can be understood as a blind area of video surveillance.

[0099] The alarm reminder of the information display device can be pushed to the staff in a timely manner to lock the target in advance. The staff can guide the person in control and the carrier of the abnormal luggage with problems, and conduct an unpacking inspection on the luggage they carry to find problems early, improving the supervision ability of contraband.

[0100] For example, in a scenario based only on video surveillance, if the item of interest appears in the blind area of video surveillance, the information display device directly determines that special inspection needs to be carried out on the item of interest and the person in control related to it.

[0101] For another example, among multiple persons under surveillance who form a paired group with a recorded piece of luggage, if any one or more of the persons under surveillance appear in a blind area of video surveillance, the information display device directly determines that a special inspection needs to be carried out on this paired group, that is, a special inspection is carried out on the luggage within the paired group and all the persons under surveillance bound to the luggage.

[0102] Considering that in the actual operation process, there is a certain distance between the conveyor belt from the nondestructive testing device 110 to the luggage carousel 150. Since there are significant differences between the light or other environmental factors from the nondestructive testing device 110 to the luggage carousel 150 and the light or environmental factors of the item of interest on the luggage carousel 150, after the nondestructive testing device 110 determines the item of interest through X-rays, the difference in light on the conveyor belt and the light on the luggage carousel 150 will cause the appearance features of the recorded item of interest not to match the appearance features under the actual light, resulting in incorrect marking or missed marking, and the video surveillance device 140 cannot accurately track.

[0103] According to an embodiment of the present application, in order to avoid the occurrence of the above problems, the item image marking and assigning device 120 includes a first capture camera and a second capture camera. The first capture camera is arranged between the nondestructive testing device and the second capture camera and is adjacent to the nondestructive testing device, and is used to obtain a first image of the item of interest and record the first features of the item of interest. The second capture camera is arranged at the entrance of the luggage carousel and is used to obtain a second image of the item of interest according to the first features of the item of interest and record the second features of the item of interest.

[0104] It can be understood that the first capture camera is adjacent to the nondestructive testing device 110, which is equivalent to the first capture camera capturing the first image of the item of interest just conveyed out from the nondestructive testing device 110 and inputting the first features of the item of interest at the first time. The second capture camera is arranged at the entrance of the luggage carousel 150, and its light and other environmental factors are between the nondestructive testing device 110 and the luggage carousel 150, playing a connecting role. It can identify and capture the second image of the item of interest through the first image and record the second features. Since it is arranged at the entrance of the luggage carousel, the second image of the item of interest captured and the second features recorded have a high appearance consistency with the item of interest on the luggage carousel 150, which is conducive to later capturing the persons under surveillance according to the second image and the second features and the tracking of the item of interest by the video surveillance device 140.

[0105] It should be noted that both the first capture camera and the second capture camera can be configured as one or more cameras for taking pictures, and the quantity is not limited here.

[0106] Based on the above-mentioned article tracking system 100, the present application also provides an article tracking method that only utilizes video monitoring.

[0107] Combined with Figure 1 the scenario in Figure 2 FIG. schematically shows a flowchart of an article tracking method according to an embodiment of the present application.

[0108] As Figure 1 shown, the article tracking method of this embodiment includes operations S210 to S240.

[0109] In operation S210, security checks are performed on the articles upstream of the baggage carousel to determine whether the articles are articles of interest.

[0110] In operation S220, when the article is an article of interest, an image of the article of interest is acquired and the characteristics of the article of interest are recorded.

[0111] In one embodiment, when the article is an article of interest, a first image of the article of interest is acquired and a first characteristic of the article of interest is recorded; a second image of the article of interest is acquired according to the first characteristic of the article of interest and a second characteristic of the article of interest is recorded.

[0112] In operation S230, when the article is an article of interest, the image of the person under surveillance corresponding to the article of interest is continuously updated on the path between the baggage carousel and the exit, and the characteristics of the person under surveillance are recorded, where the person under surveillance is all pedestrians who have come into contact with the article of interest.

[0113] In operation S240, the article of interest and the person under surveillance corresponding to the article of interest are tracked on the path between the baggage carousel and the exit to determine whether the article of interest and the person under surveillance corresponding to the article of interest need special inspection.

[0114] All luggage is subject to security inspection before the luggage carousel, and it is analyzed whether it belongs to an item of interest. The analysis process can perform manual or automatic analysis on the perspective image locally or remotely. The item of interest can be suspected luggage. When it belongs to an item of interest, an image of the luggage is obtained and the characteristics of the luggage are recorded. The characteristics of the luggage can be the shape, size, color, texture, material, etc. of the luggage. The luggage is transported to the luggage carousel and retrieved by a pedestrian. Since there is physical contact during the action, this pedestrian is determined to be a person under surveillance. While the person under surveillance retrieves the item of interest, a photo of the person under surveillance of the item of interest is taken, and the characteristics of the person under surveillance are recorded. The item of interest and the person under surveillance are monitored in real time through video surveillance. When it is found in the video surveillance that the item of interest is touched by other passing pedestrians, the pedestrian who touches the item of interest is determined to be one of the persons under surveillance related to the item of interest, and their characteristics are captured and the record in the database is updated. When an abnormality of the person under surveillance or an abnormality of the item of interest is detected in the video surveillance, a special inspection is performed. The special inspection can be to guide the pedestrian to open the box for inspection.

[0115] Of course, the way of video surveillance can enhance the supervision ability. However, when there is usually no abnormality, the luggage determined to be an item of interest also needs to be subject to special inspection at the exit.

[0116] According to the item tracking method of the present application, through continuous tracking and data update of the item of interest and the persons under surveillance who have come into contact with the item of interest, the behaviors of all persons under surveillance during the period from when the item of interest is picked up to the exit can be comprehensively and effectively supervised, helping the staff to lock in the target in advance. This not only improves the customs clearance efficiency but also avoids the situation of missed inspection, thereby reducing the work difficulty of the staff while enhancing the supervision intensity.

[0117] According to an embodiment of the present application, while updating the image of the person under surveillance corresponding to the item of interest, an image of the item of interest is obtained, and the characteristics of the item of interest are updated and recorded.

[0118] In this embodiment, the appearance image of the item of interest is updated regularly. Specifically, each time the image of the person under surveillance corresponding to the item of interest is updated, the item of interest is captured, the image of the item of interest is extracted, and the characteristics of the item of interest are updated and recorded.

[0119] According to an embodiment of the present application, in operation S210, performing a security inspection on the item to determine whether the item belongs to an item of interest includes:

[0120] Obtaining a perspective image of the item.

[0121] Judging whether it belongs to an item of interest according to the perspective image.

[0122] An X-ray scanning unit is provided on the conveyor belt upstream of the baggage carousel to perform fluoroscopy on each item on the conveyor belt, obtain a fluoroscopic image of each item, and then determine whether it belongs to an item of interest based on the fluoroscopic image. The process of determining the image of an item of interest is a method in the prior art and will not be elaborated in this embodiment.

[0123] According to an embodiment of the present application, in operation S230, when the item belongs to an item of interest, the image of the person under surveillance corresponding to the item of interest is continuously updated on the path between the baggage carousel and the exit, and the characteristics of the person under surveillance are recorded, including:

[0124] When a pedestrian comes into contact with an item of interest, the pedestrian is regarded as the person under surveillance of the item of interest, and the image of the person under surveillance is captured.

[0125] In one embodiment, the image of the person under surveillance may include: the facial image of the person under surveillance corresponding to the item of interest and the body image of the person under surveillance corresponding to the item of interest.

[0126] When a pedestrian extracts an item of interest on the baggage carousel, the pedestrian is considered as the person under surveillance of the item of interest, the image of the person under surveillance is captured, and the characteristics of the person under surveillance are extracted, such as facial features, body features, and dressing style, etc.

[0127] The characteristics of the facial image and the characteristics of the body image are recorded, and the characteristics of the facial image, the characteristics of the body image, and the characteristics of the item of interest are associated to generate a paired group.

[0128] It can be understood that the facial features of the person under surveillance, the body features of the person under surveillance, and the characteristics of the item of interest are associated through image re-identification technology. When any data information in the paired group is extracted, the other two data information can be obtained. For example, when viewing the facial features of the person under surveillance, the body features and the characteristics of the item of interest that the person under surveillance should possess will be associated.

[0129] In another embodiment, the image of the person under surveillance may only include the body image of the person under surveillance corresponding to the item of interest.

[0130] Since the roof of the airport hall is relatively high, the video surveillance device cannot accurately obtain or extract the facial features of pedestrians. Therefore, only the body features are used as the reference standard, that is, the body features and appearance identification pictures such as dressing style of the person under surveillance are extracted and captured.

[0131] The characteristics of the body image are recorded, and the characteristics of the body image and the characteristics of the item of interest are associated to generate a paired group. After the body features of the person under surveillance and the characteristics of the item of interest are associated, when viewing the body features of the person under surveillance, the characteristics of the item of interest will be associated.

[0132] In operation 240, when only using video surveillance to track the item of interest and the person under surveillance corresponding to the item of interest, there are four specific embodiments for determining whether special inspection is required.

[0133] Embodiment 1: The video surveillance locks the item of interest. When the correspondence between the item of interest and the person under surveillance is abnormal, special inspection is required. Specifically:

[0134] Lock the item of interest based on the image of the item of interest, and provide the body image of the carrier in real time.

[0135] According to the characteristics of the item of interest, obtain the data information of the pairing group associated with the characteristics of the item of interest.

[0136] When the characteristics of the body image of the carrier are different from the data information of the pairing group, it is determined that the item of interest requires special inspection.

[0137] It can be understood that when the recorded luggage is locked and it is found in the video surveillance that the recorded luggage is taken over by other unrecorded pedestrians other than the person under surveillance, the luggage needs to be inspected by opening the case.

[0138] In this embodiment, the facial features of the carrier can also be obtained simultaneously, and the facial features can also be used as one of the reference data of the pairing group.

[0139] Embodiment 2: The video surveillance locks the person under surveillance. When the correspondence between the item of interest and the person under surveillance is abnormal, special inspection is required. Specifically:

[0140] Lock the person under surveillance according to the characteristics of the person under surveillance, and provide the image of the items carried by the person under surveillance in real time.

[0141] According to the characteristics of the person under surveillance, obtain the data information of the pairing group associated with the characteristics of the person under surveillance.

[0142] When the image of the items carried by the person under surveillance is different from the data information of the pairing group, it is determined that the items carried by the person under surveillance require special inspection, where the carried items are the items of interest.

[0143] It can be understood that when the person under surveillance is locked and it is found in the video surveillance that the number of luggage items and the appearance of the person under surveillance have changed, all the luggage currently carried by the person under surveillance needs to be inspected by opening the case.

[0144] Embodiment 3: When the item of interest is lost, it is determined that the item of interest and the person under surveillance corresponding to the item of interest require special inspection.

[0145] When the item of interest cannot be locked in the video surveillance and does not appear again later, it can be understood that the item of interest is lost, and all the persons under surveillance forming the pairing group with the item of interest require special inspection.

[0146] Example 4. When an item of interest appears or approaches a concealed area, special inspection is required.

[0147] For example, if the item of interest cannot be locked in the video surveillance and then appears in the video surveillance later, all the persons under surveillance who form a paired group with the item of interest and the item of interest need special inspection.

[0148] Another example is that when it can be known from the video surveillance that the item of interest is about to move into the blind area of surveillance or has just emerged from the blind area of surveillance, all the persons under surveillance who form a paired group with the item of interest need special inspection.

[0149] According to an embodiment of the present application, in order to let the staff know in advance the luggage that needs special inspection, operations such as early interception can be implemented. The item tracking method further includes:

[0150] When the item of interest needs special inspection, an alarm message is pushed. The alarm message includes the image of the item of interest and the movement trajectory of the item of interest, the image of the person under surveillance and the movement trajectory of the person under surveillance, as well as the perspective image of the item of interest and the judgment result. The movement trajectory of the item of interest and the movement trajectories of all the persons under surveillance are obtained through cross-camera video surveillance.

[0151] The alarm message can show the activity locations and trajectories of abnormal pedestrians and abnormal luggage. The staff can predict the trajectories of abnormal pedestrians and luggage based on the alarm reminder and intercept and inspect them at any time, which is conducive to discovering problems early.

[0152] It should be noted that, according to the actual situation, the method based on entity label tracking and positioning has obvious limitations, such as label detachment, damage, signal shielding and interference caused by the metal material on the surface of the luggage. While the maintenance and consumption costs are relatively high, the positioning function is poor. Therefore, in the early stage of machine inspection in this embodiment, entity labels for the item of interest, such as sealing measures, are not adopted.

[0153] Figure 3 is combined with Figure 1 in the scenario, schematically showing a flowchart of using the item tracking method according to this embodiment.

[0154] In operation S301, the luggage is placed on the conveyor belt.

[0155] In operation S302, the non-destructive testing equipment scans the luggage to generate a perspective image.

[0156] In operation S303, according to the perspective image, when the luggage is determined to be suspicious luggage, the characteristics of the suspicious luggage are recorded.

[0157] In operation S304, the video surveillance device is installed at multiple collection points between the baggage carousel and the exit to monitor the suspicious baggage and pedestrians, and determine whether a pedestrian comes into contact with the suspicious baggage.

[0158] In operation S305, when there is contact, capture the characteristics of the body image of the person under surveillance and update the record in the database.

[0159] In operation S306, associate the characteristics of the body image of the person under surveillance with the characteristics of the suspicious baggage.

[0160] In operation S307, the video surveillance device tracks the person under surveillance and the suspicious baggage in real time, and determines whether an abnormality occurs. Abnormalities include when the video surveillance device discovers a change in the corresponding relationship between the suspicious pedestrian and the suspicious baggage, when the suspicious baggage is lost, when the person under surveillance is at the edge of or enters the monitoring blind area, and when the suspicious baggage is at the edge of or enters the monitoring blind area.

[0161] In operation S308, push the alarm information.

[0162] In operation S309, when passing through the exit, if there is an alarm prompt, special inspection is required for the suspicious baggage or the carried baggage of the person under surveillance.

[0163] In operation S310, when passing through the exit, ordinary baggage passes through directly or undergoes ordinary inspection.

[0164] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present application can be combined or combined in various ways, even if such combinations or combinations are not explicitly recited in the present application. In particular, without departing from the spirit and teachings of the present application, the features recited in the various embodiments and / or claims of the present application can be combined and combined in various ways. All such combinations and / or combinations fall within the scope of the present application.

[0165] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "instances", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0166] The embodiments of the present application have been described above. However, these embodiments are merely for illustrative purposes and not for limiting the scope of the present application. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present application is defined by the appended claims and their equivalents. Without departing from the scope of the present application, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present application.

Claims

1. An early-stage machine inspection item tracking method, characterized in that, It includes the following steps: Conduct a security check on the items upstream of the baggage carousel to determine whether the items are of interest; When the item is of interest, obtain an image of the item of interest and record the characteristics of the item of interest; When the item is of interest, continuously update the image of the person under surveillance corresponding to the item of interest on the path between the baggage carousel and the exit, and record the characteristics of the person under surveillance, where the person under surveillance is all pedestrians who have come into contact with the item of interest, and the image of the person under surveillance includes the body image of the person under surveillance; and Track the item of interest and the person under surveillance corresponding to the item of interest on the path between the baggage carousel and the exit; Among them, when the item is of interest, continuously updating the image of the person under surveillance corresponding to the item of interest on the path between the baggage carousel and the exit and recording the characteristics of the person under surveillance includes: Extract and record the characteristics of the body image; Associate the characteristics of the body image with the characteristics of the item of interest and generate a pairing group.

2. The item tracking method according to claim 1, characterized in that, While updating the image of the person under surveillance corresponding to the item of interest, obtain an image of the item of interest and update and record the characteristics of the item of interest.

3. The item tracking method according to claim 1 or 2, characterized in that, Conducting a security check on the item to determine whether the item is of interest includes: Obtain a perspective image of the item; and Judge whether it is of interest according to the perspective image.

4. The item tracking method according to claim 3, characterized in that, When the item is of interest, continuously updating the image of the person under surveillance corresponding to the item of interest on the path between the baggage carousel and the exit and recording the characteristics of the person under surveillance includes: When a pedestrian comes into contact with the item of interest, regard the pedestrian as the person under surveillance of the item of interest and capture an image of the person under surveillance.

5. The item tracking method according to claim 1, characterized in that, Tracking the item of interest and the person under surveillance corresponding to the item of interest includes: Lock the item of interest according to the image of the item of interest and provide the body image of the carrier in real time; According to the characteristics of the item of interest, obtain the data information of the pairing group associated with the characteristics of the item of interest; When the characteristics of the body image of the carrier are different from the data information of the pairing group, determine that the item of interest and the person under surveillance corresponding to the item of interest need special inspection.

6. The item tracking method according to claim 1, characterized in that, Tracking the item of interest and the person under surveillance corresponding to the item of interest includes: Lock the person under surveillance according to the characteristics of the body image of the person under surveillance and provide the image of the items carried by the person under surveillance in real time; According to the characteristics of the body image of the person under surveillance, obtain the data information of the pairing group associated with the characteristics of the body image of the person under surveillance; When the image of the items carried by the person under surveillance is different from the data information of the pairing group, determine that the items carried by the person under surveillance need special inspection.

7. The item tracking method according to claim 1, characterized in that, Tracking the item of interest and the person under surveillance corresponding to the item of interest includes: When the item of interest is lost, determine that the item of interest and the person under surveillance corresponding to the item of interest need special inspection.

8. The item tracking method according to claim 1, characterized in that, Tracking the item of interest and the person under surveillance corresponding to the item of interest includes: When the person under surveillance or the item of interest approaches a concealed area, determining that the item of interest and the person under surveillance corresponding to the item of interest require special inspection.

9. The item tracking method according to any one of claims 1-2 and 4-8, characterized in that, Capturing an image of the person under surveillance also includes capturing a facial image of the person under surveillance corresponding to the item of interest.

10. The item tracking method according to claim 9, characterized in that, When the item belongs to an item of interest, continuously updating an image of the person under surveillance corresponding to the item of interest on the path between the baggage carousel and the exit, and recording the characteristics of the person under surveillance, further includes: Extracting and recording the characteristics of the facial image; Associating the characteristics of the facial image, the characteristics of the body image, and the characteristics of the item of interest, and generating a paired group.

11. The article tracking method according to any one of claims 1-2, 4-8, and 10, characterized in that, The method further includes: When the item of interest and / or the person under surveillance corresponding to the item of interest requires special inspection, pushing an alarm message.

12. The article tracking method according to claim 11, characterized in that, The alarm message includes: an image of the item of interest and the movement trajectory of the item of interest, an image of the person under surveillance and the movement trajectory of the person under surveillance, and a perspective image of the item of interest and a judgment result.

13. The article tracking method according to claim 1, characterized in that, When the item belongs to an item of interest, obtaining an image of the item of interest and recording the characteristics of the item of interest, specifically including: When the item belongs to an item of interest, obtaining a first image of the item of interest and recording the first characteristics of the item of interest; Obtaining a second image of the item of interest according to the first characteristics of the item of interest and recording the second characteristics of the item of interest.

14. An early-stage machine-inspected article tracking system, characterized in that, Includes: A non-destructive testing device, which is arranged upstream of the baggage carousel, is used to obtain a perspective image of the item, and judge whether it belongs to an item of interest according to the perspective image; An item image marking and assigning device, which is arranged upstream of the baggage carousel, is used to obtain an image of the item of interest and record the characteristics of the item of interest; A video surveillance device, which is arranged on the path between the baggage carousel and the exit, is used to continuously track and update and record an image of the item of interest and an image of the person under surveillance corresponding to the item of interest, wherein the person under surveillance is all pedestrians who have come into contact with the item of interest, and the image of the person under surveillance includes the body image of the person under surveillance; Wherein, the video surveillance device includes: A body feature acquisition device, which is used to capture the body image of the person under surveillance corresponding to the item of interest and record the characteristics of the body image; and An association device, which is used to associate the characteristics of the body image and the characteristics of the item of interest and generate a paired group.

15. The article tracking system according to claim 14, characterized in that, The video surveillance device is configured to: When a pedestrian comes into contact with the item of interest, taking the pedestrian as the person under surveillance of the item of interest and capturing an image of the person under surveillance; and Extracting the characteristics of the person under surveillance and updating and recording them in the database.

16. The article tracking system according to claim 15, characterized in that, The video surveillance device is further configured to: Timely capturing an image of the item of interest; Extracting the characteristics of the item of interest and updating and recording them in the database.

17. The article tracking system according to claim 14, characterized in that, The video surveillance device further includes a facial feature acquisition device, which is used to capture the facial image of the person under surveillance corresponding to the item of interest and record the features of the facial image.

18. The article tracking system according to claim 17, characterized in that, At least one of the video surveillance devices is arranged near the exit.

19. The article tracking system according to claim 18, characterized in that, It further includes: An information display device, which is used to display the image of the item of interest and the movement trajectory of the item of interest, the image of the person under surveillance and the movement trajectory of the person under surveillance, as well as the perspective image and judgment result of the item of interest. The information display device is configured to be activated when there is an abnormality in the item of interest and / or the person under surveillance, where the abnormality includes at least one of the following: the data information of the pairing group changes, the item of interest is lost, the position of the item of interest is close to a concealed area, and the position of the person under surveillance is close to a concealed area.

20. The article tracking system according to claim 14, characterized in that, The item image marking and assigning device includes a first capture camera and a second capture camera. The first capture camera is arranged between the non-destructive testing device and the second capture camera and is adjacent to the non-destructive testing device, and is used to obtain the first image of the item of interest and record the first features of the item of interest. The second capture camera is arranged at the entrance of the luggage carousel and is used to obtain the second image of the item of interest and record the second features of the item of interest according to the first features of the item of interest.

Images