A bag association method based on full-segment fuzzy matching

Abstract

The application discloses a bag association method based on full-section fuzzy matching, which comprises the following steps: S1: continuously obtaining a current camera picture and extracting package image data, buffering the package image data, and obtaining a package image data queue M; S2: obtaining a package X-ray picture completely imaged in a security inspection machine, and extracting imaging time of the package X-ray picture, which is recorded as a time point t2; S3: performing time interval operation according to the time point t2 and a time point t, and obtaining a (t2-△t, t2) time section; S4: at the time point t2, updating a package image data queue M', M' comprising package image data of the (t2-△t, t2) time section, and performing fuzzy matching on the package image data in the (t2-△t, t2) time section and the package X-ray picture; and S5: according to a fuzzy matching result, obtaining package image data with the best fitting degree as final bag association image information, and performing bag association binding on the package X-ray picture.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, specifically to a method, device, and medium for associating packages based on full-segment fuzzy matching. Background Technology

[0002] In existing technologies, during parcel security checks in subways, airports, or other security scenarios, after passengers place their parcels on the conveyor belt into the security scanner, the parcels undergo a prohibited item identification process. When a prohibited item is identified, the corresponding parcel needs to be immediately identified. Therefore, it is necessary to correlate the parcel image from the security scanner with the RGB image of the parcel to facilitate confirmation by security personnel. Current solutions use the X-ray image imaging time and the time it takes for the parcel to move from the bag inlet to the X-ray imaging position to calculate the parcel's time on the conveyor belt and thus find the corresponding visible light image. This method has the following problems: 1. Due to the varying lengths of parcels and the deviations in conveyor belt speed, the time it takes for the parcel to travel from the bag inlet to the X-ray imaging position varies. Fixing it to a fixed value will lead to matching errors. 2. Furthermore, the visible light image of a parcel is a panoramic image, which may include other parcels, interfering with parcel matching. To address these issues, this invention proposes a parcel association method based on full-segment fuzzy matching. Summary of the Invention

[0003] This invention provides a method, device, and medium for bag association based on full-segment fuzzy matching. It utilizes fuzzy matching between package X-ray images and package image data in a cached package image data queue M to obtain the package image data with the best fit. Based on the matched package image data and package X-ray images, bag association is performed, so that when contraband is identified in the package X-ray image, the purpose of the corresponding package can be determined immediately.

[0004] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0005] A bag association method based on full-segment fuzzy matching is applied to a security inspection machine system. The security inspection machine has an X-ray imaging area, and a camera is installed above the conveyor belt in front of the security inspection machine entrance. The camera's shooting direction is perpendicular to the conveyor belt, and the camera's shooting range completely covers the conveyor belt. The method includes the following steps:

[0006] S1: Continuously acquire the current camera image and extract the package image data, cache the package image data, and obtain the package image data queue M. M includes package image data for the time period (t-△t, t), where △t is the package image data caching time, △t is the time it takes for the package to move from one end of the conveyor belt to the other end, and t is the time point at which the current camera image is extracted.

[0007] S2: Obtain a fully imaged package X-ray image from the security inspection machine and extract the image imaging time of the package X-ray image, denoted as time point t2;

[0008] S3: Calculate the time interval between time point t2 and time point t to obtain the time interval (t2-△t, t2), where time point t is within the time interval (t2-△t, t2);

[0009] S4: At time point t2, the package image data queue M is updated to the package image data queue M'. M' includes package image data for the time period (t2-△t, t2). The package image data for the time period (t2-△t, t2) is used to perform fuzzy matching with the package X-ray image.

[0010] S5: Based on the fuzzy matching results, obtain the package image data with the best fit as the final package association image information, and bind the final package association image information with the package X-ray image.

[0011] Furthermore, executing S1 includes the following steps:

[0012] S11: Obtain the RGB values ​​of the pixels that make up the current camera image;

[0013] S12: Obtain the wrapping image data based on the RGB values. The wrapping image data includes, but is not limited to, the outer contour of the wrapping, the visible light image of the wrapping, the coordinates of the leftmost and rightmost parts of the wrapping in the current camera image, and the length of the wrapping in the current camera image.

[0014] Furthermore, it also includes the following steps:

[0015] S1a: Obtain the package image data B at the initial moment;

[0016] S1b: Track the package from the camera images and update the package image data to obtain package image data B' for fuzzy matching.

[0017] Furthermore, setting the initial time to T0, the package image data B at the initial time includes:

[0018] B = {id, img0, xL0, xR0, LB, T0}, where id is the package ID, img0 is the visible light image of the package extracted from the camera image at time T0, xL0 is the leftmost coordinate of the package in the camera image at time T0, xR0 is the rightmost coordinate of the package in the camera image at time T0, LB is the length of the package in the camera image at time T0, and LB = xR0 - xL0.

[0019] Track the package from the camera image and obtain the package image data B' at time T1, B'={id, img1, xL1,xR1, LB1, T1}, where id is the package ID, img1 is the visible light image of the package extracted from the camera image at time T1, xL1 is the leftmost coordinate of the package in the camera image at time T1, xR1 is the rightmost coordinate of the package in the camera image at time T1, LB1 is the length of the package in the camera image at time T1, LB1= xR1- xL1;

[0020] When LB1 > LB, the package image data is updated. At this time, the package image data used for fuzzy matching is package image data B', which is cached in the package image data queue.

[0021] Furthermore, when executing S4, the following steps are also included: capturing the package image data of M' in the time period (t2-△t, t), and blurring the package image data and package X-ray image in the time period (t2-△t, t).

[0022] The present invention also provides an electronic device for bag association based on full-segment fuzzy matching, comprising:

[0023] One or more processors;

[0024] A storage unit is used to store one or more programs. When one or more programs are executed by one or more processors, they enable one or more processors to implement the above-mentioned packet association method based on full-segment fuzzy matching.

[0025] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the above-described packet association method based on full-segment fuzzy matching.

[0026] The beneficial effects of this invention are as follows: This invention utilizes fuzzy matching between package X-ray images and package image data in a cached package image data queue M to obtain package image data with the best fit. Based on the matched package image data and package X-ray images, the package is associated with the package, so that when contraband is identified in the package X-ray image, the purpose of the corresponding package can be determined immediately. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0028] Figure 2 This is an illustration of the process of acquiring the current camera image;

[0029] Figure 3 This is a schematic diagram of acquiring the current camera image at time T0.

[0030] Figure 4 The current camera image acquired at time T0;

[0031] Figure 5 This is a schematic diagram of acquiring the current camera image at time T1.

[0032] Figure 6 This is the current camera image captured at time T1. Detailed Implementation

[0033] To make the technical solution, the technical problem solved, and the technical effects of the present invention clearer, the technical solution of the present invention will be thoroughly and completely described below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments:

[0035] Example 1

[0036] like Figure 1 and Figure 2 As shown, the security inspection machine has an X-ray imaging area, and a camera is mounted above the conveyor belt. The camera's shooting direction is perpendicular to the conveyor belt, and its field of view covers the conveyor belt. This allows it to capture images of packages on the conveyor belt, with the camera's shooting direction perpendicular to the packages. The resulting camera image, in visible light, is a top-down view of the package.

[0037] In this embodiment, the conveyor belt operates from point A to point D, where A and D are the two ends of the conveyor belt. The conveyor belt's operating speed is v. Point B is the bag inlet of the security scanner, and the length of AB is L. Point C is the bag outlet of the security scanner, and the BC section of the conveyor belt is inside the security scanner. Point X indicates that when the package reaches this position, scanning begins to create an X-ray image. The length of BX is S. There is a camera above the bag inlet conveyor belt AB, which can capture the entire AB conveyor belt.

[0038] A packet association method based on full-segment fuzzy matching includes the following steps:

[0039] S1: Continuously acquire the current camera image and extract the package image data, cache the package image data, and obtain the package image data queue M. M includes package image data for the time period (t-△t, t), where △t is the package image data caching time and △t is the time it takes for the package to travel from one end of the conveyor belt to the other. Assuming a package moves from A to D, then △t=L / v is the longest time the package travels on the conveyor belt, and t is the time point at which the current camera image is acquired.

[0040] Specifically, the extracted package image data is denoted as B, and M = {B0, B1, B2, ..., Bi, ..., Bn}. When extracting the package image data, the RGB values ​​of the pixels constituting the current camera image are first obtained. Then, the package image data is calculated based on these RGB values. The package image data includes, but is not limited to, the outer contour of the package, the visible light image of the package, the coordinates of the leftmost and rightmost parts of the package within the current camera image, and the length of the package within the current camera image.

[0041] S2: Obtain a fully imaged package X-ray image from the security inspection machine and extract the image imaging time of the package X-ray image, denoted as time point t2;

[0042] S3: Calculate the time interval between time point t2 and time point t to obtain the time interval (t2-△t, t2), where time point t is within the time interval (t2-△t, t2);

[0043] Specifically, let t1 be the time it takes for the package to travel from the current time point t until the X-ray image of the package is completed, where t1 = t2 - t. The time interval (t - Δt, t) becomes (t - Δt + t1, t + t1) after time t1. Substituting t1 = t2 - t, we can obtain the time interval (t2 - Δt, t2). Since t1 is the starting point X of the package moving from a point on the conveyor belt to the starting point X for X-ray image formation, the time point t shown is within the time interval (t2 - Δt, t2).

[0044] S4: At time point t2, the package image data queue M is updated to the package image data queue M'. M' includes package image data for the time period (t2-△t, t2). The package image data for the time period (t2-△t, t2) is used to perform fuzzy matching with the package X-ray image.

[0045] Specifically, while the package image data is being cached, it is also being cleared in real time. The clearing method is as follows:

[0046] For Bi in M:

[0047] If the time ti of Bi is less than (t-Δt):

[0048] Clear package information Bi.

[0049] Therefore, at time point t2, the package image data queue will clear the package image data before t2-Δt. However, since time point t is within the time period (t2-Δt, t2), the package image data obtained at time point t is located in the updated package image data queue M'. The package image data in M' is used to perform fuzzy matching with the package X-ray image. The obtained X-ray image is denoted as XB, and max_conf is the maximum matching degree. max_conf is initialized to 0, and BB is the most suitable visible light package information.

[0050] For Bi in M':

[0051] Calculate the similarity conf between Bi and XB, comparing them from dimensions such as size and area.

[0052] When conf > max_conf

[0053] max_conf = conf

[0054] BB = Bi

[0055] Finally, remove Bi from M' to avoid duplicate matching.

[0056] In addition, in order to reduce the package image data used for fuzzy matching and shorten the traversal time, package image data of M' in the time interval (t2-△t, t) can be extracted and blurred with package X-ray images in the time interval (t2-△t, t).

[0057] S5: Based on the fuzzy matching results, obtain the package image data with the best fit as the final package association image information, and bind the final package association image information with the package X-ray image.

[0058] Example 2

[0059] The main difference between the technical solution in this embodiment and Embodiment 1 lies in the extraction of package image data. To obtain accurate package image data, such as... Figure 3 and Figure 4 As shown, due to the actual size of the package, the camera image captured when the package first enters the conveyor belt does not completely capture the package. This results in the visible light image of the package extracted from the camera image not including the entire package, or the package not being directly below the camera in the captured image, leading to errors in the acquired package image data. To avoid this, a technique for tracking the package in the camera image is adopted, such as... Figure 6 As shown in Figure 9, the technical solution is as follows:

[0060] S1a: Obtain the package image data B at the initial moment;

[0061] Specifically, such as Figure 3 and Figure 4 As shown, the initial time is set to T0, and the package image data B at the initial time includes: B={id, img0, xL0, xR0, LB, T0}, where id is the package ID, img0 is the visible light image of the package extracted from the camera image at time T0, xL0 is the leftmost coordinate of the package in the camera image at time T0, xR0 is the rightmost coordinate of the package in the camera image at time T0, LB is the length of the package in the camera image at time T0, and LB= xR0- xL0;

[0062] S1b: Track the package from the camera images and update the package image data to obtain package image data B' for calculating time point T.

[0063] Specifically, such as Figure 5 and Figure 6 As shown, the package is tracked in real time from the camera image. At time T1, the package image data B' is obtained, where B'={id, img1, xL1, xR1, LB1, T1}, where id is the package ID, img1 is the visible light image of the package extracted from the camera image at time T1, xL1 is the leftmost coordinate of the package in the camera image at time T1, xR1 is the rightmost coordinate of the package in the camera image at time T1, LB is the length of the package in the camera image at time T0, and LB1= xR1-xL1;

[0064] When LB1>LB, the package image data is updated, and the package image data used to calculate time point T is package image data B'.

[0065] Repeat this step to obtain package image data at times T2, T3...TN. When it is found that the length of the package in the camera image no longer changes at subsequent times, take the package image data B' obtained at the initial time T1 for subsequent fuzzy matching.

[0066] The present invention also provides an electronic device for bag association based on full-segment fuzzy matching, comprising:

[0067] One or more processors;

[0068] A storage unit is used to store one or more programs. When one or more programs are executed by one or more processors, they enable one or more processors to implement the above-mentioned packet association method based on full-segment fuzzy matching.

[0069] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the above-described packet association method based on full-segment fuzzy matching.

[0070] The aforementioned computer-readable storage media can be implemented in any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0071] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including specified means, the means of instruction being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] In a typical configuration, a computing device includes one or more processors, input / output interfaces, network interfaces, and memory.

[0076] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0077] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0078] It should also be noted that the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0079] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are only used to help understand the core technical content of the present invention and are not intended to limit the scope of protection of the present invention. The technical solution of the present invention is not limited to the above specific embodiments. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A bag association method based on full-segment fuzzy matching, applied to a security inspection machine system, wherein the security inspection machine is equipped with an X-ray imaging area, and a camera is installed above the conveyor belt in front of the security inspection machine entrance, the camera's imaging direction is perpendicular to the conveyor belt and the camera's imaging range completely covers the conveyor belt, characterized in that, The method includes the following steps: S1: Continuously acquire the current camera image and extract the package image data, cache the package image data, and obtain the package image data queue M. M includes package image data for the time period (t-△t, t), where △t is the package image data caching time, △t is the time it takes for the package to run from one end of the conveyor belt to the other end, and t is the time point at which the current camera image is extracted. When executing S1, the following steps are included: S11: Obtain the RGB values ​​of the pixels that make up the current camera image; S12: Obtain the package image data based on the RGB values. The package image data includes the outer contour of the package, the visible light image of the package, the coordinates of the leftmost and rightmost parts of the package in the current camera image, and the length of the package in the current camera image. It also includes the following steps: S1a: Obtain the package image data B at the initial moment; S1b: Track the package from the camera images and update the package image data to obtain package image data B' for fuzzy matching; The initial time is set to T0, and the initial package image data B includes: B = {id, img0, xL0, xR0, LB, T0}, where id is the package ID, img0 is the visible light image of the package extracted from the camera image at time T0, xL0 is the leftmost coordinate of the package in the camera image at time T0, xR0 is the rightmost coordinate of the package in the camera image at time T0, LB is the length of the package in the camera image at time T0, and LB = xR0 - xL0. Track the package from the camera image and obtain the package image data B' at time T1, B'={id, img1, xL1, xR1,LB1, T1}, where id is the package ID, img1 is the visible light image of the package extracted from the camera image at time T1, xL1 is the leftmost coordinate of the package in the camera image at time T1, xR1 is the rightmost coordinate of the package in the camera image at time T1, LB1 is the length of the package in the camera image at time T1, LB1= xR1- xL1; When LB1>LB, the package image data is updated. At this time, the package image data used for fuzzy matching is package image data B', which is cached in the package image data queue. S2: Obtain a fully imaged X-ray image of the package from the security inspection machine, and extract the imaging time of the package X-ray image, denoted as time point t2; S3: Calculate the time interval between time point t2 and time point t to obtain the time interval (t2-△t, t2), where time point t is within the time interval (t2-△t, t2); S4: At time point t2, the package image data queue M is updated to the package image data queue M', M' includes package image data in the time period (t2-△t, t2), and the package image data in the time period (t2-△t, t2) is used to perform fuzzy matching with the package X-ray image; S5: Based on the fuzzy matching results, obtain the package image data with the best fit as the final package association image information, and bind the final package association image information with the package X-ray image.

2. The packet association method based on full-segment fuzzy matching according to claim 1, characterized in that, When executing S4, the following steps are also included: capturing the package image data of M' in the time period (t2-△t, t), and blurring the package image data in the time period (t2-△t, t) with the package X-ray image.

3. An electronic device for bag association based on full-segment fuzzy matching, characterized in that, include: One or more processors; A storage unit is used to store one or more programs, which, when executed by one or more processors, enable the one or more processors to implement a packet association method based on full-segment fuzzy matching according to any one of claims 1 to 2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it can implement a packet association method based on full-segment fuzzy matching as described in any one of claims 1 to 2.

Images