Method and system for object detection and tracking

Abstract

Disclosed is a method and system for target detection and tracking. The method comprises: extracting multiple frames of images from real-time video at intervals; performing grayscale processing on each frame of image to obtain multiple grayscale images; in the multiple grayscale images, performing target detection according to grayscale values and target tracking based on grayscale values and distances between targets to obtain position information of multiple targets; and extracting images of the multiple targets from the multiple frames of images based on the position information of the multiple targets. The method does not use a target recognition model but realizes target detection and tracking based on grayscale values of images and distance judgment between targets, so that target detection and tracking can be completed without occupying a large amount of computing resources.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision, and more specifically, to a method and system for object detection and tracking. Background Technology

[0002] Security signal acquisition equipment used to detect various dangerous goods is commonly seen in airports, subways, museums, embassies, customs, train stations, and ports. The security procedures in these locations involve passing items through security signal acquisition equipment (including X-ray machines and infrared detectors) to create images, then using a real-time video analysis system to analyze the images. Finally, the analysis results are provided to professional security personnel for further processing. The real-time video analysis system is deployed on intelligent security equipment and, based on technologies such as computer vision, image processing, and deep learning, extracts target image information from real-time video using methods such as target detection and localization. It then tracks the target and ultimately identifies dangerous goods from the video.

[0003] Existing real-time video analytics systems consume significant resources during operation. To improve efficiency, these systems often require deployment on high-performance machines, such as computers with graphics processing units (GPUs) that execute deep learning models for object detection and tracking. However, this hardware configuration results in high deployment and maintenance costs, making it unsuitable for certain scenarios (e.g., community security checks). Summary of the Invention

[0004] In view of this, the purpose of this disclosure is to provide a method and system for target detection and tracking to solve the problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, a method for target detection and tracking is provided, comprising:

[0006] Extract multiple frames of images from real-time video at intervals;

[0007] Each frame of the image is processed to grayscale, resulting in multiple grayscale images;

[0008] In the multiple grayscale images, target detection is performed based on grayscale values, and target tracking is performed based on grayscale values ​​and the distance between each target to obtain the position information of multiple targets;

[0009] Images of the multiple targets are extracted from the multiple frames of images based on the location information of the multiple targets.

[0010] Optionally, the method further includes: providing images of the plurality of targets to a target recognition model for recognition, so as to identify a predetermined target therein.

[0011] Optionally, the step of performing target detection based on grayscale values ​​in the plurality of grayscale images includes:

[0012] In each grayscale image, a continuous region with a grayscale value greater than or equal to 0 and less than 255 is considered as a target.

[0013] Optionally, the target tracking based on grayscale values ​​and the distance between each target includes:

[0014] Based on the positional relationship between two targets, determine whether the two targets could be the same target; and

[0015] For two targets that may be the same target, their grayscale values ​​are compared, and the two targets with the same grayscale value are considered as the same target.

[0016] Optionally, determining whether two targets might be the same target based on their positional relationship includes:

[0017] If the distance between two targets is equal to or approximately equal to a set value, then the two targets may be the same target. The set value is calculated based on the track speed of the security inspection signal acquisition device and the time interval between the frame images of the two targets.

[0018] Optionally, the method further includes: comparing the position of each target with the edge of the grayscale image in which the target is located to determine whether the target is completely contained in the grayscale image in which it is located;

[0019] The target detection and target tracking described herein include:

[0020] Target detection and tracking are performed only on targets that are fully contained in each grayscale image.

[0021] Optionally, the method further includes: comparing the position of each target with the edge of the grayscale image in which the target is located to determine whether the target is completely contained in the grayscale image and whether the length of the target fills the grayscale image.

[0022] The target detection and target tracking include:

[0023] If the target is not completely contained in the grayscale image but its length fills the entire grayscale image, then target detection and target tracking are performed on it.

[0024] Optionally, when the target is a hazardous object, the hazard level is determined and an alarm message is given.

[0025] Optionally, the method further includes: determining the target movement direction based on the positional relationship of the same target, wherein the target movement direction includes a forward direction that is the same as the movement direction of the machine's tracks and a reverse direction that is opposite; then, when performing target detection and target tracking, target detection and target tracking are only performed on targets in the frame image of forward movement.

[0026] Secondly, embodiments of this disclosure provide a target detection and tracking method applied to customs security checks, including:

[0027] Multiple frames of images are extracted intermittently from the real-time video of the customs security check signal acquisition equipment;

[0028] Each frame of the image is processed to grayscale, resulting in multiple grayscale images;

[0029] In the multiple grayscale images, target detection is performed based on grayscale values, and target tracking is performed based on grayscale values ​​and the distance between each target to obtain the position information of multiple targets;

[0030] Extract images of the multiple targets from the multiple frames of images based on the location information of the multiple targets; and

[0031] Images of the multiple targets are provided to a target recognition model for identification, in order to identify the designated target among them.

[0032] Thirdly, embodiments of this disclosure provide a target detection and tracking system, including a frame image extraction module for extracting multiple frames of images from real-time video at intervals;

[0033] The image preprocessing module is used to scale and grayscale each frame image to obtain multiple grayscale images.

[0034] The target detection and tracking module is used to detect targets in the multiple grayscale images based on grayscale values, and simultaneously track targets based on grayscale values ​​and the distance between each target to obtain the position information of multiple targets.

[0035] The target extraction module is used to extract images of the multiple targets from the multiple frames of images based on the location information of the multiple targets;

[0036] The target recognition module is used to provide images of the multiple targets to the target recognition model for recognition, so as to identify the designated target among them.

[0037] Optionally, the target detection and tracking module includes:

[0038] In each grayscale image, a continuous region with a grayscale value greater than or equal to 0 and less than 255 is considered as a target; and based on the positional relationship between two targets, it is determined whether the two targets may be the same target. For two targets that may be the same target, grayscale values ​​are compared, and two targets with the same grayscale value are considered as the same target.

[0039] Optionally, determining whether two targets might be the same target based on their positional relationship includes:

[0040] If the distance between two targets is equal to or approximately equal to a set value, then the two targets may be the same target. The set value is calculated based on the track speed of the security inspection signal acquisition device and the time interval between the frame images of the two targets.

[0041] Optionally, the system further includes: a target integrity judgment module, used to compare the position of each target with the edge of the grayscale image in which the target is located, to determine whether the target is completely contained in the grayscale image in which it is located; then the target detection and tracking module only performs target detection and target tracking on targets that are completely contained in each grayscale image.

[0042] Optionally, the system further includes: a special target judgment module, used to compare the grayscale position of each target with the edge of the grayscale image where the target is located, to determine whether the target is completely contained in the grayscale image where it is located, and to determine whether the length of the target fills the grayscale image where it is located; then the target detection and tracking module includes: if the target is not completely contained in each grayscale image but the length of the target fills the grayscale image where it is located, then it is detected and tracked.

[0043] Optionally, the system further includes a marking module, used to determine the hazard level and provide alarm information when the set target is a set hazardous object.

[0044] Fourthly, embodiments of this disclosure provide an intelligent security inspection device, including a memory and a processor. The memory further stores computer instructions executable by the processor, which, when executed, implement the method described in any of the above-mentioned embodiments.

[0045] Fifthly, embodiments of this disclosure provide a computer-readable medium storing computer instructions executable by an electronic device, wherein when executed, the computer instructions implement the method described in any of the preceding claims.

[0046] Sixthly, embodiments of this disclosure provide an intelligent security inspection system, including:

[0047] Security inspection signal acquisition equipment is used to acquire security inspection video signals;

[0048] The frequency division device is used to divide the security inspection video signal into a first video signal and a second video signal.

[0049] A first display is configured to display the first video signal as video;

[0050] A capture card is used to convert the second video signal into a digital signal if the second video signal is an analog signal;

[0051] Intelligent security inspection equipment, equipped with any of the systems described above, to identify a set target from a real-time second video signal;

[0052] The second display is used to display the second video signal as video, and mark the set target or give an alarm in the video according to the recognition result.

[0053] The embodiments disclosed herein have the following advantages: First, target detection and tracking are performed using a combination of simple strategies, which can complete target detection and tracking without consuming a large amount of computing resources; second, instead of using all frame images in the real-time video, a set number of frame images are extracted from the real-time video and grayscale processed before target detection and tracking, thereby saving computing resources. Attached Figure Description

[0054] The above and other objects, features, and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0055] Figure 1 This is a schematic diagram of the X-ray detection scheme deployment;

[0056] Figure 2 This is an example image based on targets detected in a single frame of an image;

[0057] Figure 3 This is a flowchart illustrating the X-ray detection scheme;

[0058] Figure 4 This is a schematic diagram of the structure of a target detection and tracking system according to an embodiment of the present disclosure;

[0059] Figure 5 This is a structural diagram of an intelligent security inspection device used to deploy and implement the target detection and tracking system provided in the embodiments of this disclosure. Detailed Implementation

[0060] The present disclosure is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present disclosure below, certain specific details are described in detail. Those skilled in the art will fully understand the present disclosure even without these details. To avoid obscuring the substance of the present disclosure, well-known methods, processes, and procedures are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0061] The following terms are used in this document.

[0062] Dangerous goods, in this context, refer to those identified by X-ray detection systems, including different categories of items or specific materials agreed upon in different scenarios. For example, in airport passenger security checks, this includes lighters (all types), controlled knives, firearms, ammunition, and other civil aviation-controlled items, as well as common items that may interfere with security personnel's observation, such as laptops, cameras, power banks, lithium batteries, and umbrellas. In logistics scenarios, this includes various batteries, magnets, liquids, and other goods.

[0063] Figure 1 This is a deployment diagram of an X-ray detection scheme. As shown in the diagram, scheme 100 involves an X-ray machine 101, a frequency divider 102, a data acquisition card 103, an intelligent security inspection device 104, a network 105, a server 106, a display 107, and a display 108.

[0064] X-ray machine 101 acquires X-ray equipment signals. Frequency divider 102 splits the X-ray equipment signal in two, converting one input video signal into two identical output video signals. One signal is provided to display 107, and the other to acquisition card 103. Acquisition card 103 receives the video signal, which can be analog or digital. After converting the analog video signal into a digital signal, it provides it to intelligent security inspection equipment 104 via network 105.

[0065] The intelligent security inspection equipment 104 is equipped with an X-ray video analysis system. This system detects and tracks targets in the X-ray video, providing the detected target image information (usually packages, bags, and other items in the security inspection scenario) to an AI algorithm for target recognition. The algorithm identifies the items in the X-ray video, analyzes whether any dangerous items are present, and transmits the target image and algorithm recognition results to the security inspection management system.

[0066] Network 105 is one or a combination of various communication technologies based on switched signals, including but not limited to wired technologies using electrical and / or optical cables, and wireless technologies using infrared, radio frequency, and / or other forms. In different application scenarios, Network 105 can be the Internet, a wide area network (WAN), or a local area network (LAN), such as a company's private network. Network 105 can also be a wired network or a wireless network.

[0067] The security inspection management system is deployed on server 106 and is used to collect, store, and manage target information identified by the X-ray video analysis system and information on dangerous goods extracted from it.

[0068] Display 107 receives video signals via a video interface and displays the received video signals on a screen. Display 107 includes at least one video interface that is compatible with the video interface of the frequency divider 102. The video interface types include analog signal interfaces and digital signal interfaces. Analog signal interfaces include VGA (also known as D-Sub) interfaces and DVI-A (DVI-Analog) interfaces. Digital signal interfaces include DVI-D (DVI-Digital) interfaces, DVI-I (DVI-Integrated) interfaces, and HDMI interfaces. The X-ray equipment signal is one of VGA, DVI-A, DVI-D, DVI-I, and HDMI. The security inspection management system can transmit target information and extracted hazardous materials from the X-ray video to display 107. When displaying the X-ray video output by X-ray machine 101, display 107 marks the target information and extracted hazardous materials. Display 108 has similar functions to display 107, except that display 108 receives digital video signals. Of course, if the X-ray machine outputs a digital video signal, the acquisition card 103 in the system can be omitted, while the displays 107 and 108 are digital video signal display devices.

[0069] Figure 2 This is an example image based on targets detected in a single frame of an image. As shown in the figure above, image 200 is... Figure 1 The image shown is a frame of X-ray video output by X-ray machine 100. 201, 202, and 203 are different targets marked on image 200 based on target information provided by the X-ray video analysis system. 204 and 205 are bounding boxes in image 200, which are typically added by the X-ray machine's imaging system. Existing X-ray video analysis systems typically use a deep learning model for target detection and tracking, and then use another deep learning model to identify hazardous materials. To ensure execution efficiency, the deep learning models are deployed on an image processor, a deployment scheme that places high demands on hardware configuration.

[0070] Figure 3 This is a flowchart illustrating an X-ray detection scheme. The following is based on... Figure 3 The present disclosure provides a detailed description of a target detection and tracking method according to an embodiment. (See reference...) Figure 3As shown, the scheme includes four stages: video acquisition, target detection, target tracking, and AI algorithm recognition. The target detection and tracking method of this disclosure includes three stages: target detection, target tracking, and AI algorithm recognition. The video acquisition stage includes steps S301 and S302. Step S301 outputs analog or digital video signals through an X-ray machine. Step S302 converts the analog video signal into a digital video signal.

[0071] The target detection stage includes steps S303, S304, S305, and S306. Step S303 is used to receive a digital video signal and extract frame images from the digital video signal at intervals as target images. For example, in a digital video signal per second, one frame image is extracted as the target image every 3-12 frames. The target image is simultaneously provided to steps S304 and S310. In this way, the number of images that need to be processed in the target detection, target tracking, and AI algorithm recognition stages will be greatly reduced, thereby saving computing resources. For example, if the digital video signal per second contains 60 frame images, and one frame image is extracted every 12 frames for image processing, then compared to processing 60 frame images per second, the amount of data computation and storage is reduced by 12 times.

[0072] Step S304 involves preprocessing the target image. One purpose of preprocessing is to further reduce the computational and storage requirements for processing the target image. For example, the target image can be scaled proportionally by 50%. Alternatively, the target image can be converted to a grayscale image. For instance, in an RGB image, each pixel is assigned three color components, while in a grayscale image, each pixel has only one grayscale level. Therefore, converting an RGB image to a grayscale image saves at least one-third of the computational and storage requirements. Another purpose of preprocessing is to facilitate the processing of the target image. For example, the target image can be converted from one color space to another. For instance, in video and image processing pipelines, YUV is used for easier encoding and transmission, reducing bandwidth consumption and information errors. Therefore, in some cases, it is necessary to convert the target image format from other formats to YUV.

[0073] Steps S305 to S309 continuously perform target detection on the target image. If a target is detected, it is determined whether the target is a new target. If not, the target information of the target in the target tracking list is updated. If it is a new target, the target information of the new target is added to the target tracking list.

[0074] Object detection and tracking are fundamental tasks in computer vision. Currently, object detection in some complex scenarios (such as face recognition in the field of surveillance) uses deep learning models such as convolutional neural networks (CNN) for object detection.

[0075] However, this embodiment does not use any deep learning model but instead uses a simple combination of strategies for target detection and tracking. First, targets in the target image are selected based on the grayscale values ​​of all pixels in the image. Specifically, continuous regions with grayscale values ​​greater than (or equal to) 0 and less than 255 are selected as a single target. 0 represents black, and 255 represents white. There is usually a blank area (i.e., 0) between two consecutive regions, thus this method can distinguish between two adjacent targets. Of course, this method may mistake two overlapping targets for one; in such cases, the subsequent target recognition model will be used to differentiate them. Second, the relationship between each target and the edges of the target image is examined to determine whether the target has completely passed through the X-ray machine. We know that when using an X-ray machine, the object to be inspected is placed on the machine's conveyor belt and passes through the machine at a constant speed. During this time, the X-ray machine's imaging system generates an X-ray video (the target image is a frame image of the X-ray video). When a target does not completely enter the X-ray machine, its edge will overlap with the edge of the target image. Therefore, conversely, we can determine whether a target has completely entered the X-ray machine by comparing the position of the target's edge with the edge of the target image, and only select targets that have completely entered the X-ray machine. Third, if multiple targets overlap, or if a very large target enters the X-ray machine, its length will fill the entire target image. If the two sides of the target overlap with the two sides of the target image (judged by the target's length and positional relationship), it means that the size of this target is equal to or exceeds the length of the target image, and it is extracted as a target. Alternatively, if the length exceeds the product of the target image's length and a set ratio, the target is also extracted from the image. Fourth, for step S307, determining whether a target is a new target, its grayscale value is first compared with the grayscale values ​​of existing targets to determine if it is a new target.

[0076] The above strategy ensures that even if the security inspector drags the target in the video, the target in the grayscale image can still be detected.

[0077] Step S310 involves extracting each target from the target image based on the target tracking list, that is, cutting out each target from the target image and then sending it to the AI ​​algorithm for recognition.

[0078] AI algorithm recognition includes steps S311 and S312. Step S311 involves using a target recognition algorithm to identify the target type; this algorithm is typically a deep learning model. Step S312 involves recording the recognition results into the management system.

[0079] The embodiments disclosed herein have the following advantages: First, target detection and tracking are performed using a combination of simple strategies, which can be completed without consuming a large amount of computing resources; second, instead of using all frame images in the X-ray video, a set number of frame images are extracted from the X-ray video and grayscale processed before being used for target detection and tracking, thereby saving computing resources.

[0080] Furthermore, based on the above advantages, target detection and tracking only require intelligent security inspection equipment with a central processing unit (CPU) to perform the calculations to meet the performance requirements, while using deep learning models usually requires intelligent security inspection equipment with a graphics processing unit (GPU) to meet the performance requirements. Therefore, from a hardware deployment perspective, this embodiment has a cost advantage.

[0081] In some embodiments, determining whether two targets in two adjacent frame images might be the same target can be achieved by judging the positional relationship between the two targets. Specifically, since the X-ray machine's conveyor belt moves at a constant speed, the distance between the same target in adjacent frame images can be calculated. When the distance between a target and a target in an adjacent frame image is outside a certain set range, it can be determined that the two targets are unlikely to be the same target. For example, if an X-ray video has 60 frame images per second, the time interval between the first and third frame images is 2 / 60 seconds. Multiplying this time interval by the X-ray machine's conveyor belt speed gives the distance between the same target in the first and third frame images, denoted as D. If the distance between two targets is equal to or approximately equal to D, it indicates that the two targets might be the same target. This step can be performed before or after step S307.

[0082] The strategy described above, which determines whether two targets might be the same based on their positional relationship, ensures that target detection and tracking will not lose the target even after security personnel perform various operations on the X-ray video. These operations include: using the X-ray machine to zoom in or out on the target in the video; and performing image enhancement and penetration transformation.

[0083] In some embodiments, before performing target detection on the frame image, the target movement direction is determined based on the positional relationship of the same target. The target movement direction is determined by setting a forward direction that is the same as the movement direction of the X-ray machine's conveyor belt, and a reverse direction that is opposite to the movement direction of the X-ray machine's conveyor belt. Before performing target detection on the frame image, if the first and second targets representing the same target are not moving in a forward direction, it indicates that the security personnel have performed a backward movement operation on the X-ray video.

[0084] Figure 4This is a schematic diagram of the structure of a target detection and tracking system according to an embodiment of the present disclosure. As shown in the figure, the system 400 includes a frame image extraction module 401, an image preprocessing module 402, a target detection and tracking module 403, a target extraction module 404, and a target recognition module 405.

[0085] The frame image extraction module 401 receives X-ray video 410 as input data and is used to extract and output frame images 411 from the X-ray video at intervals. This interval extraction can be either extracting a frame image at fixed intervals or extracting a frame image from the X-ray video at random intervals.

[0086] The input data for the image preprocessing module 402 is the frame image 411, which is used to scale and perform grayscale processing on each frame image. The scaling can be proportional or non-proportional. Before or after scaling, the frame image is processed to obtain multiple grayscale images 412. The grayscale images 412 serve as input data for the object detection and tracking module 403.

[0087] The target detection and tracking module 403 is used to detect targets in multiple grayscale images based on grayscale values, and simultaneously track targets based on grayscale values ​​and the distances between each target, recording the position information of multiple targets. The grayscale image values ​​range from [0, 255], and a continuous region refers to a region with a grayscale value greater than (or equal to) 0 and less than 255. By polling and judging the grayscale value of each pixel in the grayscale image, multiple continuous regions can be obtained, each continuous region being a target, thereby obtaining information about multiple targets (including position, size, etc.), and recording the information of multiple targets as 413.

[0088] The target extraction module 404 uses information 413 about multiple targets to extract images of the corresponding multiple targets from each frame image. In other words, it extracts multiple targets from each frame image and provides the images 414 of the multiple targets to the target recognition module 405.

[0089] The target recognition module 405 is used to provide images of multiple targets to the target recognition model for recognition, so as to identify whether the multiple targets are dangerous objects.

[0090] The target detection and tracking system provided in this embodiment reduces the number of frame images that need to be processed by the frame image extraction module and further reduces the amount of data of the frame images that need to be processed by the image preprocessing module. As a result, the amount of data of the frame images transmitted to the target extraction module 404 is greatly reduced. The target detection and tracking module 403 uses a simple strategy combination instead of a deep learning model to realize target detection and tracking, which also greatly reduces the amount of computation in the target detection and tracking steps, thereby saving computing and storage resources.

[0091] Furthermore, compared with existing technologies, the target detection and tracking system provided in this embodiment can be deployed on machines with lower hardware configurations, thereby helping to reduce deployment and maintenance costs.

[0092] In some embodiments, the specific steps of the target detection and tracking module 403 include: in each grayscale image, taking a continuous region with a grayscale value greater than or equal to 0 and less than 255 as a target; for each target in each grayscale image, comparing its grayscale value with each target in a previous, adjacent grayscale image, and taking two targets with the same grayscale value as the same target.

[0093] In some embodiments, the target detection and tracking module 403 further includes: determining whether the two targets might be the same target based on their positional relationship. Determining whether the two targets might be the same target based on their positional relationship includes: if the distance between the two targets is equal to or approximately equal to a preset value, then the two targets might be the same target, where the preset value is calculated based on the X-ray machine's track speed and the time interval between the frames containing the two targets.

[0094] In some embodiments, the system 400 further includes a target integrity determination module, used to compare the position of each target with the edge of the grayscale image in which the target is located, to determine whether the target is completely contained in the grayscale image. The target detection and tracking module 403 should only detect and track targets that are completely contained in each grayscale image.

[0095] In some embodiments, the system 400 further includes a special target determination module, used to compare the position of each target with the edge of the frame image in which the target is located to determine whether the target is completely contained in the grayscale image and whether the length of the target fills the grayscale image. The target detection and tracking module 403 also detects and tracks targets that are not completely contained in each grayscale image but whose length fills the grayscale image, or whose length exceeds a set proportion of the length of the grayscale image.

[0096] In some embodiments, the system 400 further includes a marking module, used to determine the hazard level and provide alarm information when the set target is a set hazard.

[0097] In some embodiments, the image preprocessing module 402 further includes: removing borders added by the imaging system of the X-ray machine in each frame image before performing grayscale processing on each frame image.

[0098] Since the target detection and tracking system provided in this embodiment corresponds to the method described above, the system provided in this embodiment will be described in a relatively simple manner, and can be understood by referring to the method described above.

[0099] It should be understood that although the above embodiments are mostly described in terms of X-ray machines or X-ray videos, they are not limited to this. In fact, X-ray machines can be other security signal acquisition devices, such as infrared detectors. The security signals obtained by these devices can also generate videos, and target detection and tracking can be performed on these videos based on the above embodiments. Furthermore, although we emphasize that the above embodiments are more suitable for lightweight locations such as communities and hospitals, they can also be used in high-traffic areas such as high-speed rail stations, airports, and customs. When deployed in high-traffic areas, staff can combine the system's identification results with manual judgment to obtain the final result, thus balancing the efficiency and accuracy of target detection and tracking.

[0100] This disclosure also provides an intelligent security inspection device, which can be considered as a lightweight computer. In this intelligent security inspection device, reference... Figure 5 As shown, at the hardware level, it includes memory 502 and processor 501. In addition, in some cases, it also includes input / output devices 503 and other hardware 504. Memory 502 is, for example, high-speed random-access memory (RAM), or it may be non-volatile memory, such as at least one disk storage device. Input / output devices 503 include, for example, a monitor, keyboard, mouse, network controller, etc. Processor 501 can be built based on various processor models currently on the market. Processor 501, memory 502, input / output devices 503, and other hardware 504 are interconnected via a bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0101] Memory 502 is used to store programs. Specifically, the program may include program code, which includes computer instructions. The memory may include main memory and non-volatile memory, and provides computer instructions and data to processor 501. Processor 501 reads the corresponding computer program from memory 502 into main memory and then runs it, implementing the various methods provided in the embodiments of this disclosure at the logical level. These methods have been described above and will not be repeated here.

[0102] Those skilled in the art will understand that this disclosure can be implemented as a system, method, and computer program product. Therefore, this disclosure can be implemented as entirely hardware, entirely software (including firmware, resident software, and microcode), or a combination of software and hardware. Furthermore, in some embodiments, this disclosure can also be implemented as a computer program product contained in one or more computer-readable media, the computer-readable media containing computer-readable program code.

[0103] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium is, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium include: an electrical connection of one or more wires, a portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with a processing unit, apparatus, or device.

[0104] Computer-readable signal media may include data signals propagated in baseband or as part of a chopped signal, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any other suitable combination. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction system, apparatus, or device.

[0105] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, and any suitable combination thereof.

[0106] Computer program code for executing embodiments of this disclosure can be written in one or more programming languages ​​or combinations thereof. The programming languages ​​include object-oriented programming languages ​​such as JAVA and C++, and may also include conventional procedural programming languages ​​such as C. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0107] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for object detection and tracking, comprising: extracting multiple frames of images from real-time video at intervals; performing gray processing on each frame of image to obtain multiple gray images; comparing the position of each object with the edge of the gray image in which the object is located to determine whether the object is completely contained in the gray image, wherein, in the case that the edge of the object overlaps with the edge of the gray image in which the object is located, it is determined that the object is not completely contained in the gray image, and in the case that the edge of the object does not overlap with the edge of the gray image in which the object is located, it is determined that the object is completely contained in the gray image; performing object detection and object tracking only on the objects completely contained in each gray image; performing object detection according to gray values and object tracking based on the distance between objects in the multiple gray images to obtain the position information of multiple objects; extracting the images of the multiple objects from the multiple frames of images based on the position information of the multiple objects; wherein the object tracking based on the distance between objects comprises judging whether two objects are likely to be the same object based on the positional relationship between the two objects.

2. The method of claim 1, further comprising: providing the images of the multiple objects to an object recognition model for recognition to identify a set object among them.

3. The method of claim 1, wherein, the object detection according to gray values in the multiple gray images comprises: in each gray image, regarding a continuous region with a gray value greater than or equal to 0 and less than 255 as an object. 4.The method of claim 3, wherein, for two objects that are likely to be the same object, performing gray value comparison, and regarding two objects with the same gray value as the same object.

5. The method of claim 4, wherein, the judgment of whether two objects are likely to be the same object based on the positional relationship between the two objects comprises: if the distance between the two objects is equal to or approximately equal to a set value, the two objects are likely to be the same object, and the set value is calculated according to the moving speed of the track of the security signal acquisition device and the time interval between the frame images in which the two objects are located.

6. The method of claim 1, further comprising: comparing the position of each object with the edge of the gray image in which the object is located to determine whether the object is completely contained in the gray image, and judging whether the length of the object occupies the gray image in which the object is located; then the object detection and object tracking comprises: if the object is not completely contained in the gray image in which the object is located but the length of the object occupies the gray image, performing object detection and object tracking on the object.

7. The method of claim 2, wherein, when the set object is a set dangerous object, determining the danger level and giving an alarm information.

8. The method of claim 1, further comprising: based on the positional relationship of the same object, determining the moving direction of the object, wherein the moving direction of the object comprises a forward direction and an opposite direction, and performing object detection and object tracking only on the objects in the frame images moving in the forward direction. 9.A method for object detection and tracking, comprising: extracting multiple frames of images from real-time video of a security signal acquisition device of a customs at intervals; performing gray processing on each frame of image to obtain multiple gray images; The position of each target is compared with the edge of the gray image in which the target is located to determine whether the target is completely contained in the gray image, wherein, in the case that the edge of the target overlaps with the edge of the gray image in which the target is located, it is determined that the target is not completely contained in the gray image in which the target is located, and in the case that the edge of the target does not overlap with the edge of the gray image in which the target is located, it is determined that the target is completely contained in the gray image in which the target is located; target detection and target tracking are performed only on the targets completely contained in each gray image; In the plurality of gray images, target detection is performed according to gray values, and target tracking is performed based on the distances between the targets to obtain position information of the plurality of targets; Images of the plurality of targets are extracted from the plurality of frames of images based on the position information of the plurality of targets; and The images of the plurality of targets are provided to a target recognition model for recognition to identify a set target therein. The target tracking based on the distances between the targets includes judging whether two targets are likely to be the same target based on the position relationship between the two targets.

10. A system for target detection and tracking, comprising a frame image extraction module configured to extract a plurality of frames of images from a real-time video at intervals; an image preprocessing module configured to perform scaling and gray processing on each frame of image to obtain a plurality of gray images; a target completeness judgment module configured to compare the position of each target with the edge of the gray image in which the target is located to determine whether the target is completely contained in the gray image, wherein, in the case that the edge of the target overlaps with the edge of the gray image in which the target is located, it is determined that the target is not completely contained in the gray image in which the target is located, and in the case that the edge of the target does not overlap with the edge of the gray image in which the target is located, it is determined that the target is completely contained in the gray image in which the target is located; target detection and target tracking are performed only on the targets completely contained in each gray image; a target detection and tracking module configured to, in the plurality of gray images, perform target detection according to gray values, and perform target tracking based on the distances between the targets to obtain position information of the plurality of targets; a target extraction module configured to extract images of the plurality of targets from the plurality of frames of images based on the position information of the plurality of targets; a target recognition module configured to provide the images of the plurality of targets to a target recognition model for recognition to identify a set target therein; The target detection and tracking module includes judging whether two targets are likely to be the same target based on the position relationship between the two targets.

11. The system of claim 10, wherein, The target detection and tracking module includes: In each gray image, a continuous region with a gray value greater than or equal to 0 and less than 255 is regarded as a target; and for two targets that are likely to be the same target, gray value comparison is performed, and two targets with the same gray value are regarded as the same target.

12. The system of claim 11, wherein, The judgment of whether two targets are likely to be the same target based on the position relationship between the two targets includes: If the distance between two targets is equal to or approximately equal to a set value, the two targets can be the same target, and the set value is calculated according to the track moving speed of the machine and the time interval between the frame images where the two targets are located.

13. The system of claim 11, further comprising: The special target judging module is configured to compare the position of the gray value of each target with the edge of the gray image where the target is located, to determine whether the target is completely contained in the gray image where the target is located, and to determine whether the length of the target occupies the entire gray image; and the target detection and tracking module is configured to detect and track the target if the target is not completely contained in each gray image but the length of the target occupies the entire gray image.

14. The system of claim 11, further comprising: The marking module is configured to determine the danger level and give an alarm when the set target is a set dangerous object.

15. An intelligent security inspection device, comprising a memory and a processor, wherein the memory further stores computer instructions executable by the processor, and the computer instructions are executed to implement the method according to any one of claims 1 to 9.

16. A computer readable medium, wherein the computer readable medium stores computer instructions executable by an electronic device, and the computer instructions are executed to implement the method according to any one of claims 1 to 9.

17. An intelligent security inspection system, comprising: a security inspection signal acquisition device configured to acquire a security inspection video signal; a frequency dividing device configured to divide the security inspection video signal into a first video signal and a second video signal; a first display configured to display the first video signal as a video; an acquisition card configured to convert the second video signal into a digital signal if the second video signal is an analog signal; an intelligent security inspection device deployed with the system according to any one of claims 10 to 14 to identify a set target from the real-time second video signal; a second display configured to display the second video signal as a video and mark the set target or give an alarm according to the identification result.

Images