A new package detection method, system, device and storage medium of a video

Abstract

The application discloses a new parcel detection method, system and device of a video and a storage medium. The new parcel detection method of the video comprises the following steps: calculating a second parcel gap detection range value of a second to-be-detected image based on a parcel moving direction and a first parcel gap detection range value; intercepting a first parcel gap image of the first to-be-detected image based on the first parcel gap detection range value; intercepting a second parcel gap image of the second to-be-detected image based on the second parcel gap detection range value; calculating a first image standard deviation and a second image standard deviation; determining all first parcel images in the second to-be-detected image and the coordinates of each first parcel image; screening all the first parcel images through a preset screening condition to obtain a second parcel image, and taking the second parcel image as a new parcel image of a to-be-detected video, thereby improving the efficiency and accuracy of parcel detection.

Description

Technical Field

[0001] This application relates to the technical field of new package detection in video, and in particular to a method, system, device and storage medium for new package detection in video. Background Technology

[0002] With the rapid development of my country's civil aviation and rail transit industries, more and more people are choosing to travel by plane, high-speed rail, and subway. When passengers enter stations, their carry-on bags must undergo security checks to ensure safety. However, due to my country's large population and the high concentration of passengers during rush hours, security personnel need to check a large number of passengers' bags in a short period. Currently, the mainstream solution to this problem is to segment the packages in the video stream of security scanners and then use AI to identify prohibited items from the segmented images. Therefore, before AI image analysis is implemented, improving the efficiency and accuracy of package segmentation in security scanner videos has become a very urgent issue.

[0003] In existing technologies, package segmentation is mainly performed using the mean shift algorithm. However, this method involves a large amount of computation, has a slow segmentation efficiency, and a low accuracy, which reduces the accuracy and efficiency of package detection. Summary of the Invention

[0004] This application aims to at least address the technical problems existing in the prior art. To this end, this application proposes a novel video-based package detection method, system, device, and storage medium, which can improve the efficiency and accuracy of package detection.

[0005] A first aspect of this application provides a novel method for package detection in video, comprising the following steps:

[0006] The following steps are taken: obtain the package movement direction of the video to be detected, the first image to be detected, the second image to be detected, and the first package gap detection range value corresponding to the first image to be detected. The first image to be detected is the video frame image of the previous moment of the video to be detected, the video to be detected is the video of the package passing through the security inspection machine, and the second image to be detected is the video frame image of the current moment of the video to be detected.

[0007] Based on the package movement direction and the first package gap detection range value, calculate the second package gap detection range value of the second image to be detected;

[0008] A first package gap image is extracted from the first image to be detected based on the first package gap detection range value; a second package gap image is extracted from the second image to be detected based on the second package gap detection range value.

[0009] Calculate the first image standard deviation of the first package gap image; calculate the second image standard deviation of the second package gap image;

[0010] If the standard deviation of the first image is less than a preset standard deviation threshold and the standard deviation of the second image is greater than or equal to the preset standard deviation threshold, determine all first package images in the second image to be detected and the coordinates of each first package image;

[0011] Based on the coordinates of each first package image and the direction of package movement, all first package images are filtered using preset filtering conditions to obtain second package images, and the second package images are used as new package images in the video to be detected.

[0012] The novel video package detection method according to the embodiments of this application has at least the following beneficial effects:

[0013] This method obtains the package movement direction, a first image to be detected, a second image to be detected, and the first package gap detection range value corresponding to the first image to be detected from the video to be detected. The first image to be detected is the video frame image of the previous moment in the video to be detected, the video to be detected is the video of the package passing through the security scanner, and the second image to be detected is the video frame image of the video to be detected at the current moment. Based on the package movement direction and the first package gap detection range value, the second package gap detection range value of the second image to be detected is calculated. A first package gap image is extracted from the first image to be detected based on the first package gap detection range value. A second package gap image is extracted from the second image to be detected based on the second package gap detection range value. A first image standard deviation of the first package gap image is calculated. A second image standard deviation of the second package gap image is calculated. If the first image standard deviation is less than a preset image standard deviation threshold and the second image standard deviation is greater than or equal to the preset image standard deviation threshold, all first package images in the second image to be detected and the coordinates of each first package image are determined. Based on each first package... The coordinates of the image and the direction of package movement are used to filter all first package images according to preset filtering conditions to obtain second package images. The second package image is then used as a new package image in the video to be detected. This application first detects all package images in the second image to be detected, and then filters all package images according to the direction of package movement and the coordinates of the first package to obtain new package images that exist in the second image to be detected but were not detected in the first image to be detected. By segmenting the new package in the second image to be detected, and then only detecting the new package image, the repeated detection of the already detected package images in the first image to be detected at the current moment is avoided, thereby improving the efficiency and accuracy of package detection.

[0014] According to some embodiments of this application, the first package gap detection range value includes a first package gap detection ordinate range value, and the step of calculating the second package gap detection range value of the second image to be detected based on the package movement direction and the first package gap detection range value includes:

[0015] Obtain the image width of the second image to be detected;

[0016] The second package gap detection abscissa range value is calculated based on the package movement direction, the image width, and the preset gap width value;

[0017] The range of the first package gap detection vertical coordinate and the range of the second package gap detection horizontal coordinate are used as the second package gap detection range value.

[0018] According to some embodiments of this application, the step of calculating the second package gap detection abscissa range value based on the package movement direction, the image width, and the preset gap width value includes:

[0019] When the direction of package movement is a first preset direction, the range of horizontal coordinates from zero to the preset gap width value is taken as the range of horizontal coordinates for the second package gap detection.

[0020] When the package movement direction is a second preset direction, the width difference between the image width and the preset gap width value is calculated, and the horizontal coordinate range corresponding to the width difference and the image width is used as the horizontal coordinate range value of the second package gap detection.

[0021] According to some embodiments of this application, the step of filtering all first package images based on the coordinates of each first package image and the direction of package movement using preset filtering conditions to obtain a second package image, and using the second package image as a new package image for the video to be detected, includes:

[0022] Based on the direction of package movement and the coordinates of the first package image, the first package image is sorted to obtain sorted package images;

[0023] Calculate the first horizontal axis projection range value of the first package image and the second horizontal axis projection range value of the second package image in the sorted package images;

[0024] Calculate the intersection of the first horizontal axis projection range value and the second horizontal axis projection range value, and use it as the first horizontal axis projection intersection;

[0025] If the intersection of the first horizontal axis projections is an empty set, the traversal ends, and the first packaged image in the sorted packaged images is taken as the second packaged image.

[0026] If the first horizontal axis projection intersection is not empty, calculate the union of the first horizontal axis projection range values ​​and the second horizontal axis projection range values, and use this union as the second horizontal axis projection union; calculate the third horizontal axis projection range value of the third wrapped image in the sorted wrapped images on the horizontal axis; calculate the intersection of the second horizontal axis projection union and the third horizontal axis projection range values, and use this intersection as the third horizontal axis projection intersection; if the third horizontal axis projection intersection is not empty, calculate the union of the second horizontal axis projection union and the third horizontal axis projection range values, and use this union as the third horizontal axis projection union; calculate the fourth horizontal axis projection range value of the fourth wrapped image in the sorted wrapped images on the horizontal axis; calculate the intersection of the third horizontal axis projection intersection and the fourth horizontal axis projection range values, and use this intersection as the fourth horizontal axis projection intersection;

[0027] This process continues until the intersection of the i-th horizontal axis projections is empty or all the first packages have been traversed. If the intersection of the i-th horizontal axis projections is empty, the traversal ends, and the first package image to the i-th minus 1 package image in the sorted package images are taken as the second package image. If all the first package images have been traversed, all the first package images are taken as the second package image, where i is the iteration number.

[0028] According to some embodiments of this application, the method further includes:

[0029] When the intersection of the first horizontal axis projections is an empty set, the first vertical axis projection range value of the first packaged image in the sorted packaged images is calculated on the vertical axis, and the first vertical axis projection range value is used as the package gap detection vertical axis range value of the next moment of the video to be detected.

[0030] When the first horizontal axis projection intersection is not an empty set and the i-th horizontal axis projection intersection is an empty set, calculate the union of the vertical axis projection range values ​​from the first packaged image to the i-th minus 1 packaged image in the sorted packaged images, and use the union of the vertical axis projection range values ​​from the first packaged image to the i-th minus 1 packaged image as the package gap detection vertical coordinate range value of the next moment of the video to be detected.

[0031] If the intersection of the first horizontal axis projections is not empty and all the first package images have been traversed, calculate the union of the vertical axis projection range values ​​of all the sorted package images, and use the union of the vertical axis projection range values ​​of all the sorted package images as the package gap detection vertical coordinate range value of the next moment of the video to be detected.

[0032] According to some embodiments of this application, determining all first package images and the coordinates of each first package image in the second image to be detected includes:

[0033] Gaussian blur is applied to the second image to be detected to obtain the Gaussian blurred image;

[0034] The Gaussian blurred image is detected by an edge detection algorithm to obtain an edge-detected image;

[0035] The edge-detected image is dilated to obtain the dilated image;

[0036] The contour extraction algorithm is used to determine all initial wrapped images in the dilated image and the set of point coordinates for each initial wrapped image.

[0037] The minimum bounding rectangle of each initial package image is calculated based on the set of point coordinates of each initial package image;

[0038] Calculate the diagonal length of each of the minimum bounding rectangles;

[0039] Traverse each of the initial package images, and filter all the initial package images whose diagonal length is less than a preset diagonal length threshold to obtain all the first package images;

[0040] Obtain the coordinates of all images of the first package.

[0041] According to some embodiments of this application, before acquiring the second image to be detected from the video to be detected, the method further includes:

[0042] Receive the initial video frame image and the first sequence number of the initial video frame image sent by the security inspection machine;

[0043] Obtain the second sequence number of the first image to be detected;

[0044] The acquisition of the second image to be detected from the video to be detected includes:

[0045] If the initial video frame image meets the preset image standard and the difference between the first sequence number and the second sequence number is greater than the preset sequence number interval, the initial video frame image is used as the second image to be detected.

[0046] A second aspect of this application provides a new video package detection system, the video package detection system comprising:

[0047] The data acquisition module is used to acquire the package movement direction, the first image to be detected, the second image to be detected, and the first package gap detection range value corresponding to the first image to be detected in the video to be detected. The first image to be detected is the video frame image of the previous moment of the video to be detected, the video to be detected is the video of the package passing through the security inspection machine, and the second image to be detected is the video frame image of the current moment of the video to be detected.

[0048] The package gap detection range value calculation module is used to calculate the second package gap detection range value of the second image to be detected based on the package movement direction and the first package gap detection range value;

[0049] The package gap image cropping module is used to crop a first package gap image of the first image to be detected based on the first package gap detection range value; and to crop a second package gap image of the second image to be detected based on the second package gap detection range value.

[0050] The image standard deviation calculation module is used to calculate the first image standard deviation of the first package gap image and to calculate the second image standard deviation of the second package gap image.

[0051] The first package determination module is used to determine all first package images and the coordinates of each first package image in the second image to be detected when the standard deviation of the first image is less than a preset image standard deviation threshold and the standard deviation of the second image is greater than or equal to the preset image standard deviation threshold.

[0052] The package filtering module is used to filter all the first package images based on the coordinates of each first package image and the direction of package movement, and obtain a second package image, and use the second package image as a new package image of the video to be detected.

[0053] This system acquires the package movement direction, a first image to be detected, a second image to be detected, and the first package gap detection range value corresponding to the first image to be detected from the video to be detected. The first image to be detected is the video frame image of the previous moment in the video to be detected, the video to be detected is the video of the package passing through the security scanner, and the second image to be detected is the video frame image of the video to be detected at the current moment. Based on the package movement direction and the first package gap detection range value, the system calculates the second package gap detection range value of the second image to be detected; it then extracts the first package gap image from the first image to be detected based on the first package gap detection range value; it also extracts the second package gap image from the second image to be detected based on the second package gap detection range value; it calculates the first image standard deviation of the first package gap image; it calculates the second image standard deviation of the second package gap image; and if the first image standard deviation is less than a preset image standard deviation threshold and the second image standard deviation is greater than or equal to the preset image standard deviation threshold, it determines all first package images in the second image to be detected and the coordinates of each first package image; based on each first package... The coordinates of the image and the direction of package movement are used to filter all first package images according to preset filtering conditions to obtain second package images. The second package image is then used as a new package image in the video to be detected. This application first detects all package images in the second image to be detected, and then filters all package images according to the direction of package movement and the coordinates of the first package to obtain new package images that exist in the second image to be detected but were not detected in the first image to be detected. By segmenting the new package in the second image to be detected, and then only detecting the new package image, the repeated detection of the already detected package images in the first image to be detected at the current moment is avoided, thereby improving the efficiency and accuracy of package detection.

[0054] A third aspect of this application provides a novel video package detection electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enables the at least one control processor to perform the aforementioned novel video package detection method.

[0055] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the novel video package detection method described above.

[0056] It should be noted that the beneficial effects of the second to fourth aspects of this application with respect to the prior art are the same as the beneficial effects of the new video package detection system described above with respect to the prior art, and will not be described in detail here.

[0057] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0058] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0059] Figure 1 This is a flowchart illustrating an embodiment of the novel video package detection method provided in this application;

[0060] Figure 2 This is a schematic diagram of an embodiment of the new video package detection system provided in this application;

[0061] Figure 3 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0062] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0063] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0064] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0065] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0066] With the rapid development of my country's civil aviation and rail transit industries, more and more people are choosing to travel by plane, high-speed rail, and subway. When passengers enter stations, their carry-on bags must undergo security checks to ensure safety. However, due to my country's large population and the high concentration of passengers during rush hours, security personnel need to check a large number of passengers' bags in a short period. Currently, the mainstream solution to this problem is to segment the packages in the video stream of security scanners and then use AI to identify prohibited items from the segmented images. Therefore, before AI image analysis is implemented, improving the efficiency and accuracy of package segmentation in security scanner videos has become a very urgent issue.

[0067] In existing technologies, package segmentation is mainly performed using the mean shift algorithm. However, this method involves a large amount of computation, has a slow segmentation efficiency, and a low accuracy, which reduces the accuracy and efficiency of package detection.

[0068] To address the aforementioned technical deficiencies, embodiments of this application provide a novel method, system, device, and storage medium for video package detection.

[0069] Please see Figure 1 This is a flowchart illustrating a novel video package detection method provided in an embodiment of this application. The method is applied to an electronic device, which may be a server, etc. Figure 1 As shown in the video, the new package detection method includes:

[0070] Step S101: Obtain the package movement direction of the video to be detected, the first image to be detected, the second image to be detected, and the first package gap detection range value corresponding to the first image to be detected, wherein the first image to be detected is the video frame image of the previous moment of the video to be detected, the video to be detected is the video of the package passing through the security inspection machine, and the second image to be detected is the video frame image of the current moment of the video to be detected.

[0071] Step S102: Based on the package movement direction and the first package gap detection range value, calculate the second package gap detection range value of the second image to be detected;

[0072] Step S103: Extract the first package gap image of the first image to be detected based on the first package gap detection range value; extract the second package gap image of the second image to be detected based on the second package gap detection range value;

[0073] Step S104: Calculate the first image standard deviation of the first package gap image; calculate the second image standard deviation of the second package gap image;

[0074] Step S105: If the standard deviation of the first image is less than the preset standard deviation threshold and the standard deviation of the second image is greater than or equal to the preset standard deviation threshold, determine all first package images in the second image to be detected and the coordinates of each first package image.

[0075] Step S106: Based on the coordinates and movement direction of each first package image, all first package images are filtered using preset filtering conditions to obtain second package images, and the second package images are used as new package images for the video to be detected.

[0076] The aforementioned range of values ​​for detecting the gap in the first package includes the range of values ​​for the horizontal axis of the gap detection in the first package and the range of values ​​for the vertical axis of the gap detection in the first package.

[0077] The aforementioned preset image standard deviation threshold is a constant value set in advance according to actual needs.

[0078] In step S103 above, the first package gap image is cropped from the first image to be detected based on the first package gap detection range value; the second package gap image is cropped from the second image to be detected based on the second package gap detection range value. This can be done by cropping an image of the same size as the first package gap detection range value from the first image to be detected as the first package gap image; and cropping an image of the same size as the second package gap detection range value from the second image to be detected as the second package gap image.

[0079] This method obtains the package movement direction, a first image to be detected, a second image to be detected, and the first package gap detection range value corresponding to the first image to be detected from the video to be detected. The first image to be detected is the video frame image of the previous moment in the video to be detected, the video to be detected is the video of the package passing through the security scanner, and the second image to be detected is the video frame image of the video to be detected at the current moment. Based on the package movement direction and the first package gap detection range value, the second package gap detection range value of the second image to be detected is calculated. A first package gap image is extracted from the first image to be detected based on the first package gap detection range value. A second package gap image is extracted from the second image to be detected based on the second package gap detection range value. A first image standard deviation of the first package gap image is calculated. A second image standard deviation of the second package gap image is calculated. If the first image standard deviation is less than a preset image standard deviation threshold and the second image standard deviation is greater than or equal to the preset image standard deviation threshold, all first package images in the second image to be detected and the coordinates of each first package image are determined. Based on each first package... The coordinates of the image and the direction of package movement are used to filter all first package images according to preset filtering conditions to obtain second package images. The second package image is then used as a new package image in the video to be detected. This application first detects all package images in the second image to be detected, and then filters all package images according to the direction of package movement and the coordinates of the first package to obtain new package images that exist in the second image to be detected but were not detected in the first image to be detected. By segmenting the new package in the second image to be detected, and then only detecting the new package image, the repeated detection of the already detected package images in the first image to be detected at the current moment is avoided, thereby improving the efficiency and accuracy of package detection.

[0080] In some embodiments, step S102 may include, but is not limited to, steps S201 to S203:

[0081] Step S201: Obtain the image width of the second image to be detected;

[0082] Step S202: Calculate the horizontal coordinate range value of the second package gap detection based on the package movement direction, image width, and preset gap width value;

[0083] Step S203: Use the vertical coordinate range value of the first package gap detection and the horizontal coordinate range value of the second package gap detection as the second package gap detection range value.

[0084] The above-mentioned preset gap width value is a constant value set in advance according to actual needs.

[0085] This application improves the accuracy of package detection by using the vertical coordinate range of the first package gap detection and the horizontal coordinate range of the second package gap detection as the second package gap detection range value, and using the second package gap detection range value as the data basis for subsequent screening of new package images.

[0086] In some embodiments, step S202 may include, but is not limited to, steps S301 to S302:

[0087] Step S301: When the package movement direction is the first preset direction, the horizontal coordinate range corresponding to zero to the preset gap width value is taken as the second package gap detection horizontal coordinate range value.

[0088] Step S302: When the package movement direction is the second preset direction, calculate the width difference between the image width and the preset gap width value, and take the width difference to the horizontal coordinate range corresponding to the image width as the second package gap detection horizontal coordinate range value.

[0089] The first preset direction and the second preset direction can be opposite to each other. The first preset direction and the second preset direction are preset directions according to actual needs.

[0090] This application improves the accuracy of calculating the horizontal coordinate range of the second package gap detection by using different methods for calculating the horizontal coordinate range of the second package gap detection under different package movement directions, thereby improving the accuracy of package detection.

[0091] In some embodiments, step S106 may include, but is not limited to, steps S401 to S406:

[0092] Step S401: Based on the direction of package movement and the coordinates of the first package image, sort the first package image to obtain sorted package images;

[0093] Step S402: Calculate the first horizontal axis projection range value of the first package image and the second horizontal axis projection range value of the second package image in the sorted package images;

[0094] Step S403: Calculate the intersection of the first horizontal axis projection range value and the second horizontal axis projection range value, and use it as the first horizontal axis projection intersection;

[0095] Step S404: If the intersection of the first horizontal axis projections is an empty set, end the traversal and take the first package image in the sorted package images as the second package image.

[0096] Step S405: If the intersection of the first horizontal axis projections is not empty, calculate the union of the first and second horizontal axis projection range values, and use it as the second horizontal axis projection union; calculate the third horizontal axis projection range value of the third wrapped image in the sorted wrapped images on the horizontal axis; calculate the intersection of the second horizontal axis projection union and the third horizontal axis projection range value, and use it as the third horizontal axis projection intersection; if the intersection of the third horizontal axis projections is not empty, calculate the union of the second and third horizontal axis projection union and the third horizontal axis projection range value, and use it as the third horizontal axis projection union; calculate the fourth horizontal axis projection range value of the fourth wrapped image in the sorted wrapped images on the horizontal axis; calculate the intersection of the third horizontal axis projection intersection and the fourth horizontal axis projection range value, and use it as the fourth horizontal axis projection intersection.

[0097] Step S406: Continue in this manner until the intersection of the i-th horizontal axis projections is an empty set or all first packages have been traversed. If the intersection of the i-th horizontal axis projections is an empty set, end the traversal and take the first package image to the i-th minus 1 package image in the sorted package images as the second package images. If all first package images have been traversed, take all first package images as the second package images, where i is the iteration number.

[0098] In step S401 above, the first package image is sorted based on the package movement direction and the coordinates of the first package image to obtain a sorted package image. This can be achieved by sorting the first package image in ascending order according to its horizontal coordinate when the package movement direction is a first preset direction, and by sorting the first package image in descending order according to its horizontal coordinate when the package movement direction is a second preset direction.

[0099] In step S402 above, calculating the first horizontal axis projection range value of the first package image and the second horizontal axis projection range value of the second package image in the sorted package images can be done by obtaining the horizontal axis range value of the first package image and the horizontal axis range value of the second package image in the sorted package images, using the horizontal axis range value of the first package image as the first horizontal axis projection range value, and using the horizontal axis range value of the second package image as the second horizontal axis projection range value.

[0100] This application improves the efficiency of package detection by filtering out all second package images from the first package image and using the second package images as newly detected package images.

[0101] In some embodiments, the method further includes:

[0102] Step S501: When the intersection of the first horizontal axis projections is an empty set, calculate the first vertical axis projection range value of the first packaged image in the sorted packaged images on the vertical axis, and use the first vertical axis projection range value as the package gap detection vertical axis range value of the next moment of the video to be detected.

[0103] Step S502: When the intersection of the first horizontal axis projection is not empty and the intersection of the i-th horizontal axis projection is empty, calculate the union of the vertical axis projection range values ​​of the first packaged image to the i-th minus 1 packaged image in the sorted packaged images, and use the union of the vertical axis projection range values ​​of the first packaged image to the i-th minus 1 packaged image as the package gap detection vertical coordinate range value of the next moment of the video to be detected.

[0104] Step S503: If the intersection of the first horizontal axis projections is not empty and all first package images have been traversed, calculate the union of the vertical axis projection range values ​​of all sorted package images, and use the union of the vertical axis projection range values ​​of all sorted package images as the package gap detection vertical coordinate range value of the next moment of the video to be detected.

[0105] In some embodiments, obtaining the first package gap detection ordinate range value of the video to be detected can be achieved by taking the image height of the second image to be detected from zero to the image height of the second image to be detected in advance, when the current time is the initial time.

[0106] This application uses the range of the parcel gap detection ordinate from the previous moment as the range of the parcel gap detection ordinate from the current moment, and uses the range of the parcel gap detection ordinate from the current moment as the data basis for subsequent screening of new parcel images, thereby improving the accuracy of parcel detection.

[0107] In some embodiments, step S105 may include, but is not limited to, steps S601 to S608:

[0108] Step S601: Apply Gaussian blur to the second image to be detected to obtain the Gaussian blurred image;

[0109] Step S602: Detect the Gaussian blurred image using an edge detection algorithm to obtain the edge-detected image;

[0110] Step S603: Dilate the image after edge detection to obtain the dilated image;

[0111] Step S604: Determine all initial wrapping images in the dilated image and the set of point coordinates for each initial wrapping image using a contour extraction algorithm;

[0112] Step S605: Calculate the minimum bounding rectangle of each initial package image based on the set of point coordinates of each initial package image;

[0113] Step S606: Calculate the diagonal length of each minimum bounding rectangle;

[0114] Step S607: Traverse each initial package image, filter all initial package images whose diagonal length is less than a preset diagonal length threshold, and obtain all first package images;

[0115] Step S608: Obtain the coordinates of all first package images.

[0116] The edge detection algorithm described above can be the Canny edge detection algorithm.

[0117] The aforementioned preset diagonal length threshold is a constant value set in advance according to actual needs.

[0118] In step S604 above, the set of point coordinates of all initial wrapping images and each initial wrapping image in the dilated image determined by the contour extraction algorithm can be determined by the findContours function of OpenCV (Open Source Computer Vision Library).

[0119] In step S605 above, the calculation of the minimum bounding rectangle of each initial package image based on the set of point coordinates of each initial package image can be achieved by inputting the set of point coordinates of each initial package image into the boundingRect function of OpenCV, and obtaining the minimum bounding rectangle of each initial package image output by the boundingRect function of OpenCV.

[0120] This embodiment improves the efficiency of package detection by filtering the initial package images to obtain all first package images.

[0121] In some embodiments, before acquiring a second image to be detected from the video to be detected, the method further includes:

[0122] Step S701: Receive the initial video frame image and the first sequence number of the initial video frame image sent by the security inspection machine;

[0123] Step S702: Obtain the second sequence number of the first image to be detected;

[0124] Obtain the second image to be detected from the video to be detected, including:

[0125] Step S703: If the initial video frame image meets the preset image standard and the difference between the first sequence number and the second sequence number is greater than the preset sequence number interval, the initial video frame image is used as the second image to be detected.

[0126] The aforementioned preset image standard can be that the video frame image has no afterimage and no ghosting.

[0127] The above are preset serial number intervals, which are constant values ​​set in advance according to actual needs.

[0128] The first serial number mentioned above is the numerical code that the security inspection machine assigns to the initial video frame image.

[0129] In some embodiments, the method further includes:

[0130] Step S801: If the standard deviation of the first image is greater than or equal to the preset standard deviation threshold of the image, or the standard deviation of the second image is less than the preset standard deviation threshold of the image, then it is determined that there is no new package image in the second image to be detected, and the detection of new packages in the second image to be detected ends.

[0131] Additionally, refer to Figure 2 One embodiment of this application provides a novel video package detection system, including a data acquisition module 1100, a package gap detection range value calculation module 1200, a package gap image cropping module 1300, an image standard deviation calculation module 1400, a first package determination module 1500, and a package screening module 1600, wherein:

[0132] The data acquisition module 1100 is used to acquire the package movement direction of the video to be detected, the first image to be detected, the second image to be detected, and the first package gap detection range value corresponding to the first image to be detected. The first image to be detected is the video frame image of the previous moment of the video to be detected, the video to be detected is the video of the package passing through the security inspection machine, and the second image to be detected is the video frame image of the current moment of the video to be detected.

[0133] The package gap detection range value calculation module 1200 is used to calculate the second package gap detection range value of the second image to be detected based on the package movement direction and the first package gap detection range value.

[0134] The package gap image cropping module 1300 is used to crop a first package gap image of a first image to be detected based on a first package gap detection range value; and to crop a second package gap image of a second image to be detected based on a second package gap detection range value.

[0135] The image standard deviation calculation module 1400 is used to calculate the first image standard deviation of the first package gap image and the second image standard deviation of the second package gap image.

[0136] The first package determination module 1500 is used to determine all first package images and the coordinates of each first package image in the second image to be detected when the standard deviation of the first image is less than a preset image standard deviation threshold and the standard deviation of the second image is greater than or equal to the preset image standard deviation threshold.

[0137] The package filtering module 1600 is used to filter all first package images based on the coordinates and movement direction of each first package image and by using preset filtering conditions to obtain second package images, and to use the second package images as new package images for the video to be detected.

[0138] This system acquires the package movement direction, a first image to be detected, a second image to be detected, and the first package gap detection range value corresponding to the first image to be detected from the video to be detected. The first image to be detected is the video frame image of the previous moment in the video to be detected, the video to be detected is the video of the package passing through the security scanner, and the second image to be detected is the video frame image of the video to be detected at the current moment. Based on the package movement direction and the first package gap detection range value, the system calculates the second package gap detection range value of the second image to be detected; it then extracts the first package gap image from the first image to be detected based on the first package gap detection range value; it also extracts the second package gap image from the second image to be detected based on the second package gap detection range value; it calculates the first image standard deviation of the first package gap image; it calculates the second image standard deviation of the second package gap image; and if the first image standard deviation is less than a preset image standard deviation threshold and the second image standard deviation is greater than or equal to the preset image standard deviation threshold, it determines all first package images in the second image to be detected and the coordinates of each first package image; based on each first package... The coordinates of the image and the direction of package movement are used to filter all first package images according to preset filtering conditions to obtain second package images. The second package image is then used as a new package image in the video to be detected. This application first detects all package images in the second image to be detected, and then filters all package images according to the direction of package movement and the coordinates of the first package to obtain new package images that exist in the second image to be detected but were not detected in the first image to be detected. By segmenting the new package in the second image to be detected, and then only detecting the new package image, the repeated detection of the already detected package images in the first image to be detected at the current moment is avoided, thereby improving the efficiency and accuracy of package detection.

[0139] It should be noted that the system embodiments described above are based on the same inventive concept as the method embodiments described above. Therefore, the relevant content of the method embodiments described above is also applicable to the system embodiments described above, and will not be repeated here.

[0140] Figure 3 A schematic diagram of the hardware structure for new package detection in video provided in an embodiment of this application is shown.

[0141] The new package detection device in the video may include a processor 301 and a memory 302 storing computer program instructions.

[0142] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0143] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0144] In some embodiments, memory 302 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0145] The processor 301 implements any of the new video package detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 302.

[0146] In one example, the video-based new package detection device may also include a communication interface 303 and a bus 310. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0147] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0148] Bus 310 includes hardware, software, or both, that couples components of a video new package detection device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0149] The new package detection device in this video can execute the new package detection method in the video embodiment of this application based on a 3D design model, thereby achieving a combination of... Figure 1 and Figure 2 The video describes a new method and system for package detection.

[0150] Furthermore, in conjunction with the new package detection method for video in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the new package detection methods for video in the above embodiments.

[0151] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0152] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0153] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0154] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0155] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for new package detection of a video, characterized in that, The new package detection method of the video comprises: obtaining a package moving direction, a first to-be-detected image, a second to-be-detected image and a first package gap detection range value corresponding to the first to-be-detected image of a to-be-detected video, wherein the first to-be-detected image is a video frame image of a previous moment of the to-be-detected video, the to-be-detected video is a video of a package passing through a security inspection machine, the second to-be-detected image is a video frame image of a current moment of the to-be-detected video, and the first package gap detection range value comprises a first package gap detection longitudinal coordinate range value; based on the package moving direction and the first package gap detection range value, calculating a second package gap detection range value of the second to-be-detected image, specifically as follows: obtaining an image width of the second to-be-detected image; based on the package moving direction, the image width and a preset gap width value, calculating a second package gap detection transverse coordinate range value, wherein the preset gap width value is a constant value set in advance according to actual needs, specifically as follows: in the case that the package moving direction is a first preset direction, taking a transverse coordinate range corresponding to zero to the preset gap width value as the second package gap detection transverse coordinate range value; in the case that the package moving direction is a second preset direction, calculating a width difference between the image width and the preset gap width value, and taking a transverse coordinate range corresponding to the width difference to the image width as the second package gap detection transverse coordinate range value; taking the first package gap detection longitudinal coordinate range value and the second package gap detection transverse coordinate range value as the second package gap detection range value; based on the first package gap detection range value, intercepting a first package gap image of the first to-be-detected image; based on the second package gap detection range value, intercepting a second package gap image of the second to-be-detected image; calculating a first image standard deviation of the first package gap image; calculating a second image standard deviation of the second package gap image; in the case that the first image standard deviation is less than a preset image standard deviation threshold value and the second image standard deviation is greater than or equal to the preset image standard deviation threshold value, determining all first package images in the second to-be-detected image and coordinates of each first package image; based on the coordinates of each first package image and the package moving direction, screening all the first package images through a preset screening condition to obtain a second package image, and taking the second package image as a new package image of the to-be-detected video, specifically as follows: based on the package moving direction and the coordinates of the first package image, sorting the first package image to obtain a sorted package image; calculating a first horizontal axis projection range value of a first package image in the sorted package image on a horizontal coordinate and a second horizontal axis projection range value of a second package image on the horizontal coordinate; calculating an intersection of the first horizontal axis projection range value and the second horizontal axis projection range value, and taking the intersection as a first horizontal axis projection intersection; based on the first horizontal axis projection intersection, determining the second package image.

2. The method of claim 1, wherein, The determining the second parcel image based on the first horizontal axis projection intersection comprises: In the case that the first horizontal axis projection intersection is an empty set, ending the iteration, and taking the first parcel image in the sorted parcel images as the second parcel image; In the case that the first horizontal axis projection intersection is not an empty set, calculating the union of the first horizontal axis projection range value and the second horizontal axis projection range value as a second horizontal axis projection union, calculating a third horizontal axis projection range value of a third parcel image in the sorted parcel images on the horizontal coordinate, calculating the intersection of the second horizontal axis projection union and the third horizontal axis projection range value as a third horizontal axis projection intersection, in the case that the third horizontal axis projection intersection is not an empty set, calculating the union of the second horizontal axis projection union and the third horizontal axis projection range value as a third horizontal axis projection union, calculating a fourth horizontal axis projection range value of a fourth parcel image in the sorted parcel images on the horizontal coordinate, calculating the intersection of the third horizontal axis projection intersection and the fourth horizontal axis projection range value as a fourth horizontal axis projection intersection; By analogy, until the ith horizontal axis projection intersection is an empty set or all the first parcel images have been iterated, in the case that the ith horizontal axis projection intersection is an empty set, ending the iteration, and taking the first parcel image to the (i-1)th parcel image in the sorted parcel images as the second parcel image, in the case that all the first parcel images have been iterated, taking all the first parcel images as the second parcel image, wherein i is the iteration number.

3. The method of claim 2, wherein, The method further comprises: In the case that the first horizontal axis projection intersection is an empty set, calculating a first vertical axis projection range value of the first parcel image in the sorted parcel images on the vertical coordinate, and taking the first vertical axis projection range value as the parcel gap detection vertical coordinate range value of the next moment of the to-be-detected image of the to-be-detected video; In the case that the first horizontal axis projection intersection is not an empty set and the ith horizontal axis projection intersection is an empty set, calculating the union of the vertical axis projection range values of the first parcel image to the (i-1)th parcel image in the sorted parcel images, and taking the union of the vertical axis projection range values of the first parcel image to the (i-1)th parcel image as the parcel gap detection vertical coordinate range value of the next moment of the to-be-detected image of the to-be-detected video; In the case that the first horizontal axis projection intersection is not an empty set and all the first parcel images have been iterated, calculating the union of the vertical axis projection range values of all the sorted parcel images, and taking the union of the vertical axis projection range values of all the sorted parcel images as the parcel gap detection vertical coordinate range value of the next moment of the to-be-detected image of the to-be-detected video.

4. The method of claim 1, wherein, The determining the first parcel images in the second to-be-detected image and the coordinates of each first parcel image comprises: Performing Gaussian blurring on the second to-be-detected image to obtain a Gaussian blurred image; Detecting the Gaussian blurred image by an edge detection algorithm to obtain an edge detected image; Performing inflation on the edge detected image to obtain an inflated image; Determine all initial parcel images in the expanded image and a point coordinate set of each initial parcel image through a contour extraction algorithm; Calculate a minimum circumscribed rectangle of each initial parcel image based on the point coordinate set of each initial parcel image; Calculate a diagonal line length of each minimum circumscribed rectangle; Iterate through each initial parcel image, filter all initial parcel images with a diagonal line length less than a preset diagonal line length threshold, and obtain all first parcel images; Obtain coordinates of all first parcel images.

5. The method of claim 1, wherein, Before obtaining a second to-be-detected image of a to-be-detected video, the method further comprises: Receiving an initial video frame image sent by the security inspection machine and a first serial number of the initial video frame image; Obtaining a second serial number of the first to-be-detected image; The method of obtaining a second to-be-detected image of a to-be-detected video comprises: In the case that the initial video frame image reaches a preset image standard and the difference between the first serial number and the second serial number is greater than a preset serial number interval, the initial video frame image is taken as the second to-be-detected image.

6. A new package detection system for video, characterized by, The new parcel detection system of the video comprises: A data acquisition module is configured to acquire a parcel moving direction, a first to-be-detected image, a second to-be-detected image, and a first parcel gap detection range value corresponding to the first to-be-detected image, wherein the first to-be-detected image is a video frame image at a previous time of the to-be-detected video, the to-be-detected video is a video of a parcel passing through a security inspection machine, the second to-be-detected image is a video frame image at a current time of the to-be-detected video, and the first parcel gap detection range value includes a first parcel gap detection longitudinal coordinate range value; A parcel gap detection range value calculation module is configured to calculate a second parcel gap detection range value of the second to-be-detected image based on the parcel moving direction and the first parcel gap detection range value, specifically as follows: Obtaining an image width of the second to-be-detected image; Calculating a second parcel gap detection horizontal coordinate range value based on the parcel moving direction, the image width, and a preset gap width value, wherein the preset gap width value is a constant value set in advance according to actual needs, specifically as follows: In the case that the parcel moving direction is a first preset direction, taking a horizontal coordinate range corresponding to zero to the preset gap width value as the second parcel gap detection horizontal coordinate range value; In the case that the parcel moving direction is a second preset direction, calculating a width difference between the image width and the preset gap width value, and taking a horizontal coordinate range corresponding to the width difference to the image width as the second parcel gap detection horizontal coordinate range value; Taking the first parcel gap detection longitudinal coordinate range value and the second parcel gap detection horizontal coordinate range value as the second parcel gap detection range value; A parcel gap image cropping module is configured to crop a first parcel gap image of the first to-be-detected image based on the first parcel gap detection range value, and crop a second parcel gap image of the second to-be-detected image based on the second parcel gap detection range value. an image standard deviation calculation module, configured to calculate a first image standard deviation of the first parcel gap image; and calculate a second image standard deviation of the second parcel gap image; a first parcel determination module, configured to determine all first parcel images in the second to-be-detected image and coordinates of each first parcel image, in a case where the first image standard deviation is less than a preset image standard deviation threshold and the second image standard deviation is greater than or equal to the preset image standard deviation threshold; a parcel screening module, configured to screen all the first parcel images through a preset screening condition based on the coordinates of each first parcel image and the parcel moving direction, to obtain second parcel images, and take the second parcel images as new parcel images of the to-be-detected video, specifically: sort the first parcel images based on the parcel moving direction and the coordinates of the first parcel images, to obtain sorted parcel images; calculate a first horizontal axis projection range value of a first parcel image in the sorted parcel images on a horizontal coordinate and a second horizontal axis projection range value of a second parcel image on the horizontal coordinate; calculate an intersection of the first horizontal axis projection range value and the second horizontal axis projection range value, and take the intersection as a first horizontal axis projection intersection; determine the second parcel images based on the first horizontal axis projection intersection.

7. A new package detection apparatus for video, characterized by, at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform a new parcel detection method of a video according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer executable instructions for causing a computer to perform a new parcel detection method of a video according to any one of claims 1 to 5.

Images