An airport luggage empty basket detection method, system, medium and electronic equipment

Abstract

The present application relates to a kind of airport luggage empty basket detection method, system, medium and electronic equipment, by obtaining the color image and depth image of security basket, and first by the depth information in depth image, the security basket is positioned, then according to depth information, object detection is carried out, when depth image does not include object, detection is carried out again using color image, using positioning feature, the security basket in color image is positioned first, then object detection is carried out to color image, and empty basket determination is completed.The present application first detects the object in the security basket that protrusion is more obvious by depth image, then low object in the security basket is detected by color image, and detection efficiency is high and not prone to error.

Description

Technical Field

[0001] This invention belongs to the field of security inspection technology, and in particular relates to a method, system, medium and electronic equipment for detecting empty baggage baskets at airports. Background Technology

[0002] Airport terminal security checkpoints are one of the most crucial components of airport operations, serving as the most critical checkpoints for safety and directly impacting airport security. Ensuring a seamless one-to-one correspondence between passengers and their security checks, while meeting security requirements, and retaining various information about passengers carrying hand luggage, is a vital aspect of the security process.

[0003] In current security screening processes, passengers' luggage needs to be placed in a security tray, and after passing through security, the luggage in the tray is taken away. It is crucial to ensure that passengers retrieve all their luggage from the tray to prevent lost items. Current technology typically uses manual verification of the security trays, which is inefficient and prone to errors. Summary of the Invention

[0004] This invention provides a method, system, medium, and electronic device for detecting empty baggage baskets at airports, in order to solve the problems of low efficiency and error-proneness in the prior art when manually checking empty baskets.

[0005] A method for detecting empty baggage baskets at airports, the method comprising:

[0006] Acquire a color image and a depth image of the detection area containing the security basket; and obtain positioning features from the color image and depth information of pixels from the depth image;

[0007] The outline of the security basket in the depth image is identified based on the depth information of the pixels to obtain the first edge outline;

[0008] Object detection is performed on the region within the first edge contour to obtain a first detection result; the first detection result includes whether the region within the first edge contour contains an object or whether the region within the first edge contour does not contain an object.

[0009] When the first detection result indicates that the area within the first edge contour does not contain an object, the contour of the security basket in the color image is located and identified according to the positioning features to obtain the second edge contour.

[0010] Object detection is performed on the region within the second edge contour to obtain a second detection result; the second detection result includes whether the region within the second edge contour contains an object or whether the region within the second edge contour does not contain an object.

[0011] When the second detection result indicates that the area within the second edge contour does not contain an object, the security inspection basket is determined to be empty.

[0012] In one embodiment of the present invention, an RFID tag is pre-installed inside the security inspection basket, and color and depth images of the inspection area containing the security inspection basket are acquired, including:

[0013] The detection area is sensed by a pre-set RFID reading device;

[0014] When the RFID reader senses an RFID tag, it determines that the security basket is in the detection area and acquires depth and color images of multiple detection areas.

[0015] In one embodiment of the present invention, obtaining depth information of pixels from the depth image includes:

[0016] Obtain the coordinates and depth values ​​of pixels in multiple depth images;

[0017] Multiple pixels with the same coordinates are extracted from multiple depth images, and a depth information sequence is constructed using the depth values ​​of the multiple pixels with the same coordinates.

[0018] Calculate the median of the depth information sequence, and use the median as the depth information of the corresponding pixel.

[0019] In one embodiment of the present invention, the contour of the security basket in the depth image is identified based on the depth information of the pixels to obtain a first edge contour, including:

[0020] The depth information value of the pixel is compared with a preset depth threshold;

[0021] When the depth information value of a pixel is less than the depth threshold, the pixel is regarded as an edge pixel.

[0022] The edge pixels are fitted to obtain an initial contour;

[0023] The size of the initial contour is obtained and compared with the size of a preset contour template. When the size of the target contour is greater than the size of the contour template, it is determined that multiple security baskets are stuck together, and a prompt signal is generated to prompt the separation of the multiple security baskets. When the size of the target contour is the same as the size of the contour template, the initial contour is used as the first edge contour.

[0024] In one embodiment of the present invention, object detection is performed on the region within the first edge contour to obtain a first detection result, including:

[0025] The depth information values ​​of the pixels within the first edge contour are compared with a preset depth threshold range;

[0026] When the depth information value of any pixel within the first edge contour is within a preset depth threshold range, that pixel is taken as an object pixel.

[0027] The number of object pixels is counted. If the number of object pixels is less than a preset first threshold, it is determined that the region within the first edge contour does not contain an object; otherwise, it is determined that the region within the first edge contour contains an object.

[0028] In one embodiment of the present invention, the edge of the security inspection basket is pre-set with a positioning strip including a target color; the outline of the security inspection basket in the color image is located and identified according to the positioning feature to obtain a second edge outline, including:

[0029] Perform color recognition on the color image to obtain the region containing only the target color;

[0030] Obtain the region containing only the target color, and compare the region containing only the target color with a preset positioning strip template. When the region containing only the target color matches the preset positioning strip template, use the region containing only the target color as the positioning strip.

[0031] The number of positioning strips is counted. When the number of positioning strips is greater than a preset value, it is determined that multiple security baskets are stuck together, and a prompt signal is generated to prompt the separation of multiple security baskets. When the number of positioning strips is the same as the preset value, the positioning strips are fitted to generate a second edge contour.

[0032] In one embodiment of the present invention, the second detection result includes: the region within the second edge contour contains an object, and the region within the second edge contour does not contain an object; object detection is performed on the region within the second edge contour to obtain the second detection result, including:

[0033] The area within the second edge contour is segmented to obtain the basket edge area and the basket bottom area;

[0034] The edge region of the basket is binarized to obtain the grayscale value of the pixels in the edge region; and the bottom region of the basket is binarized to obtain the grayscale value of the pixels in the bottom region.

[0035] The grayscale values ​​of pixels in the edge region of the basket are compared with a preset first grayscale value range. If the grayscale value of a pixel in the edge region is within the first grayscale value range, the pixel in the edge region is taken as a pixel of the edge object. The grayscale values ​​of pixels in the bottom region of the basket are compared with a preset second grayscale value range. If the grayscale value of a pixel in the bottom region of the basket is within the second grayscale value range, the pixel in the bottom region of the basket is taken as a pixel of the bottom object.

[0036] Count the number of pixels of the objects around the basket and the number of pixels of the objects at the bottom of the basket;

[0037] The number of pixels of the object at the edge of the basket is compared with a preset second threshold, and the number of pixels of the object at the bottom of the basket is compared with a preset third threshold. When the number of pixels of the object at the edge of the basket is less than the second threshold and the number of pixels of the object at the bottom of the basket is less than the third threshold, it is determined that the area within the second edge contour does not contain an object; otherwise, it is determined that the area within the second edge contour contains an object.

[0038] The present invention also provides an airport baggage empty basket detection system, the system comprising:

[0039] The acquisition module is used to acquire color images and depth images of the detection area containing the security inspection basket; and to acquire positioning features from the color images and depth information of pixels from the depth images.

[0040] The first edge recognition module is used to recognize the outline of the security basket in the depth image based on the depth information of the pixels, and obtain the first edge outline;

[0041] The first object detection module is used to perform object detection on the region within the first edge contour and obtain a first detection result; the first detection result includes whether the region within the first edge contour contains an object or whether the region within the first edge contour does not contain an object.

[0042] The second edge recognition module is used to locate and recognize the outline of the security basket in the color image based on the positioning features when the first detection result is that the area within the first edge contour does not contain an object, thereby obtaining the second edge contour.

[0043] The second object detection module performs object detection on the region within the second edge contour to obtain a second detection result; the second detection result includes whether the region within the second edge contour contains an object or whether the region within the second edge contour does not contain an object.

[0044] The empty basket detection module is used to determine that the security basket is empty when the second detection result is that the area within the second edge contour does not contain an object.

[0045] The present invention also provides an electronic device, the electronic device comprising:

[0046] One or more processors;

[0047] A storage device for storing one or more programs, which, when executed by one or more processors, enable the electronic device to implement an airport baggage basket detection method as described above.

[0048] The present invention also provides a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed by a computer processor, causes the computer to perform an airport baggage empty basket detection method as described above.

[0049] This invention provides a method, system, medium, and electronic device for detecting empty baggage bins at airports, which has the following advantages: By acquiring color and depth images of the security bin, the bin is first located using depth information from the depth image. Then, object detection is performed based on the depth information. If the depth image does not contain any objects, the color image is used for detection. The bin is first located in the color image using localization features, and then object detection is performed on the color image to determine if it is empty. This invention first detects objects with obvious protrusions in the security bin using the depth image, and then detects low-lying objects using the color image, resulting in high detection efficiency and reduced error rate. Attached Figure Description

[0050] Figure 1 This is a schematic diagram illustrating the implementation environment of an airport baggage empty basket detection process, as shown in an exemplary embodiment of this application.

[0051] Figure 2 This is a flowchart illustrating an exemplary embodiment of the airport baggage empty basket detection method.

[0052] Figure 3 yes Figure 2 The flowchart of step S210 in the illustrated embodiment is shown in an exemplary embodiment;

[0053] Figure 4 yes Figure 2 A flowchart of step S210 in another exemplary embodiment;

[0054] Figure 5 yes Figure 2 Step S220 in the illustrated embodiment is a flowchart of an exemplary embodiment;

[0055] Figure 6 yes Figure 2 The flowchart of step S230 in the illustrated embodiment is shown in an exemplary embodiment;

[0056] Figure 7 yes Figure 2 The flowchart of step S240 in the illustrated embodiment is shown in an exemplary embodiment;

[0057] Figure 8 yes Figure 2 The flowchart of step S250 in the illustrated embodiment is shown in an exemplary embodiment;

[0058] Figure 9 This is a structural diagram of an airport baggage empty basket detection system, illustrating an exemplary embodiment of this application;

[0059] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0060] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0062] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0063] Figure 1 This is an exemplary embodiment of the present application illustrating an application scenario of an airport baggage empty basket detection method, such as... Figure 1As shown, this application uses a conveyor belt to transport the security inspection basket 110 to the inspection area. An RFID reader 140 is installed above the inspection area to read the security inspection basket. The security inspection basket 110 has a pre-installed RFID tag. When the security inspection basket 110 arrives at the inspection area, it can be sensed by the RFID reader 140. At the same time, a depth information camera 120 and a regular camera 130 are installed above the inspection area. The depth information camera 120 captures the depth image of the inspection area, and the regular camera 130 captures the color image of the inspection area. The RFID reader 140, the depth information camera 120, and the regular camera 130 are all connected to a processing device 150. The processing device 150 can be any device capable of image processing and calculation, such as a PC host, server, or smart terminal.

[0064] like Figure 2 As shown, in an exemplary embodiment, the airport baggage empty basket detection method includes at least steps S210 to S260, which are described in detail below:

[0065] S210. Obtain a color image and a depth image of the detection area containing the security check basket; and obtain the positioning features from the color image and the depth information of the pixels from the depth image;

[0066] In this embodiment, when capturing images of the security basket, the background area where the security basket is located will inevitably be captured. Therefore, the images captured by the depth information camera and the ordinary camera actually include the detection area of ​​the security basket. The security basket has a red positioning bar pre-set at its edge, so the red positioning bar is used as a positioning feature in the color image. Each pixel in the depth image contains depth information.

[0067] S220. Based on the depth information of the pixels, identify the outline of the security basket in the depth image to obtain the first edge outline;

[0068] In step S220, since the security basket is placed in the detection area, the detection area is generally flat, meaning that the security basket protrudes from the detection area. The depth information in the depth image records the feature that the edge of the security basket protrudes from the detection area. Therefore, the depth information can be used to identify the outline of the security basket, thereby obtaining the first edge outline.

[0069] S230. Perform object detection on the region within the first edge contour to obtain a first detection result; the first detection result includes whether the region within the first edge contour contains an object or whether the region within the first edge contour does not contain an object.

[0070] In step S230, since the first edge contour of the security basket has been extracted using depth information, it is only necessary to perform object detection on the area within the first edge contour to determine whether there is an object in the security basket based on the recognition results.

[0071] S240. When the first detection result is that the area within the first edge contour does not contain an object, the contour of the security basket in the color image is located and identified according to the positioning features to obtain the second edge contour.

[0072] In step S240, when the first detection result is that the area within the first edge contour does not contain an object, it means that the depth information indicates that there is no object in the security basket. Since the depth information can only detect taller objects, it cannot effectively detect thinner objects close to the bottom of the security basket. Therefore, it is necessary to use a color image for detection. Since the edge of the security basket has a positioning strip attached in advance, the position and contour of the security basket can be obtained by recognizing the positioning features (i.e., the positioning strip), thereby obtaining the second edge contour.

[0073] S250. Perform object detection on the region within the second edge contour to obtain a second detection result; the second detection result includes whether the region within the second edge contour contains an object or whether the region within the second edge contour does not contain an object.

[0074] In step S250, object detection is performed on the region within the second edge contour to detect whether there is an object in the security basket from the color image.

[0075] S260. When the second detection result indicates that the area within the second edge contour does not contain an object, the security basket is determined to be empty;

[0076] In step S260, if both the first and second detection results indicate that the security basket is empty, the security basket is determined to be empty, thus completing the empty basket detection. Otherwise, the security basket is determined not to be empty.

[0077] like Figure 3 As shown, in one embodiment of this application, an RFID tag is pre-installed inside the security inspection basket. The process of acquiring a color image and a depth image of the inspection area containing the security inspection basket further includes steps S310 to S320, which are described in detail below:

[0078] S310. The detection area is sensed by a preset RFID reading device;

[0079] In this embodiment, the detection area is sensed by radio frequency identification (RFID) technology. RFID can also be replaced by communication and identification technologies such as NFC and Bluetooth.

[0080] S320. When the RFID reader senses the RFID tag, it determines that the security basket is in the detection area and acquires depth and color images of multiple detection areas.

[0081] In step S320, when the RFID reader senses the RFID tag, it means that the security basket with the RFID tag has arrived at the detection area. At this time, multiple depth images and color images are continuously captured by the depth information camera and the ordinary camera.

[0082] In one embodiment of the present invention, obtaining depth information of pixels from a depth image includes:

[0083] like Figure 4 As shown, in one embodiment of this application, the process of obtaining the depth information of pixels from a depth image further includes steps S410 to S430, which are described in detail below:

[0084] S410. Obtain the coordinates and depth values ​​of pixels in multiple depth images;

[0085] In this embodiment, multiple depth images are acquired to eliminate depth value deviations of individual pixels in the depth images.

[0086] S420. Extract multiple pixels with the same coordinates from multiple depth images, and construct a depth information sequence using the depth values ​​of the multiple pixels with the same coordinates;

[0087] In step S420, the pixels of multiple depth images are placed into a depth information sequence according to their coordinates, and each depth information sequence corresponds to a pixel coordinate of a depth image.

[0088] S430. Calculate the median of the depth information sequence and use the median as the depth information of the corresponding pixel.

[0089] In step S430, since a depth information sequence is established, it is equivalent to statistically analyzing multiple depth information for each pixel coordinate. By calculating the median, large deviations in the depth information of the pixel are avoided.

[0090] like Figure 5 As shown, in one embodiment of this application, the process of identifying the outline of the security basket in the depth image based on the depth information of the pixels to obtain the first edge outline further includes steps S510 to S540, which are described in detail below:

[0091] S510. Compare the depth information value of the pixel with a preset depth threshold;

[0092] In this embodiment, since the security basket is placed in a flat detection area, the acquired depth image includes both the depth information of the detection area and the depth information of the security basket. Since the security basket protrudes from the detection area, the depth information of a portion of the security basket is less than the depth information of the detection area. In order to obtain the edge contour of the security basket, a depth threshold is set to filter the depth information of the detection area.

[0093] S520. When the depth information value of a pixel is less than the depth threshold, the pixel is treated as an edge pixel;

[0094] In step S520, since the depth information of a portion of the security basket is less than the depth information of the detection area, a depth threshold is set to determine which pixels have a depth information greater than the threshold. The portion greater than the threshold indicates that the depth information value is large and belongs to the pixels in the detection area, while the portion less than the threshold indicates that the depth information value is small and belongs to the pixels in the security basket. The edge pixels of the security basket can be obtained through the above principle.

[0095] In addition, the outline of the security basket can be obtained through existing contour detection functions, such as the findContours() contour detection function.

[0096] S530. Fit the edge pixels to obtain the initial contour;

[0097] In step S530, edge pixels are connected by fitting, and the fitted shape is consistent with the top view shape of the security basket, such as a rectangle.

[0098] S540. Obtain the size of the initial contour and compare it with the size of the preset contour template; when the size of the target contour is greater than the size of the contour template, determine that multiple security baskets are stuck together and generate a prompt signal to prompt the separation of multiple security baskets; when the size of the target contour is the same as the size of the contour template, use the initial contour as the first edge contour.

[0099] In step S540, the size of the initial contour is obtained by counting the number of pixels within the initial contour. Since the distance between pixels is constant and the area occupied by each pixel is fixed, the area size within the initial contour can be obtained by counting the number of pixels. Simultaneously, since multiple security baskets may be stuck together (i.e., placed close together), relying solely on shape to determine the size of the security basket is inaccurate. Because the distance between the depth information camera and the security basket is fixed, the size of the security basket in the depth image is fixed. The size of the security basket is obtained by counting the number of pixels, and then compared with the size of a pre-established standard contour template to determine whether multiple security baskets are stuck together. If the size of the target contour is larger than the size of the contour template, it is determined that multiple security baskets are stuck together, and the determination result is output to notify staff to separate the security baskets. If the size of the target contour is the same as the contour template, it means that the initial contour corresponds to a single security basket, and the initial contour can be used as the first edge contour.

[0100] like Figure 6 As shown, in one embodiment of this application, the process of performing object detection on the region within the first edge contour and obtaining the first detection result further includes steps S610 to S630, which are described in detail below:

[0101] S610. Compare the depth information values ​​of the pixels within the first edge contour with a preset depth threshold range;

[0102] In this embodiment, if there is an object in the security inspection basket, the depth information of the object will differ significantly from the depth information of the bottom of the basket. Therefore, a depth threshold range is preset. The minimum value of the range is the depth information of the bottom of the basket, and the maximum value is a custom value. Pixels are identified by using the depth threshold range.

[0103] S620. When the value of the depth information of any pixel within the first edge contour is within a preset depth threshold range, the pixel is taken as an object pixel.

[0104] In step S620, when the depth information value of any pixel within the first edge contour matches the preset depth threshold range, it indicates that the depth information of the pixel differs significantly from the depth information of the bottom of the basket, and therefore the pixel is determined to be an object pixel.

[0105] S630. Count the number of object pixels. If the number of object pixels is less than a preset first threshold, determine that the area within the first edge contour does not contain an object; otherwise, determine that the area within the first edge contour contains an object.

[0106] In step S630, since there may be a large deviation in the depth information of a single pixel, it is determined that there is an object in the security basket only when the number of object pixels is greater than the preset first number threshold; otherwise, if the number of object pixels is less than the preset first number threshold, it is determined that the area within the first edge contour does not contain an object.

[0107] like Figure 7 As shown, in one embodiment of this application, the edge of the security inspection basket is pre-set with a positioning strip including a target color. The process of locating and identifying the outline of the security inspection basket in the color image based on the positioning features to obtain the second edge outline further includes steps S710 to S730, which are described in detail below:

[0108] S710. Perform color recognition on a color image to obtain the region containing only the target color;

[0109] In this embodiment, by pre-setting positioning strips on the edge of the security basket, it is easy to identify and locate the security basket through color images; for example, if the security basket is red, when identifying the color image, the red area in the color image will be identified specifically.

[0110] S720. Obtain the area containing only the target color, and compare the area containing only the target color with the preset positioning bar template. If the area containing only the target color matches the preset positioning bar template, use the area containing only the target color as the positioning bar.

[0111] In step S720, since the objects in the security basket may also be red, in order to eliminate the interference of other red objects, it is necessary to perform geometric feature recognition on the area containing red. The positioning strip template is a long rectangle, so when the area containing red is also a long rectangle, this area is used as the positioning strip.

[0112] S730. Count the number of positioning strips. When the number of positioning strips is greater than the preset value, it is determined that multiple security baskets are stuck together, and a prompt signal is generated to prompt the separation of multiple security baskets. When the number of positioning strips is the same as the preset value, the positioning strips are fitted to generate a second edge profile.

[0113] In step S730, since the positioning strips are pre-set on the edge of the security basket, taking a rectangular security basket as an example, there are a total of 4 sides, so the number of positioning strips should be 4, and the preset value is set to 4; when the number of positioning strips is greater than the preset value, it is determined that multiple security baskets are stuck together, and the security baskets are separated by staff according to the determination result; when the number of positioning strips is 4, by fitting the 4 positioning strips, the second edge contour used to represent the outline of the security basket can be obtained.

[0114] like Figure 8As shown, in one embodiment of this application, the process of performing object detection on the region within the second edge contour and obtaining the second detection result further includes steps S810 to S830, which are described in detail below:

[0115] S810. Divide the area within the second edge contour to obtain the basket edge area and the basket bottom area;

[0116] In this embodiment, since the color of the basket edge area is different from the color of the basket bottom area, the color difference between the two can be used to divide the basket edge area and the basket bottom area into independent areas.

[0117] S820. Perform binarization processing on the edge region of the basket to obtain the grayscale values ​​of the pixels in the edge region of the basket; and perform binarization processing on the bottom region of the basket to obtain the grayscale values ​​of the pixels in the bottom region of the basket;

[0118] In step S820, since the color of the basket edge area is different from the color of the basket bottom area, the gray value of the basket edge area is also different from the gray value of the basket bottom area after binarization.

[0119] S830. Compare the grayscale values ​​of the pixels in the edge region of the basket with a preset first grayscale value range. If the grayscale values ​​of the pixels in the edge region are within the first grayscale value range, then the pixels in the edge region are taken as edge object pixels. Compare the grayscale values ​​of the pixels in the bottom region of the basket with a preset second grayscale value range. If the grayscale values ​​of the pixels in the bottom region of the basket are within the second grayscale value range, then the pixels in the bottom region of the basket are taken as bottom object pixels.

[0120] In step S830, since the gray values ​​of the edge region and the bottom region are different, it is necessary to set different gray value ranges. By matching the gray value range with the gray value of the pixel, it is determined whether the pixel belongs to the edge region, the bottom region, or an object.

[0121] S840. Count the number of pixels of objects around the basket and the number of pixels of objects at the bottom of the basket;

[0122] In step S840, since individual pixels may have large grayscale value deviations, it is not possible to determine whether there is an object in the security basket based on just one pixel. However, if there are a large number of pixels that differ significantly from the bottom or edge area of ​​the basket, it can be basically determined that there is an object in the bottom or edge area of ​​the basket.

[0123] S850. Compare the number of pixels of the object at the edge of the basket with a preset second threshold, and compare the number of pixels of the object at the bottom of the basket with a preset third threshold; if the number of pixels of the object at the edge of the basket is less than the second threshold and the number of pixels of the object at the bottom of the basket is less than the third threshold, determine that the area within the second edge contour does not contain an object; otherwise, determine that the area within the second edge contour contains an object.

[0124] In step S850, a second and a third quantity threshold are set to determine whether objects exist in the basket edge and bottom regions. Finally, the result of determining whether objects exist in the color image is obtained based on the comparison results.

[0125] This invention provides a method for detecting empty baggage bins at airports. It acquires color and depth images of the security bin, first locating the bin using depth information from the depth image, then performing object detection based on the depth information. If the depth image does not contain any objects, the color image is then used for detection. The bin is first located in the color image using localization features, and then object detection is performed on the color image to determine if it is empty. This invention first detects objects with noticeable protrusions in the bin using the depth image, and then detects low-lying objects using the color image, resulting in high detection efficiency and a low error rate.

[0126] like Figure 9 As shown, the present invention also provides an airport baggage empty basket detection system, characterized in that the system includes:

[0127] The acquisition module is used to acquire color images and depth images of the detection area containing the security inspection basket; and to acquire positioning features from the color images and depth information of pixels from the depth images.

[0128] The first edge recognition module is used to recognize the outline of the security basket in the depth image based on the depth information of the pixels, and obtain the first edge outline;

[0129] The first object detection module is used to perform object detection on the region within the first edge contour and obtain a first detection result; the first detection result includes whether the region within the first edge contour contains an object or whether the region within the first edge contour does not contain an object.

[0130] The second edge recognition module is used to locate and recognize the outline of the security basket in the color image based on the positioning features when the first detection result is that the area within the first edge contour does not contain an object, thereby obtaining the second edge contour.

[0131] The second object detection module performs object detection on the region within the second edge contour to obtain a second detection result; the second detection result includes whether the region within the second edge contour contains an object or whether the region within the second edge contour does not contain an object.

[0132] The empty basket detection module is used to determine that the security basket is empty when the second detection result is that the area within the second edge contour does not contain an object.

[0133] This invention provides an airport baggage empty basket detection system. It acquires color and depth images of the security basket, first locating the basket using depth information from the depth image, then performing object detection based on the depth information. If the depth image does not contain an object, the color image is then used for detection. The system first locates the basket in the color image using localization features, and then performs object detection on the color image to determine if the basket is empty. This invention first detects objects with noticeable protrusions in the security basket using the depth image, and then detects low-lying objects using the color image, resulting in high detection efficiency and a low error rate.

[0134] It should be noted that the airport baggage empty basket detection system and the airport baggage empty basket detection method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the airport baggage empty basket detection system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0135] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement an airport baggage empty basket detection method provided in the above embodiments.

[0136] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 10As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0138] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0139] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0140] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0142] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0143] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the airport baggage empty basket detection method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0144] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform an airport baggage empty basket detection method provided in the various embodiments described above.

[0145] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. An airport baggage empty basket detection method characterized by, The method comprises: acquiring a color image and a depth image of a detection area containing a security basket; and acquiring a positioning feature from the color image and depth information of a pixel point from the depth image; identifying the outline of the security basket in the depth image according to the depth information of the pixel point to obtain a first edge outline; performing object detection on the area within the first edge outline to obtain a first detection result; the first detection result comprises that the area within the first edge outline contains an object or does not contain an object; when the first detection result is that the area within the first edge outline does not contain an object, positioning and identifying the outline of the security basket in the color image according to the positioning feature to obtain a second edge outline; performing object detection on the area within the second edge outline to obtain a second detection result; the second detection result comprises that the area within the second edge outline contains an object or does not contain an object; when the second detection result is that the area within the second edge outline does not contain an object, determining that the security basket is empty.

2. The method of claim 1, wherein, The security basket is provided with a preset RFID tag, and acquiring a color image and a depth image of a detection area containing a security basket comprises: sensing the detection area by a preset RFID reading device; when the RFID reading device senses the RFID tag, determining that the security basket is in the detection area, and collecting depth images and color images of multiple detection areas.

3. The method of claim 2, wherein The depth information of a pixel point is acquired from the depth image, comprising: acquiring the coordinates and depth values of the pixel points in multiple depth images; extracting multiple pixel points with the same coordinates from the multiple depth images, and constructing a depth information sequence with the depth values of the multiple pixel points with the same coordinates; calculating the median of the depth information sequence, and taking the median as the depth information of the corresponding pixel point.

4. The method of claim 1, wherein The outline of the security basket in the depth image is identified according to the depth information of the pixel point to obtain a first edge outline, comprising: comparing the value of the depth information of the pixel point with a preset depth threshold value; when the value of the depth information of the pixel point is less than the depth threshold value, taking the pixel point as an edge pixel point; fitting the edge pixel points to obtain an initial outline; obtaining the size of the initial outline, and comparing the size of the initial outline with the size of a preset outline template; when the size of the target outline is greater than the size of the outline template, determining that multiple security baskets are adhered, generating a prompt signal for prompting to separate the multiple security baskets; when the size of the target outline is consistent with the size of the outline template, taking the initial outline as the first edge outline.

5. The method of claim 1, wherein The area within the first edge outline is subjected to object detection to obtain a first detection result, comprising: comparing the value of the depth information of the pixel point within the first edge outline with a preset depth threshold range; when the value of the depth information of any pixel point within the first edge outline is within the preset depth threshold range, taking the pixel point as an object pixel point; The number of the object pixels is counted, and when the number of the object pixels is less than a preset first number threshold, it is determined that the area within the first edge contour does not contain an object; otherwise, it is determined that the area within the first edge contour contains an object.

6. The method of claim 1, wherein The edge of the security basket is provided with a positioning strip including a target color; the contour of the security basket in the color image is positioned and recognized according to the positioning feature, and a second edge contour is obtained, including: Color recognition is performed on the color image to obtain an area containing only the target color; The area containing only the target color is obtained, and the area containing only the target color is compared with a preset positioning strip template; when the area containing only the target color is consistent with the preset positioning strip template, the area containing only the target color is taken as the positioning strip; The number of the positioning strips is counted, and when the number of the positioning strips is greater than a preset value, it is determined that multiple security baskets are adhered, and a prompt signal for prompting to separate the multiple security baskets is generated; when the number of the positioning strips is consistent with the preset value, the positioning strips are fitted to generate a second edge contour.

7. The method of claim 1, wherein, The second detection result includes that the area within the second edge contour contains an object or does not contain an object; object detection is performed on the area within the second edge contour to obtain a second detection result, including: The area within the second edge contour is segmented to obtain a basket edge region and a basket bottom region; The basket edge region is subjected to binarization processing to obtain the gray value of the pixel points in the basket edge region, and the basket bottom region is subjected to binarization processing to obtain the gray value of the pixel points in the basket bottom region; The gray value of the pixel points in the basket edge region is compared with a preset first gray value range, and when the gray value of the pixel points in the basket edge region is within the first gray value range, the pixel points in the basket edge region are taken as basket edge object pixel points; the gray value of the pixel points in the basket bottom region is compared with a preset second gray value range, and when the gray value of the pixel points in the basket bottom region is within the second gray value range, the pixel points in the basket bottom region are taken as basket bottom object pixel points; The number of the basket edge object pixel points and the number of the basket bottom object pixel points are counted; The number of the basket edge object pixel points is compared with a preset second number threshold, and the number of the basket bottom object pixel points is compared with a preset third number threshold; when the number of the basket edge object pixel points is less than the second number threshold and the number of the basket bottom object pixel points is less than the third number threshold, it is determined that the area within the second edge contour does not contain an object; otherwise, it is determined that the area within the second edge contour contains an object.

8. An airport baggage empty carton detection system characterized by, The system includes: A collection module is configured to acquire a color image and a depth image of a detection area containing a security basket, acquire a positioning feature from the color image, and acquire depth information of pixel points from the depth image; A first edge recognition module is configured to recognize a contour of the security basket in the depth image according to the depth information of the pixel points to obtain a first edge contour. The first object detection module is configured to perform object detection on the region within the first edge contour to obtain a first detection result. The first detection result includes that the region within the first edge contour contains an object or that the region within the first edge contour does not contain an object. The second edge identification module is configured to, when the first detection result is that the region within the first edge contour does not contain an object, identify and locate the contour of the security basket in the color image according to the positioning feature to obtain a second edge contour. The second object detection module is configured to perform object detection on the region within the second edge contour to obtain a second detection result. The empty basket detection module is configured to determine whether the security basket is empty according to the first detection result and the second detection result to complete empty basket detection.

9. An electronic device, comprising: The electronic device includes: one or more processors; a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement an airport luggage empty basket detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, a computer program is stored thereon, which, when executed by a processor of a computer, causes the computer to perform an airport luggage empty basket detection method according to any one of claims 1 to 7.

Images