Image Processing Method, Apparatus, System, Medium and Electronic Device

Abstract

Embodiments of the present disclosure provide an image processing method, apparatus, system, medium, and electronic device, belonging to the field of computer technology. The method includes: obtaining a background image of a capture area; obtaining an initial moment when a calibrated item starts to be scanned by a security inspection device; obtaining a capture image in which at least a part of the calibrated item is in the capture area; determining, according to the background image and the capture image, a target image including the complete appearance of the calibrated item from the capture image; obtaining an interception moment of the target image; and obtaining a capture delay time from when the calibrated item passes through the security inspection device to the capture area of the security inspection device according to the interception moment of the target image and the initial moment. Through the technical solution provided by the embodiments of the present disclosure, by automatically obtaining the capture delay time from when the calibrated item starts to be scanned by the security inspection device to the capture area of the security inspection device, the working efficiency and accuracy of calibrating the capture delay time of the security inspection device can be improved in a large-scale deployment scenario.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method, device and system, a computer-readable storage medium and an electronic device. Background Art

[0002] At present, image processing and analysis technology is widely used in the security inspection of goods in areas such as airports, railway stations, customs, border inspection, docks and subway stations. The security inspection equipment generates a scanned image by scanning the inspected items, which helps to identify the contraband (including but not limited to dangerous goods) inside. If it is necessary to obtain the appearance image of the inspected item at the same time, such as when it is necessary to locate the unpacked items through the appearance image, an image acquisition device (such as a camera) is generally installed above the conveying device (such as a conveyor belt) at the exit of the security inspection equipment, and the appearance of the inspected item is captured by snapshot.

[0003] Under normal circumstances, the rotation speed of the conveyor belt of the security inspection equipment fluctuates relatively little, and the conveyor belt speed can be considered to be constant. Therefore, the time from the start of scanning the inspected object to the time when the inspected object reaches a suitable position on the conveyor belt for shooting is basically fixed. The delay time required for capturing can be roughly estimated by the speed of the conveyor belt, the installation position and angle of the camera, and then the appropriate capture delay time can be finally determined through actual testing and adjustment, so that the inspected object can appear in the capture area at the right time. In this way, the appearance image of the inspected object captured by the camera can correctly correspond to the scanned image of the inspected object.

[0004] However, due to the differences in the models, sizes, and conveyor belt speeds of different security inspection equipment, as well as the differences in the installation positions and angles of the cameras that capture the appearance of the inspected items in actual situations, it is necessary to estimate the capture delay time for each security inspection equipment, and to conduct tests and adjustments. This will bring a huge workload in large-scale deployment scenarios.

[0005] At the same time, in the existing solutions, the adjustment of the capture delay time needs to be done manually, and any manual intervention work may have human errors. If the deviation of the capture delay time setting is too large, it will directly lead to the appearance image of the inspected object cannot be captured, or cause the scanned image and the appearance image of the inspected object to correspond incorrectly.

[0006] Therefore, a new image processing method, image processing device, image processing system, computer-readable storage medium and electronic device are needed.

[0007] It should be noted that the invention in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the Invention

[0008] An object of embodiments of the present disclosure is to provide a new image processing method, an image processing apparatus, an image processing system, a computer-readable storage medium, and an electronic device, thereby at least to some extent overcoming the problems of low work efficiency and low accuracy rate existing in manual calibration of security inspection equipment.

[0009] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be learned in part through the practice of the present disclosure.

[0010] Embodiments of the present disclosure provide an image processing method, the method including: acquiring a background image of a capture area; acquiring an initial moment when a calibrated item starts to be scanned by a security inspection device; acquiring a capture image in which at least a part of the calibrated item is in the capture area; determining, according to the background image and the capture image, a target image including a complete appearance of the calibrated item from the capture image; acquiring an interception moment of the target image; and obtaining a capture delay time from when the calibrated item passes through the security inspection device to the capture area according to the interception moment of the target image and the initial moment.

[0011] Embodiments of the present disclosure provide an image processing apparatus, the apparatus including: a background image acquisition unit configured to acquire a background image of a capture area; an initial moment acquisition unit configured to acquire an initial moment when a calibrated item starts to be scanned by a security inspection device; a capture image acquisition unit configured to acquire a capture image in which at least a part of the calibrated item is in the capture area; a target image acquisition unit configured to determine, according to the background image and the capture image, a target image including a complete appearance of the calibrated item from the capture image; an interception moment acquisition unit configured to acquire an interception moment of the target image; and a capture delay time acquisition unit configured to obtain a capture delay time from when the calibrated item passes through the security inspection device to the capture area according to the interception moment of the target image and the initial moment.

[0012] An embodiment of the present disclosure provides an image processing system, which includes: a security inspection device including a conveying device and a scanning device; an image acquisition device located above the conveying device at the exit of the security inspection device, and the area where the image acquisition device acquires images is a capture area; an image processing device for obtaining a background image of the capture area; obtaining an initial time when a calibration item starts to be scanned through the security inspection device; obtaining a capture image when at least a part of the calibration item is in the capture area; determining a target image including the complete appearance of the calibration item from the capture image according to the background image and the capture image; obtaining an interception time of the target image; and obtaining a capture delay time of the calibration item from passing through the security inspection device to the capture area according to the interception time of the target image and the initial time.

[0013] An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image processing method as described in any one of the above embodiments.

[0014] An embodiment of the present disclosure provides an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the at least one processor implements the image processing method as described in any one of the above embodiments.

[0015] In the technical solutions provided by some embodiments of the present disclosure, a target image including the complete appearance of a calibration item is obtained from a capture image through a background image and a capture image of the capture area, and according to the interception time corresponding to the target image and the initial time when the calibration item starts to be scanned through the security inspection device, the capture delay time of the calibration item from passing through the security inspection device to the capture area can be automatically obtained. In an application scenario with large-scale deployment, applying the technical solution provided by the embodiment of the present disclosure to automatically calibrate the capture delay time of a security inspection device can improve the accuracy and working efficiency of calibrating the capture delay time of the security inspection device.

[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the following drawings are only some embodiments of the present disclosure, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings. In the drawings:

[0018] Figure 1A schematic diagram schematically shows an exemplary system architecture to which the image processing method or the image processing apparatus according to the embodiments of the present disclosure can be applied.

[0019] Figure 2 A schematic diagram schematically shows the structure of an electronic device suitable for implementing the embodiments of the present disclosure.

[0020] Figure 3 A flowchart schematically shows an image processing method according to an embodiment of the present disclosure.

[0021] Figure 4 A schematic diagram schematically shows an image capture process according to an embodiment of the present disclosure.

[0022] Figure 5 A schematic diagram schematically shows a method for sampling a captured image according to an embodiment of the present disclosure.

[0023] Figure 6 A schematic diagram schematically shows a method for determining the position of a calibration item in a captured area according to an embodiment of the present disclosure.

[0024] Figure 7 A block diagram schematically shows an image processing apparatus according to an embodiment of the present disclosure.

[0025] Figure 8 A schematic diagram schematically shows an image processing system according to an embodiment of the present disclosure. Detailed implementation manners

[0026] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0027] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or can be implemented using other methods, components, devices, steps, etc. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0028] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all the content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0030] Figure 1 The schematic diagram schematically shows an exemplary system architecture 100 to which the image processing method or image processing device according to embodiments of the present disclosure can be applied.

[0031] As Figure 1 shown, the system architecture 100 may include a security inspection device 101, an image acquisition device (such as a camera) 102, a network 103, a terminal device 104, and a server 105. Among them, the image acquisition device 102 is installed above the conveyor device at the exit of the security inspection device 101. Staff can use the security inspection device 101 and the camera 102 to obtain the captured video stream, and interact with the terminal device 104 or the server 105 through the network 103 to perform operations such as intercepting and processing the video stream.

[0032] It should be understood that Figure 1 the numbers of the image acquisition device, terminal device, network, and server in

[0033] are merely illustrative. According to the implementation requirements, there can be any number of cameras, terminal devices, networks, and servers. For example, the camera 102 can be a dual-camera composed of two cameras.

[0034] Figure 2 The schematic diagram schematically shows the structure of an electronic device suitable for implementing embodiments of the present disclosure.

[0035] It should be noted that Figure 2The illustrated electronic device 200 is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0036] As Figure 2 shown, the electronic device 200 includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 202 or the program loaded from the storage section 208 into the random access memory (RAM) 203. In the RAM 203, various programs and data required for system operation are also stored. The CPU 201, ROM 202, and RAM 203 are connected to each other via a bus 204. The input / output (I / O) interface 205 is also connected to the bus 204.

[0037] The following components are connected to the I / O interface 205: an input section 206 including a keyboard, a mouse, etc.; an output section 207 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed so that a computer program read from it can be installed into the storage section 208 as needed.

[0038] Specifically, according to the embodiments of the present disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), various functions defined in the system of the present application are executed.

[0039] It should be noted that the computer-readable storage medium shown in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable storage medium other than the computer-readable storage medium, and this computer-readable storage medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0040] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0041] The units involved in the embodiments of the present disclosure can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.

[0042] As another aspect, the present application also provides a computer-readable storage medium, which can be included in the electronic device described in the above embodiments; or can exist alone without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device is caused to implement the methods described in the following embodiments. For example, the electronic device can implement the Figure 3 various steps shown.

[0043] In an embodiment of the present disclosure, an image processing method is proposed to optimize the technical problems existing in the above related technologies. Specifically, refer to Figure 3 shown, the image processing method is applicable to the electronic device described in the foregoing embodiments, and can be executed by Figure 1 either the terminal device 104 or the server 105 in, and the present disclosure does not make any limitation thereto. Figure 3 The method provided by the embodiment may include the following steps S310 to step S360:

[0044] In step S310, a background image of the capture area is acquired.

[0045] In an exemplary embodiment, acquiring a background image of the capture area may include: acquiring a video stream collected by an image acquisition device of the capture area; and selecting an image frame that does not include the calibration item from the video stream as the background image.

[0046] The method provided by the embodiments of the present disclosure can be applied to an image processing system, and may include: a security inspection device, which includes a conveying device and a scanning device; an image acquisition device, which is located above the conveying device at the exit of the security inspection device, and the area where the image acquisition device acquires images is the capture area.

[0047] In the embodiments of the present disclosure, the calibration item refers to a standard item used when calibrating the capture delay time of the security inspection device. When the calibration item passes through the security inspection device, the appearance of the calibration item needs to be captured. In order to be able to clearly distinguish the calibration item from the conveying device, so as to better capture the appearance of the calibration item, an item with an obvious difference from the conveying device is selected as the calibration item.

[0048] In the embodiments of the present disclosure, a camera is taken as an example of the image acquisition device for illustration. Figure 8A schematic diagram of an image processing system according to an embodiment of the present disclosure is schematically shown. As Figure 8 shown, the security inspection device body 801 includes a scanning device 802, and an image acquisition device (such as a camera) 803 is installed above the conveying device 804 at the exit of the security inspection device. The calibration article 805 on the conveying device 804 gradually enters the image acquisition area 806 of the image acquisition device 803, that is, the capture area, after being scanned by the security inspection device along the security inspection direction 807. By intercepting the video stream obtained by the image acquisition device 803, each captured image frame can be obtained, and the obtained image frames are subjected to image recognition processing in the server 808 or the terminal device 809, and an image frame that does not include the appearance of the calibration article is selected as the background image. The present disclosure is not limited thereto. When the technical solution provided by the embodiments of the present disclosure is applied to different scenarios, the background image can be an image in any format.

[0049] In step S320, obtain the initial moment when the calibration article starts to be scanned by the security inspection device.

[0050] In the embodiments of the present disclosure, when the calibration article starts to be scanned by the security inspection device, the scanning device of the security inspection device can generate a corresponding scanning image through scanning imaging technology (including but not limited to spectral imaging technology). Contraband (including but not limited to dangerous goods) can be identified according to the scanning image. When contraband is found in the scanning image, in order to quickly locate the position of the article including the contraband, it is necessary to obtain the initial moment when the article starts to be scanned by the security inspection device. The initial moment when the calibration article starts to be scanned by the security inspection device can be obtained through the scanning image. The present disclosure can use any recording storage method or device to record the initial moment when the calibration article starts to be scanned by the security inspection device.

[0051] In step S330, obtain a captured image in which at least a part of the calibration article is in the capture area.

[0052] In the embodiments of the present disclosure, during the process of the calibration article leaving the exit of the security inspection device and completely entering the capture area, the display range of the calibration article in the captured image gradually increases. During this increasing process, it is considered that the part of the calibration article in the capture area is continuously increasing; when the display range of the calibration article in the captured image stops increasing, as Figure 4 shown in Figure D, it is considered that the calibration article completely enters the capture area; during the process of the calibration article gradually leaving the capture area, the display range of the calibration article in the captured image gradually decreases. During this decreasing process, before the display range of the calibration article in the captured image decreases to 0, it is considered that the part of the calibration article in the capture area is continuously decreasing, as Figure 4Figure E, and at least part of the appearance of the calibrated item is within the target image; when the display range of the calibrated item in the captured image is reduced to 0, it is considered that the calibrated item has completely left the capture area, and the appearance of the calibrated item is not included in the target image.

[0053] Specifically, image processing and analysis methods can be used to perform image recognition and analysis on the captured image, and the process of the change of the displayed part of the calibrated item in the captured image can be analyzed in detail. For example Figure 4 Part A to Figure 4 Part E. Figure 4 Part A schematically shows a schematic diagram of the image capture process according to an embodiment of the present disclosure. Among them, the security inspection device 401 includes a main body 402 and a conveying device 403 (hereinafter, the conveyor belt is taken as an example for illustration, but the present disclosure is not limited thereto). A scanning device is included inside the main body 402. When the calibrated item 405 passes through the exit 406 of the security inspection device along the conveying direction 404, it gradually reaches the capture area 407, that is, the image capture area of the image capture device. In Figure 4 In Part A, Part A shows a schematic diagram in which the calibrated item 405 is not included in the capture area 407; the calibrated item 405 gradually leaves the exit 406 of the security inspection device and enters the capture area 407 as the conveying device 403 conveys, as shown in Part B; when the calibrated item 405 starts to enter the capture area 407, a part of the appearance of the calibrated item 405 is included in the captured image, as Figure 4 shown in Part C; then, the calibrated item 405 completely enters the capture area 407, as Figure 4 shown in Part D, the calibrated item 405 first completely enters the capture area 407, and the complete appearance of the calibrated item 405 is included in the captured image; finally, the calibrated item 405 gradually leaves the capture area 407 as the conveying device 403 conveys, as Figure 4 shown in Part E.

[0054] In step S340, according to the background image and the captured image, a target image including the complete appearance of the calibrated item is determined from the captured image.

[0055] In an exemplary embodiment, according to the background image and the captured image, determining a target image including the complete appearance of the calibrated item from the captured image may include: calculating the difference value between each frame of the captured image and the background image respectively; selecting any frame of the captured image with the largest difference value as the target image.

[0056] In the embodiments of the present disclosure, the target image including the complete appearance of the calibrated item can be determined according to the maximum difference value between the background image and the captured image. To obtain the difference value between each frame of the captured image and the background image, image processing and analysis algorithms can be used to analyze the captured image. Any image analysis method (such as the image grayscale algorithm) can be used to obtain the pixel difference between each pixel point of the background image and the captured image including the calibrated item, add up this difference value and take the average of the obtained sum to get the difference value. When the difference value stops growing and remains unchanged, it is the maximum value of the difference. When the maximum difference value is obtained, it means that the calibrated item is completely included in the current target image.

[0057] In the embodiments of the present disclosure, the target image including the complete appearance of the calibrated item can also be determined according to the maximum area ratio between the background image and the captured image. The determination method is the same as the method of using the maximum difference value between the background image and the captured image to determine that the target image includes the complete appearance of the calibrated item as described above.

[0058] In an exemplary embodiment, according to the background image and the captured image, determining a target image including the complete appearance of the calibrated item from the captured image may include: respectively identifying the contour of the calibrated item in each frame of the captured image; respectively calculating the area ratio of the contour of the calibrated item in each frame of the captured image to the corresponding frame of the captured image; selecting any frame of the captured image with the largest area ratio as the target image.

[0059] However, due to scratches on the conveying device (such as a conveyor belt), background noise, or noise in the video stream obtained by the image acquisition device (such as a camera), the maximum difference value or the maximum area ratio cannot be used as the sole basis to identify the complete appearance of the calibrated item in the captured area.

[0060] In an exemplary embodiment, according to the background image and the captured image, determining a target image including the complete appearance of the calibrated item from the captured image may include: respectively calculating the difference value between each frame of the captured image and the background image; determining, from the captured images, a frame of the captured image with the largest difference value that first appears as the first captured image; obtaining a continuous plurality of frames of captured images whose capture time is after the first captured image; sampling the pixel values of the side of each frame of the continuous plurality of frames of captured images close to the security inspection device; if the pixel values of the side of the continuous plurality of frames of captured images close to the security inspection device meet a predetermined condition, then determining any frame of the continuous plurality of frames of captured images as the target image.

[0061] In the embodiments of the present disclosure, the target image including the complete appearance of the calibrated item may also be determined according to the area ratio between the background image and the captured image. The area ratio refers to the ratio of the area of the complete contour of the calibrated item to the area of the captured image in the target image including the complete appearance of the calibrated item.

[0062] In an exemplary embodiment, determining a target image including the complete appearance of the calibrated item from the captured image according to the background image and the captured image may include: calculating the area ratio between each frame of the captured image and the background image respectively; determining, from the captured images, a frame of the captured image with the largest area ratio that appears for the first time as the second captured image; obtaining a continuous plurality of frames of captured images whose capture time is after the second captured image; sampling the pixel values of the side close to the security inspection device in each frame of the continuous plurality of frames of captured images; and if the pixel values of the side close to the security inspection device in the continuous plurality of frames of captured images meet a predetermined condition, determining any frame in the continuous plurality of frames of captured images as the target image.

[0063] In the embodiments of the present disclosure, by sampling the pixels of the side close to the security inspection device in the captured image, a target image including the complete appearance of the calibrated item is determined. After finding the maximum difference value or the maximum area ratio, the embodiments of the present disclosure combine the image sampling analysis algorithm and the contour recognition algorithm to confirm that the contour of the entire calibrated item has completely appeared in the captured area, that is, the pixel sampling of the side close to the security inspection device outlet in the captured image does not include the pixels of the calibrated item, then the top contour of the calibrated item can be identified.

[0064] The method for distinguishing the calibrated item and the background pixels is to continuously sample the pixel values of the captured images close to the security inspection device outlet for a certain number of frames, and track the changes of each sampling line. As Figure 5 shown in part C, the square frame line close to the security inspection device side is the sampling line 501. If the predetermined condition that the captured image has a complete calibrated item is met, that is, the sampling values obtained by the sampling line 501 fluctuate within a certain threshold range or do not change, it can be considered that the content in the sampling line 501 no longer changes, and all the pixels in the sampling come from the background picture, and it is considered that the calibrated item has completely entered the captured area. If the sampling values change within a continuous certain number of frames, then a partial appearance of the calibrated item is included in the captured image, as Figure 5 shown in part A; if the sampling does not change within a continuous certain number of frames, it can be determined that the captured image includes the complete appearance of the calibrated item, as Figure 5 shown in parts B to D.

[0065] In an exemplary embodiment, according to the background image and the captured image, determining a target image including the complete appearance of the calibrated item from the captured image may include: identifying the contour of the target captured image; obtaining a circumscribed regular geometric shape of the contour of the target captured image; obtaining the center point of the circumscribed regular geometric shape of the contour of the target captured image; and determining that the target captured image in which the center point of the circumscribed regular geometric shape is on the center line of the captured area perpendicular to the central axis direction of the conveyor belt of the security inspection device is the target image.

[0066] In the embodiments of the present disclosure, the circumscribed regular geometric shape of the contour of the target captured image is any one of polygons including but not limited to squares, pentagons, etc., and circles. In the present disclosure, a rectangle is taken as an example for illustration. The center point of the circumscribed rectangle of the contour of the target captured image can be calculated by the following method:

[0067] The above rectangle includes 4 vertices (x0, y0), (x1, y1), (x2, y2), (x3, y3), and the area A of the polygon is:

[0068]

[0069] The center point (C x , C y ) of the polygon is given by the following formula:

[0070]

[0071]

[0072] According to the position of the calibrated item contour in the captured image, a target image with a better captured position where the calibrated item is centered in the captured image can be obtained. Identify the contour of the calibrated item in the captured image, and obtain the center point of the circumscribed rectangle of the calibrated item contour. Determine the position of the calibrated item in the captured area according to the position of the center point of the circumscribed rectangle in the central axis direction of the conveyor belt, as shown in part A of Figure 6 .

[0073] In step S350, obtain the capture time of the target image.

[0074] In the embodiments of the present disclosure, obtain the capture time of the target image. Calculate the difference value or area ratio between each frame of the captured image and the background image, and select any frame of the captured image with the largest difference value or area ratio as the target image including the complete appearance of the calibrated item. As shown in part D of Figure 4 , when the calibrated item first appears completely in the captured image, it can be used as the first captured image, and the corresponding capture time can be obtained.

[0075] In the embodiments of the present disclosure, the capture moment of the target image is obtained. A method of sampling the pixels on the side close to the security inspection device in the captured image is adopted to determine the target image including the complete appearance of the calibrated item. When the pixel values within the sampling line remain dynamically fluctuating or unchanged within a certain threshold, it indicates that the calibrated item appears completely in the captured image, such as Figure 5 from part B to part D. Among them, Figure 5 compared with part B and part C of Figure 5 , part D shows the target image in which the calibrated item is centered in the captured image. At this time, the target image with the capture effect can be obtained, and the capture moment corresponding to the target image is recorded.

[0076] In the embodiments of the present disclosure, the capture moment of the target image is obtained. By adopting a relatively better capture position, a captured image can be obtained when the calibrated item is as much as possible centered in the capture area. As shown in Figure 6 shown, Figure 6 FIG. schematically shows a schematic diagram of a method for determining the position of a calibrated item in a capture area according to an embodiment of the present disclosure. In Figure 6 part B of Figure 6 , as each captured image changes, the center point 602 of the circumscribed rectangle 601 of the contour of the calibrated item moves along the central axis direction 604 of the conveyor belt. When it reaches and falls on the center line 603 perpendicular to the central axis direction of the conveying device (such as the conveyor belt) in the capture area, the image processing system records the capture moment of the corresponding captured image. Among them Figure 6 part B of Figure 6 adopts a relatively better capture position and has a better capture effect compared with part A.

[0077] In step S360, according to the capture moment of the target image and the initial moment, the capture delay time of the calibrated item from passing through the security inspection device to the capture area is obtained.

[0078] In an exemplary embodiment, obtaining the capture delay time of the calibrated item from passing through the security inspection device to the capture area according to the capture moment of the target image and the initial moment includes: obtaining the initial capture delay time of the calibrated item from passing through the security inspection device to the capture area according to the capture moment of the target image and the initial moment; taking the average value of the initial capture delay time as the capture delay time.

[0079] In the embodiments of the present disclosure, the initial capture delay time of the calibrated item from passing through the security inspection device to the capture area is obtained according to the capture moment of the target image and the initial moment. By taking the initial moment as t1 and the capture time of the above target image as t2, the capture delay time (t2 - t1) of the calibrated item from passing through the security inspection device to the capture area can be obtained.

[0080] In the embodiments of the present disclosure, the average value of the initial capture delay time is taken as the capture delay time. Since there may be errors in the capture delay time obtained each time, in order to obtain a more accurate capture delay time, the capture delay time (t2 - t1) is calibrated multiple times in the present disclosure, and the average value is taken to obtain a more accurate capture delay time.

[0081] The image processing method proposed in the embodiments of the present disclosure may be a method for calibrating the capture delay time of the appearance of a calibrated item based on security inspection equipment. The initial moment when the calibrated item starts to be scanned by the security inspection equipment is obtained. By intercepting the video stream captured by the camera of the security inspection equipment, a series of captured image frames can be obtained. Any image frame that does not include the calibrated item is selected as the background image. Among the continuously captured image frames after this background image, the captured image that includes the complete calibrated item is selected as the target image. The difference obtained by subtracting the interception time of the target image from the initial moment is the corresponding capture delay time.

[0082] According to the method proposed in the embodiments of the present disclosure, it can be used to calibrate the capture delay time of the item to be detected by the security inspection equipment in an actual application scenario. The item to be detected refers to an item that needs to be detected by the security inspection equipment.

[0083] In an exemplary embodiment, it includes: obtaining the starting moment when the item to be detected starts to be scanned by the security inspection equipment, and obtaining the scanned image of the item to be detected; identifying the scanned image to determine that the item to be detected includes contraband; according to the starting moment and the capture delay time, determining the arrival time when the item to be detected is in the capture area; obtaining a target image including the complete appearance of the item to be detected according to the arrival time; using the target image to determine that the item to be detected is the target item to be unpacked.

[0084] In the embodiments of the present disclosure, the scanned image of the item to be detected can be obtained through an image acquisition device, and at the same time, the starting moment when the item to be detected starts to be scanned by the security inspection equipment is obtained. When identifying the scanned image and determining that the item to be detected includes contraband, the item to be detected needs to be unpacked for inspection. Among them, the item to be detected that needs to be unpacked for inspection is the target item to be unpacked. Adding the starting moment and the capture delay time of the item to be detected can obtain the arrival time when the item to be detected reaches the capture area. According to the arrival time, the corresponding target image including the complete appearance of the item to be detected can be obtained. After determining that the complete appearance of the item to be detected in the target image is consistent with the appearance of the target item to be unpacked, the target item to be unpacked can be unpacked for inspection.

[0085] The method provided by the embodiments of the present disclosure improves the work efficiency in large-scale deployment scenarios and can reduce human errors to a certain extent.

[0086] The following introduces the device embodiments of the present disclosure, which can be used to execute the above-mentioned image processing method of the present disclosure. For details not disclosed in the device embodiments of the present disclosure, please refer to the embodiments of the above-mentioned image processing method of the present disclosure.

[0087] Figure 7 A block diagram of an image processing device according to an embodiment of the present disclosure is schematically shown. Referring to Figure 7 As shown, an image processing device 700 according to an embodiment of the present disclosure may include: a background image acquisition unit 710, an initial time acquisition unit 720, a captured image acquisition unit 730, a target image acquisition unit 740, an interception time acquisition unit 750, and a capture delay time acquisition unit 760.

[0088] Among them, the background image acquisition unit 710 may be used to acquire the background image of the capture area. The initial time acquisition unit 720 may be used to acquire the initial time when the calibrated item starts to be scanned by the security inspection device. The captured image acquisition unit 730 may be used to acquire a captured image in which at least a part of the calibrated item is in the capture area. The target image acquisition unit 740 may be used to determine a target image including the complete appearance of the calibrated item from the captured image according to the background image and the captured image. The interception time acquisition unit 750 may be used to acquire the interception time of the target image. The capture delay time 760 acquisition unit may be used to obtain the capture delay time from when the calibrated item passes through the security inspection device to the capture area according to the interception time of the target image and the initial time.

[0089] In an exemplary embodiment, the target image acquisition unit 740 may include: a difference value calculation unit, which may be used to calculate the difference value between each frame of the captured image and the background image respectively; a target selection unit, which may be used to select any one frame of the captured image with the largest difference value as the target image.

[0090] In an exemplary embodiment, the target image acquisition unit 740 may include: a contour recognition unit, which respectively recognizes the contour of the calibrated item in each frame of the captured image; an area ratio calculation unit, which respectively calculates the area ratio of the contour of the calibrated item in each frame of the captured image to the corresponding frame of the captured image; a target selection unit, which selects any one frame of the captured image with the largest area ratio as the target image.

[0091] In an exemplary embodiment, the target image acquisition unit 740 may include: a difference value calculation unit that calculates the difference value between each captured image and the background image respectively; a first captured image acquisition unit that determines, from the captured images, a captured image with the largest difference value that appears for the first time as the first captured image; a continuous multi-frame captured image acquisition unit that acquires a continuous multi-frame of captured images whose capture time is after the first captured image; a sampling unit that samples the pixel values of the side of each captured image in the continuous multi-frame of captured images close to the security inspection device; and a target selection unit that, if the pixel values of the side of the continuous multi-frame of captured images close to the security inspection device meet a predetermined condition, determines any one of the continuous multi-frame of captured images as the target image.

[0092] In an exemplary embodiment, the target image acquisition unit 740 may include: an area ratio calculation unit that calculates the area ratio between each captured image and the background image respectively; a second captured image acquisition unit that determines, from the captured images, a captured image with the largest area ratio that appears for the first time as the second captured image; a continuous multi-frame captured image acquisition unit that acquires a continuous multi-frame of captured images whose capture time is after the second captured image; a sampling unit that samples the pixel values of the side of each captured image in the continuous multi-frame of captured images close to the security inspection device; and a target selection unit that, if the pixel values of the side of the continuous multi-frame of captured images close to the security inspection device meet a predetermined condition, determines any one of the continuous multi-frame of captured images as the target image.

[0093] In an exemplary embodiment, the target image acquisition unit 740 may include: a contour recognition unit that recognizes the contour of the target captured image; a contour shape acquisition unit that obtains the circumscribed regular geometric shape of the contour of the target captured image; a contour shape center point acquisition unit that obtains the center point of the circumscribed regular geometric shape of the contour of the target captured image; and a target selection unit that determines the target captured image whose center point of the circumscribed regular geometric shape is on the center line in the direction perpendicular to the central axis of the conveyor of the security inspection device in the capture area as the target image.

[0094] In an exemplary embodiment, the background image acquisition unit 710 may include:

[0095] a video stream acquisition unit that acquires the video stream captured by the image acquisition device in the capture area; and a background selection unit that selects an image frame not including the calibrated item from the video stream as the background image.

[0096] In an exemplary embodiment, the capture delay time acquisition unit 760 may include:

[0097] An initial capture delay time acquisition unit obtains the initial capture delay time for the calibrated item to pass through the security inspection device to the capture area according to the capture time of the target image and the initial time; an average value acquisition unit takes the average value of the initial capture delay time as the capture delay time.

[0098] In an exemplary embodiment, the image processing device of the method further includes: a scanned image acquisition unit that acquires the starting time when the item to be detected starts to be scanned by the security inspection device and acquires the scanned image of the item to be detected; a scanned image recognition unit that recognizes the scanned image and determines that the item to be detected includes prohibited items; an arrival time acquisition unit that determines the arrival time of the item to be detected in the capture area according to the starting time and the capture delay time; a target acquisition unit that acquires a target image including the complete appearance of the item to be detected according to the arrival time; a target unpacking item acquisition unit that determines the item to be detected as a target unpacking item by using the target image.

[0099] It should be noted that although several units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0100] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0101] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0102] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the claims.

Claims

1. An image processing method, characterized in that, Including: Obtain a background image of the capture area, where the background image does not include a calibration item, and the capture area is the area where an image acquisition device acquires images; The image acquisition device is installed above the conveyor device at the exit of the security inspection equipment; Obtain a scanned image of the calibration item generated by the security inspection equipment, and obtain the initial moment when the calibration item starts to be scanned through the scanned image; Obtain a capture image in which at least part of the calibration item is in the capture area; According to the background image and the capture image, determine a target image including the complete appearance of the calibration item from the capture image; Obtain the capture moment of the target image; According to the capture moment of the target image and the initial moment, obtain the capture delay time of the calibration item from passing through the security inspection equipment to the capture area.

2. The method according to claim 1, wherein According to the background image and the capture image, determining a target image including the complete appearance of the calibration item from the capture image includes: Calculate the difference value between each frame of the capture image and the background image respectively; Select any frame of the capture image with the largest difference value as the target image.

3. The method according to claim 1, wherein According to the background image and the capture image, determining a target image including the complete appearance of the calibration item from the capture image includes: Identify the contour of the calibration item in each frame of the capture image respectively; Calculate the area ratio of the contour of the calibration item in each frame of the capture image to the area of the corresponding frame of the capture image respectively; Select any frame of the capture image with the largest area ratio as the target image.

4. The method according to claim 1, characterized in that According to the background image and the capture image, determining a target image including the complete appearance of the calibration item from the capture image includes: Calculate the difference value between each frame of the capture image and the background image respectively; Determine a frame of the capture image with the largest difference value that first appears from the capture image as the first capture image; Obtain a continuous multiple frames of capture images whose capture moments are after the first capture image; Sample the pixel values of the side of each frame of the continuous multiple frames of capture images close to the security inspection equipment; If the pixel values of the side of the continuous multiple frames of capture images close to the security inspection equipment meet a predetermined condition, determine that any frame in the continuous multiple frames of capture images is the target image.

5. The method according to claim 1, wherein According to the background image and the capture image, determining a target image including the complete appearance of the calibration item from the capture image includes: Calculate the area ratio between each frame of the capture image and the background image respectively; Determine a frame of the capture image with the largest area ratio that first appears from the capture image as the second capture image; Obtain a continuous multiple frames of capture images whose capture moments are after the second capture image; Sample the pixel values of the side of each frame of the continuous multiple frames of capture images close to the security inspection equipment; If the pixel values of the side of the continuous multiple frames of capture images close to the security inspection equipment meet a predetermined condition, determine that any frame in the continuous multiple frames of capture images is the target image.

6. The method according to claim 1, wherein Determining a target image including the complete appearance of the calibrated item from the captured image according to the background image and the captured image includes: Identifying the contour of the captured image; Obtaining a circumscribed regular geometric shape of the contour of the captured image; Obtaining the center point of the circumscribed regular geometric shape of the contour of the captured image; Determining that the captured image in which the center point of the circumscribed regular geometric shape is on the center line in the direction perpendicular to the central axis of the conveying device of the security inspection device in the capture area is the target image.

7. The method according to claim 1, characterized in that, Obtaining the background image of the capture area, including: Obtaining the video stream collected by the image acquisition device in the capture area; Selecting an image frame that does not include the calibrated item from the video stream as the background image.

8. The method according to any one of claims 1 to 7, characterized in that, Obtaining the capture delay time of the calibrated item from passing through the security inspection device to the capture area according to the capture time of the target image and the initial time, including: Obtaining the initial capture delay time of the calibrated item from passing through the security inspection device to the capture area according to the capture time of the target image and the initial time; Taking the average value of the initial capture delay time as the capture delay time.

9. The method according to claim 1, wherein Further includes: Obtaining the starting time when the item to be detected starts to be scanned by the security inspection device, and obtaining the scanned image of the item to be detected; Identifying the scanned image and determining that the item to be detected includes contraband; Determining the arrival time of the item to be detected in the capture area according to the starting time and the capture delay time; Obtaining a target image including the complete appearance of the item to be detected according to the arrival time; Using the target image to determine that the item to be detected is the target item to be unpacked.

10. An image processing apparatus, characterized in that, Includes: A background image acquisition unit for acquiring a background image of the capture area, the background image not including the calibrated item, and the capture area being the area where the image acquisition device acquires images; the image acquisition device is installed above the conveying device at the exit of the security inspection device; An initial time acquisition unit for acquiring the scanned image generated by the calibrated item passing through the security inspection device, and acquiring the initial time when the calibrated item starts to be scanned through the scanned image; A captured image acquisition unit for acquiring a captured image in which at least part of the calibrated item is in the capture area; A target image acquisition unit for determining a target image including the complete appearance of the calibrated item from the captured image according to the background image and the captured image; An interception time acquisition unit for acquiring the interception time of the target image; A capture delay time acquisition unit for obtaining the capture delay time of the calibrated item from passing through the security inspection device to the capture area according to the interception time of the target image and the initial time.

11. An image processing system, characterized in that, Includes: A security inspection device, which includes a conveying device and a scanning device; An image acquisition device, which is located above the conveying device at the exit of the security inspection device, and the area where the image acquisition device acquires images is the capture area; An image processing device is configured to obtain a background image of the capture area, where the background image does not include a calibration item; obtain a scanned image of the calibration item generated by the security inspection device, and obtain an initial time when the calibration item starts to be scanned by the security inspection device through the scanned image; obtain a capture image in which at least a part of the calibration item is in the capture area; determine a target image including the complete appearance of the calibration item from the capture image according to the background image and the capture image; obtain an interception time of the target image; and obtain a capture delay time from when the calibration item passes through the security inspection device to when it is captured in the capture area according to the interception time of the target image and the initial time.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the image processing method according to any one of claims 1 to 9.

13. An electronic device, characterized in that, Comprising: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the image processing method according to any one of claims 1 to 9.

Images