Image processing method and device

Abstract

The embodiment of the present application provides an image processing method and device. The method includes: obtaining a strip image of a detection object, wherein the strip image is obtained by scanning and detecting the detection object with a security inspection device; determining at least one reference image corresponding to the strip image, and splicing the strip image and the at least one reference image to obtain an intermediate image corresponding to the detection object; processing the intermediate image through a target model to determine a target area and a non-target area in the strip image, wherein the target area refers to an image sub-area including the detection object. The accuracy of image processing is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of security inspection technology, and in particular to an image processing method and device. Background Art

[0002] The security inspection equipment can scan objects on the conveyor belt and identify whether the object is a target object, or whether there is a target object (for example, dangerous goods, liquid containers, etc.) among the objects.

[0003] In actual application, the scanning component of the security inspection equipment can perform X-ray scanning on objects on the conveyor belt and generate corresponding images. The image processing system of the security inspection equipment can segment the image through a preset algorithm. However, the current security inspection image segmentation algorithms all have the problem of low segmentation accuracy. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and device to solve the problem of low accuracy of image processing performed by a security inspection image segmentation algorithm.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Acquire a strip image of the detection object, wherein the strip image is obtained by scanning and detecting the detection object with a security inspection device;

[0007] Determine at least one reference image corresponding to the strip image, and perform splicing processing on the strip image and the at least one reference image to obtain an intermediate image corresponding to the detection object, wherein the reference image includes feature information for assisting in segmenting the strip image;

[0008] The intermediate image is processed by a target model to determine a target region and a non-target region in the strip image, wherein the target region refers to an image sub-region including the detection object.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0010] An acquisition module, used to acquire a strip image of the detection object, wherein the strip image is obtained by scanning and detecting the detection object through security inspection equipment;

[0011] a determination module, configured to determine at least one reference image corresponding to the strip image and perform splicing processing on the strip image and the at least one reference image to obtain an intermediate image, wherein the reference image includes feature information for assisting in segmenting the strip image;

[0012] The first processing module is used to process the intermediate image through a target model to determine a target area and a non-target area in the strip image, wherein the target area refers to an image sub-area including the detection object.

[0013] In a third aspect, an embodiment of the present application provides a security inspection device, including:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform any method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any method in the first aspect when executed by a processor.

[0019] The image processing method and device provided in the embodiment of the present application, after the security inspection device acquires the strip image, determines at least one reference image corresponding to the strip image, and the reference image includes feature information for assisting in segmenting the strip image. The strip image and at least one reference image are spliced ​​into an intermediate image, and the intermediate image is processed by the target model. In this way, the target area and non-target area can be determined in the strip image based on the feature information in the strip image and at least one reference image, avoiding the situation where the result of image segmentation processing by the preset algorithm has errors when the security inspection device acquires less feature information corresponding to the strip image, thereby improving the accuracy of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application;

[0021] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a process for determining an initial image provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a target model provided in an embodiment of the present application;

[0024] Figure 5 A flowchart of another image processing method provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of a process for determining at least one reference image provided by an embodiment of the present application;

[0026] Figure 7 A schematic diagram of another process of determining at least one reference image provided by an embodiment of the present application;

[0027] Figure 8 A schematic diagram of the image processing process provided by an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0029] Fig.10 A schematic diagram of the structure of another image processing device provided in an embodiment of the present application;

[0030] Fig.11 A schematic diagram of the structure of the security inspection equipment provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0032] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] For ease of understanding, below, combined Figure 1 , the application scenarios to which the embodiments of the present application are applicable are described.

[0035] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 , including a security inspection device 101. The security inspection device 101 includes a scanning component, a conveying device, an image processing device and a display. The security inspection device 101 can be an X-ray security inspection device, the scanning component includes a ray source and a detector, the conveying device can be a conveyor belt, and a preset algorithm is set in the image processing device. A plurality of objects are placed on the conveying device of the security inspection device 101, and the conveying device sequentially conveys the plurality of objects to the detection area of ​​the scanning component (the area between the two dotted lines). The scanning component scans and processes the objects in the detection area, generates a corresponding strip image, and sends the strip image to the image processing device. The image processing device of the security inspection device 101 processes the strip image by a preset algorithm to divide the strip image into a target area and a non-target area, and displays the processed image through the display.

[0036] In the above process, the strip image is directly used as the processing object. Since the strip image has less feature information corresponding to it, there is a problem of inaccurate image segmentation results.

[0037] In an embodiment of the present application, after obtaining a strip image, at least one reference image corresponding to the strip image is determined, and the reference image includes feature information for assisting in segmenting the strip image. The strip image and at least one reference image are spliced ​​into an intermediate image, and the intermediate image is processed by a target model. In this way, the target area and the non-target area can be determined in the strip image based on the feature information in the strip image and at least one reference image, thereby avoiding the situation where the result of image segmentation processing by a preset algorithm has errors when the security inspection equipment collects less feature information corresponding to the strip image, thereby improving the accuracy of image processing. The solution of the embodiment of the present application can be accurately used for ultra-thin package segmentation, dangerous goods segmentation, bottled liquid segmentation, etc.

[0038] The method shown in the present application is described below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be described repeatedly in different embodiments.

[0039] Figure 2 This is a flowchart of an image processing method provided in an embodiment of the present application. Figure 2 , the method may include:

[0040] S201, obtaining a strip image of a detection object.

[0041] The execution subject of the embodiment of the present application may be a security inspection device, or an image processing device arranged in the security inspection device, or a processing device independent of the security inspection device. The image processing device may be implemented by software, or by a combination of software and hardware. The security inspection device may be an X-ray security inspection device.

[0042] The strip image is obtained by scanning and detecting the detection object through security inspection equipment.

[0043] In an optional embodiment, the strip image acquired by the security inspection device can be obtained in the following manner: initial scanning data acquired by the security inspection device during the scanning and detection process of the inspection object is acquired; a first template image and a second template image are acquired, the first template image is image data acquired by the detector in the security inspection device when the radiation source in the security inspection device is turned on and there is no inspection object, and the second template image is image data acquired by the detector in the security inspection device when the radiation source in the security inspection device is not turned on and there is no inspection object; based on the first template image and the second template image, the initial scanning data is corrected to obtain a strip image.

[0044] The initial scanning data includes at least one pixel value. The ray source in the security inspection device may be an X-ray source.

[0045] Next, combine Figure 3 , the process of determining the initial image is explained. Figure 3 This is a schematic diagram of the process of determining the initial image provided by the embodiment of the present application. Figure 3 , including a transmission device 301. The transmission device 301 is set in the security inspection device. The transmission device 301 transmits the object A1 to the detection range (the area between the two dotted lines) of the scanning component of the security inspection device. The scanning component of the security inspection device collects the object A1 within the detection range to obtain an initial image. The initial image is a columnar strip image, and the initial image includes at least one pixel value.

[0046] The correction parameters can be determined according to the first template image and the second template image, and the initial scan data can be corrected according to the first template image, the second template image and the correction parameters to obtain a strip image.

[0047] The first template image and the second template image are column-type strip images. The first template image and the second template image each include at least one pixel value.

[0048] Assume that the first template image includes at least one pixel value The second template image includes at least one pixel value of .

[0049] The following formula 1 can be used to determine the initial image and the Correction parameters for pixel values:

[0050]

[0051] in, For the strip image, Correction parameters for pixel values; is the first template image pixel value; is the first pixel value; Take 1, 2, ..., .

[0052] According to the above formula 1, the correction parameters can be determined to include .

[0053] The following formula 2 can be used to determine the strip image. Pixel values:

[0054]

[0055] in, For the strip image, pixel value; is the correction scaling factor. The explanations of other parameters are as above.

[0056] The correction scaling factor can be determined according to the detection range of the scanning component. The correction scaling factor can be set in advance and stored in a preset storage space of the security inspection device.

[0057] For example, the correction scaling factor may be 65535.

[0058] According to the above formula 2, it can be determined that at least one pixel value in the strip image corresponding to the initial image includes .

[0059] Exemplarily, at least one pixel value in the initial image, the first template image, the second template image, and the strip image may be a grayscale value.

[0060] S202: Determine at least one reference image corresponding to the strip image, and perform splicing processing on the strip image and the at least one reference image to obtain an intermediate image corresponding to the detection object.

[0061] Among them, the reference image includes feature information for assisting in segmenting the strip image. For example, the reference image may include a strip image obtained by capturing an object on a conveying device before capturing the current strip image, that is, a historical strip image captured before the current strip image.

[0062] In an optional implementation, at least one reference image corresponding to the strip image may be determined in the following manner: determining whether there is a preset number of historical strip images in the first buffer where the strip image is located, wherein the preset number refers to the number of reference images required in the segmentation process of the current strip image, and the specific value may be reasonably determined according to actual conditions; the acquisition time of the historical strip images is earlier than the acquisition time of the currently acquired strip image, and the historical strip images are cached in sequence according to the acquisition time, and the historical strip images are subsequently acquired in the order of the acquisition time, so as to ensure as much as possible that the historical strip images participating in the segmentation process of the current strip image and the current strip image correspond to the same detection object;

[0063] If it is determined that there are a preset number of historical strip images in the first buffer where the strip image is located, the preset number of historical strip images in the first buffer are determined as at least one reference image; in the order of acquisition time, the last historical strip image in the historical strip images is the previous strip image of the currently acquired strip image;

[0064] and / or

[0065] If it is determined that the number of historical strip images in the first buffer is less than the preset number, determine whether there are historical strip images in the second buffer; the historical strip images stored in the second buffer are obtained by copying all the historical strip images in the first buffer to the second buffer after the first buffer is full; if there are historical strip images in the second buffer, and the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer meet the preset number requirements, the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer are determined as at least one reference image. The historical strip images in the second buffer are also stored in sequence according to the acquisition time, and the acquisition time is earlier than the historical strip images in the first buffer. The second number of historical strip images in the second buffer, the first number of historical strip images in the first buffer, and the current strip image are sequentially spliced ​​in the acquisition order to reproduce the process of the detection object being scanned by the scanning component.

[0066] If it is determined that there is no historical strip image in the second buffer, a third number of preset images and a first number of historical strip images in the first buffer are determined as at least one reference image; the sum of the third number and the first number is equal to the preset number. At this time, the third number of preset images can be some pre-set general images, such as blank images.

[0067] The scan data obtained after the security inspection device scans and inspects the inspection object can be stored in a cache, for example, a first buffer and a second buffer are set in the embodiment of the present application. The currently acquired strip image and at least one reference image are cached in the first buffer and the second buffer in turn for subsequent processing. In this way, it is avoided that at least one reference image needs to be obtained from the memory each time at least one reference image corresponding to the strip image is obtained, saving the overhead of multiple memory copies.

[0068] The first buffer is used to store the strip images acquired within the most recent preset number of times.

[0069] For example, assuming that the strip image currently acquired is image A', the preset number of required reference images is 20. The current strip image A' is stored in the first buffer, and it is determined that there are 30 historical strip images in the first buffer, namely, images A1 to A30. According to the acquisition time of the 30 historical strip images, it is determined that at least one reference image includes images A11 to A30 in the first buffer. Among them, images A11 to A30 are strip images cached in the first buffer in the order of the acquisition time. The last historical strip image A30 is the previous strip image of the currently acquired strip image A'.

[0070] The strip image and at least one reference image are spliced ​​so that when the target model is processed, the strip image can be segmented according to the feature information of the strip image and at least one reference image to obtain the target area and non-target area determined in the strip image, thereby avoiding the situation where the feature information of a single strip image is small, resulting in difficulty in model segmentation processing and inaccurate segmentation results.

[0071] For example, according to the above example, the strip image is determined to be image A', and at least one reference image corresponding to the strip image includes images A11 to A30. Images A11 to A30 and the currently acquired strip image can be spliced ​​according to the acquisition time to obtain an intermediate image, and then the target model is used to segment the intermediate image.

[0072] S203: Process the intermediate image using the target model to determine the target area and the non-target area in the strip image.

[0073] The target region refers to the image subregion that includes the detected object.

[0074] The target model has an image segmentation function, for example, the target model may include at least one downsampling module, at least one upsampling module, and a segmentation module. Exemplarily, the target model may be implemented based on a U-Net model or other network models, which may be reasonably determined according to image processing requirements.

[0075] In an optional embodiment, the intermediate image can be processed by the target model in the following manner to determine the target area and the non-target area in the strip image: down-sample the intermediate image by at least one down-sampling module to obtain a first sampling image, the size of which is smaller than the size of the intermediate image; up-sample the first sampling image by at least one up-sampling module and fuse the up-sampled image with the intermediate image to obtain a second sampling image, the size of which is the same as the size of the intermediate image; segment the second sampling image by a segmentation module to determine the target area and the non-target area in the strip image. The segmentation module can be implemented based on a traditional image segmentation algorithm such as a threshold segmentation algorithm, or can be implemented based on a related segmentation function in a neural network.

[0076] Next, combine Figure 4 , describe the target model. Figure 4 A schematic diagram of a target model provided in an embodiment of the present application. Figure 4, including a target model 401. The target model 401 includes an input layer, at least one downsampling module, at least one upsampling module, a segmentation module, and an output layer. The number of at least one downsampling module and at least one upsampling module is M. M can be determined according to the data type, usage scenario, etc. specifically processed by the target model 401. At least one downsampling module of the target model 401 is used to extract feature information of the intermediate image and gradually reduce the resolution to obtain a first sampled image. At least one upsampling module of the target model 401 gradually restores the resolution of the first sampled image. The target model 401 fuses the first sampled image processed by at least one upsampling module with the intermediate image to obtain a second sampled image. The segmentation module of the target model 401 performs refined segmentation on the second sampled image and outputs the processing result through the output layer of the target model 401. The processing result includes the target area and the non-target area in the strip image.

[0077] The non-target area is used to indicate the non-object area in the strip image, such as the conveyor belt, blank area, etc. The target area is used to indicate the area corresponding to each detection object in the strip image.

[0078] The image processing method provided in the embodiment of the present application obtains a strip image of the detection object. Determine at least one reference image corresponding to the strip image. And perform splicing processing on the strip image and at least one reference image to obtain an intermediate image. The intermediate image is processed by a target model to determine the target area and the non-target area in the strip image. In the above process, the target area and the non-target area can be determined in the strip image based on the feature information in the strip image and at least one reference image, thereby avoiding the situation where the result of the image segmentation processing by the preset algorithm has errors when the security inspection equipment collects less feature information corresponding to the strip image, thereby improving the accuracy of image processing.

[0079] Based on any of the above embodiments, Figure 5 , the detailed process of image processing is explained.

[0080] Figure 5 This is a flowchart of another image processing method provided in an embodiment of the present application. Figure 5 , the method comprising:

[0081] S501, obtaining a strip image of the detection object.

[0082] It should be noted that the execution process of S501 can refer to S201 and will not be repeated here.

[0083] S502: Determine whether there are a preset number of historical strip images in the first buffer where the strip images are located.

[0084] The memory size of the first buffer and the second buffer is the same. At least one strip image can be stored, and the number of the at least one strip image is the target number corresponding to the memory size. The target number can be the preset number+1.

[0085] After obtaining the strip image of the detection object, the strip image can be stored in the first buffer. The strip image can be stored in the first buffer in the following manner: determine whether the strip image is stored in the first buffer; if so, determine the storage quantity of the strip image stored in the first buffer, and store the strip image in the first buffer according to the storage quantity; if not, store the strip image in the storage space corresponding to the first address of the first buffer. The intermediate storage image is the strip image stored in the first buffer most recently.

[0086] The strip image can be stored in the first buffer according to the storage quantity in the following manner: determine whether the storage quantity is equal to N, where N is the target quantity corresponding to the memory size of the first buffer; if so, copy all the historical strip images already stored in the first buffer to the second buffer, and store the currently acquired strip image in the storage space corresponding to the first address in the first buffer; if not, determine the first strip image that was most recently stored in the first buffer, and store the strip image in the first buffer according to the tail address of the first strip image.

[0087] For example, assume that the number of targets is 20. After acquiring the strip image B11 of the detection object, it is determined that there are strip images stored in the first buffer. It is determined that the number of images stored in the first buffer is 10, and these 10 strip images are strip images B1 to B10. It is determined that the first strip image most recently stored in the first buffer is strip image B10. Therefore, according to the tail address of strip image B10, strip image B11 is stored in the first buffer.

[0088] If the storage quantity of the images stored in the first buffer is the target quantity, it can be determined that there is no remaining storage space in the first buffer.

[0089] For example, assume that the target number is 20. After acquiring the strip image B21 of the detection object, it is determined that there are strip images stored in the first buffer. It is determined that the storage number of images stored in the first buffer is 20, and these 20 strip images are strip images B1 to strip images B20. If the first number is determined to be the target number, it can be determined that there is no remaining storage space in the first buffer. At this time, the strip images B1 to strip images B20 in the first buffer are all copied to the second buffer, and the currently acquired strip image B21 is stored in the storage space corresponding to the first address in the first buffer.

[0090] S503: If yes, determine a preset number of historical strip images in the first buffer as at least one reference image.

[0091] According to the order of acquisition time, the last historical strip image in the historical strip images is the previous strip image of the currently acquired strip image.

[0092] If the number of historical strip images stored in the first buffer is a preset number, the preset number of historical strip images in the first buffer can be determined as at least one reference image. If the number of historical strip images stored in the first buffer is greater than the preset number, the preset number of historical strip images most recently stored in the first buffer are determined as at least one reference image in the order of acquisition time.

[0093] Determining the strip image in the first buffer as the reference image can more accurately provide feature information near the detection object in the strip image.

[0094] S504: If the number of historical strip images in the first buffer is less than a preset number, determine whether there is a historical strip image in the second buffer.

[0095] The historical strip images stored in the second buffer are obtained by copying all the historical strip images in the first buffer to the second buffer after the first buffer is full.

[0096] S505: If there are historical strip images in the second buffer, and a second number of historical strip images in the second buffer and a first number of historical strip images in the first buffer meet a preset quantity requirement, the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer are determined as at least one reference image.

[0097] The sum of the first quantity and the second quantity is the preset quantity.

[0098] The first number of historical strip images in the first buffer are not historical strip images stored in the second buffer.

[0099] Next, combine Figure 6 , a process of determining at least one reference image in a first buffer and a second buffer is described. Figure 6 A schematic diagram of a process for determining at least one reference image provided by an embodiment of the present application. Figure 6, including a first buffer 601 and a second buffer 602. The first buffer 601 and the second buffer 602 can be set in a security inspection device or a processing device. The memory size of the first buffer 601 and the second buffer 602 is the same. The area in the dotted box in each process is the image area corresponding to the intermediate image. The intermediate image includes a strip image and N reference images, where N is a preset number. When N+1 images are stored in the first buffer 601, each historical strip image in the first buffer 601 is copied to the second buffer 602.

[0100] Please refer to process 1, the second buffer 602 stores L images copied from the first buffer 601. The strip image of the detection object currently obtained is image L+1, and image L+1 is stored in the first buffer 601 after the copy operation is completed. The second buffer 602 and the first buffer 601 can both use an overwrite write method to store the strip images that need to be stored. For example, image L+1 can be written into the storage space corresponding to the first address of the first buffer 601 in an overwrite write method to replace image 1 originally stored in the first buffer 601. The preset number is determined to be L-1. At this time, the first number of historical strip images stored in the first buffer 601 is 0. Therefore, it is necessary to determine L-1 images as at least one reference image in the second buffer 602. According to the order of acquisition time, it is determined that the most recently acquired L-1 images in the second buffer 602 include image 2 to image L. Therefore, it is determined that at least one reference image includes image 2 to image L in the second buffer 602.

[0101] Please refer to process 2, the second buffer 602 stores L images copied from the first buffer 601. At this time, the strip image to be processed is continued to be obtained as image L+2, and image L+2 is stored in the first buffer 601. Since the storage capacity in the first buffer 601 is full, and there is image L+1 which is the image acquired at the last acquisition moment. Therefore, image L+2 is stored in the storage space corresponding to the tail address of image L+1 in the first buffer 601 to replace image 2 originally stored in the first buffer 601. The preset number is determined to be L-1, and in the first buffer 601, it is determined that image L+1 and image L+2 are not historical strip images stored in the second buffer 602. Since image L+2 is the currently stored strip image, the historical strip image stored in the first buffer 601 is image L+1, and the first number of historical strip images stored in the first buffer 601 is 1. Therefore, it is determined that the first number of historical strip images in the first buffer 601 is image L+1. At this time, it is necessary to determine L-2 images as at least one reference image in the second buffer 602. According to the order of acquisition time, the L-2 historical strip images recently acquired in the second buffer 602 are determined to include image 3 to image L. Therefore, the at least one reference image is determined to include image L+1 in the first buffer 601 and image 3 to image L in the second buffer 602.

[0102] S506: If there is no historical strip image in the second buffer and the number of historical strip images in the first buffer is less than a preset number, determine a third number of preset images and the first number of historical strip images in the first buffer as at least one reference image.

[0103] The sum of the third quantity and the first quantity is equal to the preset quantity.

[0104] The pixel value of each pixel in the preset image is a preset pixel value. For example, the preset pixel value is 255.

[0105] Next, combine Figure 7 , a process of determining at least one reference image in a first buffer is described. Figure 7 A schematic diagram of another process for determining at least one reference image provided by an embodiment of the present application. Figure 7, including a first buffer 701 (diagonal line filling area) and a second buffer 702 (horizontal line filling area). The first buffer 701 and the second buffer 702 can be set in the security inspection device, and the memory size of the first buffer 701 and the second buffer 702 is the same. The first buffer 701 and the second buffer 702 can store at least one image, and the number of at least one image is the target number X. The area in the dotted box in each process is the image area corresponding to the intermediate image. The intermediate image includes a strip image and N reference images, and N is a preset number.

[0106] Please refer to process 1, obtain the strip image to be processed as image 1, and store image 1 in the first buffer 701. At this time, no image is stored in the first buffer 701. Therefore, image 1 is stored in the storage space corresponding to the first address of the first buffer 701. Since image 1 is the currently stored image, it is determined that the first number of historical strip images stored in the first buffer 701 is 0, then it can be determined that the first number of images stored in the first buffer 701 is less than the preset number, and there is no historical strip image in the second buffer 702. The preset number N obtained in the preset storage space is 31. Therefore, it is determined that at least one reference image includes 31 preset images.

[0107] Please refer to process 2, continue to obtain the strip image to be processed as image 2, and store image 2 in the first buffer 701. At this time, image 1 is stored in the first buffer 701. Therefore, image 2 is stored in the storage space corresponding to the tail address of image 1 in the first buffer 701. It is determined that the historical strip image stored in the first buffer 701 is image 1, and the first number of stored historical strip images is 1. Therefore, it is determined that at least one reference image includes image 1 in the first buffer 701 and 30 preset images. And so on, until 32 images are stored in the first buffer 701 (that is, the first buffer 701 is fully stored).

[0108] Please refer to process 3, continue to obtain the strip image to be processed until the strip image obtained is image 32, and store image 32 in the first buffer 701. At this time, images 1 to 31 are stored in the first buffer 701. Therefore, image 32 is stored in the storage space corresponding to the tail address of image 31 in the first buffer. It is determined that the historical strip images stored in the first buffer 701 are images 1 to 31, and the first number of the stored historical strip images is 31. Therefore, it is determined that at least one reference image includes images 1 to 31 in the first buffer 701.

[0109] S507: Perform splicing processing on the strip image and at least one reference image to obtain an intermediate image corresponding to the detection object.

[0110] For example, according to the above Figure 6 As shown in process 1, it is determined that at least one reference image corresponding to image L+1 includes image 2 to image L. Therefore, the strip image and at least one reference image are spliced ​​to obtain an intermediate image including image 2 to image L+1. The intermediate image can be the above Figure 6 In process 1 shown, the area corresponding to the dotted box.

[0111] For example, according to the above Figure 7 As shown in process 1, determining at least one reference image corresponding to image 1 includes determining that at least one reference image includes 31 preset images. Therefore, the strip image and the at least one reference image are spliced ​​to obtain an intermediate image including 31 preset images and image 1.

[0112] S508. Perform feature extraction processing on the intermediate image through at least one down-sampling module of the target model to obtain a first sampled image.

[0113] Before the intermediate image is processed by the target model, the intermediate image is processed by at least two image processing methods to obtain multiple processed input images, so that the multiple input images are input into the target model for processing; the at least two image processing methods include at least two of the following: image noise reduction processing, image detail contrast enhancement processing and image edge enhancement processing.

[0114] The image detail contrast enhancement process can be an image grayscale stretching process, an image histogram equalization process, etc. The image edge enhancement process can enhance the wrapping edge of the detected object. By processing the intermediate image by at least two image processing methods, the model input image can be a multi-channel image, which can enhance the image features of the area of ​​interest on the image, and help improve the accuracy of the model processing.

[0115] By processing the intermediate image using at least two image processing methods, image features in the intermediate image can be enhanced, making the output result after the target model processing more accurate.

[0116] The intermediate image can be downsampled by at least one downsampling module of the target model to obtain a first sampling image in the following manner: the intermediate image is subjected to feature extraction by the first downsampling module to obtain the first downsampling image; the x-1th downsampling image is subjected to feature extraction by the xth downsampling module to obtain the xth downsampling image, where x is 2, 3, ..., S in sequence. When x is S, the xth downsampling image is the first sampling image, the size of the first sampling image is smaller than the size of the intermediate image, and S is the number of downsampling modules.

[0117] The downsampling module may include a first convolutional activation layer and a maximum pooling layer.

[0118] The x-1th down-sampled image may be obtained by performing feature extraction processing on the x-1th down-sampled image through the x-th down-sampled module in the following manner: performing convolution processing on the x-1th down-sampled image through a convolution activation layer to obtain at least one first feature corresponding to the x-1th down-sampled image; and processing the at least one first feature through a maximum pooling layer to obtain the x-th down-sampled image.

[0119] Optionally, a convolutional layer can be added before the first downsampling module.

[0120] S509: Upsample the first sampled image through at least one upsampling module of the target model and fuse the upsampled image with the intermediate image to obtain a second sampled image.

[0121] The first sampled image can be upsampled by at least one upsampling module of the target model to obtain an upsampled image in the following manner: the first sampled image is upsampled by the first upsampling module to obtain the first upsampled image; the y-1th upsampled image is upsampled by the yth downsampling module to obtain the yth upsampled image, where y is 2, 3, ..., S in sequence. When y is S, the yth upsampled image is the upsampled image, and S is the number of upsampling modules.

[0122] The number of upsampling modules is the same as the number of downsampling modules. The upsampling module may include a second convolution activation layer, a convolution transpose layer, and a fully connected layer.

[0123] The y-1th upsampled image can be upsampled by the y-th downsampling module in the following manner to obtain the y-th upsampled image: the y-1th upsampled image is convolved by a convolution activation layer to obtain at least one second feature corresponding to the y-1th upsampled image; at least one second feature is upsampled by a convolution transpose layer to obtain at least one third feature corresponding to each second feature; at least one third feature corresponding to each second feature is processed by a fully connected layer to obtain the y-th upsampled image.

[0124] The up-sampled image is fused with the intermediate image to obtain a second sampled image. The size of the second sampled image is the same as that of the intermediate image.

[0125] S510 , performing segmentation processing on the second sample image by using a segmentation module of the target model to determine a target area and a non-target area in the strip image.

[0126] The segmentation module may include a third convolutional activation layer, a convolutional layer, and a segmentation layer.

[0127] The segmentation module can be implemented based on threshold segmentation algorithm, traditional segmentation algorithm, neural network model, etc.

[0128] The second sampling image can be segmented by the segmentation module of the target model to determine the target area and the non-target area in the strip image in the following manner: the second sampling image is processed by the third convolution activation layer to obtain at least one fourth feature corresponding to the second sampling image; the at least one fourth feature is processed by the convolution layer to obtain an adjusted image; the adjusted image is segmented by the segmentation layer to determine the target area and the non-target area in the strip image.

[0129] If the image output by the segmentation layer is a single-channel image, each pixel value in the image is used to indicate the area of ​​the pixel. If the pixel value of the pixel is a first preset value, it can be determined that the pixel is located in the target area in the strip image. If the pixel value of the pixel is a second preset value, it can be determined that the pixel is located in the non-target area in the strip image.

[0130] For example, the first preset value is 0, and the second preset value is 1. For pixel 1 in the target image, the pixel value of pixel 1 is determined to be 0. Therefore, it can be determined that pixel 1 is located in the target area in the strip image. For pixel 10 in the target image, the pixel value of pixel 10 is determined to be 1. Therefore, it can be determined that pixel 10 is located in the non-target area in the strip image.

[0131] If the image output by the segmentation layer is a multi-channel image, in each channel image, each pixel value is used to indicate the area of ​​the pixel. For any pixel point, the pixel value of the pixel point in each channel image is determined, and the target pixel value corresponding to the pixel point is determined according to the pixel value of the pixel point in each channel image. If the target pixel value is a first preset value, it can be determined that the pixel is located in the target area in the strip image. If the target pixel value is a second preset value, it can be determined that the pixel is located in the non-target area in the strip image.

[0132] For example, the number of channels of the target image is 2, and the images corresponding to each channel are channel image 1 and channel image 2. For pixel 1, the pixel value of pixel 1 in channel image 1 is a1, and the pixel value of pixel 1 in channel image 1 is a2. According to a1 and a2, the target pixel value of pixel 1 is determined, and according to the target pixel value of pixel 1, the area corresponding to pixel 1 in the strip image is determined.

[0133] After the target area and the non-target area are determined in the strip image, they can be stored or displayed on a display so that the user can perform subsequent operations according to the display.

[0134] The target model provided by the embodiment of the present application has a smaller number of parameters than the downsampling operation of the classic Unet network. The use of the LeakyRelu activation function can avoid the problem that the gradient of some network parameters is zero and invalid at a larger or smaller learning rate. Using the convolutional transpose layer instead of the upsampling operation can make the upsampling operation always be inferred on the device side during the inference process, saving the copying overhead between the host and the device side.

[0135] The image processing method provided by the embodiment of the present application obtains a strip image of the detection object. Determine whether there is a preset number of historical strip images in the first buffer where the strip image is located. If so, determine the preset number of historical strip images in the first buffer as at least one reference image. If the number of historical strip images in the first buffer is less than the preset number, determine whether there is a historical strip image in the second buffer. If there is a historical strip image in the second buffer, and the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer meet the preset number requirements, then the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer are determined as at least one reference image. If there is no historical strip image in the second buffer, and the number of historical strip images in the first buffer is less than the preset number, the third number of preset images and the first number of historical strip images in the first buffer are determined as at least one reference image. The strip image and at least one reference image are spliced ​​to obtain an intermediate image. The intermediate image is processed by the target model to determine the target area and the non-target area in the strip image. In the above process, the target area and the non-target area can be determined in the strip image based on the feature information in the strip image and at least one reference image, so as to avoid the situation where the security inspection equipment collects less feature information corresponding to the strip image and the result of image segmentation processing using the preset algorithm contains errors, thereby improving the accuracy of image processing.

[0136] Based on any of the above embodiments, Figure 8 , the process of image processing is illustrated with examples.

[0137] Figure 8 This is a schematic diagram of the image processing process provided by the embodiment of the present application. Figure 8 , including a security inspection device 801. The security inspection device 801 may be an X-ray security inspection device. The security inspection device 801 is provided with a scanning component, an image processing device, a first buffer and a second buffer. The security inspection device 801 obtains an initial image acquired by the scanning component from an object on a conveying device. The security inspection device 801 obtains a first template image and a second template image, and performs correction processing on the initial image according to the first template image and the second template image, and obtains a strip image to be processed as image B3.

[0138] The image processing device of the security inspection device 801 determines that image B1 and image B2 are stored in the first buffer. The image processing device of the security inspection device 801 stores image B3 in the storage area corresponding to the tail address of image B2 in the first buffer. The image processing device of the security inspection device 801 determines that there is no historical strip image in the second buffer. Therefore, the image processing device of the security inspection device 801 determines that the historical strip images stored in the first buffer are image B1 and image B2, and the first number of stored historical strip images is 2. The image processing device of the security inspection device 801 obtains the preset number of 63. If the image processing device of the security inspection device 801 determines that the first number is less than the preset number and there is no historical strip image in the second buffer, it can be determined that at least one reference image includes image B1, image B2, and 61 preset images. The pixel value of each pixel in the preset image is 255.

[0139] The image processing device of the security inspection device 801 performs stitching processing on at least one reference image to obtain an intermediate image B. The image processing device of the security inspection device 801 performs image noise reduction processing, image detail contrast enhancement processing and image edge enhancement processing on the intermediate image B to obtain an intermediate processed image B1. The image processing device of the security inspection device 801 obtains a target model, and processes the intermediate processed image B1 through the target model to obtain an image B31 corresponding to the target image of image B3, and image B31 is a single-channel image. The image processing device of the security inspection device 801 determines whether the pixel value of each pixel in image B31 is 0 or 1. Among them, the pixel value 0 is used to indicate that the corresponding pixel in the strip image is located in the target area, and the pixel value 1 is used to indicate that the corresponding pixel in the strip image is located in the non-target area.

[0140] The image processing method provided in the embodiment of the present application obtains a strip image of the detection object. Determine at least one reference image corresponding to the strip image. And perform splicing processing on the strip image and at least one reference image to obtain an intermediate image. The intermediate image is processed by a target model to determine the target area and the non-target area in the strip image. In the above process, the target area and the non-target area can be determined in the strip image based on the feature information in the strip image and at least one reference image, thereby avoiding the situation where the result of the image segmentation processing by the preset algorithm has errors when the security inspection equipment collects less feature information corresponding to the strip image, thereby improving the accuracy of image processing.

[0141] Fig. 9 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Fig. 9 , the image processing device 10 may include:

[0142] An acquisition module 11 is used to acquire a strip image of the detection object, wherein the strip image is obtained by scanning and detecting the detection object with a security inspection device;

[0143] A first determination module 12 is used to determine at least one reference image corresponding to the strip image and to perform splicing processing on the strip image and the at least one reference image to obtain an intermediate image, wherein the reference image includes feature information for assisting in segmenting the strip image;

[0144] The first processing module 13 is used to process the intermediate image through a target model to determine a target area and a non-target area in the strip image, wherein the target area refers to an image sub-area including the detection object.

[0145] In a possible implementation manner, the first determining module 12 is specifically configured to:

[0146] Determine whether there are a preset number of historical strip images in the first buffer where the strip image is located; the acquisition time of the historical strip images is earlier than the acquisition time of the currently acquired strip image, and cache them in sequence according to the acquisition time;

[0147] If yes, a preset number of historical strip images in the first buffer are determined as the at least one reference image; in the order of acquisition time, the last historical strip image in the historical strip images is the previous strip image of the currently acquired strip image;

[0148] and / or

[0149] If the number of historical strip images in the first buffer is less than the preset number, determining whether there is a historical strip image in the second buffer; the historical strip images stored in the second buffer are obtained by copying all the historical strip images in the first buffer to the second buffer after the first buffer is full;

[0150] If there are historical strip images in the second buffer, and the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer meet the preset quantity requirement, the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer are determined as the at least one reference image.

[0151] In a possible implementation manner, the first processing module 13 is specifically configured to:

[0152] Performing feature extraction processing on the intermediate image by the at least one down-sampling module to obtain a first sampled image, wherein the size of the first sampled image is smaller than the size of the intermediate image;

[0153] Performing upsampling processing on the first sampled image and fusing the upsampled image with the intermediate image through the at least one upsampling module to obtain a second sampled image, wherein the size of the second sampled image is the same as that of the intermediate image;

[0154] The second sample image is segmented by the segmentation module to determine the target area and the non-target area in the strip image.

[0155] In a possible implementation, the downsampling module includes a first convolution activation layer and a maximum pooling layer;

[0156] The upsampling module includes a second convolution activation layer, a convolution transpose layer and a fully connected layer.

[0157] In a possible implementation manner, the acquisition module 11 is specifically used for:

[0158] Obtaining initial scanning data collected by security inspection equipment during the scanning and inspection process of the inspection object;

[0159] Acquire a first template image and a second template image, wherein the first template image is image data collected by the detector in the security inspection device when the radiation source in the security inspection device is turned on and there is no detection object; and the second template image is image data collected by the detector in the security inspection device when the radiation source in the security inspection device is not turned on and there is no detection object;

[0160] Based on the first template image and the second template image, the initial scan data is corrected to obtain the strip image.

[0161] The image processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0162] Fig.10 A schematic diagram of the structure of another image processing device provided in an embodiment of the present application. Fig. 9 Based on the examples shown, see Fig.10 The image processing device 10 also includes a second determining module 14 , a second processing module 15 and a storage module 16 .

[0163] Wherein, the second determining module 14 is used for:

[0164] If there is no historical strip image in the second buffer, a third number of preset images and a first number of historical strip images in the first buffer are determined as the at least one reference image; the sum of the third number and the first number is equal to the preset number.

[0165] The second processing module 15 is specifically used for:

[0166] Processing the intermediate image using at least two image processing methods to obtain a plurality of processed input images, and inputting the plurality of input images into the target model for processing;

[0167] The at least two image processing methods include at least two of the following: image noise reduction processing, image detail contrast enhancement processing and image edge enhancement processing.

[0168] The storage module 16 is specifically used for:

[0169] Determining whether there is remaining storage space in the first buffer;

[0170] If not, all the historical strip images stored in the first buffer are copied to the second buffer, and the currently acquired strip image is stored in the storage space corresponding to the first address in the first buffer.

[0171] The image processing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0172] Fig.11 This is a schematic diagram of the structure of the security inspection equipment provided in the embodiment of this application. Fig.11 The security inspection device 20 may include: a memory 21 and a processor 22. Exemplarily, the memory 21 and the processor 22 are interconnected via a bus 23.

[0173] The memory 21 is used to store program instructions;

[0174] The processor 22 is used to execute the program instructions stored in the memory, so as to enable the security inspection device 20 to execute the method shown in the above method embodiment.

[0175] The security inspection equipment provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0176] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above method.

[0177] The embodiment of the present application may also provide a computer program product, including a computer program, which can implement the above method when executed by a processor.

[0178] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The above-mentioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the above-mentioned memory (storage medium) includes: read-only memory (ROM), random access memory (RAM), flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0179] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0182] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0183] In the present application, the term "include" and its variations may refer to non-restrictive inclusion; the term "or" and its variations may refer to "and / or". The terms "first", "second", etc. in the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In the present application, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

Claims

1. An image processing method, characterized in that: include: Acquire a strip image of the detection object, wherein the strip image is obtained by scanning and detecting the detection object with a security inspection device; Determine at least one reference image corresponding to the strip image, and perform splicing processing on the strip image and the at least one reference image to obtain an intermediate image corresponding to the detection object; Processing the intermediate image by means of a target model to determine a target region and a non-target region in the strip image, wherein the target region refers to an image subregion including the detection object; Determining at least one reference image corresponding to the strip image includes: Determine whether there is a preset number of historical strip images in the first buffer where the strip image is located, and the acquisition time of the historical strip images is before the current strip image; If yes, determining a preset number of historical strip images in the first buffer as at least one reference image, wherein the reference image includes feature information for assisting in segmenting the strip images; and / or, if not, supplementing a second number of historical strip images from a second buffer to a preset number to determine the reference image; If there is no historical strip image in the second buffer, a third number of preset images are used to supplement the preset number to determine the reference image.

2. The method according to claim 1, characterized in that in, The historical strip images are cached in sequence according to the acquisition time, and the last historical strip image in the historical strip images is the previous strip image of the currently acquired strip image.

3. The method according to claim 1, characterized in that The method of supplementing the second number of historical strip images from the second buffer to a preset number to determine the reference image includes: If there are historical strip images in the second buffer, and the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer meet the preset quantity requirement, the second number of historical strip images in the second buffer and the first number of historical strip images in the first buffer are determined as the at least one reference image; the historical strip images stored in the second buffer are obtained by copying all the historical strip images in the first buffer to the second buffer after the first buffer is full.

4. The method according to claim 1, characterized in that: The using the third number of preset images to supplement the preset number to determine the reference image includes: A third number of preset images and a first number of historical strip images in the first buffer are determined as the at least one reference image.

5. The method according to claim 1, characterized in that The target model includes at least one down-sampling module, at least one up-sampling module, and a segmentation module; Processing the intermediate image by the target model to determine the target area and the non-target area in the strip image includes: Performing feature extraction processing on the intermediate image by the at least one down-sampling module to obtain a first sampled image, wherein the size of the first sampled image is smaller than the size of the intermediate image; Performing upsampling processing on the first sampled image and fusing the upsampled image with the intermediate image through the at least one upsampling module to obtain a second sampled image, wherein the size of the second sampled image is the same as that of the intermediate image; The second sample image is segmented by the segmentation module to determine the target area and the non-target area in the strip image.

6. The method according to claim 5, characterized in that The downsampling module includes a first convolution activation layer and a maximum pooling layer; The upsampling module includes a second convolution activation layer, a convolution transpose layer and a fully connected layer.

7. The method according to claim 1, characterized in that Before processing the intermediate image by the target model, the method further includes: Processing the intermediate image using at least two image processing methods to obtain a plurality of processed input images, and inputting the plurality of input images into the target model for processing; The at least two image processing methods include at least two of the following: image noise reduction processing, image detail contrast enhancement processing and image edge enhancement processing.

8. The method according to claim 1, characterized in that Get the strip image of the detected object, including: Obtaining initial scanning data collected by security inspection equipment during the scanning and inspection process of the inspection object; Acquire a first template image and a second template image, wherein the first template image is image data collected by the detector in the security inspection device when the radiation source in the security inspection device is turned on and there is no detection object; and the second template image is image data collected by the detector in the security inspection device when the radiation source in the security inspection device is not turned on and there is no detection object; Based on the first template image and the second template image, the initial scan data is corrected to obtain the strip image.

9. The method according to claim 1, characterized in that: The method further comprises: Determining whether there is remaining storage space in the first buffer; If not, all the historical strip images stored in the first buffer are copied to the second buffer, and the currently acquired strip image is stored in the storage space corresponding to the first address in the first buffer.

10. An image processing device, characterized in that: The device comprises: An acquisition module is used to acquire a strip image of the detection object, wherein the strip image is obtained by scanning and detecting the detection object with a security inspection device; a determination module, configured to determine at least one reference image corresponding to the strip image, and perform splicing processing on the strip image and the at least one reference image to obtain an intermediate image corresponding to the detection object; A processing module, configured to process the intermediate image by using a target model to determine a target area and a non-target area in the strip image, wherein the target area refers to an image sub-area including the detection object; The determination module is specifically used to determine whether there are a preset number of historical strip images in the first buffer where the strip image is located, and the acquisition time of the historical strip images is before the current strip image; If yes, determining a preset number of historical strip images in the first buffer as at least one reference image, wherein the reference image includes feature information for assisting in segmenting the strip images; and / or, if not, supplementing a second number of historical strip images from a second buffer to a preset number to determine the reference image; If there is no historical strip image in the second buffer, a third number of preset images are used to supplement the preset number to determine the reference image.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images