Image processing method and device, electronic equipment and computer readable storage medium
By generating synthetic images and using the target recognition difficulty parameter to determine the fusion of sensitive and background images, the problem of insufficient identification capability of prohibited items in customs inspections is solved, and the training effect is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NUCTECH JIANGSU CO LTD
- Filing Date
- 2019-11-05
- Publication Date
- 2026-07-31
AI Technical Summary
The types of prohibited items found during customs inspections are diverse, and they are concealed in various ways. The existing image database is insufficient to meet the needs of training customs inspectors, resulting in inadequate identification capabilities.
Based on the target recognition difficulty parameter, target sensitive images and background images are determined. Synthetic images are generated through image transformation and fusion, and the recognition difficulty is adjusted to improve the training effect.
The generated synthetic images can effectively improve the ability of customs inspectors to identify prohibited items and adapt to the training needs of prohibited items of different types and postures.
Smart Images
Figure CN112766294B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, and more particularly to an image processing apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] During customs inspections of prohibited items, image recognition is usually performed on the scanned images of the items to be inspected in order to determine whether any prohibited items are concealed within the items.
[0003] The process involves manually identifying images of the items to be inspected, requiring customs inspectors to have sufficient experience in image recognition (i.e., having identified different types and postures of prohibited items).
[0004] To train customs inspectors to identify prohibited items in images of items to be inspected, they can be trained using images of known items that contain prohibited items, thereby improving their identification capabilities.
[0005] Currently, due to the diverse types and concealment methods of prohibited items during customs inspections, the existing images in the customs image inspection database are insufficient to meet the training requirements for customs inspectors.
[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0007] This disclosure provides an image processing method and apparatus, an electronic device, and a computer-readable storage medium, which can determine a target-sensitive image and a target background image based on a target recognition difficulty parameter, and generate synthetic samples with different recognition difficulty coefficients based on the target-sensitive image and the target background image.
[0008] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.
[0009] This disclosure proposes an image recognition method, which includes: obtaining a target recognition difficulty parameter; determining a target sensitive image based on the target recognition difficulty parameter, and obtaining complexity information of the target sensitive image; obtaining a target background image based on the target recognition difficulty parameter and the complexity information of the target sensitive image; and fusing the target sensitive image and the target background image to generate a synthetic image.
[0010] In some embodiments, fusing the target sensitive image with the target background image to generate a composite image includes: obtaining the size of the target sensitive image and its edge pixels; determining a region to be fused in the target background image based on the size of the target sensitive image and its edge pixels; and replacing the pixels of the region to be fused with the pixels of the target sensitive image.
[0011] In some embodiments, determining a region to be fused in a target background image based on the size of the target sensitive image and its edge pixels includes: determining a sliding window in the target background image based on the size of the target sensitive image; and determining the region to be fused based on the edge pixels of the target sensitive image and the sliding window.
[0012] In some embodiments, the image processing method further includes: smoothing the edges of the fused target sensitive image.
[0013] In some embodiments, determining a target-sensitive image based on the target recognition difficulty parameter includes: constructing a target distribution function centered on the target recognition difficulty parameter; randomly sampling the target distribution function to obtain sampling results; and determining the target-sensitive image based on the sampling results.
[0014] In some embodiments, obtaining a target recognition difficulty parameter includes: obtaining the recognition result of a target object undergoing recognition training based on an image to be recognized, and the training difficulty parameter of the recognition training; determining the performance parameter of the target object based on the recognition result; determining the recognition capability parameter of the target object based on the training difficulty parameter of the recognition training and the performance parameter of the target object; and determining the target recognition difficulty parameter based on the recognition capability parameter of the target object.
[0015] In some embodiments, the image to be recognized includes a sample sensitivity map and a sample background map. The method further includes: acquiring complexity information of the sample sensitivity map and the sample background map; determining a synthesis difficulty parameter for synthesizing the image to be recognized based on the sample sensitivity map and the sample background map; determining a recognition difficulty parameter for the image to be recognized based on the complexity information of the sample sensitivity map and the sample background map and the synthesis difficulty parameter for synthesizing the image to be recognized based on the sample sensitivity map and the sample background map; and determining a training difficulty parameter for the recognition training based on the recognition difficulty parameter for the image to be recognized.
[0016] This disclosure provides an image processing apparatus, which includes: a parameter acquisition module, a target sensitive image acquisition module, a target background image acquisition module, and a synthesis module.
[0017] The parameter acquisition module can be configured to acquire a target recognition difficulty parameter; the target sensitive image acquisition module is configured to determine a target sensitive image based on the target recognition difficulty parameter and acquire the complexity information of the target sensitive image; the target background image acquisition module is configured to acquire a target background image based on the target recognition difficulty parameter and the complexity information of the target sensitive image; and the synthesis module is configured to fuse the target sensitive image and the target background image to generate a synthesized image.
[0018] This disclosure provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method described above.
[0019] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described in any of the preceding embodiments.
[0020] The image recognition method, apparatus, electronic device, and computer-readable storage medium provided in this disclosure determine a target-sensitive image based on a target recognition difficulty parameter. Then, based on the target recognition difficulty parameter and the target-sensitive image, a target background image with a high degree of consistency in complexity information with the target-sensitive image is determined. Finally, the target-sensitive image and the target background image are fused to obtain a synthetic sample with predictable recognition difficulty. The technical solution provided in this disclosure allows for the determination of a target-sensitive image and a target background image with similar backgrounds based on the target recognition difficulty parameter. Fusing the target-sensitive image with the target background image can effectively hide the target-sensitive image within the target background image. Furthermore, the recognition difficulty of the synthetic sample generated based on the target-sensitive image and the target background image can be adjusted by adjusting the target recognition difficulty parameter.
[0021] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description
[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. The drawings described below are merely some embodiments of this disclosure, and those skilled in the art will be able to derive other drawings from these drawings without any inventive effort.
[0023] Figure 1A schematic diagram of an exemplary system architecture that can be applied to the image processing method or image processing apparatus of the present disclosure is shown.
[0024] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment.
[0025] Figure 3 This is a schematic diagram illustrating a target-sensitive image according to an exemplary embodiment.
[0026] Figure 4 This is a schematic diagram illustrating a target background image according to an exemplary embodiment.
[0027] Figure 5 This is a schematic diagram illustrating a composite image according to an exemplary embodiment.
[0028] Figure 6 yes Figure 2 The flowchart of step S1 in some embodiments.
[0029] Figure 7 yes Figure 6 The flowchart of step S11 in an exemplary embodiment.
[0030] Figure 8 yes Figure 2 The flowchart of step S2 in an exemplary embodiment.
[0031] Figure 9 yes Figure 2 The flowchart of step S4 in some embodiments.
[0032] Figure 10 yes Figure 9 The flowchart of step S42 in some embodiments.
[0033] Figure 11 This is a flowchart illustrating a method for extracting sensitive images according to an exemplary embodiment.
[0034] Figure 12 This is a flowchart illustrating another image processing method according to an exemplary embodiment.
[0035] Figure 13 This is a schematic diagram illustrating an image processing system according to an exemplary embodiment.
[0036] Figure 14 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
[0037] Figure 15 This is a schematic diagram of the structure of a computer system applied to an image processing apparatus according to an exemplary embodiment. Detailed Implementation
[0038] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.
[0039] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.
[0040] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0041] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0042] In this specification, the terms “a,” “an,” “the,” “the,” and “at least one” are used to indicate the presence of one or more elements / components / etc.; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.
[0043] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0044] Figure 1A schematic diagram of an exemplary system architecture that can be applied to the image processing method or image processing apparatus of the present disclosure is shown.
[0045] like Figure 1 As shown, the system architecture 100 may include an image acquisition device 101, terminal devices 102 and 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the scanning device 101, terminal devices 102 and 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0046] Users can use the image acquisition device 101, terminal devices 102 and 103 to interact with the server 105 via the network 104 to receive or send messages, etc. The terminal devices 102 and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, virtual reality devices, smart home devices, etc.
[0047] The image acquisition device 101 may be, for example, a laser scanning device, and can be used to acquire images of a target object (including, but not limited to, binary images, color images, etc.). The target object may be, for example, a vehicle, a container, etc.
[0048] Server 105 can be a server that provides various services, such as a backend management server that supports the devices operated by users using terminal devices 101, 102, and 103. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal devices.
[0049] Server 105 may, for example, acquire a target recognition difficulty parameter; server 105 may, for example, determine a target sensitive image based on the target recognition difficulty parameter and acquire the complexity information of the target sensitive image; server 105 may, for example, acquire a target background image based on the target recognition difficulty parameter and the complexity information of the target sensitive image; server 105 may, for example, fuse the target sensitive image with the target background image to generate a composite image.
[0050] It should be understood that Figure 1 The number of scanning devices, terminal equipment, networks and servers in the diagram is merely illustrative. Server 105 can be a single physical server or a combination of multiple servers. Depending on actual needs, it can have any number of terminal equipment, networks and servers.
[0051] Figure 2This is a flowchart illustrating an image processing method according to an exemplary embodiment. The method provided in this disclosure can be processed by any electronic device with computing power, such as the one described above. Figure 1 In the embodiments, server 105 and / or terminal devices 102 and 103 are used as examples in the following embodiments, but this disclosure is not limited thereto.
[0052] During customs inspections, containers are typically scanned, and customs officers perform manual image recognition based on the scan results to determine whether contraband is concealed within the container. However, due to the limited experience of customs officers, they often cannot immediately identify contraband in the container.
[0053] To train customs inspectors to identify contraband, they are typically trained using images of known contraband items. For example, images of containers known to contain firearms concealed within cotton, apples, or timber can be used for training.
[0054] However, due to the varying postures of contraband in containers and the different types of items concealing contraband, there are not enough training samples in the customs image database to train customs inspectors.
[0055] In order to provide a sufficient number of samples for training customs inspectors, the following technical solutions are proposed in this disclosure.
[0056] Reference Figure 2 The image processing method provided in this disclosure may include the following steps.
[0057] In step S1, the target recognition difficulty parameter is obtained.
[0058] In some embodiments, the target recognition difficulty parameter can be a specified difficulty parameter value. The specified difficulty parameter value can be set according to the specific training scenario and the recognition ability of the customs inspection personnel to be trained. This disclosure does not impose any restrictions on this.
[0059] In some customs inspection fields, the target recognition difficulty parameter can be slightly greater than the ability of the customs inspector being trained to identify contraband. For example, assuming the ability of the customs inspector being trained to identify contraband is 2, the target recognition difficulty parameter can be set to 2*1.2, etc., so that the difficulty of identifying contraband in the generated synthetic image is slightly higher than the actual identification ability of the customs inspector being trained.
[0060] In step S2, a target-sensitive image is determined based on the target recognition difficulty parameter, and the complexity information of the target-sensitive image is obtained.
[0061] In some embodiments, images of sensitive items can be extracted from images containing sensitive items to form a sensitive image library. Before extracting the images of sensitive items, the images containing sensitive items can be transformed as required (e.g., enlargement, reduction, clockwise rotation, counterclockwise rotation, flipping, high-energy, low-energy, color inversion, grayscale, linear transformation, logarithmic transformation, histogram equalization, etc.), then the area and shape to be extracted (e.g., quadrilaterals, etc.) are selected, and finally the image pixels of the selected area are copied to generate a sensitive image in the required image format.
[0062] In some embodiments, the sensitive images generated according to the above steps can be defined with attributes (name, type, and description, etc.), and the sensitive images can be uploaded and saved to a sensitive item image library for later use.
[0063] In some embodiments, the sensitive object image library may include images of sensitive objects of different types, poses, and materials.
[0064] In some embodiments, the complexity information of a sensitive image can describe the identifiability of the sensitive image.
[0065] In some embodiments, the texture complexity of the sensitive image can be described by the second moment (variance) of the gray-level histogram of the sensitive image, and the texture complexity of the sensitive image can be normalized to the complexity information of the sensitive image, i.e., the complexity information.
[0066] Normalization, in particular, uses the invariant moments of an image to find a set of parameters that can eliminate the influence of other transformation functions on the image transformation. That is, it transforms the image into a unique standard form to resist affine transformations. Image normalization makes images resistant to geometric transformation attacks; it can identify the invariants in the image, thus revealing that these images were originally identical or part of a series.
[0067] In some embodiments, the target recognition difficulty parameter and the complexity information of sensitive images in the sensitive object image library can be used to determine, for example, Figure 3 The target sensitive image is shown, and the actual complexity information of the target sensitive image is obtained.
[0068] In some embodiments, a first difficulty parameter can be determined based on the target recognition difficulty parameter, and then the target sensitive image can be determined based on the first difficulty parameter.
[0069] For example, the first difficulty parameter can be determined according to formula (1), and the target sensitive image can be determined according to the first difficulty parameter.
[0070] y = 1 - (1 - x) 1 / 3(1)
[0071] Where y is the first difficulty parameter and x is the target recognition difficulty parameter.
[0072] In some embodiments, one or more images whose complexity information is close to the first difficulty parameter can be selected from among the sensitive images in the sensitive object image library as the target sensitive image.
[0073] It is understood that the complexity information of the target sensitive image may not be exactly the same as the first difficulty parameter, so after obtaining the target sensitive image, it is also necessary to obtain the actual complexity information of the target sensitive image.
[0074] In step S3, the target background image is obtained based on the target recognition difficulty parameter and the complexity information of the target sensitive image.
[0075] In some embodiments, the background image (e.g., including an image of the item to be inspected) can be transformed as required (e.g., zoom in, zoom out, rotate clockwise, rotate counterclockwise, flip, high energy, low energy, invert colors, grayscale, linear transformation, logarithmic transformation, histogram equalization, etc.), and then the attributes (name, type, and description, etc.) of the transformed background image can be defined, and the background image can be uploaded and saved to the background item image library for later use.
[0076] In some embodiments, a second difficulty parameter can be determined based on the target recognition difficulty parameter and the complexity information of the target sensitive image, and the target background image can be obtained based on the second difficulty parameter.
[0077] For example, the second difficulty parameter can be determined according to formula (2).
[0078] z = 1 - ((1-x) / (1-y)) 1 / 2 (2)
[0079] Where z is the second difficulty parameter, x is the target recognition difficulty parameter, and y is the complexity information of the target sensitive image.
[0080] In some embodiments, one or more images whose complexity information is close to the second difficulty parameter can be selected from the background images in the background object image library as such. Figure 4 The target background image shown.
[0081] In step S4, the target sensitive image is fused with the target background image to generate a synthetic image.
[0082] In some embodiments, techniques such as Poisson fusion can be used to combine... Figure 3The target sensitive image shown is similar to... Figure 4 The target background image shown is fused to generate, as shown Figure 5 The synthesized image shown. In some embodiments, by... Figure 5 Image recognition allows customs inspectors to pinpoint the location of sensitive item 501.
[0083] The image recognition method provided in this disclosure determines a target sensitive image based on a target recognition difficulty parameter. Then, based on the target recognition difficulty parameter and the target sensitive image, a target background image with a high degree of consistency in complexity information with the target sensitive image is determined. Finally, the target sensitive image and the target background image are fused to obtain a synthetic sample with predictable recognition difficulty. The technical solution provided in this disclosure allows for the determination of a target sensitive image and a target background image with similar backgrounds based on the target recognition difficulty parameter. Fusing the target sensitive image with the target background image effectively hides the target sensitive image within the target background image. Furthermore, the recognition difficulty of the synthetic sample generated from the target sensitive image and the target background image can be adjusted by adjusting the target recognition difficulty parameter. Finally, the technical solution provided in this disclosure allows for the fusion of sensitive items of different types and postures with images of different types of background items. In the field of customs inspection, the technical solution provided in this disclosure can generate images of different items to be inspected that contain contraband, which is beneficial for improving the image recognition capabilities of customs inspectors.
[0084] Figure 6 yes Figure 2 A flowchart of step S1 in some embodiments. See reference. Figure 6 Step S1 above may include the following steps.
[0085] In step S11, the recognition result of the target object based on the image to be recognized and the training difficulty parameter of the recognition training are obtained.
[0086] In some embodiments, the target object may refer to the person who needs to undergo image recognition training, such as a customs inspector.
[0087] In some embodiments, the customs inspector may be trained using the image to be identified and the identification result obtained by the customs inspector may be obtained, the identification result including the image of the image to be identified by the customs inspector that contains sensitive items, and the correctly identified image.
[0088] In some embodiments, the recognition results of the target object in the past n recognition training processes and the training difficulty parameters of the n recognition training processes can be obtained.
[0089] In step S12, the performance parameters of the target object are determined based on the recognition results.
[0090] In some embodiments, the recognition result includes the number K of images to be recognized in the recognition training, the number M of images including sensitive items found by the target object in the recognition training, and the number R of images including sensitive items correctly recognized by the target object.
[0091] In some embodiments, the performance parameters of the target object in the recognition training can be determined based on the number K of images to be recognized in the recognition training, the number M of images including sensitive items found by the target object in the recognition training, and the number R of images including sensitive items correctly recognized by the target object.
[0092] In some embodiments, the performance parameters of the target object in a single training session can be determined according to formula (3).
[0093]
[0094] Among them, c j R represents the performance parameters of the target object in its j-th recognition training. j M represents the number of images containing sensitive items that the target object correctly identified during its j-th recognition training. j K represents the number of images containing sensitive items identified by the target object during its j-th recognition training. j The number of images containing sensitive items to be identified in the j-th recognition training of the target object is represented by j, which is a positive integer greater than or equal to 1 and less than or equal to n, and n is the number of recognition training times for the selected target object.
[0095] In step S13, the recognition capability parameter of the target object is determined based on the training difficulty parameter of the recognition training and the performance parameter of the target object.
[0096] In some embodiments, the training difficulty parameter of the recognition training can be determined based on the complexity information x(i) of each image to be recognized in the recognition training.
[0097] For example, the training difficulty parameter of the recognition training can be determined by formula (4).
[0098]
[0099] Among them, b j The training difficulty parameter representing the j-th recognition training of the target object; x j(i) represents the complexity information of the i-th image to be recognized in the j-th recognition training of the target object; K represents the number of images to be recognized in the j-th recognition training of the target object; C is a time coefficient that can be set manually; T represents the time limit for the j-th recognition training of the target object. The shorter the time limit, the greater the difficulty of the recognition training; N represents the number of images to be recognized in the j-th recognition training of the target object; j is a positive integer greater than 1 and less than or equal to n, and n is the number of recognition training sessions of the selected target object; i is a positive integer greater than or equal to 1 and less than or equal to K.
[0100] In some embodiments, the recognition capability parameters of the target object can be determined based on the performance parameters of the target object in each recognition training and the recognition difficulty parameters of the corresponding recognition training.
[0101] For example, the weighted product of the target object's performance parameters in each recognition training session and the training difficulty parameters of that training session can be used as the target object's recognition ability parameter.
[0102] For example, the recognition capability parameter of the target object can be determined by formula (5).
[0103]
[0104] Where 'a' represents the target object's recognition capability parameter, and 'h' represents the target object's recognition capability parameter. j This represents the weights, where h1 to h2 are the weights. N You can set it to p in sequence. n-1 (n = 1, 2, 3, ..., N), c j b represents the performance parameters of the target object in its j-th recognition training. j The training difficulty parameter represents the j-th recognition training of the target object, where j is a positive integer greater than or equal to 1 and less than or equal to n, and n is the number of recognition training sessions for the selected target object.
[0105] In step S14, the target recognition difficulty parameter is determined based on the target object recognition capability parameter.
[0106] In some embodiments, the target recognition difficulty parameter can be determined based on the target object's recognition capability parameter. For example, the target recognition difficulty parameter can be set slightly larger than the recognition capability parameter, so that the recognition difficulty parameter of the final generated synthetic image is greater than the target object's recognition capability parameter. Training the target object's recognition based on the generated synthetic image can improve the target object's recognition capability to some extent.
[0107] The technical solution provided in this embodiment can determine the recognition ability of the target object based on the past recognition training results of the target object and the training difficulty parameter of the recognition training, and determine a target recognition difficulty parameter based on the recognition ability of the target object, so as to synthesize a synthetic sample image with known recognition difficulty based on the target recognition difficulty parameter.
[0108] Figure 7 yes Figure 6 A flowchart of step S11 in an exemplary embodiment. (See reference...) Figure 7 Step S11 above may include the following steps.
[0109] In some embodiments, Figure 6 The image to be identified in step S11 may include a sample sensitivity map and a sample background map. Figure 5 The synthesized image shown can be the image to be identified, which includes the sample sensitivity map 501 and the sample background map (the part other than 501).
[0110] In step S111, the synthesis difficulty parameter for synthesizing the image to be identified based on the sample sensitivity map and the sample background map is determined.
[0111] In some embodiments, a grayscale image of the sample sensitive item can be obtained using scanning technology as the sensitive image, and a grayscale image of the sample item can be obtained as the sample background image.
[0112] In some embodiments, the complexity information of a sensitive image can describe the identifiability of the sensitive image.
[0113] In some embodiments, the texture complexity of the sensitive image can be described by the second moment (variance) of the gray-level histogram of the sensitive image, and the texture complexity of the sensitive image can be normalized to the complexity information of the sensitive image, i.e., the complexity information.
[0114] In step S112, the synthesis difficulty parameter for synthesizing the image to be identified based on the sample sensitivity map and the sample background map is determined.
[0115] In some embodiments, a first matrix can be obtained based on the sample sensitivity map, and a second matrix can be obtained based on the sample background map.
[0116] In some embodiments, the sample sensitivity map can be negatively sliced and normalized to obtain the first matrix A.
[0117] In related technologies, performing negative image processing can be understood as performing color inversion processing on the image.
[0118] In some embodiments, the portion of the sample background image that overlaps with the sample sensitivity image (by fusing the sample sensitivity image and the sample background image, a portion of the original sample background image must overlap with the sample sensitivity image) can be subjected to negative slicing and normalization processing to obtain the second matrix B.
[0119] In some embodiments, after negativeing and normalization, a completely white sample sensitivity map can generate a zero matrix, and a completely black sample sensitivity map can generate a 1 matrix.
[0120] In some embodiments, the synthesis difficulty parameter of the image to be identified synthesized based on the sample sensitivity map and the sample background map can be determined according to formula (6).
[0121]
[0122] Wherein, P1 represents the synthesis difficulty parameter of the image to be identified synthesized based on the sample sensitivity map and the sample background map, A represents the first matrix, and B represents the second matrix.
[0123] In step S113, the recognition difficulty parameter of the image to be recognized is determined based on the complexity information of the sample sensitivity map and the sample background map, as well as the synthesis difficulty parameter of synthesizing the image to be recognized based on the sample sensitivity map and the sample background map.
[0124] In some embodiments, the recognition difficulty parameter of the image to be recognized can be determined according to formula (7).
[0125] L=1-(1-P2)*(1-P3)*(1-P1) (7)
[0126] Wherein, L can represent the recognition difficulty parameter of the image to be recognized, P1 can represent the synthesis difficulty parameter of the image to be recognized synthesized from the sample sensitivity image and the sample background image, P2 can represent the complexity information of the sample sensitivity image, and P3 can represent the complexity information of the sample background image.
[0127] In step S114, the training difficulty parameter for the recognition training is determined based on the recognition difficulty parameter of the image to be recognized.
[0128] In some embodiments, each recognition training of the target object may include M images, and the recognition difficulty parameter of each image is x(i) (i is a positive integer greater than or equal to 1 and less than or equal to M). Then the training difficulty parameter of each recognition training can be expressed as formula (4).
[0129] The technical solution provided in this embodiment determines the recognition difficulty parameter of the image to be recognized based on the complexity information of the sample sensitivity map and sample background map in the image to be recognized, and determines the training difficulty parameter of the recognition training based on the recognition difficulty parameter of each image to be recognized in the recognition training. This embodiment of the disclosure provides a method for quantifying the recognition difficulty of an image to be recognized and the training difficulty of the recognition training.
[0130] Figure 8 yes Figure 2 A flowchart of step S2 in an exemplary embodiment. (See reference...) Figure 8 Step S2 above may include the following steps.
[0131] In step S21, a target distribution function is constructed with the target recognition difficulty parameter as the center.
[0132] In some embodiments, the target distribution function may be a Gaussian distribution function, and the target distribution function may be constructed with the target recognition difficulty parameter as the center.
[0133] In step S22, the target distribution function is randomly sampled to obtain the sampling results.
[0134] In step S23, the target sensitive image is determined based on the sampling results.
[0135] In some embodiments, the sensitive image whose identification difficulty parameter is closest to the sampling result can be selected as the target sensitive image.
[0136] In some other embodiments, a sensitive image that is closest to the target recognition difficulty parameter can be directly identified as the target sensitive image.
[0137] This disclosure provides a method for constructing a target distribution function based on a target recognition difficulty parameter and sampling the target distribution function to determine a target-sensitive image.
[0138] Figure 9 yes Figure 2 A flowchart of step S4 in some embodiments. See reference. Figure 9 Step S4 above may include the following steps.
[0139] In step S41, the size of the target sensitive image and its edge pixels are obtained.
[0140] In some embodiments, the target sensitive image may be a regularly shaped image, such as a rectangle, rhombus, or circle.
[0141] In some embodiments, the shape and size of the target sensitive image and the edge pixels of the target sensitive image can be obtained based on the target sensitive image.
[0142] In step S42, the region to be fused is determined in the target background image based on the size of the target sensitive image and its edge pixels.
[0143] In some embodiments, a region in the target background image that is closer to the target sensitive image can be determined as the region to be fused based on the shape, size, and edge pixels of the target sensitive graphic.
[0144] In step S43, the pixels of the region to be fused are replaced with the pixels of the target sensitive image.
[0145] In some embodiments, the pixels of the region to be fused can be directly replaced with the pixels of the target sensitive image, and the edges of the fused target sensitive image can be smoothed.
[0146] In this embodiment, on the one hand, based on the size and edge pixels of the target sensitive image, a region to be fused is determined in the target background image that is the same size as the target sensitive image and has similar edge pixels. This method can select a pre-fusion region in the background image that can effectively hide the target sensitive image. On the other hand, the pixels of the region to be fused are replaced with those similar to those of the target sensitive image, and smoothing processing is used to process the edge regions of the fused target sensitive image. This makes the fusion fast and convenient, and also makes the fusion more natural and with a high degree of concealment.
[0147] Figure 10 yes Figure 9 A flowchart of step S42 in some embodiments. See reference. Figure 9 Step S24 above may include the following steps.
[0148] In step S421, a sliding window is determined in the target background image according to the size of the target sensitive image.
[0149] In some embodiments, a sliding window may be determined in the target background image according to the shape and size of the target sensitive image, and the shape and size of the sliding window may be the same as the shape and size of the target sensitive image.
[0150] In step S422, the region to be fused is determined based on the edge pixels of the target sensitive image and the sliding window.
[0151] In some embodiments, the sliding window can be slid across the non-blank area (i.e., the area excluding the blank area) of the target background image until the average value of the edge pixels of the target background image covered by the sliding window is equal to the average value of the edge pixels of the target sensitive image, and the area in the target background image covered by the sliding window at this time can be considered as the area to be fused.
[0152] Figure 11 This is a flowchart illustrating a method for extracting sensitive images according to an exemplary embodiment. (Reference) Figure 11 The method for extracting sensitive images may include the following steps.
[0153] In step S111, the image containing the sensitive item is opened.
[0154] In step S112, the image including the sensitive item is subjected to image transformation processing.
[0155] In step S113, the sensitive item is extracted from the image including the sensitive item to generate a sensitive image.
[0156] In step S114, the attribute information of the sensitive image is filled in, including the image name, description, image type, and image status.
[0157] In step S115, the sensitive image and its attribute information are uploaded to the sensitive item image library.
[0158] In this embodiment, the sensitive image is obtained by image cutout, which is both convenient and quick.
[0159] Figure 12 This is a flowchart illustrating another image processing method according to an exemplary embodiment. (Reference) Figure 12 The image processing method may include the following steps.
[0160] In step S121, the recognition result of the target object based on the image to be recognized and the training difficulty parameter of the recognition training are obtained.
[0161] In step S122, the performance parameters of the target object are determined based on the recognition results.
[0162] In step S123, the recognition capability parameter of the target object is determined based on the training difficulty parameter of the recognition training and the performance parameter of the target object.
[0163] In step S124, the target recognition difficulty parameter is determined based on the target object recognition capability parameter.
[0164] In step S125, a target sensitive image is determined based on the target recognition difficulty parameter, and the complexity information of the target sensitive image, as well as the size and edge pixels of the target sensitive image, are obtained.
[0165] In step S126, the target background image is obtained based on the target recognition difficulty parameter and the complexity information of the target sensitive image.
[0166] In step S127, a sliding window is determined in the background image according to the size of the sensitive image, and a region to be fused is determined in the background image according to the edge pixels of the sensitive image and the sliding window.
[0167] In step S128, the pixels of the region to be fused are replaced with the pixels of the sensitive image.
[0168] In step S129, the edges of the fused sensitive image are smoothed.
[0169] The image recognition method provided in this disclosure determines a target sensitive image based on a target recognition difficulty parameter. Then, based on the target recognition difficulty parameter and the target sensitive image, a target background image with a high degree of consistency in complexity information with the target sensitive image is determined. Finally, the target sensitive image and the target background image are fused to obtain a synthetic sample with predictable recognition difficulty. The technical solution provided in this disclosure allows for the determination of a target sensitive image and a target background image with similar backgrounds based on the target recognition difficulty parameter. Fusing the target sensitive image with the target background image effectively hides the target sensitive image within the target background image. Furthermore, the recognition difficulty of the synthetic sample generated from the target sensitive image and the target background image can be adjusted by adjusting the target recognition difficulty parameter. Finally, the technical solution provided in this disclosure allows for the fusion of sensitive items of different types and postures with images of different types of background items. In the field of customs inspection, the technical solution provided in this disclosure can generate images of different items to be inspected that contain contraband, which is beneficial for improving the image recognition capabilities of customs inspectors.
[0170] Figure 13 This is a schematic diagram illustrating an image processing system according to an exemplary embodiment. For example... Figure 13 As shown, the image processing system 130 may include: a login / logout module 131, an image extraction module 132, an image fusion module 133, and a fused image library 134.
[0171] In some embodiments, users can log in to the image processing system through the login / logout module 131.
[0172] In some embodiments, the image extraction module 132 can be used to extract images of sensitive items from images that include sensitive items.
[0173] In some embodiments, the image fusion module 133 can be used to obtain target recognition difficulty parameters; determine a target sensitive image based on the target recognition difficulty parameters, and obtain recognition parameters of the target sensitive image; obtain a target background image based on the target recognition difficulty parameters and the recognition parameters of the target sensitive image; and fuse the target sensitive image with the target background image to generate a synthetic image.
[0174] In some embodiments, the fused image library 134 may be used to store the synthesized image.
[0175] Figure 14 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. (Refer to...) Figure 14 The image processing apparatus 1400 provided in this embodiment may include: a parameter acquisition module 1401, a target sensitive image acquisition module 1402, a target background image acquisition module 1403, and a synthesis module 1404.
[0176] The parameter acquisition module 1401 can be configured to acquire a target recognition difficulty parameter; the target sensitive image acquisition module 1402 can be configured to determine a target sensitive image based on the target recognition difficulty parameter and acquire the complexity information of the target sensitive image; the target background image acquisition module 1403 can be configured to acquire a target background image based on the target recognition difficulty parameter and the complexity information of the target sensitive image; and the synthesis module 1404 can be configured to fuse the target sensitive image and the target background image to generate a synthesized image.
[0177] In some embodiments, the compositing module 1404 may include: an edge pixel acquisition submodule, a region to be merged acquisition submodule, and a pixel replacement module.
[0178] The edge pixel acquisition submodule can be configured to acquire the size of the target sensitive image and its edge pixels; the region to be merged acquisition submodule can be configured to determine the region to be merged in the target background image based on the size of the target sensitive image and its edge pixels; and the pixel replacement module can be configured to replace the pixels of the region to be merged with the pixels of the target sensitive image.
[0179] In some embodiments, the submodule for obtaining the region to be merged may include a sliding window determination unit and a region to be merged determination unit.
[0180] The sliding window determination unit can be configured to determine a sliding window in the target background image based on the size of the target sensitive image; the region to be fused determination unit can be configured to determine the region to be fused based on the edge pixels of the target sensitive image and the sliding window.
[0181] In some embodiments, the pixel replacement module may also be configured to smooth the edges of the fused target sensitive image.
[0182] In some embodiments, the target sensitive image acquisition module may include: a target distribution function determination submodule, a sampling submodule, and a target sensitive region determination submodule.
[0183] The target distribution function determination submodule can be configured to construct a target distribution function centered on the target recognition difficulty parameter; the sampling submodule can be configured to randomly sample the target distribution function to obtain sampling results; and the target sensitive region determination submodule can be configured to determine the target sensitive image based on the sampling results.
[0184] In some embodiments, the parameter acquisition module may include: a training parameter acquisition submodule, a performance parameter acquisition submodule, a recognition ability parameter acquisition submodule, and a target recognition difficulty parameter acquisition submodule.
[0185] The training parameter acquisition submodule can be configured to acquire the recognition result of the target object based on the image to be recognized and the training difficulty parameter of the recognition training; the performance parameter acquisition submodule can be configured to determine the performance parameter of the target object based on the recognition result; the recognition ability parameter acquisition submodule can be configured to determine the recognition ability parameter of the target object based on the training difficulty parameter of the recognition training and the performance parameter of the target object; and the target recognition difficulty parameter acquisition submodule can be configured to determine the target recognition difficulty parameter based on the recognition ability parameter of the target object.
[0186] In some embodiments, the training parameter acquisition submodule may include: a subsample recognition difficulty parameter determination unit, a synthesis difficulty parameter determination unit, a sample recognition difficulty parameter determination unit, and a training difficulty parameter determination unit.
[0187] The subsample recognition difficulty parameter determination unit can be configured to acquire the complexity information of the sample sensitivity map and the sample background map; the synthesis difficulty parameter determination unit can be configured to determine the synthesis difficulty parameter of synthesizing the image to be recognized based on the sample sensitivity map and the sample background map; the sample recognition difficulty parameter determination unit can be configured to determine the recognition difficulty parameter of the image to be recognized based on the complexity information of the sample sensitivity map and the sample background map and the synthesis difficulty parameter of synthesizing the image to be recognized based on the sample sensitivity map and the sample background map; and the training difficulty parameter determination unit can be configured to determine the training difficulty parameter of the recognition training based on the recognition difficulty parameter of the image to be recognized.
[0188] Since the functional modules of the image processing apparatus 1400 in the example embodiments of this disclosure correspond to the steps of the example embodiments of the image processing method described above, they will not be described again here.
[0189] The following is for reference. Figure 15 It shows a schematic diagram of the structure of a computer system 1500 suitable for implementing a terminal device according to the embodiments of this application. Figure 15 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0190] like Figure 15 As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1502 or programs loaded from storage portion 1508 into random access memory (RAM) 1503. The RAM 1503 also stores various programs and data required for the operation of the system 1500. The CPU 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.
[0191] The following components are connected to I / O interface 1505: an input section 1506 including a keyboard, mouse, etc.; an output section 1507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN card, modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to I / O interface 1505 as needed. Removable media 1511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1510 as needed so that computer programs read from them can be installed into storage section 1508 as needed.
[0192] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1509, and / or installed from removable medium 1511. When the computer program is executed by central processing unit (CPU) 1501, it performs the functions defined above in the system of this application.
[0193] It should be noted that the computer-readable storage medium shown in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0194] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0195] The modules and / or sub-modules and / or units described in the embodiments of this application can be implemented in software or hardware. The described modules and / or sub-modules and / or units can also be housed in a processor; for example, a processor can be described as including a sending unit, an acquiring unit, a determining unit, and a first processing unit. The names of these modules and / or sub-modules and / or units do not, in certain circumstances, constitute a limitation on the module and / or sub-module and / or unit itself.
[0196] In another aspect, this application also provides a computer-readable storage medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable storage medium carries one or more programs that, when executed by the device, enable the device to perform the following functions: acquiring a target recognition difficulty parameter; determining a target sensitive image based on the target recognition difficulty parameter and acquiring complexity information of the target sensitive image; acquiring a target background image based on the target recognition difficulty parameter and the complexity information of the target sensitive image; and fusing the target sensitive image and the target background image to generate a composite image.
[0197] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) and includes several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or smart device, etc.) to execute the method according to the embodiments of this disclosure, for example... Figure 2 One or more of the steps shown.
[0198] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.
[0199] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.
[0200] It should be understood that this disclosure is not limited to the detailed structures, drawing arrangements or implementations shown herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.
Claims
1. An image processing method, characterized in that, include: The target recognition difficulty parameter is obtained based on the recognition ability parameter of the customs inspection personnel to be trained, and the target recognition difficulty parameter is greater than the recognition ability parameter of the customs inspection personnel to be trained. The target recognition difficulty parameter is nonlinearly transformed to determine the first difficulty parameter; the texture complexity of the sensitive image is described by the second moment of the gray-level histogram of the sensitive image, and the texture complexity of the sensitive image is normalized to the complexity information of the sensitive image. The target sensitive image is determined based on the first difficulty parameter and the complexity information of the sensitive image, and the complexity information of the target sensitive image is obtained. A second difficulty parameter is determined based on the target recognition difficulty parameter and the complexity information of the target sensitive image, and a target background image is obtained based on the second difficulty parameter and the complexity information of the background image. The process of fusing the target sensitive image with the target background image to generate a synthetic image for training the customs inspection personnel to be trained includes: acquiring the shape, size and edge pixels of the target sensitive image; A sliding window is determined in the target background image based on the shape and size of the target sensitive image, the shape and size of the sliding window being the same as that of the target sensitive image; the sliding window is slid across a non-blank area of the target background image until the average value of the edge pixels of the target background image covered by the sliding window is equal to the average value of the edge pixels of the target sensitive image, thus determining the area in the target background image covered by the sliding window as the area to be fused; the pixels of the area to be fused are replaced with the pixels of the target sensitive image.
2. The method according to claim 1, characterized in that, Also includes: The edges of the fused target sensitive image are smoothed.
3. The method according to claim 1, characterized in that, Determining the target-sensitive image based on the target recognition difficulty parameter includes: Construct a target distribution function centered on the target recognition difficulty parameter; The target distribution function is randomly sampled to obtain the sampling results; The target sensitive image is determined based on the sampling results.
4. The method according to claim 1, characterized in that, The target recognition difficulty parameters are obtained based on the recognition ability parameters of the customs inspectors to be trained, including: Obtain the recognition results of the customs inspection personnel to be trained based on the image to be recognized, as well as the training difficulty parameters of the recognition training; Based on the identification results, determine the performance parameters of the customs inspection personnel to be trained; The identification ability parameters of the customs inspector to be trained are determined based on the training difficulty parameters of the identification training and the performance parameters of the customs inspector to be trained. The target recognition difficulty parameter is determined based on the recognition ability parameters of the customs inspection personnel to be trained.
5. The method according to claim 4, characterized in that, The image to be identified includes a sample sensitivity map and a sample background map, and the method further includes: Obtain the complexity information of the sample sensitivity map and the sample background map; Determine the synthesis difficulty parameter for synthesizing the image to be identified based on the sample sensitivity map and the sample background map; The recognition difficulty parameter of the image to be recognized is determined based on the complexity information of the sample sensitivity map and the sample background map, as well as the synthesis difficulty parameter of synthesizing the image to be recognized based on the sample sensitivity map and the sample background map. The training difficulty parameters for the recognition training are determined based on the recognition difficulty parameters of the image to be recognized.
6. An image processing apparatus, characterized in that, include: The parameter acquisition module is configured to acquire a target recognition difficulty parameter based on the recognition ability parameter of the customs inspection personnel to be trained, wherein the target recognition difficulty parameter is greater than the recognition ability parameter of the customs inspection personnel to be trained. The target sensitive image acquisition module is configured to perform a nonlinear transformation on the target recognition difficulty parameter to determine a first difficulty parameter; describe the texture complexity of the sensitive image based on the second-order distance of the gray-level histogram of the sensitive image, normalize the texture complexity of the sensitive image to the complexity information of the sensitive image; determine the target sensitive image based on the first difficulty parameter and the complexity information of the sensitive image, and acquire the complexity information of the target sensitive image. The target background image acquisition module is configured to determine a second difficulty parameter based on the target recognition difficulty parameter and the complexity information of the target sensitive image, and to acquire the target background image based on the second difficulty parameter and the complexity information of the background image. The synthesis module is configured to fuse the target sensitive image with the target background image to generate a synthesized image for training the customs inspection personnel to be trained. The synthesis module includes: The edge pixel acquisition submodule is configured to acquire the shape, size, and edge pixels of the target sensitive image; The submodule for obtaining the region to be merged is configured to determine a sliding window in the target background image based on the shape and size of the target sensitive image, wherein the shape and size of the sliding window are the same as the shape and size of the target sensitive image; slide the sliding window in a non-blank area of the target background image until the average value of the edge pixels of the target background image covered by the sliding window is equal to the average value of the edge pixels of the target sensitive image, thereby determining the area in the target background image covered by the sliding window as the region to be merged; The pixel replacement module is configured to replace the pixels in the region to be fused with the pixels in the target sensitive image.
7. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.