Image processing model acquisition, image processing method and device

Abstract

The application relates to the technical field of image processing, and discloses an image processing model acquisition method and device and an image processing method and device. In the method, a sample image is preprocessed by using a current image processing model to obtain a corresponding representation image; the preprocessing comprises at least one of the following four operations: a clipping operation on an edge region with a gray value less than a first threshold value, an inhibition operation on a feature response of a region with a texture change degree less than a second threshold value, a brightness equalization operation on a global region, and a decision processing operation on a region with a color component variance less than a third threshold value; the representation image is hashed by using the current image processing model to obtain a hash vector of the sample image; and the current image processing model is adjusted in parameters based on a contrast loss function until a target image processing model is obtained. The above method weakens redundant information in the sample image by preprocessing the sample image, and realizes more accurate and effective hash vector output.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing model acquisition, image processing method and apparatus. Background Technology

[0002] Image hash vectors are frequently used as the basis for image matching. Related technologies often employ cryptographic hash algorithms such as MD5 and SHA, or locality-sensitive hashing algorithms like Simhash, to determine image hash vectors. However, such methods for determining image hash vectors have limitations; the accuracy of the hash vector is easily affected by redundant information in the image, thus impacting the image matching results. Therefore, a more accurate and effective method for determining image hash vectors is needed. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, this application provides an image processing model acquisition method and apparatus. The technical solution of this application is as follows.

[0004] According to a first aspect of the embodiments of this application, an image processing model acquisition method is provided, the method comprising: For each sample image in the sample image set, the sample image is preprocessed using the current image processing model to obtain the corresponding representation image; the preprocessing includes performing at least one of the following four operations: cropping operation for edge regions with gray values ​​less than a first threshold, suppressing operation for feature responses in regions with texture change less than a second threshold, brightness equalization operation for global regions, and decision processing operation for regions with color component variance less than a third threshold. The current image processing model is used to perform hash encoding on the representation image to obtain the hash vector of the sample image; A contrastive loss function is constructed based on the first hash vector similarity and the second hash vector similarity; the first hash vector similarity represents the hash vector similarity corresponding to the positive sample image group to which the sample image belongs, and the second hash vector similarity represents the hash vector similarity corresponding to the negative sample image group to which the sample image belongs. The parameters of the current image processing model are adjusted based on the contrast loss function until the target image processing model is obtained.

[0005] According to a second aspect of the embodiments of this application, an image processing method is provided, the method comprising: Obtain the image to be processed; The hash vector of the image to be processed is determined using the target image processing model obtained by training according to the image processing model acquisition method described in the first aspect.

[0006] According to a third aspect of the embodiments of this application, an image processing model acquisition apparatus is provided, the apparatus comprising: The preprocessing module is used to preprocess each sample image in the sample image set using the current image processing model to obtain the corresponding representation image. The preprocessing includes performing at least one of the following four operations: cropping operation for edge regions with gray values ​​less than a first threshold, suppressing operation for feature responses in regions with texture changes less than a second threshold, brightness equalization operation for global regions, and decision processing operation for regions with color component variance less than a third threshold. The hash encoding module is used to perform hash encoding on the representation image using the current image processing model to obtain the hash vector of the sample image; The loss construction module is used to construct a contrastive loss function based on a first hash vector similarity and a second hash vector similarity; the first hash vector similarity represents the hash vector similarity corresponding to the positive sample image group to which the sample image belongs, and the second hash vector similarity represents the hash vector similarity corresponding to the negative sample image group to which the sample image belongs. The parameter adjustment module is used to adjust the parameters of the current image processing model based on the contrast loss function until the target image processing model is obtained.

[0007] According to a fourth aspect of the embodiments of this application, an image processing apparatus is provided, the apparatus comprising: The image acquisition module is used to acquire the image to be processed. An image processing module is used to determine the hash vector of the image to be processed using a target image processing model trained according to the image processing model acquisition method described in the first aspect.

[0008] According to a fifth aspect of the embodiments of this application, a computing device is provided, the computing device including a processor and a memory, the memory being used to store at least one computer program, the at least one computer program being loaded and executed by the processor to implement the image processing model acquisition method as described in the first aspect, or the image processing method as described in the second aspect.

[0009] According to a sixth aspect of the embodiments of this application, a computer-readable storage medium is provided, which stores at least one computer program, which is loaded and executed by a processor to implement the image processing model acquisition method as described in the first aspect or the image processing method as described in the second aspect.

[0010] According to a seventh aspect of the present application, a computer program product is provided, the computer program product comprising at least one computer program, the at least one computer program being loaded and executed by a processor to implement the image processing model acquisition method as described in the first aspect, or the image processing method as described in the second aspect.

[0011] In an image processing model acquisition method provided in this application embodiment, during the training process of obtaining a target image processing module, the current image processing model is used as the base model. The sample image is preprocessed using the current image processing model, which can weaken specific regions providing redundant information or weaken the redundant information itself, thereby identifying regions of interest or information of interest (i.e., representation images) in the sample image. Then, the hash vector of the sample image is represented by the hash encoding result of the current image processing model for the representation image. This avoids the interference of redundant information in the sample image, which affects the accuracy of the determined hash vector of the sample image. Based on this, a contrastive loss function is constructed according to the hash vector similarity (i.e., the first hash vector similarity) corresponding to the positive sample image group to which the sample image belongs, and the hash vector similarity (i.e., the second hash vector similarity) corresponding to the negative sample image group to which the sample image belongs. The parameters of the current image processing model are then adjusted based on the contrastive loss function until the target image processing model is obtained. Guided by the contrastive loss function, the model can be guided to adjust parameters with the aim of increasing the first hash vector similarity and decreasing the second hash similarity, thereby supporting the trained target image processing model to achieve accurate and effective hash vector output. Meanwhile, for sample image preprocessing, multiple optional preprocessing operations are provided, supporting image optimization across different visual dimensions such as grayscale, texture, brightness, and color components. Each preprocessing operation focuses on locating specific redundant regions or information and configuring specific weakening methods. This allows for flexible selection and combination of preprocessing operations, thereby improving the adaptability of determining image hash vectors for different business scenarios.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0013] Figure 1 This illustration shows an application environment diagram provided by an embodiment of this application; Figure 2 A flowchart of the image processing model acquisition method provided in an embodiment of this application is shown; Figure 3 A schematic diagram of the solution architecture provided in the embodiments of this application is shown; Figure 4 A flowchart of the image processing method provided in an embodiment of this application is shown; Figure 5 This paper shows a structural block diagram of the image processing model acquisition device provided in an embodiment of the present application; Figure 6 A structural block diagram of the image processing apparatus provided in an embodiment of this application is shown; Figure 7 A structural block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0016] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or at least two processors or memory) can be used to implement one or at least two modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0017] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0018] The term “exemplary” as used herein means “serving as an example or embodiment.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0019] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or at least two elements selected from the set consisting of A, B, and C. Additionally, "multiple" means two or more.

[0020] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed description. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0021] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0022] Contrastive learning: an unsupervised / self-supervised learning paradigm that learns effective representations of data by constructing positive sample pairs (samples with consistent or similar visual content) and negative sample pairs (samples with significantly different visual content), maximizing the feature similarity of positive sample pairs and minimizing the feature similarity of negative sample pairs, without the need for manual data annotation.

[0023] Hamming distance: The number of distinct characters at corresponding positions in two strings of equal length. In other words, it is the number of characters that need to be replaced to transform one string into another. For example, the Hamming distance between 1011101 and 1001001 is 2.

[0024] Hash embedding (also known as hash vector): A low-dimensional binary vector (e.g., 128 bits) generated by a deep neural network that can represent the core visual content of an image. Hash embeddings generated from images with consistent visual content have high consistency (e.g., Hamming distance ≤ 3), while hash embeddings generated from images with different visual content have significant differences (e.g., Hamming distance > 3).

[0025] Hash vector buckets: Index structures used to store hash vectors. In this embodiment, hash vectors are grouped by fixed length to construct multi-level hash vector buckets, enabling fast indexing and grouping of hash vectors and avoiding efficiency losses caused by global comparisons. This is the core structure for improving the efficiency of clustering massive images.

[0026] Invalid hash vector: refers to the case where the hash embedding is a vector of all zeros, corresponding to invalid images with no valid visual content, such as solid color filling, all black, all white, etc.

[0027] Figure 1 The illustration shows an application environment provided by an embodiment of this application. The application environment may include a terminal and a server. The terminal and the server can be directly or indirectly connected via wired or wireless communication.

[0028] The terminal can be a mobile phone, computer (such as a desktop computer, tablet computer, or laptop computer), augmented reality (AR) / virtual reality (VR) device, digital assistant, smart voice interaction device (such as a smart speaker), smart wearable device, smart home appliance, in-vehicle terminal, etc. It can run programs or web pages that support image processing model acquisition and / or image processing. The program supporting "image processing model acquisition" can be an application or app developed specifically for "image processing model acquisition" function, or it can be an application or app with "image processing model acquisition" functionality. Similarly, the program supporting "image processing" can be an application or app developed specifically for "image processing" function, or it can be an application or app with "image processing" functionality. In practical applications, this program supporting image processing model acquisition and / or image processing can be an AI assistant application, instant messaging application, news feed application, video application, social application, etc.

[0029] The server can be a standalone physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server may include network communication units, processors, and memory.

[0030] In practical applications, the image processing model acquisition method provided in this application embodiment can be executed independently by the terminal, independently by the server, or collaboratively by the terminal and the server through interaction.

[0031] It should be noted that when sample images, reference images, and images to be processed that are related to user information are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0032] Figure 2 and Figure 3 A flowchart illustrating an image processing model acquisition method provided in an embodiment of this application is shown. This method is used by an electronic device (such as...) Figure 1 Taking the terminal or server shown as an example, the method includes the following steps: Step 201: For each sample image in the sample image set, preprocess the sample image using the current image processing model to obtain the corresponding representation image; the preprocessing includes performing at least one of the following four operations: cropping operation for edge regions with gray values ​​less than a first threshold, suppressing operation for feature responses in regions with texture variation less than a second threshold, brightness equalization operation for global regions, and decision processing operation for regions with color component variance less than a third threshold.

[0033] The sample image set can include multiple sample images. Multiple sample images can support the construction of multiple positive sample image groups and multiple negative sample image groups. For example, the image similarity between two sample images constituting a positive sample image group is greater than or equal to an image similarity threshold; the image similarity between two sample images constituting a negative sample image group is less than an image similarity threshold.

[0034] The sample images may be in formats including, but are not limited to, JPG, PNG, WEBP, and BMP. The sample images must conform to the input requirements of the current image processing model, which can be achieved through transcoding methods adapted to the format.

[0035] The current image processing model's input requirements can be tailored to the scale of the sample images, which can be flexibly set according to actual business needs. For example, it can adapt to 128×128 images deployed on the edge or 224×224 images for high-precision cloud scenarios. In practical applications, constraining the scale of the input images ensures scale consistency, thereby improving the stability of model training and inference. If the current scale of the input image is smaller than the preset scale, bilinear interpolation can be used for magnification to avoid stretching distortion.

[0036] The preprocessing involves four optional preprocessing operations: a) cropping of edge regions with grayscale values ​​less than a first threshold, b) suppression of feature responses in regions with texture variation less than a second threshold, c) brightness equalization of the global region, and d) decision processing for regions with color component variance less than a third threshold. At least one of these four preprocessing operations can be selected for execution, and the execution order of the selected operations can be flexibly arranged. It should be noted that when at least two preprocessing operations are selected, the preprocessing object of the subsequent operation is the preprocessing result of the preceding operation, according to the arranged execution order.

[0037] The preprocessing of sample images aims to weaken specific redundant regions or information, resulting in a weakened representation image. By weakening specific redundant regions, regions of interest (ROIs) can be identified in the sample image, and a representation image can be obtained based on these ROIs. For example, preprocessing operation a) supports image optimization in the grayscale visual dimension, where the specific redundant region is an edge region with a grayscale value less than a first threshold, and the weakening method is cropping. Preprocessing operation b) supports image optimization in the texture visual dimension, where the specific redundant region is a region with a texture variation less than a second threshold, and the weakening method is feature response suppression, such as setting the pixel value of relevant pixels to 0 or reducing their feature weights. Preprocessing operation d) supports image optimization in the color component (RGB) visual dimension, where the specific redundant region is a region with a color component variance less than a third threshold, and the weakening method is processing to adapt decision information. By weakening specific redundant information, ROIs can be identified in the sample image, and a representation image can be obtained based on these ROIs. Taking the preprocessing operation c) above as an example, it supports image optimization in the visual dimension of brightness. Here, the specific redundant information is abnormal brightness information in the global region (such as indicating an uneven brightness distribution in the global region). The weakening method is equalization, which aims to eliminate abnormal brightness information. Accordingly, because the abnormal brightness information is eliminated, the information of interest in the sample image is identified and retained to obtain a representation image.

[0038] In some embodiments, the preprocessing includes sequentially performing the cropping operation, the suppression operation, the brightness equalization operation, and the decision processing operation; the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image may include the following steps: First, using the current image processing model to crop the edge regions in the sample image whose grayscale values ​​are less than the first threshold to obtain a first image; then, using the current image processing model to suppress the feature response of regions in the first image whose texture variation is less than the second threshold to obtain a second image; furthermore, using the current image processing model to equalize the brightness performance of the second image to obtain a third image; finally, for the background regions in the third image whose color component variance is less than the third threshold, using the current image processing model to process the third image based on decision information to obtain the representation image; the decision information is determined based on the proportion and position of the background region relative to the third image.

[0039] This provides a path for sequentially optimizing sample images using four preprocessing operations. On one hand, incorporating all four optional preprocessing operations enhances the strength and comprehensiveness of weakening redundant information in the sample image, ensuring that the representation image, as a hash-encoded object, contains no redundant information, or even only core visual information. On the other hand, prioritizing the cropping operation allows for filtering of physically redundant regions, improving the efficiency and reducing resource consumption of subsequent suppression, brightness equalization, and decision processing operations. Placing the suppression operation before the brightness equalization operation further reduces the global area that brightness equalization needs to focus on. Considering the decision information's focus on the proportion and position of the background region relative to the current image, placing the decision processing operation later is more adaptable. This path is suitable for processing vertical and horizontal video screenshots, prioritizing the removal of black borders at the top and bottom of the screenshot through cropping.

[0040] It should be noted that the process of performing a cropping operation on the sample image to obtain the first image can be referred to the process of performing a cropping operation on the sample image to obtain the representation image described later, and will not be repeated here. Similarly, the process of performing a suppression operation on the first image to obtain the second image can be referred to the process of performing a suppression operation on the sample image to obtain the representation image described later, and will not be repeated here. Likewise, the process of performing a brightness equalization operation on the second image to obtain the third image can be referred to the process of performing a brightness equalization operation on the sample image to obtain the representation image described later, and will not be repeated here. Finally, the process of performing a decision processing operation on the third image to obtain the representation image can be referred to the process of performing a decision processing operation on the sample image to obtain the representation image described later, and will not be repeated here.

[0041] In some embodiments, when the preprocessing includes the cropping operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image may include the following steps: First, the sample image is processed using the current image processing model based on a preset edge detection algorithm to obtain at least one first candidate region containing the true edge of the sample image; then, for each first candidate region, the first candidate region is cropped from the sample image using the current image processing model when the gray value of the first candidate region is less than the first threshold, to obtain the representation image.

[0042] The true edge can refer to the physical border of the sample image, or it can refer to the adjacent edge whose distance from the physical border is less than a preset distance (such as 1-5 pixels).

[0043] The preset edge detection algorithm can be the Canny edge detection algorithm, the Sobel edge detection algorithm, etc.; or it can be the findContours function in the OpenCV library.

[0044] The gray value of the first candidate region can be the mean, median, or other values ​​of the gray values ​​of all pixels in the first candidate region.

[0045] The first threshold can be set to 10, 11, 12, 13, etc. The first threshold can be flexibly set according to actual business needs.

[0046] The pre-defined edge detection algorithm focuses on locating regions in an image that exhibit abrupt changes in visual appearance. By using this algorithm to select at least one first candidate region from the sample image, it improves the convenience and efficiency of locating specific redundant regions (edge ​​regions with grayscale values ​​less than a first threshold). Furthermore, it focuses on determining whether the grayscale value of the first candidate region is less than the first threshold and crops out these regions from the sample image. Simultaneously, the first candidate region must also satisfy the condition of containing the true edges of the sample image, meaning it is located in the boundary region of the sample image. Correspondingly, cropping out first candidate regions with grayscale values ​​less than the first threshold has minimal impact on the compositional continuity of the sample image. The smaller the grayscale value, the darker the color. By using the constraint that the grayscale value is less than the first threshold, dark gray edges and even black edges in the sample image can be cropped to avoid feature redundancy caused by these regions.

[0047] For example, the current image processing model processes the sample image based on a preset edge detection algorithm to obtain four first candidate regions. These four first candidate regions are the upper boundary region, lower boundary region, left boundary region, and right boundary region of the sample image. If the gray values ​​of the upper and lower boundary regions are both less than a first threshold (e.g., 10), while the gray values ​​of the left and right boundary regions are both greater than or equal to the first threshold (e.g., 10), then the upper and lower boundary regions are cropped from the sample image to obtain the representation image.

[0048] In a further embodiment, at least one region to be cropped is determined from at least one first candidate region, wherein the grayscale value of the region to be cropped is less than a first threshold. If the proportion of at least one region to be cropped relative to the sample image is greater than a preset proportion (e.g., 20%), the cropping operation can be stopped, and the sample image can be filtered (i.e., no further image processing steps are performed on the sample image). In practical applications, the sample image can be removed from the sample image set, or the sample image and images with an image similarity greater than or equal to the image similarity threshold can be removed simultaneously from the sample image set.

[0049] In one specific embodiment, the cropping operation described above can be performed by a cropping network in the current image processing model. The cropping network can be obtained by training a preset network using multiple training images. The training images are obtained by randomly adding black borders to the original images. The constraints for randomly adding black borders are as follows: the width of the black border is 5%-20% of the image's side length, and any one or more directions (top, bottom, left, right) are randomly selected. During the training process to obtain the cropping network, the preset network outputs cropped images corresponding to the training images. Based on the differences between the cropped images and the original images, the parameters of the preset network are adjusted until the cropping network is obtained.

[0050] By simulating black border interference scenarios, the network's ability to handle black borders is improved. In this case, the cropped network can be a network whose parameters were frozen during the training of the current image model, or a network whose parameters are being further tuned during the training of the current image processing model.

[0051] In another specific embodiment, the above cropping operation can be performed by a preset network in the current image processing model, where the preset network is the network whose parameters are tuned in the current image processing model.

[0052] In some embodiments, when the preprocessing includes the suppression operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image may include the following steps: First, processing the sample image using the current image processing model based on a preset edge detection algorithm to obtain at least one second candidate region; then, for each second candidate region, using the current image processing model to represent the degree of texture change of the second candidate region with the edge pixel density of the second candidate region, and performing feature response suppression on the second candidate region when the edge pixel density of the second candidate region is less than a second threshold, to obtain the representation image; the edge pixel density of the second candidate region is determined based on the ratio of the number of edge pixels to the total number of pixels in the region.

[0053] The preset edge detection algorithm can be Canny edge detection algorithm, Sobel edge detection algorithm, etc.

[0054] The second candidate region can refer to the edge region formed by pixels whose gradient magnitude is greater than a threshold. This threshold can be set between 50 and 150, and can be flexibly set according to actual business needs. For example, for natural images, the threshold can be set between 50 and 80; for document images, the threshold can be set between 100 and 150.

[0055] The second threshold can be set to 0.05, 0.1, etc. The second threshold can be flexibly set according to actual business needs.

[0056] The pre-defined edge detection algorithm focuses on locating regions in an image where visual abrupt changes occur. By using this algorithm to select at least one second candidate region from the sample image, the ease and efficiency of locating specific redundant regions (regions with texture changes less than a second threshold) can be improved. Furthermore, the algorithm focuses on determining whether the edge pixel density of the second candidate region is less than the second threshold, and suppresses the feature responses of second candidate regions with density below the second threshold. Simultaneously, using edge pixel density to measure texture change reduces the complexity of measuring texture change, balancing interpretability and computational efficiency. The lower the edge pixel density (the lower the texture change), the more likely the second candidate region is a flat region without effective texture (such as areas displaying sky, water, walls, or even solid-color backgrounds in an image), rather than a textured region indicating core visual content. Compared to cropping such second candidate regions from the sample image, suppressing feature responses avoids providing redundant features for subsequent hash encoding stages without affecting the integrity of the sample image (because such second candidate regions may be located in non-boundary areas of the sample image), thus preventing mismatches in hash vectors.

[0057] In some embodiments, when the preprocessing includes the brightness equalization operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image may include the following steps: first, using the current image processing model to determine the brightness value of each pixel in the sample image; then, using the current image processing model to suppress the brightness of pixels in the sample image whose brightness value is less than a preset brightness value, so as to obtain the representation image.

[0058] The preset brightness value can be an empirical value; it can also be the mean, median, or other values ​​of brightness values ​​of all pixels in the sample image; or it can refer to a high-frequency brightness value. The process of determining the high-frequency brightness value is as follows: Based on the brightness values ​​of each pixel in the sample image, determine that the brightness value hit more than a preset number is at least one candidate brightness value; randomly select one from at least one candidate brightness value as the high-frequency brightness value, or select the one with the highest hit count among at least one candidate brightness value as the high-frequency brightness value.

[0059] Among them, brightness performance suppression can set the brightness value of pixels with a brightness value less than a preset brightness value to 0 or reduce their feature weight.

[0060] Here, abnormal brightness information in the global region focuses on pixels whose brightness values ​​are lower than a preset value. Brightness equalization is achieved by suppressing the brightness of such pixels, focusing on brightness balance at a higher level of visual perception. These pixels often constitute dark areas in the image, which are often specific redundant regions (such as noise areas). Suppressing brightness can further prevent these redundant regions from providing redundant features for subsequent hash encoding. Dark areas may distract visual attention, but further darkened dark areas may not compete for visual attention. Therefore, this approach can effectively handle dimly lit, overexposed, and unevenly lit images, ensuring that the hash encoding consistency of the same image under different lighting conditions is ≥90%.

[0061] In addition, brightness equalization can also focus on the balance of the explicit representation of brightness values ​​as a visual feature. Accordingly, the brightness equalization operation can be achieved by "darkening the bright areas of the image (the brightness value of the pixel is greater than or equal to the preset brightness value) and brightening the dark areas of the image", which can be done using the CreateCLAHE function in the OpenCV library.

[0062] In some embodiments, when the preprocessing includes the decision processing operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image may include the following steps: First, using the current image processing model to determine at least one third candidate region in the sample image; the color component variance of the third candidate region is less than the third threshold; then, for each third candidate region, if the proportion of the third candidate region relative to the sample image is less than a first proportion and the third candidate region contains the true edge of the sample image, the current image processing model is used to generate first decision information, and the third candidate region is cropped according to the first decision information; if the proportion of the third candidate region relative to the sample image is less than the first proportion and the third candidate region does not contain the true edge of the sample image, the current image processing model is used to generate second decision information, and the third candidate region is subjected to feature response suppression according to the second decision information to obtain the representation image.

[0063] In this approach, the color component of a pixel can be considered as one indicator, and the variance of the color component can be determined by calculating the variance of the color components of all pixels in the sample image. Alternatively, the color component of a pixel can be separated into three indicators (i.e., R-channel value, G-channel value, and B-channel value), and the variance of each color component can be determined based on the variances of the R-channel value, G-channel value, and B-channel value. The variance of each channel value can be determined by calculating the variance of the channel values ​​corresponding to all pixels in the sample image.

[0064] The variance of color classification can be calculated and determined using the var function in the Numpy library.

[0065] The third threshold can be set to values ​​such as 5, 10, or 15. The third threshold can be flexibly set according to actual business needs.

[0066] The first percentage can be set to 20%, 25%, 30%, etc. This first percentage can be flexibly set according to actual business needs.

[0067] The true edge can refer to the physical border of the sample image, or it can refer to the adjacent edge whose distance from the physical border is less than a preset distance (such as 1-5 pixels).

[0068] The first and second decision information may contain only attribute definition information for the third candidate region, or they may contain both attribute definition information and processing guidance information. For the first decision information, the attribute definition information can be an invalid boundary region, and the processing guidance information can indicate pruning. For the second decision information, the attribute definition information can be an invalid non-boundary region, and the processing guidance information can indicate feature response suppression.

[0069] The smaller the color component, the more likely the third candidate region is to be a near-solid color region or a solid color region, rather than a region displaying core visual content. For such regions, considering their proportion and position relative to the sample image, differential region optimization based on decision information can avoid specific redundant regions providing redundant features for subsequent hash encoding stages. This fine-grained optimization of specific redundant regions has strong adaptability.

[0070] Furthermore, if the proportion of at least one third candidate region relative to the sample image is greater than or equal to the second proportion, the current image processing model generates third decision information indicating invalid images, and the sample image is filtered according to the third decision information. A larger proportion of at least one third candidate region indicates more solid-color or near-solid-color regions in the sample image, i.e., more specific redundant regions. Timely filtering of sample images (i.e., ceasing further image processing) avoids hash encoding of such sample images, thereby reducing the generation of invalid hash vectors and preventing such images from affecting the stability of hash vector matching and clustering.

[0071] The second percentage can be 60%, 70%, 80%, etc. This second percentage can be flexibly set according to actual business needs, such as 60% for copyright protection scenarios and 80% for image deduplication scenarios.

[0072] The third decision information may contain only attribute definition information for the sample image, or it may contain both attribute definition information and processing method guidance information. The attribute definition information may be for invalid images, and the processing method guidance information may indicate filtering.

[0073] In some embodiments, the sample image set is constructed through the following steps: 1) For each of the multiple reference images, perform at least one image transformation operation on the reference image to generate a related image, and construct a positive sample image group containing the related image and the reference image; the at least one image transformation operation includes at least one of the following: an operation to instruct image cropping, an operation to instruct image flipping, an operation to instruct image rotation, an operation to instruct image brightness adjustment, and an operation to instruct the addition of preset material to the image; 2) Construct the sample image set based on the positive sample image groups in which the multiple reference images are respectively located; the negative sample image group in which the reference image is located consists of the reference image and the difference image, and the difference image is an image in the sample image set whose difference from the reference image is greater than a preset difference.

[0074] Of the five selectable image transformation operations, at least one can be selected to perform the image transformation, and the execution order of the selected at least two preprocessing operations can also be flexibly arranged. It should be noted that when at least two image transformation operations are selected, the image transformation object of the later image transformation operation is the image transformation result of the earlier image transformation operation, according to the arranged execution order.

[0075] This refers to the operation of cropping an image, which focuses on altering the content of a reference image. By cropping the reference image, certain visual elements in the reference image may be reduced. In practical applications, the reference image can be randomly cropped.

[0076] The operation that indicates image flipping focuses on rotating the reference image around a certain point by a certain angle without changing its content.

[0077] This operation, which instructs the rotation of an image, focuses on flipping the reference image along a certain axis without altering its content. In practical applications, this allows for slight rotation of the reference image.

[0078] The operation that indicates adjusting the image brightness focuses on not changing the content of the reference image, but may change the visual effect of the content.

[0079] This section describes the operation of adding preset materials to an image, focusing on altering the content of a base image. By adding preset materials, specific image elements in the base image may be added, while previously existing visual image elements may be obscured. Preset materials can include preset text, preset stickers (such as emojis, logos, etc.), preset black borders, etc. In practical applications, random black borders can be added to the base image. The constraints for randomly adding black borders are as follows: the black border width is 5%-20% of the image's side length, and it can be randomly selected in any one of the following directions: top, bottom, left, or right.

[0080] Among them, multiple reference images can come from unlabeled general image sets (covering multiple categories of images such as nature, people, objects, and documents, with a total number of images ≥ 1 million), which helps to reduce data preparation costs.

[0081] This section provides a method for constructing sample image sets. The sample image set is constructed based on multiple positive sample image groups. The construction of a single positive sample image group relies on image transformation operations performed on a baseline image, which helps to ensure that the commonalities and differences between the two images in the positive sample image group coexist. Through flexible selection and combination of image transformation operations, various image transformations in real-world business scenarios can be fully simulated, which is beneficial for exploring the diversity of variant images serving as the baseline image, thereby improving the quality of the positive sample image set. Simultaneously, the construction of the sample image set from multiple positive sample image groups also supports the efficiency and convenience of constructing negative sample image groups based on the sample image set. Therefore, this method of constructing sample image sets (including the dynamic nature of sample image group construction) is beneficial for improving the generalization ability of the target image processing model obtained through subsequent training, ensuring the adaptability of the target image processing model to various image variant scenarios.

[0082] Furthermore, such a sample image set can also support the determination of the first hash vector similarity and the second hash vector similarity in step 203 described later. In practical applications, it can be used to classify positive sample image groups in the sample image set.

[0083] The negative sample image group containing the benchmark image consists of the benchmark image and difference images. The difference images can be randomly selected from the out-of-class sample images of the benchmark image. Compared to the same-class sample images of the benchmark image, the out-of-class sample images of the benchmark image have significantly different visual content from the benchmark image.

[0084] In practical applications, the total number of images in the sample image set can be in the hundreds of thousands. The method for constructing the sample image set provided in this application can efficiently and cost-effectively adapt to new business scenarios, requiring only the import of unlabeled images that meet the requirements of the new business scenario. Correspondingly, this also helps improve the scalability of the model, eliminating the need to train specific models separately for different business scenarios.

[0085] Step 202: Use the current image processing model to perform hash encoding on the representation image to obtain the hash vector of the sample image.

[0086] Instead of directly hashing the sample image to obtain its hash vector, the representation image is hashed, and the hashing result of the representation image is used as the hash vector of the sample image.

[0087] The hash vector can be a binary vector. The two possible values ​​can be 0 and 1, or -1 and +1. The hash vector can have 64, 128, 256 bits, etc.

[0088] In some embodiments, the current image processing model includes a multi-level convolutional network, a feature transformation network, a feature pooling network, and a hash encoding network. The step of hash encoding the representation image using the current image processing model to obtain a hash vector of the sample image may include the following steps: First, using the representation image as input, the multi-level convolutional network is used to encode the representation image to obtain a first feature; then, the feature transformation network is used to serialize the first feature to obtain a second feature; furthermore, the feature pooling network is used to pool the second feature to obtain a third feature; finally, the hash encoding network is used to hash encode the third feature to obtain a hash vector of the sample image.

[0089] Multi-level convolutional networks can include multiple sets of separable convolutional modules, which are cascaded. Compared to traditional convolution, separable convolution can improve the targeting of feature extraction while reducing computational cost and parameter count. Multi-level convolutional networks focus on increasing the number of channels in the channel dimension and reducing the scale dimension.

[0090] Among them, the feature transformation network focuses on reducing the number of dimensions to 2.

[0091] Among them, the feature pooling network can perform global pooling. The feature pooling network focuses on reducing the value of a specific dimension to 1.

[0092] Among them, hash coding networks focus on feature mapping and can be implemented using the sign function sign().

[0093] This section presents the model structure of the current image processing model serving the hash encoding stage, and how it represents the flow of images within the process. This model structure accurately extracts deep image features through a multi-level convolutional network, serializes and regularizes feature dimensions through a feature transformation network, and generates compact hash vectors through a hash encoding network. This utilization of more precise features improves the accuracy of the generated hash vectors.

[0094] For example, taking six sets of cascaded separable convolutional modules as an example, each set of separable convolutional modules can alternate between 1*2 and 2*1 separable convolutions. If the tensor representation of the image is (C=24, H=128, W=128), where C represents the number of channels, and the tensor representation of the first feature output by the multi-level convolutional network is (C=128, H=2, W=2), then the input and output feature representations of the six sets of cascaded separable convolutional modules are shown in the table below:

[0095] Table 1 Using the first feature as input, the tensor representation of the second feature output by the feature transformation network is (D=128, L=4), where D represents the feature dimension and L represents the sequence length. This is equivalent to flattening a 3D feature map into a 2D sequence.

[0096] Using the second feature as input, the tensor representation of the third feature output by the feature pooling network is (D=128, L=1). This is equivalent to performing global average pooling on the sequence length dimension, compressing a sequence of length 4 into a sequence of length 1.

[0097] Using the third feature as input, the hash coding network maps the third feature into a hash vector.

[0098] In some embodiments, if the hash vector of a sample image is an invalid hash vector, the sample image can be removed from the sample image set. Alternatively, the sample image and images with an image similarity greater than or equal to the image similarity threshold can be removed from the sample image set to further improve the quality of the training data.

[0099] Step 203: Construct a contrastive loss function based on the first hash vector similarity and the second hash vector similarity; the first hash vector similarity represents the hash vector similarity corresponding to the positive sample image group to which the sample image belongs, and the second hash vector similarity represents the hash vector similarity corresponding to the negative sample image group to which the sample image belongs.

[0100] The positive sample image group to which the sample image belongs consists of: the sample image and an image whose image similarity to the sample image is greater than or equal to the image similarity threshold. The number of positive sample image groups to which the sample image belongs can be greater than or equal to 1.

[0101] The negative sample image group to which the sample image belongs consists of: the sample image and an image whose image similarity to the sample image is less than the image similarity threshold. The number of negative sample image groups to which the sample image belongs can be greater than or equal to 1.

[0102] Calculating the similarity between two hash vectors can be achieved by using Hamming distance, Euclidean distance, or cosine similarity. For example, calculating the Hamming distance between two hash vectors determines the similarity: a smaller Hamming distance indicates a higher similarity, and a larger Hamming distance indicates a lower similarity. Similarly, calculating the Euclidean distance between two hash vectors also determines the similarity: a smaller Euclidean distance indicates a higher similarity, and a larger Euclidean distance indicates a lower similarity. Finally, calculating the cosine similarity between two hash vectors determines the similarity: a smaller cosine similarity indicates a lower similarity, and a larger cosine similarity indicates a higher similarity.

[0103] Among them, the value of the contrast loss function is positively correlated with the similarity of the first hash vector and negatively correlated with the similarity of the second hash vector.

[0104] The contrast loss function can be the InfoNCE loss function. Furthermore, the temperature coefficient can be set to 0.1.

[0105] Taking the example that there is at least one positive sample image group and at least one negative sample image group to which the sample image belongs, the first hash vector similarity represents the hash vector similarity corresponding to at least one positive sample image group. The first hash vector similarity is a global representation value, while the hash vector similarity of a single positive sample image group is a local representation value.

[0106] The first hash vector similarity can be taken as the mean, extreme value, median, etc., of the hash vector similarities of at least one positive sample image group. The hash vector similarity of a single positive sample image group is determined by calculating the similarity between two hash vectors provided by two images in the positive sample image group. The second hash vector similarity characterizes the hash vector similarity corresponding to at least one negative sample image group. The second hash vector similarity is a global representation value, while the hash vector similarity of a single negative sample image group is a local representation value. The second hash vector similarity can be taken as the mean, extreme value, median, etc., of the hash vector similarities of at least one negative sample image group. The hash vector similarity of a single negative sample image group is determined by calculating the similarity between two hash vectors provided by two images in the negative sample image group.

[0107] Step 204: Adjust the parameters of the current image processing model based on the contrast loss function until the target image processing model is obtained.

[0108] Guided by the contrastive loss function, the model can be adjusted to increase the similarity of the first hash vector and decrease the similarity of the second hash vector. During the adjustment of model parameters, the similarity between the hash vectors of two images in the positive sample image group may change, and the similarity between the hash vectors of two images in the negative sample image group may change. These changes can be reflected to some extent by the value of the contrastive loss function.

[0109] It should be noted that adjusting the parameters of the current image processing model can involve at least one iteration. This means that, using a set of sample images as the training data source, and training the target image processing model based on the current model, at least one parameter adjustment can be performed. Each parameter adjustment is based on the contrastive loss function corresponding to each sample image in a subset of sample images from one iteration. A subset of sample images from one iteration can refer to all sample images in the set, or it can refer to a portion of the sample images.

[0110] For example, 1) 100 iterations can be set and an early stopping strategy can be adopted (e.g., if the accuracy of hash vector matching does not improve for 10 consecutive iterations for the validation set, training can be stopped) to avoid model overfitting and save training time; 2) The AdamW optimizer (learning rate 1e-4, weight decay 1e-5) can be used to backpropagate and update model parameters to avoid overfitting and improve the model's generalization ability.

[0111] In some embodiments, for step 202 above, the step of hashing the representation image using the current image processing model to obtain the hash vector of the sample image may include the following steps: First, extracting features from the representation image using the current image processing model to obtain a representation feature vector; the representation feature vector is a floating-point vector; then, hashing the representation feature vector using the current image processing model to obtain the hash vector of the sample image; the hash vector of the sample image is a binary vector. Correspondingly, the step of adjusting the parameters of the current image processing model based on the contrast loss function until the target image processing model is obtained may include the following steps: adjusting the parameters of the current image processing model based on the contrast loss function and the quantization loss function until the target image processing model is obtained; the quantization loss function is constructed based on the difference between the representation feature vector and the hash vector of the sample image.

[0112] The difference between the feature vector and the hash vector of the sample image is determined by calculating the Hamming distance, Euclidean distance, and cosine similarity between the two vectors. For example, the Hamming distance is calculated to determine the difference: the smaller the Hamming distance, the smaller the difference; the larger the Hamming distance, the larger the difference. Similarly, the Euclidean distance is calculated to determine the difference: the smaller the Euclidean distance, the smaller the difference; the larger the Euclidean distance, the larger the difference. Finally, the cosine similarity is calculated to determine the difference: the smaller the cosine similarity, the larger the difference; the larger the cosine similarity, the smaller the difference.

[0113] The quantization loss function can be the L2 loss function.

[0114] The quantization loss function can be combined with the contrastive loss function to form a combined loss function. The weight of the quantization loss function in the combined loss function can be adjusted according to the quantization error of the hash vector. The value is usually in the range of 0.1-1.0, such as Loss_InfoNCE (contrastive loss function) + 0.5×Loss_L2 (quantization loss function).

[0115] The introduction of the quantization loss function aims to guide the model to adjust its parameters to reduce the difference between the representation feature vector and the hash vector of the sample image, thereby supporting the trained target image processing model to achieve accurate and effective hash vector output. This helps reduce information loss during the quantization process, improves the representation accuracy of the hash vector, and thus helps ensure the consistency between the hash vectors of two similar images.

[0116] This application provides an image processing model acquisition scheme, where the trained target image processing model supports end-to-end fast hash vector determination. During the training process, a contrastive learning framework is employed to eliminate dependence on labeled data, and a combined loss function is used to achieve global optimization of feature extraction and quantization. In the preprocessing stage, redundant information is filtered to improve the robustness of the represented image against interference. In the hash encoding stage, a lightweight model structure is used, balancing low computational cost with high representation accuracy.

[0117] In some embodiments, a target image processing model is obtained by training using the image processing model acquisition method provided in this application. This model supports inference in multiple formats such as Onnx / TensorRT / OpenVINO and is compatible with various hardware environments such as CPU, GPU, and FPGA. The model has a small size and high processing throughput (e.g., image processing up to 4417 images / minute, video processing up to 556 videos / minute (each video is calculated as 8 frames)), meeting the deployment requirements of high-concurrency cloud business scenarios and resource-constrained edge scenarios.

[0118] As can be seen from the technical solutions provided in the embodiments of this application above, in the process of training the target image processing module, the current image processing model is used as the base model. Preprocessing the sample image using the current image processing model can weaken specific regions providing redundant information or weaken the redundant information itself, thereby identifying the region of interest or information of interest (i.e., the representation image) in the sample image. Then, the hash vector of the sample image is represented by the hash encoding result of the current image processing model for the representation image. This avoids the interference of redundant information in the sample image, which affects the accuracy of the determined hash vector of the sample image. Based on this, a contrastive loss function is constructed according to the hash vector similarity (i.e., the first hash vector similarity) corresponding to the positive sample image group to which the sample image belongs, and the hash vector similarity (i.e., the second hash vector similarity) corresponding to the negative sample image group to which the sample image belongs. Then, the parameters of the current image processing model are adjusted based on the contrastive loss function until the target image processing model is obtained. Under the guidance of the contrastive loss function, the model can be guided to adjust its parameters to increase the first hash vector similarity and decrease the second hash similarity, thereby supporting the trained target image processing model to achieve accurate and effective hash vector output. Meanwhile, for sample image preprocessing, multiple optional preprocessing operations are provided, supporting image optimization across different visual dimensions such as grayscale, texture, brightness, and color components. Each preprocessing operation focuses on locating specific redundant regions or information and configuring specific weakening methods. This allows for flexible selection and combination of preprocessing operations, thereby improving the adaptability of determining image hash vectors for different business scenarios.

[0119] Figure 3 and Figure 4 A flowchart illustrating an image processing method provided in an embodiment of this application is shown. This method is used by an electronic device (such as...) Figure 1 Taking the terminal or server shown as an example, the method includes the following steps: Step 401: Obtain the image to be processed; Step 402: Determine the hash vector of the image to be processed using the target image processing model obtained by training according to the image processing model acquisition method described in steps 201-204 above.

[0120] In some embodiments, the image processing method provided in this application can be used to serve hash vector matching applications, thereby providing corresponding similarity scores, clustering label services, etc., for a single related image based on the hash vector matching results, or providing corresponding group services for two related images (i.e., constructing two related images into an image group, indicating that the image similarity between the two related images is greater than or equal to the image similarity threshold), etc. Such services can then be invoked by other services.

[0121] In some embodiments, the image processing method provided in this application can be used to determine the hash vectors of two images to be matched. Based on this, a two-layer strategy of "equivalence matching + Hamming distance matching" can be adopted to achieve image similarity determination that balances efficiency and accuracy. First, a complete equality comparison can be performed on the two hash vectors to quickly determine whether the two images are completely identical (if completely identical, the Hamming distance = 0). Generally, the matching time is ≤0.1ms / image, suitable for scenarios such as image deduplication and retrieval of completely identical content. Then, Hamming distance matching is performed on the two non-equivalent hash vectors. The Hamming distance calculation speed can be optimized through bitwise operations (the calculation time for a single hash vector pair is ≤0.05ms). Two images can be determined to be similar images with a Hamming distance ≤3 (this threshold can be flexibly set according to actual business needs to adapt to different precision requirements), suitable for scenarios such as similar image clustering and copyright protection.

[0122] In some embodiments, the image processing method provided in this application can determine the hash vector of the image to be clustered, and then construct corresponding hash vector buckets based on the hash vector of the image to be clustered. Taking a hash vector of 128 bits as an example, the 128 bits can be divided into 4 groups from high to low bits, with each group of 32 bits corresponding to a hash vector bucket level, thus constructing a multi-level hash vector bucket. This enables fast indexing of hash vectors, thereby significantly improving the clustering efficiency for massive images. During the clustering process, intra-bucket matching (such as Hamming distance matching) is performed sequentially from high to low bits. Compared to matching all bits, this effectively improves clustering efficiency. After clustering a certain number of images, newly added images can be quickly inserted into the corresponding clusters without re-clustering globally, adapting to real-time incremental scenarios and improving the real-time performance and flexibility of clustering.

[0123] In some embodiments, the image processing method provided in this application includes, but is not limited to, the following scenarios: 1) Content security review scenario: A target image processing model can be used to output the hash vector of the image to be reviewed. Then, the hash vector of the image to be reviewed is compared with the hash vectors of preset abnormal images (such as images that do not conform to platform specifications) on the platform (e.g., UGC platforms, news platforms, short video platforms, etc.) to determine hash vector similarity. If the hash vector similarity is greater than a threshold, the image or video frame to be reviewed can be determined to be an abnormal image. This improves the efficiency of reviewing massive amounts of images (including video frames) and significantly reduces review costs. By clustering the hash vectors of identical / similar abnormal content, rapid identification and interception of abnormal content can be achieved, eliminating the need for repeated review of duplicate content.

[0124] 2) Image Deduplication Scenarios: Used for image / video deduplication and popularity statistics on platforms (such as social media platforms, image sharing platforms, cloud storage platforms, etc.). For two images that are visually identical (e.g., slight variations in visual content caused by scaling, lighting changes, slight noise, etc.) or have a certain transformation relationship, a target image processing model can be applied to generate the same hash vector, thereby achieving accurate filtering of duplicate content and popularity statistics. In practical applications, the deduplication time for a single image is only 1.3ms, and the image processing throughput reaches 4417 images / minute, which can meet the needs of high-concurrency deduplication.

[0125] 3) Copyright Protection Scenarios: The system can utilize a target image processing model to output hash vectors for copyrighted images (including video frames), and use these as unique hash vectors for each copyrighted image (including video frames), thereby constructing a copyright hash vector library. Based on this, infringing content on the internet (such as stolen images and infringing video clips) can be quickly matched, enabling comprehensive copyright tracking and protection across all platforms. It is compatible with copyright service platforms, image material platforms, and film and television copyright protection platforms, supporting rapid location of infringement sources and providing technical support for copyright enforcement.

[0126] 4) Hot Topic Discovery Scenario: With the support of the target image processing model, it can perform hash vector clustering on massive images from target platforms (such as social media platforms, news platforms, etc.) to quickly discover high-spreading hot images and events, achieving real-time hot topic discovery. Adaptable to news and information platforms, it can complete clustering analysis of millions of images within 10 minutes, with a hot topic recognition latency of ≤30 minutes, helping users quickly grasp the dynamics of hot topics.

[0127] It should be noted that the above description of application scenarios is merely exemplary, and the application scenarios of the embodiments of this application are not limited thereto.

[0128] As can be seen from the technical solutions provided in the embodiments of this application above, the target image processing model is a model with high generalization ability obtained through training. The target image processing model is used to process the image to be processed to determine the hash vector of the image to be processed. With the help of the adaptability and reliability of the target image processing model, the efficiency and effect of determining the hash vector of the image to be processed can be improved.

[0129] This application also provides an image processing model acquisition device, such as... Figure 5 As shown, the image processing model acquisition device 50 includes: The preprocessing module 501 is used to preprocess each sample image in the sample image set using the current image processing model to obtain a corresponding representation image. The preprocessing includes performing at least one of the following four operations: cropping operation for edge regions with gray values ​​less than a first threshold, suppressing operation for feature responses in regions with texture change less than a second threshold, brightness equalization operation for global regions, and decision processing operation for regions with color component variance less than a third threshold. The hash encoding module 502 is used to perform hash encoding on the representation image using the current image processing model to obtain the hash vector of the sample image; The loss construction module 503 is used to construct a contrastive loss function based on the first hash vector similarity and the second hash vector similarity; the first hash vector similarity represents the hash vector similarity corresponding to the positive sample image group to which the sample image belongs, and the second hash vector similarity represents the hash vector similarity corresponding to the negative sample image group to which the sample image belongs. The parameter adjustment module 504 is used to adjust the parameters of the current image processing model based on the contrast loss function until the target image processing model is obtained.

[0130] In some embodiments, the preprocessing includes sequentially performing the cropping operation, the suppression operation, the brightness equalization operation, and the decision processing operation; the preprocessing module is further configured to: The current image processing model is used to crop the edge regions in the sample image whose gray values ​​are less than the first threshold to obtain a first image. The current image processing model is used to suppress the feature response of regions in the first image whose texture changes are less than the second threshold, thereby obtaining the second image. The brightness of the second image is equalized using the current image processing model to obtain the third image; For the background region in the third image whose color component variance is less than the third threshold, the current image processing model is used to process the third image based on decision information to obtain the representation image; the decision information is determined based on the proportion and position of the background region relative to the third image.

[0131] In some embodiments, where the preprocessing includes the cropping operation, the preprocessing module is further configured to: The sample image is processed using the current image processing model based on a preset edge detection algorithm to obtain at least one first candidate region containing the true edge of the sample image; For each first candidate region, the current image processing model is used to crop the first candidate region from the sample image when the gray value of the first candidate region is less than the first threshold, so as to obtain the representation image.

[0132] In some embodiments, where the preprocessing includes the suppression operation, the preprocessing module is further configured to: The sample image is processed using the current image processing model based on a preset edge detection algorithm to obtain at least one second candidate region; For each second candidate region, the current image processing model is used to represent the degree of texture change of the second candidate region by the edge pixel density of the second candidate region, and feature response suppression is performed on the second candidate region when the edge pixel density of the second candidate region is less than the second threshold, so as to obtain the characterization image; the edge pixel density of the second candidate region is determined based on the ratio of the number of edge pixels to the total number of pixels in the region.

[0133] In some embodiments, where the preprocessing includes the brightness equalization operation, the preprocessing module is further configured to: The brightness value of each pixel in the sample image is determined using the current image processing model. The current image processing model is used to suppress the brightness of pixels in the sample image whose brightness values ​​are less than a preset brightness value, in order to obtain the characterization image.

[0134] In some embodiments, where the preprocessing includes the decision processing operation, the preprocessing module is further configured to: At least one third candidate region in the sample image is determined using the current image processing model; the color component variance of the third candidate region is less than the third threshold. For each of the third candidate regions, if the proportion of the third candidate region relative to the sample image is less than the first proportion and the third candidate region contains the true edge of the sample image, the current image processing model generates first decision information, and the third candidate region is cropped according to the first decision information; if the proportion of the third candidate region relative to the sample image is less than the first proportion and the third candidate region does not contain the true edge of the sample image, the current image processing model generates second decision information, and the third candidate region is subjected to feature response suppression according to the second decision information to obtain the representation image.

[0135] In some embodiments, the preprocessing module is further configured to: If the proportion of the at least one third candidate region relative to the sample image is greater than or equal to the second proportion, the current image processing model is used to generate third decision information indicating invalid images, and the sample image is filtered according to the third decision information.

[0136] In some embodiments, the current image processing model includes a multi-level convolutional network, a feature transformation network, and a hash coding network, wherein the hash coding module is further configured to: Using the representation image as input, the multi-level convolutional network is used to encode the representation image to obtain the first feature; The first feature is serialized using the feature transformation network to obtain the second feature; The hash vector of the sample image is obtained by hash encoding the second feature using the hash encoding network.

[0137] In some embodiments, the hash encoding module is further configured to: The current image processing model is used to extract features from the representation image to obtain a representation feature vector; the representation feature vector is a floating-point vector. The current image processing model is used to perform hash encoding on the representation feature vector to obtain the hash vector of the sample image; the hash vector of the sample image is a binary vector. The parameter adjustment module is also used for: The parameters of the current image processing model are adjusted based on the contrast loss function and the quantization loss function until the target image processing model is obtained; the quantization loss function is constructed based on the difference between the representation feature vector and the hash vector of the sample image.

[0138] In some embodiments, the sample image set is constructed through the following steps: For each of the multiple reference images, at least one image transformation operation is performed on the reference image to generate a related image and to construct a positive sample image group containing the related image and the reference image; the at least one image transformation operation includes at least one of the following: an operation to instruct image cropping, an operation to instruct image flipping, an operation to instruct image rotation, an operation to instruct image brightness adjustment, and an operation to instruct the addition of preset material to the image; The sample image set is constructed based on the positive sample image groups to which the multiple reference images are respectively located; the negative sample image group to which the reference image is located consists of the reference image and the difference image, wherein the difference image is an image in the sample image set whose difference from the reference image is greater than a preset difference.

[0139] It should be noted that the image processing model acquisition device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the corresponding steps. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing model acquisition device and the image processing model acquisition method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0140] This application also provides an image processing apparatus, such as... Figure 6 As shown, the image processing apparatus 60 includes: Image acquisition module 601 is used to acquire the image to be processed; Image processing module 602 is used to determine the hash vector of the image to be processed using the target image processing model obtained by training according to the image processing model acquisition method described in steps 201-204 above.

[0141] It should be noted that the image processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when performing the corresponding steps. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0142] Figure 7 A structural block diagram of an electronic device provided in an embodiment of this application is shown.

[0143] Typically, an electronic device 2000 includes a processor 2010 and a memory 2020.

[0144] Processor 2010 may include one or at least two processing cores, such as a quad-core processor or an octa-core processor. Processor 2010 may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 2010 may also include a main processor and coprocessors. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. Processor 2010 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content required for display on the screen. Processor 2010 may also include an artificial intelligence processor, which handles computational operations related to machine learning.

[0145] The memory 2020 may include at least one computer-readable storage medium. The computer-readable storage medium is capable of storing data and supports the reading of the stored data by a related processing module. The computer-readable storage medium can store data for a long period or a short period. The computer-readable storage medium can be a removable medium or a non-removable medium. The computer-readable storage medium can be a magnetic storage medium, an optical storage medium, a semiconductor storage medium, etc.

[0146] The electronic device 2000 may also include a power supply component 2030 (which is configured to perform power management of the electronic device 2000), a wired or wireless network interface 2040 (which is configured to connect the electronic device 2000 to a network), and an input / output (I / O) interface 2050.

[0147] In some embodiments, the memory 2020 stores a computer program that is loaded and executed by the processor 2010 to implement the methods described above.

[0148] This application also provides a computer-readable storage medium storing a computer program that is loaded and executed by a processor to implement the above-described method.

[0149] For example, computer storage media may be random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, high-density digital video discs (DVDs) or other optical storage, magnetic tape cassettes, magnetic tapes, disks, or other magnetic storage devices. Of course, those skilled in the art will understand that computer storage media are not limited to the above-mentioned types.

[0150] This application also provides a computer program product, which includes a computer program that is loaded and executed by a processor to implement the above-described method.

[0151] The computer program described above can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages. Programming languages ​​include object-oriented programming languages—such as Smalltalk, C+, etc.—and conventional procedural programming languages—such as the "C" language or similar programming languages. The computer program can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute the computer program to implement various aspects of this application by utilizing the state information of the computer program.

[0152] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, and electronic devices according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by a computer program.

[0153] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for obtaining an image processing model, characterized in that, The method includes: For each sample image in the sample image set, the sample image is preprocessed using the current image processing model to obtain the corresponding representation image; the preprocessing includes performing at least one of the following four operations: cropping operation for edge regions with gray values ​​less than a first threshold, suppressing operation for feature responses in regions with texture change less than a second threshold, brightness equalization operation for global regions, and decision processing operation for regions with color component variance less than a third threshold. The current image processing model is used to perform hash encoding on the representation image to obtain the hash vector of the sample image; A contrastive loss function is constructed based on the first hash vector similarity and the second hash vector similarity; the first hash vector similarity represents the hash vector similarity corresponding to the positive sample image group to which the sample image belongs, and the second hash vector similarity represents the hash vector similarity corresponding to the negative sample image group to which the sample image belongs. The parameters of the current image processing model are adjusted based on the contrast loss function until the target image processing model is obtained.

2. The method according to claim 1, characterized in that, The preprocessing includes sequentially performing the cropping operation, the suppression operation, the brightness equalization operation, and the decision processing operation; The step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image includes: The current image processing model is used to crop the edge regions in the sample image whose gray values ​​are less than the first threshold to obtain a first image. The current image processing model is used to suppress the feature response of regions in the first image whose texture changes are less than the second threshold, thereby obtaining the second image. The brightness of the second image is equalized using the current image processing model to obtain the third image; For the background region in the third image whose color component variance is less than the third threshold, the current image processing model is used to process the third image based on decision information to obtain the representation image; the decision information is determined based on the proportion and position of the background region relative to the third image.

3. The method according to claim 1, characterized in that, When the preprocessing includes the cropping operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image includes: The sample image is processed using the current image processing model based on a preset edge detection algorithm to obtain at least one first candidate region containing the true edge of the sample image; For each first candidate region, the current image processing model is used to crop the first candidate region from the sample image when the gray value of the first candidate region is less than the first threshold, so as to obtain the representation image.

4. The method according to claim 1, characterized in that, When the preprocessing includes the suppression operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image includes: The sample image is processed using the current image processing model based on a preset edge detection algorithm to obtain at least one second candidate region; For each second candidate region, the current image processing model is used to represent the degree of texture change of the second candidate region by the edge pixel density of the second candidate region, and feature response suppression is performed on the second candidate region when the edge pixel density of the second candidate region is less than the second threshold, so as to obtain the characterization image; the edge pixel density of the second candidate region is determined based on the ratio of the number of edge pixels to the total number of pixels in the region.

5. The method according to claim 1, characterized in that, When the preprocessing includes the brightness equalization operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image includes: The brightness value of each pixel in the sample image is determined using the current image processing model. The current image processing model is used to suppress the brightness of pixels in the sample image whose brightness values ​​are less than a preset brightness value, in order to obtain the characterization image.

6. The method according to claim 1, characterized in that, When the preprocessing includes the decision processing operation, the step of preprocessing the sample image using the current image processing model to obtain the corresponding representation image includes: At least one third candidate region in the sample image is determined using the current image processing model; the color component variance of the third candidate region is less than the third threshold. For each of the third candidate regions, if the proportion of the third candidate region relative to the sample image is less than the first proportion and the third candidate region contains the true edge of the sample image, the current image processing model generates first decision information, and the third candidate region is cropped according to the first decision information; if the proportion of the third candidate region relative to the sample image is less than the first proportion and the third candidate region does not contain the true edge of the sample image, the current image processing model generates second decision information, and the third candidate region is subjected to feature response suppression according to the second decision information to obtain the representation image.

7. The method according to claim 6, characterized in that, The method further includes: If the proportion of the at least one third candidate region relative to the sample image is greater than or equal to the second proportion, the current image processing model is used to generate third decision information indicating invalid images, and the sample image is filtered according to the third decision information.

8. The method according to any one of claims 1 to 7, characterized in that, The current image processing model includes a multi-level convolutional network, a feature transformation network, a feature pooling network, and a hash coding network. The step of using the current image processing model to hash-encode the representation image to obtain the hash vector of the sample image includes: Using the representation image as input, the multi-level convolutional network is used to encode the representation image to obtain the first feature; The first feature is serialized using the feature transformation network to obtain the second feature; The second feature is pooled using the feature pooling network to obtain the third feature; The hash vector of the sample image is obtained by hash encoding the third feature using the hash encoding network.

9. The method according to any one of claims 1 to 7, characterized in that, The step of hashing the representation image using the current image processing model to obtain the hash vector of the sample image includes: The current image processing model is used to extract features from the representation image to obtain a representation feature vector; the representation feature vector is a floating-point vector. The current image processing model is used to perform hash encoding on the representation feature vector to obtain the hash vector of the sample image; the hash vector of the sample image is a binary vector. The step of adjusting the parameters of the current image processing model based on the contrastive loss function until the target image processing model is obtained includes: The parameters of the current image processing model are adjusted based on the contrast loss function and the quantization loss function until the target image processing model is obtained; the quantization loss function is constructed based on the difference between the representation feature vector and the hash vector of the sample image.

10. The method according to any one of claims 1 to 7, characterized in that, The sample image set is constructed through the following steps: For each of the multiple reference images, at least one image transformation operation is performed on the reference image to generate a related image and to construct a positive sample image group containing the related image and the reference image; the at least one image transformation operation includes at least one of the following: an operation to instruct image cropping, an operation to instruct image flipping, an operation to instruct image rotation, an operation to instruct image brightness adjustment, and an operation to instruct the addition of preset material to the image; The sample image set is constructed based on the positive sample image groups to which the multiple reference images are respectively located; the negative sample image group to which the reference image is located consists of the reference image and the difference image, wherein the difference image is an image in the sample image set whose difference from the reference image is greater than a preset difference.

11. An image processing method, characterized in that, The method includes: Obtain the image to be processed; The hash vector of the image to be processed is determined by training a target image processing model using the image processing model acquisition method according to any one of claims 1-10.

12. An image processing model acquisition device, characterized in that, The device includes: The preprocessing module is used to preprocess each sample image in the sample image set using the current image processing model to obtain the corresponding representation image. The preprocessing includes performing at least one of the following four operations: cropping operation for edge regions with gray values ​​less than a first threshold, suppressing operation for feature responses in regions with texture changes less than a second threshold, brightness equalization operation for global regions, and decision processing operation for regions with color component variance less than a third threshold. The hash encoding module is used to perform hash encoding on the representation image using the current image processing model to obtain the hash vector of the sample image; The loss construction module is used to construct a contrastive loss function based on a first hash vector similarity and a second hash vector similarity; the first hash vector similarity represents the hash vector similarity corresponding to the positive sample image group to which the sample image belongs, and the second hash vector similarity represents the hash vector similarity corresponding to the negative sample image group to which the sample image belongs. The parameter adjustment module is used to adjust the parameters of the current image processing model based on the contrast loss function until the target image processing model is obtained.

13. An image processing apparatus, characterized in that, The device includes: The image acquisition module is used to acquire the image to be processed. An image processing module is used to determine the hash vector of the image to be processed using a target image processing model obtained by training according to the image processing model acquisition method according to any one of claims 1-10.

14. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, which is loaded and executed by the processor to implement the image processing model acquisition method as described in any one of claims 1 to 10, or the image processing method as described in claim 11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the image processing model acquisition method as described in any one of claims 1 to 10, or the image processing method as described in claim 11.

16. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the image processing model acquisition method as described in any one of claims 1 to 10, or the image processing method as described in claim 11.

Images