Image processing method, device and equipment, readable storage medium and program product

Abstract

The invention provides an image processing method and device, equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring data enhancement configuration parameters of a sample image, wherein the data enhancement configuration parameters comprise enhanced boundary control data and similarity control data; determining a data enhancement control parameter according to the enhanced boundary control data and the similarity control data, and determining a data enhancement parameter of a first sample image included in a positive sample pair corresponding to the sample image according to the data enhancement control parameter and the enhanced boundary control data; determining a data enhancement parameter of a second sample image included in the positive sample pair according to the data enhancement parameter and the data enhancement control parameter; and processing the sample image according to the data enhancement parameter of the first sample image and the data enhancement parameter of the second sample image to obtain the first sample image and the second sample image. By adopting the method and the device, relatively complex positive sample pairs can be generated, the difficulty of model training is effectively improved, and the pre-training effect of the model is further improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to image processing methods, apparatus, devices, readable storage media, and program products. Background Technology

[0002] Both large visual models and multimodal large models require large-scale labeled images for training, but the cost of labeled images is high. Therefore, methods for pre-training based on unlabeled datasets for contrastive learning are receiving increasing attention.

[0003] In contrastive learning, positive and negative samples are needed for training. Samples generated from the same image are called positive samples, and samples generated from different images are called negative samples. Existing methods for generating positive samples typically involve randomly enhancing the sample images (e.g., cropping, blurring), obtaining two sample image patches, and using these as positive sample pairs. This method generates relatively simple positive sample pairs, which reduces the difficulty of model training and consequently leads to poor pre-training performance. Summary of the Invention

[0004] This application provides image processing methods, apparatus, devices, readable storage media, and program products, which can generate more complex positive sample pairs, effectively increasing the difficulty of model training and thus improving the pre-training effect of the model.

[0005] On one hand, embodiments of this application provide an image processing method, which includes:

[0006] Data augmentation configuration parameters for sample images are obtained, including augmentation boundary control data and similarity control data;

[0007] Based on the enhanced boundary control data and the similarity control data, data augmentation control parameters are determined, and based on the data augmentation control parameters and the enhanced boundary control data, data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image are determined.

[0008] Based on the data augmentation parameters of the first sample image and the data augmentation control parameters, the data augmentation parameters of the second sample image included in the positive sample pair are determined; wherein, the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold.

[0009] The sample images are augmented using the data augmentation parameters of the first and second sample images respectively, to obtain a positive sample pair including the first and second sample images; wherein the positive sample pair is used to train a neural network model to obtain an image processing model.

[0010] Accordingly, embodiments of this application provide an image processing apparatus, which includes:

[0011] An acquisition unit is used to acquire data augmentation configuration parameters for sample images, wherein the data augmentation control parameters include augmentation boundary control data and similarity control data;

[0012] The determining unit is configured to determine data augmentation control parameters based on the augmentation boundary control data and the similarity control data, and to determine data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data.

[0013] The determining unit is further configured to determine the data augmentation parameters of the second sample image included in the positive sample pair based on the data augmentation parameters of the first sample image and the data augmentation control parameters; wherein the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold.

[0014] The processing unit is configured to perform data augmentation processing on the sample image according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, respectively, to obtain a positive sample pair corresponding to the sample image, including the first sample image and the second sample image; wherein, the positive sample pair is used to train a neural network model to obtain an image processing model.

[0015] In one implementation, the enhanced boundary control data includes a first enhanced boundary and a second enhanced boundary; the determining unit, when determining the data enhancement control parameters based on the enhanced boundary control data and the similarity control data, specifically performs the following steps:

[0016] Determine the boundary ratio data based on the first enhancement boundary and the second enhancement boundary;

[0017] Based on the boundary ratio data and the similarity control data, determine the joint probability distribution corresponding to the data augmentation control parameters;

[0018] The data augmentation control parameters are determined based on the joint probability distribution corresponding to the data augmentation control parameters.

[0019] In one implementation, the determining unit, when determining the joint probability distribution corresponding to the data augmentation control parameters based on the boundary ratio data and the similarity control data, specifically performs the following steps:

[0020] If the similarity control data is equal to the preset reference threshold, then the boundary ratio data is processed by average distribution construction to obtain the joint probability distribution corresponding to the data augmentation control parameters, and the joint probability distribution is the first average distribution;

[0021] If the similarity control data is greater than the preset reference threshold, then the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain the joint probability distribution corresponding to the data augmentation control parameters, wherein the joint probability distribution is a first normal distribution;

[0022] If the similarity control data is less than the preset reference threshold, then the absolute values ​​of the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain a second normal distribution, and the second normal distribution is flipped to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is a flipped distribution.

[0023] In one implementation, the determining unit, when determining the data augmentation control parameters based on the joint probability distribution corresponding to the data augmentation control parameters, specifically performs the following steps:

[0024] If the joint probability distribution corresponding to the data augmentation control parameters is a first average distribution or a first normal distribution, then data extraction processing is performed according to the first average distribution or the first normal distribution to obtain a first data extraction result, and the data augmentation control parameters are determined according to the first data extraction result.

[0025] If the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution, then data extraction processing is performed according to the flipped distribution to obtain the second data extraction result;

[0026] If the second data extraction result is greater than or equal to the preset value, then the data enhancement control parameter is determined based on the second data extraction result and the boundary ratio data.

[0027] If the second data extraction result is less than the preset value, then the data augmentation control parameter is determined based on the second data extraction result and the negative value of the boundary ratio data.

[0028] In one implementation, the determining unit, when determining the data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data, specifically performs the following steps:

[0029] Based on the data augmentation control parameters and the augmentation boundary control data, determine the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image;

[0030] The data augmentation parameters of the first sample image are determined based on the data augmentation information of the first sample image.

[0031] In one implementation, the determining unit, when determining the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data, specifically performs the following steps:

[0032] Determine a first ratio between the first augmentation boundary included in the augmentation boundary control data and the first data augmentation control parameter, and determine the minimum value between the first ratio and the data augmentation control parameter as the first distribution boundary;

[0033] Determine a second ratio between the second augmentation boundary included in the augmentation boundary control data and the second data augmentation control parameter, and determine the maximum value of the second ratio and the data augmentation control parameter as the second distribution boundary;

[0034] The first distribution boundary and the second distribution boundary are processed to construct an average distribution, thereby obtaining the second average distribution corresponding to the data augmentation parameters of the first sample image.

[0035] Based on the second average distribution corresponding to the data augmentation parameters of the first sample image, the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image is determined.

[0036] In one implementation, the data augmentation parameters include the cropping area; the processing unit, configured to perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the second sample image respectively, to obtain the positive sample pair corresponding to the sample images including the first sample image and the second sample image, specifically performs the following steps:

[0037] The sample image is jointly cropped based on the cropping area of ​​the first sample image to obtain the first sample image included in the positive sample pair corresponding to the sample image.

[0038] The sample images are jointly cropped based on the cropping area of ​​the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0039] In one implementation, the data augmentation parameters include target blur or target grayscale; the processing unit, configured to perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the second sample image respectively, to obtain the positive sample pair corresponding to the sample images including the first sample image and the second sample image, specifically performs the following steps:

[0040] The sample images are randomly cropped according to the enhanced boundary control data to obtain a first intermediate sample image and a second intermediate sample image.

[0041] Based on the target blur or target grayscale of the first sample image, the first intermediate sample image is blurred or grayscaled to obtain the first sample image included in the positive sample pair corresponding to the sample image;

[0042] The second intermediate sample image is blurred or grayscaled according to the target blur or target grayscale of the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0043] In one implementation, the processing unit is further configured to perform the following steps:

[0044] Obtain a positive sample set containing multiple positive sample pairs, and a negative sample set containing multiple negative sample pairs, wherein the negative sample pairs are determined based on the multiple positive sample pairs;

[0045] The positive sample set and the negative sample set are input into the neural network model for comparative learning processing to obtain comparative learning difference data.

[0046] The model parameters of the neural network model are adjusted based on the contrastive learning difference data to obtain the image processing model.

[0047] Accordingly, embodiments of this application provide a computer device, which includes:

[0048] A processor is a tool for implementing computer programs.

[0049] A computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the image processing method described above.

[0050] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when read and executed by a processor of a computer device, causes the computer device to perform the image processing method described above.

[0051] Accordingly, this application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the image processing method described above.

[0052] In this application, data augmentation configuration parameters, including enhanced boundary control data and similarity control data, can be obtained to facilitate subsequent data augmentation processing of sample images based on these parameters, thereby obtaining positive sample pairs. This application can determine data augmentation control parameters based on the enhanced boundary control data and similarity control data, and further determine the data augmentation parameters of the first sample image, which is included in the positive sample object corresponding to the sample image. After determining the data augmentation parameters of the first sample image, the data augmentation parameters of the second sample image can be determined, and then the sample images can be processed according to the data augmentation parameters to obtain positive sample pairs. Through the image processing method provided in this application, the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image can be significant, resulting in a first sample image and a second sample image with significant differences, i.e., a positive sample pair with significant differences. This allows for the generation of more complex positive sample pairs, effectively increasing the difficulty of model training and thus improving the model's pre-training effect. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram illustrating the relationship between image similarity and cropping area provided in an embodiment of this application;

[0055] Figure 2 This is a schematic diagram of the probability information of the area ratio in the random cutting method provided in the embodiments of this application;

[0056] Figure 3 This is a schematic diagram of the system architecture of an image processing system provided in an embodiment of this application;

[0057] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0058] Figure 5 This is a schematic diagram of the probability density corresponding to a joint probability distribution provided in an embodiment of this application;

[0059] Figure 6 This is a schematic diagram of an image processing method provided in an embodiment of this application;

[0060] Figure 7 This is a schematic diagram of another image processing method provided in an embodiment of this application;

[0061] Figure 8 This is a flowchart illustrating another image processing method provided in an embodiment of this application;

[0062] Figure 9 This is a schematic diagram illustrating the complexity of a positive sample pair provided in an embodiment of this application;

[0063] Figure 10 This is a structural block diagram of an image processing apparatus provided in an embodiment of this application;

[0064] Figure 11 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0065] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0066] It should be noted that the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature specified with "first" or "second" may explicitly or implicitly include at least one of those features.

[0067] To facilitate understanding, the following brief explanations are provided for some of the terms:

[0068] Data augmentation, also known as data enhancement, is a method of creating datasets by modifying existing data (e.g., cropping, stretching, grayscale, blurring, etc.). Data augmentation methods can effectively increase the size of the training set.

[0069] Contrastive learning training process: Positive and negative samples are input into the model for training to obtain difference data. The model parameters are then adjusted using the difference data so that the visual features of positive samples are closer to each other and the visual features of negative samples are farther apart, thereby achieving contrastive learning pre-training.

[0070] Positive samples: Samples generated based on the same image.

[0071] Negative samples: Samples generated based on different images.

[0072] Typically, positive samples are generated by independently and randomly enhancing the sample images (e.g., independent random cropping, independent random blurring, etc.) to obtain two sample image patches. The following example illustrates the method of obtaining positive sample pairs by independently and randomly cropping the sample images: Let the probability distribution corresponding to the cropped area of ​​the first sample image be S1~U[S min ,S max Let the probability distribution corresponding to the cropped area of ​​the second sample image be S2~U[S min ,S max ], where S1 is the cropping area of ​​the first sample image, S2 is the cropping area of ​​the second sample image, S min S represents the minimum cropping area (for example, the minimum could be 0.2, meaning the cropped area is 0.2% of the sample image area). max Let S1 be the maximum value of the cropped area (e.g., the maximum value can be 1), and let U be the average distribution. During independent random cropping, a random sample is drawn based on the probability distribution corresponding to the cropped area of ​​the first sample data to obtain the cropped area S1 of the first sample image. The same method is used to determine the cropped area S2 of the second sample image. That is, the methods for determining the cropped area of ​​the first sample image and the methods for determining the cropped area of ​​the second sample image are independent of each other.

[0073] The similarity between the first and second sample images can be defined as the cosine similarity between them, also known as the difficulty of finding a positive sample pair. The lower the cosine similarity, the greater the difficulty of finding a positive sample pair. See also... Figure 1 This figure is a schematic diagram illustrating the relationship between image similarity and cropping area provided in an embodiment of this application. Figure 1 In the diagram, the horizontal axis represents the ratio S of the cropped area of ​​the first sample image to the cropped area of ​​the second sample image. r S r =S1 / S2, where the vertical axis represents the cosine similarity between the first and second sample images. Figure 1 In the diagram, the dashed line represents the area S when the cutting area is determined using a random cutting method. r The relationship with cosine similarity, represented by the solid line S. r The relationship with cosine similarity. (From...) Figure 1 It can be seen that S r The further away from 1, the lower the similarity between the first and second sample images, meaning the greater the difficulty in finding positive sample pairs. However, when using the independent random cropping method to determine the first and second sample images, the similarity between the determined first and second sample images is relatively high.

[0074] Please see Figure 2 This figure is a schematic diagram of the probability information of the area ratio in the random cutting method provided in the embodiments of this application. Figure 2 In the diagram, the horizontal axis represents the area ratio (log(S1 / S2)) determined by the random cropping method, and the vertical axis represents the probability density of the area ratio determined by the random cropping method. Figure 2 It can be seen that in over 80% of positive sample pairs, the cropping area ratio between the first and second sample images is between 1 / 2 and 2. This means that the cropping area ratio between the first and second sample images determined using random cropping methods has a high probability of being less than 1:2 (e.g., a ratio of 1:1.5, 1:1.9, 1:1.2, etc.). Therefore, it can be concluded that the first and second sample images are likely to have a high similarity. This makes model training using positive samples easier, resulting in poorer model training performance.

[0075] Based on this, embodiments of this application provide an image processing method that can obtain data augmentation configuration parameters for sample images. The data augmentation control parameters include augmentation boundary control data and similarity control data. The method determines data augmentation control parameters based on the augmentation boundary control data and similarity control data, and determines data augmentation parameters for a first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and augmentation boundary control data. It then determines data augmentation parameters for a second sample image included in the positive sample pair based on the data augmentation parameters of the first sample image and the data augmentation control parameters. The difference between the data augmentation parameters of the first sample image and the second sample image is greater than or equal to a preset difference threshold. Data augmentation processing is performed on the sample images based on the data augmentation parameters of the first and second sample images respectively, resulting in the first and second sample images included in the positive sample pair corresponding to the sample image. The positive sample pair is used to train a neural network model to obtain an image processing model. Compared to independent random augmentation schemes, this application provides a joint random augmentation scheme, which can first determine the data augmentation control parameters, and then determine the data augmentation parameters of the first sample image based on the data augmentation control parameters, thereby generating more complex positive sample pairs, effectively increasing the difficulty of model training, and thus improving the pre-training effect of the model.

[0076] The image processing method provided in this application can be applied to machine learning technology in the field of artificial intelligence. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Machine learning (ML) is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formulaic learning. When performing machine learning, it is necessary to obtain positive and negative samples. The method provided in this application can be used to obtain highly complex positive sample pairs, thereby effectively increasing the difficulty of model training and improving the pre-training effect of the model.

[0077] The architecture of the image processing system provided in the embodiments of this application will now be described with reference to the accompanying drawings.

[0078] Please see Figure 3 The figure is a schematic diagram of the system architecture of an image processing system provided in an embodiment of this application. The image processing system includes an image processing device 301, a model training device 302, and a database 303. The image processing device 301 can interact with the model training device 302 and the database 303. Wherein:

[0079] Image processing device 301 can acquire data augmentation configuration parameters for sample images and perform data augmentation processing on the sample images based on these parameters to obtain positive sample pairs with higher complexity. It can then send these positive sample pairs to model training device 302 for model training. Image processing device 301 can be a device independent of model training device 302, or it can be a module deployed within model training device 302. Image processing device 301 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0080] The model training device 302 can receive positive sample pairs sent by the image processing device 301 and perform model training processing based on these positive sample pairs. The model training device 302 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0081] Database 303 is used to store data related to the image processing device 301, such as sample images and data augmentation configuration data for the sample images. Database 303 can be a local database within the image processing device 301, or a cloud database associated with the image processing device 301 (i.e., a database deployed in the cloud). Specifically, it can be deployed based on any of the following: private cloud, public cloud, hybrid cloud, edge cloud, etc., thus allowing the cloud database to focus on different functions. For example, a database deployed in a private cloud uses personal cloud hardware and primarily serves a small group of users. A database deployed in a public cloud, on the other hand, is based on a third-party cloud platform, enabling data sharing; any user's data can be stored in this database, and any user can use the data in the database.

[0082] The following will elaborate on such matters. Figure 1 The working principle of the image processing system shown:

[0083] Image processing device 301 can obtain data augmentation configuration parameters for sample images from database 303. These data augmentation control parameters may include augmentation boundary control data and similarity control data. Image processing device 301 can determine the data augmentation control parameters based on the augmentation boundary control data and similarity control data, and determine the data augmentation parameters for the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and augmentation boundary control data.

[0084] Image processing device 301 can determine the data enhancement parameters of the second sample image included in the positive sample pair based on the data enhancement parameters and data enhancement control parameters of the first sample image; and the difference between the data enhancement parameters of the first sample image and the data enhancement parameters of the second sample image is greater than or equal to a preset difference threshold. Image processing device 301 can perform data enhancement processing on the sample images according to the data enhancement parameters of the first sample image and the data enhancement parameters of the second sample image respectively, to obtain the first sample image and the second sample image included in the positive sample pair corresponding to the sample images.

[0085] Image processing device 301 can send the identified positive sample pairs to model training device 302. Model training device 302 uses the positive and negative sample pairs to train a neural network model to obtain an image processing model. The image processing method provided in this application embodiment can determine high-complexity positive sample pairs through a joint random augmentation method, thereby effectively increasing the difficulty of model training and improving the pre-training effect of the model.

[0086] It is understood that the schematic diagrams of the image processing system described in the embodiments of this application are for the purpose of more clearly illustrating the image processing method of the embodiments of this application, and do not constitute a limitation on the image processing method provided in the embodiments of this application. For example, the image processing method provided in the embodiments of this application can be executed not only by the image processing device 301, but also by the model training device 302, and can also be executed by other devices different from the image processing device 301 that are capable of communicating with the model training device 302 and the database 303. Those skilled in the art will understand that... Figure 1 The number of image processing device 301, model training device 302, and database 303 shown in the embodiments is merely illustrative. Any number of devices can be configured according to business needs. Furthermore, as system architecture evolves and new business scenarios emerge, the image processing method provided in this application embodiment is also applicable to similar technical problems.

[0087] It should be noted that the collection and processing of relevant data (e.g., sample images) in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0088] Please see Figure 4 , Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application. The image processing method can be implemented by the image processing device 301, the model training device 302, or other devices capable of implementing the image processing method. The following description uses the image processing device 301 as an example. The flow of the image processing method provided in this embodiment includes, but is not limited to:

[0089] S401. Obtain data augmentation configuration parameters for the sample image, wherein the data augmentation control parameters include augmentation boundary control data and similarity control data.

[0090] In this embodiment, the image processing device can acquire data augmentation configuration parameters for sample images. These parameters may include augmentation boundary control data and similarity control data. The augmentation boundary control data indicates the boundary between the data augmentation parameters of the subsequently generated first sample image and the data augmentation parameters of the second sample image. For example, when the data augmentation method is cropping, the data augmentation parameters may include the cropping area, and the augmentation boundary control data may indicate the maximum and minimum values ​​of the cropping area. Similarly, when the data augmentation method is blurring, the data augmentation parameters may include blurriness, and the augmentation boundary control data may indicate the maximum and minimum values ​​of the blurriness. The similarity control data is used to adjust the similarity between the subsequently generated first and second sample images, and can be represented by β. It should be noted that the data augmentation configuration parameters can be adaptively adjusted according to different application requirements. In practical applications, the data augmentation configuration parameters for the same batch of sample images can be the same, and a batch of sample images may contain multiple sample images. The method provided in this application embodiment can control the similarity between subsequently generated positive sample pairs by using data augmentation configuration parameters, thereby improving the controllability of positive sample pairs and facilitating the generation of positive sample pairs that meet application requirements, which in turn helps improve the training effect of the model.

[0091] S402. Determine data augmentation control parameters based on the enhanced boundary control data and the similarity control data, and determine data augmentation parameters for the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the enhanced boundary control data.

[0092] Typically, when generating positive sample pairs containing a first sample image and a second sample image, the first sample image is generated first, followed by the second sample image. Therefore, relevant data of the first sample image can be obtained before generating the second sample image, and the second sample image can be generated based on this data, resulting in a lower similarity between the first and second sample images.

[0093] In this embodiment, based on the above principle, the image processing device can determine data augmentation control parameters according to the enhanced boundary control data and similarity control data. These data augmentation control parameters can indicate the correlation between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image. When the data augmentation control parameters indicate a large difference between the data augmentation parameters of the first sample image and the second sample image, the probability of a low similarity between the subsequently determined first and second sample images is higher.

[0094] Image processing equipment can determine the data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on data augmentation control parameters and augmentation boundary control data. These data augmentation parameters may include cropping area, blurring, grayscale, stretching parameters, etc. Different types of data augmentation parameters can correspond to different data augmentation methods. For example, the cropping area of ​​the first sample image corresponds to a cropping data augmentation method (also known as a joint random cropping method), that is, performing joint random cropping processing on the sample image based on the cropping area can obtain the first sample image. As another example, the blurring of the first sample image corresponds to a blurring data augmentation method (also known as a joint random blurring method), that is, performing joint random blurring processing on the sample image based on the blurring level can obtain the first sample image. Through the method provided in this application embodiment, data augmentation control parameters can be determined, thereby determining the data augmentation parameters of the first sample image, which is beneficial for subsequently determining the data augmentation parameters of the second sample image, thus resulting in a lower similarity between the first and second sample images.

[0095] In one embodiment, the enhanced boundary control data includes a first enhanced boundary and a second enhanced boundary. The specific implementation method for determining the data enhancement control parameters based on the enhanced boundary control data and the similarity control data can be as follows: determining boundary ratio data based on the first and second enhanced boundaries; determining the joint probability distribution corresponding to the data enhancement control parameters based on the boundary ratio data and the similarity control data; and determining the data enhancement control parameters based on the joint probability distribution corresponding to the data enhancement control parameters.

[0096] Specifically, the image processing device can determine boundary ratio data based on the first enhancement boundary and the second enhancement boundary. The boundary ratio data can, to a certain extent, limit the value range of the data enhancement parameters for the first and second sample images. The image processing device can determine the joint probability distribution corresponding to the data enhancement control parameters based on the boundary ratio data and the similarity control data. Since the data enhancement control parameters are used to indicate the correlation between the data enhancement parameters of the first and second sample images, the probability distribution corresponding to the data enhancement control parameters is also the probability distribution jointly corresponding to the data enhancement parameters of the first and second sample images, i.e., the joint probability distribution. After determining the joint probability distribution, the image processing device can determine the data enhancement control data based on this joint probability distribution. Through the method provided in this application embodiment, the joint probability distribution can be determined based on the boundary ratio data and the similarity control data, and the data enhancement control parameters can be determined based on the joint probability distribution. This is beneficial for subsequently determining the data enhancement parameters of the first and second sample images, which have significant differences, based on the data enhancement control parameters.

[0097] In one embodiment, the specific implementation method for determining the joint probability distribution corresponding to the data augmentation control parameters based on the boundary ratio data and similarity control data can be:

[0098] If the similarity control data is equal to the preset reference threshold, then the boundary ratio data is processed by average distribution construction to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is the first average distribution.

[0099] If the similarity control data is greater than the preset reference threshold, the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is the first normal distribution.

[0100] If the similarity control data is less than the preset reference threshold, the absolute values ​​of the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain a second normal distribution. The second normal distribution is then flipped to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is a flipped distribution.

[0101] Specifically, the preset reference threshold can be 0. When the similarity control data β equals 0, the image processing device can construct a joint probability distribution based on the boundary ratio data to obtain the data augmentation control parameters. This joint probability distribution is a first average distribution based on the boundary ratio data. In some cases, this first average distribution can be represented as U[negative value of boundary ratio data, boundary ratio data].

[0102] When the similarity control data β is greater than 0, the image processing device can construct a normal distribution from the boundary ratio data and the similarity control data to obtain the joint probability distribution corresponding to the data augmentation control parameters. This joint probability distribution is a first normal distribution. In some cases, this first normal distribution can be represented as N. T (0, (1 / similarity control data) * boundary ratio data, negative of boundary ratio data, boundary ratio data).

[0103] When the similarity control data β is less than 0, the image processing device can construct a second normal distribution from the absolute values ​​of the boundary ratio data and the similarity control data. It can then perform a flipping process based on the boundary information of the second normal distribution to obtain the joint probability distribution corresponding to the data augmentation control parameters; this joint probability distribution is a flipped distribution. For example, if the similarity control data β = -1, then a second normal distribution can be constructed from the absolute values ​​of the similarity control data and the boundary ratio data (the curve corresponding to the probability density function of this second normal distribution is...). Figure 5When β = 1, the curves corresponding to the second normal distribution are the same. At this time, the boundary information of the second normal distribution is [-log(5), log(5)], and the boundary median is log(1). The image processing device can divide the curve corresponding to the probability density function of the second normal distribution into a left half-curve (the boundary information is [-log(5), log(1)]) and a right half-curve (the boundary information is [log(1), log(5)]) based on the boundary median log(1), and perform flipping processing on the left half-curve and the right half-curve respectively to obtain the curve corresponding to the flipped distribution (such as...). Figure 5 (The curve corresponding to β = -1). In some cases, the flipped distribution can be represented as N. T (0, -(1 / similarity control data)*boundary ratio data, negative of boundary ratio data, boundary ratio data). The method provided in this application embodiment can determine the joint probability distribution corresponding to the data augmentation control parameters, which is beneficial for subsequently determining reasonable data augmentation control parameters, and thus can determine positive sample pairs with higher complexity.

[0104] It should be noted that, to ensure the reasonableness of the data, the joint probability distribution corresponding to the data augmentation control parameters can be the distribution followed by the logarithm of the data augmentation control parameters. For example, the data augmentation control parameters are represented as S. r The joint probability distribution is N T Then it can be log(S) r )~N T .

[0105] Please see Figure 5 The figure is a schematic diagram of the probability density corresponding to a joint probability distribution provided in an embodiment of this application. Figure 5 In the diagram, the horizontal axis represents the logarithm of the data augmentation control parameter, i.e., log(S). r The vertical axis represents the probability density, and the curve represents the data augmentation control parameters (S). r The probability density curve corresponding to the distribution of the logarithm of ). Figure 5 In this context, the joint probability distribution followed by the logarithms of the data augmentation control parameters is denoted as J-Crop(β), where β represents the similarity control data. The augmentation boundary control data involved in each joint probability distribution is the same. Figure 5 It can be seen that the smaller the value of β, the greater the probability that the value of the data augmentation control parameter is far away from 1, and the greater the probability that the first sample image and the second sample image have low similarity. Figure 5 The method also includes the probability density curves corresponding to the independent random augmentation method. Through the method provided in this application, the similarity between the first and second sample images can be controlled using similarity control data, which is beneficial for generating highly complex positive sample pairs and improving the training effect of the model.

[0106] In one embodiment, the specific implementation method of the data augmentation control parameters is determined based on the joint probability distribution corresponding to the data augmentation control parameters, which may be as follows:

[0107] If the joint probability distribution corresponding to the data augmentation control parameters is a first average distribution or a first normal distribution, then data extraction is performed according to the first average distribution or the first normal distribution to obtain the first data extraction result, and the data augmentation control parameters are determined based on the first data extraction result. Specifically, if the joint probability distribution corresponding to the data augmentation control parameters is a first average distribution or a first normal distribution, then data extraction is performed according to the first average distribution or the first normal distribution to obtain the first data extraction result. The data augmentation control parameters are determined based on the first data extraction result. For example, if the logarithm of the data augmentation control parameters follows a first average distribution or a first normal distribution, then after data extraction according to the first average distribution or the first normal distribution, the first data extraction result obtained is the value of the logarithm of the data augmentation control parameters. Then, exponential operations can be performed on the first data extraction result to obtain the data augmentation control parameters.

[0108] If the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution, then data extraction is performed according to the flipped distribution to obtain the second data extraction result. Specifically, if the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution, then the second data extraction result obtained by performing data extraction according to the flipped distribution has a relatively high probability of being a smaller value (as mentioned above). Figure 5 -log(3), -log(5), etc.) or larger values ​​(as mentioned above) Figure 5 If log(3), log(5), etc. are used, then the values ​​of the data augmentation control parameters determined based on the second data extraction results may be too large (e.g., e). 3 Or e 5 ) or too small (e.g., e) -3 Or e -5 This can lead to the data augmentation parameters for the first sample image, determined subsequently based on the data augmentation control parameters, being either too large or too small. This could result in the data control parameters falling outside the range indicated by the augmentation boundary control data, rendering the data augmentation process ineffective. Therefore, when the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution (i.e., the similarity control data β is less than 0), the second data extraction result can be further processed after obtaining it. The processing method is shown in the following steps.

[0109] If the second data extraction result is greater than or equal to a preset value, then the data enhancement control parameters are determined based on the second data extraction result and the boundary ratio data. Specifically, the image processing device can determine the second data extraction result. If the second data extraction result is greater than or equal to a preset value (for example, the preset value can be 0), then the data enhancement control parameters are determined based on the second data extraction result and the boundary ratio data.

[0110] If the second data extraction result is less than a preset value, then the data augmentation control parameters are determined based on the second data extraction result and the negative value of the boundary ratio data. Specifically, if the second data extraction result is less than a preset value, then the data augmentation control parameters can be determined based on the second data extraction result and the negative value of the boundary ratio data. The method provided in this application embodiment can utilize a joint probability distribution to determine the data augmentation control parameters, thereby realizing a joint random augmentation method. This is beneficial for subsequently generating highly complex positive sample pairs, increasing the training difficulty of the model. Simultaneously, it also considers possible anomalies, ensuring the rationality of the data augmentation control parameters.

[0111] The following example illustrates the method for determining the data augmentation parameters (i.e., cropping area) of the first sample image, using the cropping area as an example. Existing methods for determining the cropping area are independent random cropping methods, the specific process of which is as follows:

[0112] Step 1: Obtain the maximum value of the cutting area (S) max ) and minimum value (S) min );

[0113] Step 2: Construct the average distribution corresponding to the cropped area (S1) of the first sample image and the average distribution corresponding to the cropped area (S2) of the second sample image, i.e., S1~U[S min ,S max ], S2~U[S min ,S max ];

[0114] Step 3: Determine the cropping area of ​​the first sample image and the cropping area of ​​the second sample image based on the average distribution corresponding to the cropping area of ​​the first sample image and the average distribution corresponding to the cropping area of ​​the second sample image, respectively.

[0115] The method for determining data augmentation parameters provided in this application is a joint random pruning method, the specific process of which is as follows:

[0116] Step 1: Obtain enhanced boundary control data (i.e., the maximum value of the clipping area (S)). max ) and minimum value (S) min )), and similarity control data (β);

[0117] Step 2: Determine the boundary ratio data (S) based on the enhanced boundary control data. b ), that is, S b =log(S) max / S min );

[0118] Step 3: If the similarity control data β is greater than the preset reference threshold (0), then construct the first normal distribution, i.e., log(S r )~N T (0,(1 / β)*S b ,-S b ,S b ), and can perform data extraction processing according to the first normal distribution to obtain the first data extraction result (log(S)). r Based on the results of the first data extraction, the data augmentation control parameters (S) can be determined. r Jump to step 6;

[0119] Step 4: If the similarity control data β equals the preset reference threshold (0), then construct the first average distribution, i.e., log(S r )~U[-S b ,S b ], and can perform data extraction processing according to the first average distribution to obtain the first data extraction result (log(S) r Based on the results of the first data extraction, the data augmentation control parameters (S) can be determined. r Jump to step 6;

[0120] Step 5: If the similarity control data β is less than the preset reference threshold (0), then construct a second normal distribution based on the absolute value of the similarity control data β, and determine the flipped distribution corresponding to the data augmentation control parameters based on the second normal distribution, i.e., log(S r )~N T (0,-(1 / β)*S b ,-S b ,S b ), and data extraction can be performed based on this flipped distribution to obtain the second data extraction result (log(S)). r If the second data extraction result is greater than or equal to a preset value (which can be 0), then the data augmentation control parameter (S) can be determined based on the second data extraction result and the boundary ratio data. r ), that is, log(S) r ) = S b -log(S rIf the second data extraction result is less than the preset value (which can be 0), the data augmentation control parameter (S) can be determined based on the second data extraction result and the negative value of the boundary ratio data. r ), that is, log(S) r )=-S b -log(S r ).

[0121] Step 6: Based on the data augmentation control parameters (S) r The data augmentation parameters of the first sample image are determined based on the data augmentation parameters of the first sample image and the data augmentation control parameters, and the data augmentation parameters of the second sample image are determined based on the data augmentation parameters of the first sample image and the data augmentation control parameters.

[0122] As can be seen from the comparison of the above processes, the joint random cropping method provided in this application does not independently determine the data augmentation parameters of the two sample images in the positive sample pair. Instead, it comprehensively considers the correlation between the two sample images in the positive sample pair, first determines the data augmentation control parameters, and then determines the data augmentation parameters of the first sample image based on the data augmentation control parameters. The method provided in this application can generate positive sample pairs with higher complexity. At the same time, in the application process, the complexity of the positive sample pairs can be controlled by adjusting the similarity control data, which is conducive to generating more positive sample pairs that meet the application requirements, thereby increasing the training difficulty of the model and improving the pre-training effect of the model.

[0123] S403. Based on the data augmentation parameters of the first sample image and the data augmentation control parameters, determine the data augmentation parameters of the second sample image included in the positive sample pair; wherein the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold.

[0124] In this embodiment, the image processing device can determine the data augmentation parameters of the second sample image included in the positive sample pair based on the data augmentation parameters and data augmentation control parameters of the first sample image. The difference between the data augmentation parameters of the first and second sample images is greater than or equal to a preset difference threshold. This preset difference threshold can be the difference value between the two sample images in the positive sample pair determined using an independent random augmentation method. Compared with the independent random augmentation scheme, the joint random augmentation method provided in this embodiment can determine the data augmentation parameters of the first and second sample images with significant differences, thereby generating positive sample pairs containing these significantly different first and second sample images. This allows for the determination of more complex positive sample pairs, increasing the training difficulty of the model trained using positive sample pairs and improving the model's training effect.

[0125] S404. Perform data augmentation processing on the sample image according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image respectively, to obtain the positive sample pair corresponding to the sample image, including the first sample image and the second sample image; wherein, the positive sample pair is used to train a neural network model to obtain an image processing model.

[0126] In this embodiment, the image processing device can perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the second sample image, respectively, to obtain a positive sample pair corresponding to the sample image, including the first sample image and the second sample image, wherein the similarity between the first sample image and the second sample image is low. The determined positive sample pair can be used to train a neural network model to obtain an image processing model. The method provided in this embodiment can determine positive sample pairs with higher complexity, which is beneficial to improving the training efficiency of the model trained using the positive sample pair and also improves the training effect of the model, resulting in higher prediction accuracy of the trained model.

[0127] In one embodiment, taking cropping as an example of data augmentation method, the data augmentation parameters may include the cropping area. The specific implementation of performing data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the second sample image to obtain the first and second sample images included in the positive sample pair corresponding to the sample image can be as follows: performing joint cropping processing on the sample images according to the cropping area of ​​the first sample image to obtain the first sample image included in the positive sample pair corresponding to the sample image; and performing joint cropping processing on the sample images according to the cropping area of ​​the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0128] Specifically, after determining the cropping area S1 of the first sample image, the aspect ratio r1 of the first sample image can be randomly determined. Then, based on the cropping area S1 and the aspect ratio r1, the width of the first sample image can be determined as follows: The height of the first sample image is We can define the probability distribution corresponding to the cropping x-coordinate i1 of the first sample image as the average distribution U[0,Ww] (where W represents the width of the sample image), and the probability distribution corresponding to the cropping y-coordinate j1 as the average distribution U[0,Hh] (where H represents the height of the sample image). Based on the cropping x-coordinate i of the first sample image... nThe corresponding probability distribution determines the cropping x-coordinate i1 of the first sample image, and the cropping y-coordinate j1 of the first sample image is determined according to the probability distribution corresponding to the cropping y-coordinate j1 (the cropping x-coordinate and cropping y-coordinate of the first sample image are collectively referred to as the cropping position of the first sample image). After determining the width, height, cropping x-coordinate, and cropping y-coordinate of the first sample image, joint random cropping processing can be performed on the sample image based on these parameters to obtain the first sample image. A similar method can be used to determine the second sample image. The joint random cropping method provided in this application embodiment can obtain positive sample pairs with significantly different cropping areas, thereby increasing the complexity of the positive sample pairs and thus improving the training effect of model training using these positive sample pairs.

[0129] Please see Figure 6 This figure is a schematic diagram of an image processing method provided in an embodiment of this application. Figure 6 Figure 6a shows sample image patch 1 and sample image 2 obtained by independently and randomly cropping the sample images using the independent random cropping method. As can be seen from the figure, the cropping areas of sample image patch 1 and sample image patch 2 are quite similar. Figure 6b shows the feature Z corresponding to sample image patch 1 in the feature encoding space. q1 And the feature Z corresponding to sample image patch 2 k1 As can be seen from the figure, feature Z q1 With feature Z k1 The feature distance between them is relatively short, meaning that the similarity between sample image block 1 and sample image block 2 is high. Figure 6c shows sample image block 3 and sample image block 4 obtained by determining the data augmentation parameters using the joint random cropping method provided in this embodiment and performing joint random cropping on the sample images. As can be seen from the figure, the cropping areas of sample image block 3 and sample image block 4 differ significantly. Figure 6d shows the feature Z corresponding to sample image block 3 in the feature encoding space. q2 And the feature Z corresponding to sample image block 4 k2 As can be seen from the figure, feature Z q2 With feature Z k2 The feature distance between them is relatively large, meaning that the similarity between sample image patch 3 and sample image patch 4 is low. For example... Figure 6 As shown, the method provided in this application embodiment can identify positive sample pairs containing two sample images with low similarity, which is beneficial for subsequently using the positive sample pairs to train a model with higher prediction accuracy.

[0130] It should be noted that although the data augmentation parameters in the above process only include the cropping area, that is, only the cropping area is explicitly considered when performing data augmentation on the sample objects, the consideration of the cropping area also implicitly controls the distance between the first and second sample images. Let the cropping x-coordinate of the first sample image be i1, and the cropping y-coordinate be j1 (the cropping x-coordinate and cropping y-coordinate of the first sample image are collectively referred to as the cropping position of the first sample image); let the cropping x-coordinate of the second sample image be i2, and the cropping y-coordinate be j2, then the distance between the first and second sample images can be... The greater the distance between the first sample image and the second sample image, the higher the complexity of the positive sample pair. The joint random cropping method provided in this application can determine the cropping area S1 of the first sample image. Since the cropping abscissa and ordinate of the first sample image can be determined based on the cropping area, determining the cropping area in the method provided in this application can also affect the cropping position of the first sample image, thereby affecting the distance between the first and second sample images. In some cases, the data augmentation parameters in the image processing method provided in this application can directly include the cropping position to directly control the distance between the first and second sample images, thereby increasing the complexity of the positive sample pair.

[0131] Please see Figure 7 This figure is a schematic diagram of another image processing method provided in an embodiment of this application. Figure 7 As shown in 7a and 7b, if the first sample image and the second sample image are close in distance within the sample image, the similarity between the first sample image and the second sample image is high (as shown in 7a); if the first sample image and the second sample image are far apart in the sample image, the similarity between the first sample image and the second sample image is low. In the image processing method provided in this application embodiment, when the data augmentation parameter includes the cropping area, the method provided in this application can implicitly influence the cropping position of the first sample image and the second sample image. For example... Figure 7As shown in 7c, the horizontal axis represents the similarity control data β, and the vertical axis represents the distance between the first sample image and the second sample image. In 7c, the dashed line represents the distance between the first and second sample images determined using the independent random cropping method, and the solid line represents the distance between the first and second sample images determined using the joint random cropping method provided in this application. As can be seen from 7c, as the similarity control data β gradually increases, the distance between the first and second sample images becomes smaller; that is, the smaller the similarity control data β, the larger the distance between the first and second sample images, and the lower the similarity between them. Through the method provided in this application, the distance between the first and second sample images can be implicitly influenced by the joint random cropping method, thereby obtaining positive sample pairs with higher complexity.

[0132] In one embodiment, taking blurring or grayscale as an example of data augmentation method, the data augmentation parameters may include target blurring or target grayscale; the specific implementation of performing data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image to obtain the positive sample pair corresponding to the sample image, including the first sample image and the second sample image, can be as follows:

[0133] Based on the enhanced boundary control data, the sample images are randomly cropped to obtain a first intermediate sample image and a second intermediate sample image. Based on the target ambiguity or target grayscale of the first sample image, the first intermediate sample image is blurred or grayscaled to obtain the first sample image included in the positive sample pair corresponding to the sample image. Based on the target ambiguity or target grayscale of the second sample image, the second intermediate sample image is blurred or grayscaled to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0134] Specifically, when the data augmentation parameters include target blur or target gray level, the image processing device can directly perform joint random augmentation processing on the sample image according to the data augmentation parameters to obtain positive sample pairs. For example, if the data augmentation parameters include target blur, the image processing device can perform joint random blur processing on the sample image according to the target blur of the first sample image to obtain the first sample image, and perform joint random blur processing on the sample image according to the target blur of the second sample image to obtain the second sample image.

[0135] Image processing equipment can also employ a combination of enhancement methods to achieve data enhancement of sample images, thereby obtaining more complex positive sample pairs. The image processing equipment can use an independent random cropping method, performing independent random cropping on the sample images according to the enhancement boundary control data to obtain a first intermediate sample image and a second intermediate sample image. If the data enhancement parameters include target blur, then the first intermediate sample image can be subjected to joint random blurring based on the target blur of the first sample image to obtain the first sample image; similarly, the second intermediate sample image can be subjected to joint random blurring based on the target blur of the second sample image to obtain the second sample image. If the data enhancement parameters include target grayscale, then the first intermediate sample image can be subjected to joint random grayscale processing based on the target grayscale of the first sample image to obtain the first sample image; similarly, the second intermediate sample image can be subjected to joint random grayscale processing based on the target grayscale of the second sample image to obtain the second sample image.

[0136] It should be noted that the method for determining the target blur or target grayscale of the first sample image is similar to the method for determining the cropping area of ​​the first sample image in the above embodiments, except that the maximum area value (S) in the above method is used. max Replace with the maximum blur level or maximum gray level, minimum area (S) min The value can be replaced with the minimum blur or minimum grayscale value. Other enhancement methods (e.g., stretching, color dithering, etc.) can use the methods provided in the embodiments of this application to determine the data enhancement parameters. For example, the data enhancement method for the sample image is stretching or color dithering. Stretching can be a proportional enlargement or reduction of the sample image according to the stretching parameters. Color dithering can be an adjustment of the color data of the sample image. The data enhancement parameters may also include target stretching parameters or color dithering parameters. The method for determining the target stretching parameters or color dithering parameters of the first sample image can be similar to the method for determining the cropping area of ​​the first sample image in the above embodiments, and will not be described again here.

[0137] The method provided in this application embodiment can be used to enhance sample images by combining various enhancement methods (e.g., independent random cropping, joint random blurring, joint random grayscale, etc.), thereby further increasing the complexity of positive sample pairs and improving the training effect of the model.

[0138] In one embodiment, the image processing method provided in this application may further include the following steps: obtaining a positive sample set containing multiple positive sample pairs and a negative sample set containing multiple negative sample pairs, wherein the negative sample pairs are determined based on the multiple positive sample pairs; inputting the positive sample set and the negative sample set into a neural network model for comparative learning processing to obtain comparative learning difference data; and adjusting the model parameters of the neural network model based on the comparative learning difference data to obtain an image processing model.

[0139] Specifically, the method provided in this application embodiment can be used to determine multiple positive sample pairs and generate a positive sample set based on these positive sample pairs. Since the negative samples are generated from different sample images, the negative sample set can be directly determined based on the sample images contained in different positive sample pairs. The negative sample set may also include sample images different from those in the positive sample set. The image processing device can input the positive sample set and the negative sample set into a neural network model for comparative learning processing to obtain the model prediction result. Based on the model prediction result and the positive and negative sample sets, comparative learning difference data is determined, which can reflect the degree of recognition of the sample image by the neural network model to a certain extent. For example, if the neural network model is an image classification model, a target sample image is determined, and the positive and negative sample images of the target sample image are determined through the positive and negative sample sets. The target sample image, positive sample image, and negative sample image are input into the neural network model for comparative learning processing to obtain the category prediction result. Based on the category prediction result, the positive and negative sample images, the comparative learning difference data is determined. If the prediction accuracy of the neural network model is high, the comparative learning difference data is small.

[0140] The model parameters of the neural network model are adjusted based on the comparative learning difference data. When the number of adjustments to the model parameters reaches a preset number or the prediction accuracy of the trained neural network model reaches a preset requirement, the neural network model at this point can be identified as an image processing model. The method provided in this application embodiment can utilize highly complex positive samples for comparative learning processing of the model, thereby obtaining an image processing model with high prediction accuracy. Furthermore, because the positive sample pairs used for training have higher complexity, the model's training efficiency is higher, and the training effect is better.

[0141] The image processing method provided in this application can adjust the complexity of positive sample pairs through similarity control parameters, thereby improving the controllability of the generated positive sample pairs and facilitating the generation of positive sample pairs that meet application requirements. It can first determine the data augmentation control parameters reflecting the correlation between the data augmentation parameters of the first and second sample images, thus comprehensively considering the relationship between the first and second sample images and avoiding situations where the first and second sample images have high similarity. The method provided in this application can be applied to various data augmentation scenarios (e.g., cropping, blurring, stretching, grayscale, etc.), exhibiting good universality and meeting diverse application needs. The method provided in this application combines the joint random augmentation method and the independent random augmentation method provided in this application, further increasing the complexity of the generated positive sample pairs. Using the generated high-complexity positive samples for comparative learning of the model can improve the model's training efficiency and effect, resulting in an image processing model with high prediction accuracy.

[0142] Please see Figure 8 , Figure 8 This is a schematic flowchart of another image processing method provided in an embodiment of this application. This image processing method can be implemented by the image processing device 301 described above, by the model training device 302 described above, or by other devices capable of implementing this image processing method. The following description uses the implementation of the image processing device 301 as an example. The flow of the image processing method provided in this embodiment includes, but is not limited to:

[0143] S801. Obtain data augmentation configuration parameters for the sample image, wherein the data augmentation control parameters include augmentation boundary control data and similarity control data.

[0144] In this embodiment, the image processing device can acquire data augmentation configuration parameters for a sample image. These parameters may include augmentation boundary control data and similarity control data. The augmentation boundary control data indicates the boundary between the data augmentation parameters of the subsequently generated first sample image and the data augmentation parameters of the second sample image. The similarity control data controls the similarity between the subsequently generated first and second sample images, and can be represented by β.

[0145] S802. Determine data augmentation control parameters based on the enhanced boundary control data and the similarity control data, and determine data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the enhanced boundary control data.

[0146] In this embodiment, the image processing device can determine data augmentation control parameters based on enhanced boundary control data and similarity control data. These data augmentation control parameters can indicate the correlation between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image. For example, when the data augmentation parameter is the cropping area, if the cropping area of ​​the first sample image is S1 and the cropping area of ​​the second sample image is S2, then the data augmentation control parameter S... r It can be S r =S1 / S2, where the data augmentation control parameter indicates the ratio of the cropped area of ​​the first sample image to the cropped area of ​​the second sample image. The specific implementation method for determining the data augmentation control parameter is shown in step S402 above.

[0147] After determining the data augmentation control parameters, the data augmentation information of the first sample image, which is included in the positive sample pair corresponding to the sample image, can be determined based on the data augmentation control parameters and the augmentation boundary control data. This data augmentation information can indicate the probability distribution corresponding to the data augmentation parameters of the first sample image. For example, the data augmentation information of the first sample image can be "the probability distribution corresponding to the data augmentation parameters of the first sample image is a first probability distribution". The method provided in this application embodiment can determine the data augmentation information of the first sample image, which is beneficial for subsequently determining reasonable data augmentation parameters for the first sample image based on this data augmentation information, thereby obtaining a high-complexity positive sample pair.

[0148] In one embodiment, the specific implementation method for determining the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image, based on the data augmentation control parameters and augmentation boundary control data, can be as follows:

[0149] Determine the first ratio between the first augmentation boundary included in the augmentation boundary control data and the first data augmentation control parameter, and define the minimum value between the first ratio and the data augmentation control parameter as the first distribution boundary; for example, the first augmentation boundary can be represented as S. max The data augmentation control parameter is represented as S. r Then the first ratio is S max / S r The first distribution boundary can be min(S) max / S r ,S max The function min represents taking the minimum of two values.

[0150] Determine the second ratio between the second augmentation boundary included in the augmentation boundary control data and the second data augmentation control parameter, and define the maximum value of the second ratio and the data augmentation control parameter as the second distribution boundary; for example, the second augmentation boundary can be represented as S. min The data augmentation control parameter is represented as S.r Then the second ratio is S min / S r The second distribution boundary can be max(S) min / S r ,S min The function max represents taking the maximum of two values.

[0151] The first and second distribution boundaries are averaged to construct a second average distribution corresponding to the data augmentation parameters of the first sample image; for example, the second average distribution corresponding to the data augmentation parameters of the first sample image can be U[max(S min / S r ,S min ),min(S max / S r ,S max )).

[0152] Based on the second average distribution corresponding to the data augmentation parameters of the first sample image, the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image is determined. Specifically, the image processing device can determine the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image based on the second average distribution corresponding to the data augmentation parameters of the first sample image. For example, the data augmentation information can be S1~U[max(S min / S r ,S min ),min(S max / S r ,S max (where S1 is the data augmentation parameter of the first sample image). The method provided in this application embodiment can construct data augmentation information corresponding to the data augmentation parameters of the first sample image based on the data augmentation control parameters. This data augmentation information can indicate the probability distribution corresponding to the data augmentation parameters, which is beneficial for subsequently determining reasonable data augmentation parameters for the first sample image based on this data augmentation information, thereby generating high-complexity positive sample pairs.

[0153] S803. Determine the data augmentation parameters of the first sample image based on the data augmentation information of the first sample image.

[0154] In this embodiment, the data augmentation information of the first sample image can indicate the probability distribution corresponding to the data augmentation parameters of the first sample image. The image processing device can perform data extraction processing based on the probability distribution corresponding to the data augmentation parameters of the first sample image to obtain the data augmentation parameters of the first sample image, which are within the data range indicated by the augmentation boundary control data. Through the method provided in this embodiment, reasonable data augmentation parameters for the first sample image can be determined using the data augmentation information, which is beneficial for subsequently determining the digital augmentation parameters of the second sample image based on the data augmentation parameters of the first sample image, thereby obtaining high-complexity positive sample pairs.

[0155] S804. Based on the data augmentation parameters of the first sample image and the data augmentation control parameters, determine the data augmentation parameters of the second sample image included in the positive sample pair; wherein the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold.

[0156] In this embodiment, since the data augmentation control parameters can indicate the correlation between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, the image processing device can determine the data augmentation parameters of the second sample image based on the data augmentation parameters of the first sample image and the data augmentation control parameters. The difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold. A larger difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image indicates a larger difference between the first and second sample images. This preset difference threshold can be the difference value between two sample images in a positive sample pair determined using an independent random augmentation method. The method provided in this embodiment can determine the data augmentation parameters of the first and second sample images with significant differences, thereby facilitating the generation of positive sample pairs containing the first and second sample images with significant differences.

[0157] The following section uses data augmentation methods as an example to illustrate the joint random pruning method provided in this application, using algorithmic language:

[0158] Algorithm input: Enhanced boundary control data (i.e., S) min and S max ) and similarity control data β

[0159]

[0160]

[0161] Algorithm outputs: S1 and S2.

[0162] As can be seen from the above algorithm, the image processing method provided in this application adopts a joint random cropping method. First, the data augmentation control parameters are determined. These data augmentation control parameters can indicate the correlation between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image. Then, the data augmentation parameters of the first sample image are determined based on the data augmentation control parameters, and then the data augmentation parameters of the second sample image can be determined. This makes the difference between the first sample image and the second sample image greater, which is conducive to generating more complex positive sample pairs.

[0163] It should be noted that in some cases, the image processing method provided in this application can be used to perform joint random enhancement processing on the sample images to obtain positive sample pairs; alternatively, a processing scheme combining multiple enhancement methods can be used to enhance the sample images to obtain positive sample pairs. The enhancement methods can be any one of joint random cropping, joint random blurring, joint random grayscale, joint random stretching, independent random cropping, etc. For example, a method combining joint random cropping and joint random blurring can be used to determine positive sample pairs. Specifically, the data enhancement parameters can include the cropping area and the target blur. The sample images can first be subjected to joint random cropping processing based on the cropping area to obtain a first intermediate sample image, and then joint random blurring processing can be performed on the first intermediate sample image to obtain the first sample image. In this case, to avoid excessive differences between the first sample image and the second sample image, which could prevent the model from learning image features effectively, certain limitations can be imposed on the similarity control data. The method provided in the embodiments of this application can yield positive sample pairs with higher complexity.

[0164] S805. Perform data augmentation processing on the sample image according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image respectively, to obtain the positive sample pair corresponding to the sample image, including the first sample image and the second sample image; wherein, the positive sample pair is used to train a neural network model to obtain an image processing model.

[0165] In this embodiment, the image processing device can perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the second sample image, respectively, to obtain a positive sample pair corresponding to the sample image, including the first sample image and the second sample image, wherein the similarity between the first sample image and the second sample image is low. The determined positive sample pair can be used to train a neural network model to obtain an image processing model. The method provided in this embodiment can determine positive sample pairs with higher complexity, which is beneficial for increasing the training difficulty of the model and resulting in higher prediction accuracy of the trained model.

[0166] Please see Figure 9The figure is a schematic diagram of the complexity of a positive sample pair provided in an embodiment of this application. Figure 9 The solid line in 9a illustrates how the cosine similarity between the first and second sample images changes with the similarity control data during the process of enhancing sample images using the joint random enhancement method provided in this application to obtain positive sample pairs. As shown in 9a, the larger the similarity control data, the higher the cosine similarity between the first and second sample images; the cosine similarity between the first and second sample images obtained using the independent random enhancement method remains consistently high (as shown by the dashed line in 9a, the cosine similarity is 0.9297). Figure 9 In Figure 9b, the horizontal axis represents the training rounds, and the vertical axis represents the similarity of the generated positive sample pairs. As shown in Figure 9b, compared to independent random augmentation methods, the joint random augmentation method provided in this application can generate positive sample pairs with higher complexity and lower similarity.

[0167] The method provided in this application was tested on a small dataset, and the results are shown in Table 1 below.

[0168] Table 1:

[0169]

[0170] In Table 1 above, SimCLR, BYOL, MoCo v2, and SimSiam represent different image processing models, STL-10 and Tiny-ImageNet represent small datasets containing sample images, Baseline represents independent random augmentation methods, and J-Crop(β) represents the joint random cropping method provided in this application embodiment. In this case, the data augmentation parameters include the cropping area, and β is the similarity control data. The value of β in Table 1 is 0. As shown in Table 1, on small datasets, the image processing method provided in this application can improve the training effect of the model to a certain extent. The data in Table 1 shows that compared with independent random augmentation methods, the joint random augmentation method provided in this application embodiment can generate positive sample pairs with higher complexity and lower similarity. Using these higher-complexity positive sample pairs for model training can yield a model with better training effect and improve the model's prediction accuracy.

[0171] When using the method provided in the embodiments of this application to determine the clipping area and target ambiguity, the relevant data results of the model training effect can be shown in Table 2 below.

[0172] Table 2:

[0173]

[0174] In Table 2 above, SimCLR, SimSiam, MoCo v1, MoCo v2, and MoCo v3 represent different image processing models. Batch Size indicates the number of samples selected in one training iteration, and Epoch represents the training round. Baseline indicates an independent random augmentation method, J-Crop indicates the use of the joint random cropping method provided in this application, and J-Blur indicates the use of the joint random blurring method provided in this application. As shown in Table 1, both the joint random cropping method and the joint random blurring method can increase the complexity of positive sample pairs, thereby improving the training effect of using positive sample pairs for model training and ultimately improving the prediction accuracy of the trained model.

[0175] When the similarity control data changes, the relevant data on the model training effect can be shown in Table 3 below.

[0176] Table 3:

[0177]

[0178] In Table 3 above, Baseline represents the independent random augmentation method, J-Crop represents the joint random cropping method provided in this application, and β is the similarity control data. As shown in Table 3, when β is less than or equal to 0 (for example, when β is -1, the training effect of the SimSiam model is significantly improved; when β is -2, the training effect of the MoCO v2 model is significantly improved), compared with the independent random augmentation method, the joint random cropping method provided in this application embodiment can significantly generate positive sample pairs with higher complexity and lower similarity, thereby significantly improving the training effect of various image processing models.

[0179] When the method provided in the embodiments of this application is applied to downstream detection and segmentation tasks, the test results are shown in Table 4 below.

[0180] Table 4:

[0181]

[0182] In Table 4 above, Pre-train represents pre-training, and VOC and COCO are both datasets. J-Crop indicates the use of the joint random cropping method provided in this application, and β is the similarity control data, with a value of 0 in Table 4. As can be seen from Table 4, the image processing method provided in this application, when applied after an image processing model (such as MoCo v1, MoCo v2, etc.), can achieve good results in downstream detection and segmentation tasks.

[0183] The image processing method provided in this application can adjust the complexity of positive sample pairs through similarity control parameters, thereby improving the controllability of the generated positive sample pairs and facilitating the generation of positive sample pairs that meet application requirements. It can first determine the data augmentation control parameters reflecting the correlation between the data augmentation parameters of the first and second sample images, thus comprehensively considering the relationship between the first and second sample images and avoiding situations where the first and second sample images have high similarity. The method provided in this application can be applied to various data augmentation scenarios (e.g., cropping, blurring, stretching, grayscale, etc.), exhibiting good universality and meeting diverse application needs. The method provided in this application combines the joint random augmentation method and the independent random augmentation method provided in this application, further increasing the complexity of the generated positive sample pairs. Using the generated high-complexity positive samples for comparative learning of the model can improve the model's training efficiency and effect, resulting in an image processing model with high prediction accuracy.

[0184] Please see Figure 10 , Figure 10 This is a structural block diagram of an image processing apparatus provided in an embodiment of this application. The image processing apparatus can be disposed within the computer device provided in this embodiment of the application, and the computer device can be as described above. Figure 3 The image processing device 301 in the image processing system shown. Figure 10 The image processing device shown can be a computer program running on a computer device, which can be used to perform... Figure 4 or Figure 8 The image processing method embodiments shown include some or all of the steps. Please refer to [link / reference]. Figure 10 The image processing apparatus may include the following units:

[0185] The acquisition unit 1001 is used to acquire data augmentation configuration parameters of the sample image, wherein the data augmentation control parameters include augmentation boundary control data and similarity control data;

[0186] The determining unit 1002 is configured to determine data augmentation control parameters based on the augmentation boundary control data and the similarity control data, and to determine data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data.

[0187] The determining unit 1002 is further configured to determine the data enhancement parameters of the second sample image included in the positive sample pair based on the data enhancement parameters of the first sample image and the data enhancement control parameters; wherein the difference between the data enhancement parameters of the first sample image and the data enhancement parameters of the second sample image is greater than or equal to a preset difference threshold.

[0188] The processing unit 1003 is configured to perform data augmentation processing on the sample image according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, respectively, to obtain a positive sample pair corresponding to the sample image, including the first sample image and the second sample image; wherein, the positive sample pair is used to train a neural network model to obtain an image processing model.

[0189] In one embodiment, the enhanced boundary control data includes a first enhanced boundary and a second enhanced boundary; the determining unit 1002, when determining the data enhancement control parameters based on the enhanced boundary control data and the similarity control data, specifically performs the following steps:

[0190] Determine the boundary ratio data based on the first enhancement boundary and the second enhancement boundary;

[0191] Based on the boundary ratio data and the similarity control data, determine the joint probability distribution corresponding to the data augmentation control parameters;

[0192] The data augmentation control parameters are determined based on the joint probability distribution corresponding to the data augmentation control parameters.

[0193] In one embodiment, when determining the joint probability distribution corresponding to the data augmentation control parameters based on the boundary ratio data and the similarity control data, the determining unit 1002 specifically performs the following steps:

[0194] If the similarity control data is equal to the preset reference threshold, then the boundary ratio data is processed by average distribution construction to obtain the joint probability distribution corresponding to the data augmentation control parameters, and the joint probability distribution is the first average distribution;

[0195] If the similarity control data is greater than the preset reference threshold, then the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain the joint probability distribution corresponding to the data augmentation control parameters, wherein the joint probability distribution is a first normal distribution;

[0196] If the similarity control data is less than the preset reference threshold, then the absolute values ​​of the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain a second normal distribution, and the second normal distribution is flipped to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is a flipped distribution.

[0197] In one embodiment, the determining unit 1002, when determining the data augmentation control parameters based on the joint probability distribution corresponding to the data augmentation control parameters, specifically performs the following steps:

[0198] If the joint probability distribution corresponding to the data augmentation control parameters is a first average distribution or a first normal distribution, then data extraction processing is performed according to the first average distribution or the first normal distribution to obtain a first data extraction result, and the data augmentation control parameters are determined according to the first data extraction result.

[0199] If the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution, then data extraction processing is performed according to the flipped distribution to obtain the second data extraction result;

[0200] If the second data extraction result is greater than or equal to the preset value, then the data enhancement control parameter is determined based on the second data extraction result and the boundary ratio data.

[0201] If the second data extraction result is less than the preset value, then the data augmentation control parameter is determined based on the second data extraction result and the negative value of the boundary ratio data.

[0202] In one embodiment, the determining unit 1002, when determining the data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data, specifically performs the following steps:

[0203] Based on the data augmentation control parameters and the augmentation boundary control data, determine the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image;

[0204] The data augmentation parameters of the first sample image are determined based on the data augmentation information of the first sample image.

[0205] In one embodiment, the determining unit 1002, when determining the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data, specifically performs the following steps:

[0206] Determine a first ratio between the first augmentation boundary included in the augmentation boundary control data and the first data augmentation control parameter, and determine the minimum value between the first ratio and the data augmentation control parameter as the first distribution boundary;

[0207] Determine a second ratio between the second augmentation boundary included in the augmentation boundary control data and the second data augmentation control parameter, and determine the maximum value of the second ratio and the data augmentation control parameter as the second distribution boundary;

[0208] The first distribution boundary and the second distribution boundary are processed to construct an average distribution, thereby obtaining the second average distribution corresponding to the data augmentation parameters of the first sample image.

[0209] Based on the second average distribution corresponding to the data augmentation parameters of the first sample image, the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image is determined.

[0210] In one embodiment, the data augmentation parameters include the cropping area; the processing unit 1003 is configured to perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, respectively, to obtain the positive sample pair corresponding to the sample images, including the first sample image and the second sample image, specifically by performing the following steps:

[0211] The sample image is jointly cropped based on the cropping area of ​​the first sample image to obtain the first sample image included in the positive sample pair corresponding to the sample image.

[0212] The sample images are jointly cropped based on the cropping area of ​​the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0213] In one embodiment, the data augmentation parameters include target blur or target grayscale; the processing unit 1003 is configured to perform data augmentation processing on the sample image according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image respectively, and to obtain the positive sample pair corresponding to the sample image including the first sample image and the second sample image, specifically to perform the following steps:

[0214] The sample images are randomly cropped according to the enhanced boundary control data to obtain a first intermediate sample image and a second intermediate sample image.

[0215] Based on the target blur or target grayscale of the first sample image, the first intermediate sample image is blurred or grayscaled to obtain the first sample image included in the positive sample pair corresponding to the sample image;

[0216] The second intermediate sample image is blurred or grayscaled according to the target blur or target grayscale of the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0217] In one embodiment, the processing unit 1003 is further configured to perform the following steps:

[0218] Obtain a positive sample set containing multiple positive sample pairs, and a negative sample set containing multiple negative sample pairs, wherein the negative sample pairs are determined based on the multiple positive sample pairs;

[0219] The positive sample set and the negative sample set are input into the neural network model for comparative learning processing to obtain comparative learning difference data.

[0220] The model parameters of the neural network model are adjusted based on the contrastive learning difference data to obtain the image processing model.

[0221] Figure 10 The various units in the image processing apparatus shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the image processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0222] In one embodiment, the ability to perform such operations can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). Figure 4 or Figure 8 Computer programs for the steps involved in some or all of the methods shown, to construct, for example... Figure 10 The image processing apparatus shown herein, and the image processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computing device via the computer-readable storage medium, and executed therein.

[0223] The image processing apparatus provided in this application embodiment can adjust the complexity of positive sample pairs by using similarity control parameters, thereby improving the controllability of the generated positive sample pairs and facilitating the generation of positive sample pairs that meet application requirements. Data augmentation control parameters reflecting the correlation between the data augmentation parameters of the first and second sample images can be determined first, thus comprehensively considering the relationship between the first and second sample images and avoiding situations where the first and second sample images have high similarity. The method provided in this application embodiment can be applied to various data augmentation scenarios (e.g., cropping, blurring, stretching, grayscale, etc.), exhibiting good universality and meeting diverse application needs. The method provided in this application embodiment combines the joint random augmentation method and the independent random augmentation method provided in this application, further increasing the complexity of the generated positive sample pairs. Using the generated high-complexity positive samples for comparative learning processing of the model can improve the model's training efficiency and effect, resulting in an image processing model with high prediction accuracy.

[0224] Based on the above methods and apparatus embodiments, this application provides a computer device. Please refer to... Figure 11 , Figure 11 This is a structural block diagram of a computer device provided in an embodiment of this application. Figure 11 The computer device shown can be the one described above. Figure 3 Image processing device 301 in the middle. Figure 11 The computer device shown includes at least a processor 1101, an input interface 1102, an output interface 1103, and a computer-readable storage medium 1104. The processor 1101, input interface 1102, output interface 1103, and computer-readable storage medium 1104 can be connected via a bus or other means.

[0225] The computer-readable storage medium 1104 can be stored in the memory of a computer device. The computer-readable storage medium 1104 is used to store computer programs, which include computer instructions. The processor 1101 is used to execute the computer program stored in the computer-readable storage medium 1104. The processor 1101 (or CPU (Central Processing Unit)) is the computing and control core of the computer device. It is suitable for implementing computer programs, specifically for loading and executing computer programs to achieve corresponding methods or functions.

[0226] This application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space for storing the operating system of the computer device. Furthermore, the storage space also stores computer programs suitable for loading and execution by a processor. It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0227] In a specific implementation, the processor 1101 can load and execute the computer program stored in the computer-readable storage medium 1104 to achieve the aforementioned related... Figure 4 or Figure 8 The corresponding steps in the image processing method shown. In a specific implementation, the computer program in the computer-readable storage medium 1104 is loaded by the processor 1101 and executed as follows:

[0228] Data augmentation configuration parameters for sample images are obtained, including augmentation boundary control data and similarity control data;

[0229] Based on the enhanced boundary control data and the similarity control data, data augmentation control parameters are determined, and based on the data augmentation control parameters and the enhanced boundary control data, data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image are determined.

[0230] Based on the data augmentation parameters of the first sample image and the data augmentation control parameters, the data augmentation parameters of the second sample image included in the positive sample pair are determined; wherein, the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold.

[0231] The sample images are augmented using the data augmentation parameters of the first and second sample images respectively, to obtain a positive sample pair including the first and second sample images; wherein the positive sample pair is used to train a neural network model to obtain an image processing model.

[0232] In one embodiment, the enhanced boundary control data includes a first enhanced boundary and a second enhanced boundary; when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to determine the data enhancement control parameters based on the enhanced boundary control data and the similarity control data, it is specifically used to perform the following steps:

[0233] Determine the boundary ratio data based on the first enhancement boundary and the second enhancement boundary;

[0234] Based on the boundary ratio data and the similarity control data, determine the joint probability distribution corresponding to the data augmentation control parameters;

[0235] The data augmentation control parameters are determined based on the joint probability distribution corresponding to the data augmentation control parameters.

[0236] In one embodiment, when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to determine the joint probability distribution corresponding to the data augmentation control parameters based on the boundary ratio data and the similarity control data, it is specifically used to perform the following steps:

[0237] If the similarity control data is equal to the preset reference threshold, then the boundary ratio data is processed by average distribution construction to obtain the joint probability distribution corresponding to the data augmentation control parameters, and the joint probability distribution is the first average distribution;

[0238] If the similarity control data is greater than the preset reference threshold, then the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain the joint probability distribution corresponding to the data augmentation control parameters, wherein the joint probability distribution is a first normal distribution;

[0239] If the similarity control data is less than the preset reference threshold, then the absolute values ​​of the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain a second normal distribution, and the second normal distribution is flipped to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is a flipped distribution.

[0240] In one embodiment, when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to determine the data augmentation control parameters based on the joint probability distribution corresponding to the data augmentation control parameters, it is specifically used to perform the following steps:

[0241] If the joint probability distribution corresponding to the data augmentation control parameters is a first average distribution or a first normal distribution, then data extraction processing is performed according to the first average distribution or the first normal distribution to obtain a first data extraction result, and the data augmentation control parameters are determined according to the first data extraction result.

[0242] If the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution, then data extraction processing is performed according to the flipped distribution to obtain the second data extraction result;

[0243] If the second data extraction result is greater than or equal to the preset value, then the data enhancement control parameter is determined based on the second data extraction result and the boundary ratio data.

[0244] If the second data extraction result is less than the preset value, then the data augmentation control parameter is determined based on the second data extraction result and the negative value of the boundary ratio data.

[0245] In one embodiment, when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to determine the data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data, it is specifically used to perform the following steps:

[0246] Based on the data augmentation control parameters and the augmentation boundary control data, determine the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image;

[0247] The data augmentation parameters of the first sample image are determined based on the data augmentation information of the first sample image.

[0248] In one embodiment, when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to determine the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data, it is specifically used to perform the following steps:

[0249] Determine a first ratio between the first augmentation boundary included in the augmentation boundary control data and the first data augmentation control parameter, and determine the minimum value between the first ratio and the data augmentation control parameter as the first distribution boundary;

[0250] Determine a second ratio between the second augmentation boundary included in the augmentation boundary control data and the second data augmentation control parameter, and determine the maximum value of the second ratio and the data augmentation control parameter as the second distribution boundary;

[0251] The first distribution boundary and the second distribution boundary are processed to construct an average distribution, thereby obtaining the second average distribution corresponding to the data augmentation parameters of the first sample image.

[0252] Based on the second average distribution corresponding to the data augmentation parameters of the first sample image, the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image is determined.

[0253] In one embodiment, the data augmentation parameters include the cropping area; when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, respectively, to obtain the positive sample pair corresponding to the sample images including the first sample image and the second sample image, it is specifically used to perform the following steps:

[0254] The sample image is jointly cropped based on the cropping area of ​​the first sample image to obtain the first sample image included in the positive sample pair corresponding to the sample image.

[0255] The sample images are jointly cropped based on the cropping area of ​​the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0256] In one embodiment, the data augmentation parameters include target blur or target grayscale; when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101 to perform data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, respectively, to obtain the positive sample pair corresponding to the sample images including the first sample image and the second sample image, it is specifically used to perform the following steps:

[0257] The sample images are randomly cropped according to the enhanced boundary control data to obtain a first intermediate sample image and a second intermediate sample image.

[0258] Based on the target blur or target grayscale of the first sample image, the first intermediate sample image is blurred or grayscaled to obtain the first sample image included in the positive sample pair corresponding to the sample image;

[0259] The second intermediate sample image is blurred or grayscaled according to the target blur or target grayscale of the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

[0260] In one embodiment, when the computer program in the computer-readable storage medium 1104 is loaded and executed by the processor 1101, it is further configured to perform the following steps:

[0261] Obtain a positive sample set containing multiple positive sample pairs, and a negative sample set containing multiple negative sample pairs, wherein the negative sample pairs are determined based on the multiple positive sample pairs;

[0262] The positive sample set and the negative sample set are input into the neural network model for comparative learning processing to obtain comparative learning difference data.

[0263] The model parameters of the neural network model are adjusted based on the contrastive learning difference data to obtain the image processing model.

[0264] The computer device provided in this application embodiment can adjust the complexity of positive sample pairs through similarity control parameters, thereby improving the controllability of the generated positive sample pairs and facilitating the generation of positive sample pairs that meet application requirements. Data augmentation control parameters reflecting the correlation between the data augmentation parameters of the first and second sample images can be determined first, thus comprehensively considering the relationship between the first and second sample images and avoiding situations where the first and second sample images have high similarity. The method provided in this application embodiment can be applied to various data augmentation scenarios (e.g., cropping, blurring, stretching, grayscale, etc.), exhibiting good universality and meeting diverse application needs. The method provided in this application embodiment combines the joint random augmentation method and the independent random augmentation method provided in this application, further enhancing the complexity of the generated positive sample pairs. Using the generated high-complexity positive samples for comparative learning processing of the model can improve the model's training efficiency and effect, resulting in an image processing model with high prediction accuracy.

[0265] This application also provides a computer-readable storage medium storing computer instructions. When executed on a computer device, the instructions cause the computer device to perform the steps in the various method embodiments of this application to implement the image processing method provided in this application. Specific implementation details can be found in the foregoing description and will not be repeated here.

[0266] This application also provides a computer program product, which includes a computer program or computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the computer device to perform the steps in the various method embodiments of this application to implement the image processing method provided in this application. Specific implementation details can be found in the foregoing description and will not be repeated here.

[0267] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program with a predetermined function, which works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0268] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0269] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Data augmentation configuration parameters for sample images are obtained, including augmentation boundary control data and similarity control data; Based on the enhanced boundary control data and the similarity control data, data augmentation control parameters are determined, and based on the data augmentation control parameters and the enhanced boundary control data, data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image are determined. Based on the data augmentation parameters of the first sample image and the data augmentation control parameters, the data augmentation parameters of the second sample image included in the positive sample pair are determined; wherein, the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold. The sample images are augmented using the data augmentation parameters of the first and second sample images respectively, to obtain a positive sample pair including the first and second sample images; wherein the positive sample pair is used to train a neural network model to obtain an image processing model.

2. The method as described in claim 1, characterized in that, The enhanced boundary control data includes a first enhanced boundary and a second enhanced boundary; determining the data enhancement control parameters based on the enhanced boundary control data and the similarity control data includes: Determine the boundary ratio data based on the first enhancement boundary and the second enhancement boundary; Based on the boundary ratio data and the similarity control data, determine the joint probability distribution corresponding to the data augmentation control parameters; The data augmentation control parameters are determined based on the joint probability distribution corresponding to the data augmentation control parameters.

3. The method as described in claim 2, characterized in that, The step of determining the joint probability distribution corresponding to the data augmentation control parameters based on the boundary ratio data and the similarity control data includes: If the similarity control data is equal to the preset reference threshold, then the boundary ratio data is processed by average distribution construction to obtain the joint probability distribution corresponding to the data augmentation control parameters, and the joint probability distribution is the first average distribution; If the similarity control data is greater than the preset reference threshold, then the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain the joint probability distribution corresponding to the data augmentation control parameters, wherein the joint probability distribution is a first normal distribution; If the similarity control data is less than the preset reference threshold, then the absolute values ​​of the boundary ratio data and the similarity control data are processed to construct a normal distribution to obtain a second normal distribution, and the second normal distribution is flipped to obtain the joint probability distribution corresponding to the data augmentation control parameters. The joint probability distribution is a flipped distribution.

4. The method as described in claim 3, characterized in that, The step of determining the data augmentation control parameters based on the joint probability distribution corresponding to the data augmentation control parameters includes: If the joint probability distribution corresponding to the data augmentation control parameters is a first average distribution or a first normal distribution, then data extraction processing is performed according to the first average distribution or the first normal distribution to obtain a first data extraction result, and the data augmentation control parameters are determined according to the first data extraction result. If the joint probability distribution corresponding to the data augmentation control parameters is a flipped distribution, then data extraction processing is performed according to the flipped distribution to obtain the second data extraction result; If the second data extraction result is greater than or equal to the preset value, then the data enhancement control parameter is determined based on the second data extraction result and the boundary ratio data. If the second data extraction result is less than the preset value, then the data augmentation control parameter is determined based on the second data extraction result and the negative value of the boundary ratio data.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the data augmentation parameters of the first sample image, which is included in the positive sample pair corresponding to the sample image, based on the data augmentation control parameters and the augmentation boundary control data includes: Based on the data augmentation control parameters and the augmentation boundary control data, determine the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image; The data augmentation parameters of the first sample image are determined based on the data augmentation information of the first sample image.

6. The method as described in claim 5, characterized in that, The step of determining the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data includes: Determine a first ratio between the first augmentation boundary included in the augmentation boundary control data and the first data augmentation control parameter, and determine the minimum value between the first ratio and the data augmentation control parameter as the first distribution boundary; Determine a second ratio between the second augmentation boundary included in the augmentation boundary control data and the second data augmentation control parameter, and determine the maximum value of the second ratio and the data augmentation control parameter as the second distribution boundary; The first distribution boundary and the second distribution boundary are processed to construct an average distribution, thereby obtaining the second average distribution corresponding to the data augmentation parameters of the first sample image. Based on the second average distribution corresponding to the data augmentation parameters of the first sample image, the data augmentation information of the first sample image included in the positive sample pair corresponding to the sample image is determined.

7. The method according to any one of claims 1-4, characterized in that, The data augmentation parameters include the cropping area; the step of performing data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image to obtain the positive sample pairs corresponding to the sample images, including the first sample image and the second sample image, includes: The sample image is jointly cropped based on the cropping area of ​​the first sample image to obtain the first sample image included in the positive sample pair corresponding to the sample image. The sample images are jointly cropped based on the cropping area of ​​the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

8. The method according to any one of claims 1-4, characterized in that, The data augmentation parameters include target blur or target grayscale; the step of performing data augmentation processing on the sample images according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image to obtain the positive sample pair corresponding to the sample images, including the first sample image and the second sample image, includes: The sample images are randomly cropped according to the enhanced boundary control data to obtain a first intermediate sample image and a second intermediate sample image. Based on the target blur or target grayscale of the first sample image, the first intermediate sample image is blurred or grayscaled to obtain the first sample image included in the positive sample pair corresponding to the sample image; The second intermediate sample image is blurred or grayscaled according to the target blur or target grayscale of the second sample image to obtain the second sample image included in the positive sample pair corresponding to the sample image.

9. The method according to any one of claims 1-4, characterized in that, The method further includes: Obtain a positive sample set containing multiple positive sample pairs, and a negative sample set containing multiple negative sample pairs, wherein the negative sample pairs are determined based on the multiple positive sample pairs; The positive sample set and the negative sample set are input into the neural network model for comparative learning processing to obtain comparative learning difference data. The model parameters of the neural network model are adjusted based on the contrastive learning difference data to obtain the image processing model.

10. An image processing apparatus, characterized in that, include: An acquisition unit is used to acquire data augmentation configuration parameters for sample images, wherein the data augmentation control parameters include augmentation boundary control data and similarity control data; The determining unit is configured to determine data augmentation control parameters based on the augmentation boundary control data and the similarity control data, and to determine data augmentation parameters of the first sample image included in the positive sample pair corresponding to the sample image based on the data augmentation control parameters and the augmentation boundary control data. The determining unit is further configured to determine the data augmentation parameters of the second sample image included in the positive sample pair based on the data augmentation parameters of the first sample image and the data augmentation control parameters; wherein the difference between the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image is greater than or equal to a preset difference threshold. The processing unit is configured to perform data augmentation processing on the sample image according to the data augmentation parameters of the first sample image and the data augmentation parameters of the second sample image, respectively, to obtain a positive sample pair corresponding to the sample image, including the first sample image and the second sample image; wherein, the positive sample pair is used to train a neural network model to obtain an image processing model.

11. A computer device, characterized in that, The computer device includes: A processor is a tool for implementing computer programs. A computer-readable storage medium storing a computer program adapted to be loaded by the processor and to implement the image processing method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to implement the image processing method as described in any one of claims 1-9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the image processing method as described in any one of claims 1-9.

Images