Image processing method and related products

Abstract

The embodiment of the present application provides an image processing method and related products, wherein the method includes: determining M reference pasting positions from an image to be processed, where M is a positive integer; determining a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed; pasting the preset image block to the target pasting position to obtain a target image, thereby balancing the number of scarce targets, improving the richness of training samples, and thereby improving the accuracy of detecting rare targets.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and related products. Background Art

[0002] With the continuous development of artificial intelligence technology, target detection has also developed rapidly. Target detection refers to finding all the targets of interest in an image and determining their size and position. It is one of the most important problems in computer vision. The application of target detection is very wide, including vehicle and pedestrian detection, face detection, military target detection, etc. In recent years, deep learning methods have achieved great success in target detection tasks and are currently the most important and widely used target detection technology.

[0003] When using deep learning methods to solve target detection problems, the quality of training data is an important factor affecting detection performance. For example, if the training data is too little or too one-sided and cannot reflect various situations in reality, it will often lead to overfitting of the model and lack of generalization ability. For another example, the actual training data collected often has an imbalance in the number of target objects, with one or several categories of targets far less than other categories, which will result in low detection accuracy for these scarce targets. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and related products, which can improve the accuracy of rare target detection.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] An image processing method, characterized in that the method comprises:

[0007] Determine M reference pasting positions from the image to be processed, where M is a positive integer;

[0008] Determine a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed;

[0009] The preset image block is pasted to the target pasting position to obtain a target image.

[0010] In combination with the first aspect, in a first possible implementation manner of the first aspect, before determining M reference pasting positions from the image to be processed, the method further includes:

[0011] Segmenting the image to be processed to obtain an image block in a first area and an image block in a second area, wherein an object to be augmented in the image to be processed appears in the first area but does not appear in the second area;

[0012] The step of determining M reference pasting positions from the image to be processed includes:

[0013] M reference pasting positions are determined from the image blocks in the first area.

[0014] In combination with the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the image blocks of the first area are image blocks of a road area, and the image blocks of the second area are image blocks of a non-road area; and segmenting the image to be processed to obtain image blocks of the first area and image blocks of the second area includes:

[0015] Segmenting the image to be processed to obtain image blocks of a road area and image blocks of a non-road area, wherein the object to be augmented in the image to be processed appears in the road area but does not appear in the non-road area;

[0016] The determining M reference pasting positions from the image blocks of the first area includes:

[0017] M reference pasting positions are determined from the image blocks of the road area.

[0018] In combination with the first possible implementation manner of the first aspect or the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, determining a target pasting position of the preset image block from the M reference pasting positions includes:

[0019] The target pasting position is determined from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of N real targets, the real target is an object in the image to be processed, and N is a positive integer.

[0020] In combination with the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, determining the target pasting position from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of the N real targets includes:

[0021] Obtaining the coverage area of ​​the image block of each reference pasting position covering the first region, to obtain M target coverage areas;

[0022] According to the area of ​​each reference pasting position and the M target coverage areas, determining the area coverage rate with each reference pasting position to obtain M first coverage rates;

[0023] The target pasting position is determined from the M reference pasting positions according to the M first coverage rates.

[0024] In combination with the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the method further includes: determining, according to an area of ​​the first reference pasting position and an area of ​​the N real targets, a coverage ratio of the first reference pasting position to the N real targets, to obtain N third coverage ratios, wherein the first reference pasting position is any one of the M reference pasting positions;

[0025] Determine the maximum value among the N third coverage rates as the true coverage rate corresponding to the first reference pasting position, and obtain the second coverage rate corresponding to the first reference pasting position;

[0026] Repeat the above step of obtaining the second coverage rate of the first reference pasting position until the real coverage rate corresponding to each reference pasting position is obtained, and M second coverage rates are obtained;

[0027] The step of determining the target pasting position from the M reference pasting positions according to the M first coverage rates includes:

[0028] The target pasting position is determined from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios.

[0029] In combination with the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, determining the target pasting position from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios includes:

[0030] Sorting the M reference pasting positions according to the corresponding first coverage rates in accordance with a preset sorting method to obtain a target sequence;

[0031] The second coverage rate of the reference pasting position in the target sequence is compared with the preset coverage rate threshold according to the sequence order of the target sequence to determine the target second coverage rate, and the reference pasting position corresponding to the target second coverage rate is used as the target pasting position. The target second coverage rate is the first second coverage rate that is smaller than the preset coverage rate threshold.

[0032] In combination with the sixth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, the method further includes:

[0033] In response to the second coverage rates of the reference pasting positions in the target sequence being no less than the preset coverage rate threshold, it is determined that the target pasting position is empty.

[0034] In combination with any possible implementation manner of the first aspect to the seventh possible implementation manner of the first aspect, in an eighth possible implementation manner of the first aspect, pasting the preset image block to the target pasting position to obtain the target image includes:

[0035] Determine a first mask image of the target to be augmented in the preset image block; the first mask image is a binary image;

[0036] filtering the first mask image so as to blur the edge of the object to be augmented in the first mask image, thereby obtaining a second mask image;

[0037] According to the second mask image, the target to be augmented is pasted to the target pasting position to obtain the target image.

[0038] In combination with the eighth possible implementation manner of the first aspect, in a ninth possible implementation manner of the first aspect, pasting the target to be augmented to the target pasting position according to the second mask image to obtain the target image includes:

[0039] Acquire the pixel value of each pixel of the target to be augmented in the preset image block, and the reference pixel value of each pixel at the target pasting position;

[0040] Determine a target pixel value for each pixel at the target pasting position according to the pixel value of each pixel of the target to be augmented, the reference pixel value of each pixel at the target pasting position, and the pixel value of each pixel in the second mask image;

[0041] The reference pixel value of each pixel at the target pasting position is replaced with the corresponding target pixel value to obtain the target image.

[0042] In combination with any possible implementation manner of the first aspect to the ninth possible implementation manner of the first aspect, in a tenth possible implementation manner of the first aspect, a portion of the preset image block except for the target to be augmented is transparent.

[0043] In combination with any possible implementation manner of the first aspect to the tenth possible implementation manner of the first aspect, in an eleventh possible implementation manner of the first aspect, the method further includes:

[0044] The target image and the image to be processed are used as training samples in a training sample set to train the neural network to be trained.

[0045] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising a first determining unit, a second determining unit and a pasting unit, wherein:

[0046] The first determining unit is used to determine M reference pasting positions from the image to be processed, where M is a positive integer;

[0047] The second determining unit is used to determine a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed;

[0048] The pasting unit is used to paste the preset image block to the target pasting position to obtain a target image.

[0049] In combination with the second aspect, in a first possible implementation manner of the second aspect, the apparatus further includes a segmentation unit, wherein the segmentation unit is configured to: before determining the M reference pasting positions from the image to be processed,

[0050] Segmenting the image to be processed to obtain an image block in a first area and an image block in a second area, wherein an object to be augmented in the image to be processed appears in the first area but does not appear in the second area;

[0051] The first determining unit is used for:

[0052] M reference pasting positions are determined from the image blocks in the first area.

[0053] In combination with the first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the image blocks of the first area are image blocks of a road area, and the image blocks of the second area are image blocks of a non-road area; and the segmentation unit is used to:

[0054] Segmenting the image to be processed to obtain image blocks of a road area and image blocks of a non-road area, wherein the object to be augmented in the image to be processed appears in the road area but does not appear in the non-road area;

[0055] The first determining unit is used for:

[0056] M reference pasting positions are determined from the image blocks of the road area.

[0057] In combination with the first possible implementation manner of the second aspect or the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the second determining unit is used to:

[0058] The target pasting position is determined from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of N real targets, the real target is an object in the image to be processed, and N is a positive integer.

[0059] In combination with the third possible implementation manner of the second aspect, in a fourth possible implementation manner of the second aspect, the second determining unit is configured to:

[0060] Obtaining the coverage area of ​​the image block of each reference pasting position covering the first region, to obtain M target coverage areas;

[0061] According to the area of ​​each reference pasting position and the M target coverage areas, determining the area coverage rate with each reference pasting position to obtain M first coverage rates;

[0062] The target pasting position is determined from the M reference pasting positions according to the M first coverage rates.

[0063] In combination with the fourth possible implementation manner of the second aspect, in a fifth possible implementation manner of the second aspect, the second determining unit is further used to: determine, according to an area of ​​the first reference pasting position and an area of ​​the N real targets, a coverage ratio of the first reference pasting position to the N real targets, to obtain N third coverage ratios, and the first reference pasting position is any one of the M reference pasting positions;

[0064] Determine the maximum value among the N third coverage rates as the true coverage rate corresponding to the first reference pasting position, and obtain the second coverage rate corresponding to the first reference pasting position;

[0065] Repeat the above step of obtaining the second coverage rate of the first reference pasting position until the real coverage rate corresponding to each reference pasting position is obtained, and M second coverage rates are obtained;

[0066] In the aspect of determining the target pasting position from the M reference pasting positions according to the M first coverage rates, the second determining unit is used to:

[0067] The target pasting position is determined from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios.

[0068] In combination with the fifth possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, in determining the target pasting position from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios, the second determining unit is used to:

[0069] Sorting the M reference pasting positions according to the corresponding first coverage rates in accordance with a preset sorting method to obtain a target sequence;

[0070] The second coverage rate of the reference pasting position in the target sequence is compared with the preset coverage rate threshold according to the sequence order of the target sequence to determine the target second coverage rate, and the reference pasting position corresponding to the target second coverage rate is used as the target pasting position. The target second coverage rate is the first second coverage rate that is smaller than the preset coverage rate threshold.

[0071] In combination with the sixth possible implementation manner of the second aspect, in a seventh possible implementation manner of the second aspect, the second determining unit is further configured to:

[0072] In response to the second coverage rates of the reference pasting positions in the target sequence being no less than the preset coverage rate threshold, it is determined that the target pasting position is empty.

[0073] In combination with any possible implementation manner of the second aspect to the seventh possible implementation manner of the second aspect, in an eighth possible implementation manner of the second aspect, the pasting unit is used to:

[0074] Determine a first mask image of the target to be augmented in the preset image block; the first mask image is a binary image;

[0075] filtering the first mask image so as to blur the edge of the object to be augmented in the first mask image, thereby obtaining a second mask image;

[0076] According to the second mask image, the target to be augmented is pasted to the target pasting position to obtain the target image.

[0077] In conjunction with the eighth possible implementation manner of the second aspect, in a ninth possible implementation manner of the second aspect, in the aspect of pasting the target to be augmented to the target pasting position according to the second mask image to obtain the target image, the pasting unit is used to:

[0078] Acquire the pixel value of each pixel of the target to be augmented in the preset image block, and the reference pixel value of each pixel at the target pasting position;

[0079] Determine a target pixel value for each pixel at the target pasting position according to the pixel value of each pixel of the target to be augmented, the reference pixel value of each pixel at the target pasting position, and the pixel value of each pixel in the second mask image;

[0080] The reference pixel value of each pixel at the target pasting position is replaced with the corresponding target pixel value to obtain the target image.

[0081] In combination with any possible implementation manner of the second aspect to the ninth possible implementation manner of the second aspect, in a tenth possible implementation manner of the second aspect, a portion of the preset image block except for the target to be augmented is transparent.

[0082] In combination with any possible implementation manner of the second aspect to the tenth possible implementation manner of the second aspect, in an eleventh possible implementation manner of the second aspect, the apparatus further includes a construction unit, and the construction unit is used to:

[0083] The target image and the image to be processed are used as training samples in a training sample set to train the neural network to be trained.

[0084] A third aspect of an embodiment of the present application provides a terminal, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute some or all of the step instructions described in the first aspect of the embodiment of the present application.

[0085] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0086] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0087] Implementing the embodiments of the present application has at least the following beneficial effects:

[0088] In the embodiment of the present application, M reference pasting positions are determined from the image to be processed, where M is a positive integer, and a target pasting position of a preset image block is determined from the M reference pasting positions, wherein the preset image block includes the target to be augmented in the image to be processed, and the preset image block is pasted to the target pasting position to obtain a target image. Therefore, when the scarce targets in the actual training data are far less than other categories, the image processing method provided by the embodiment of the present application can be used to augment the training data by pasting preset image blocks containing scarce targets (that is, targets to be augmented) in the image to be processed (that is, training data), so that the number of scarce targets is balanced, thereby improving the richness of the training samples, and further improving the accuracy of detecting rare targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0090] Figure 1 A schematic diagram of an image processing method is provided for an embodiment of the present application;

[0091] Figure 2 A flowchart of an image processing method is provided for an embodiment of the present application;

[0092] Figure 3A A flowchart of an image processing method applied to a road image scene is provided for an embodiment of the present application;

[0093] Figure 3B A schematic diagram of a disparity map conversion method is provided for an embodiment of the present application;

[0094] Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0095] Figure 5 A structural schematic diagram of an image processing device is provided for an embodiment of the present application;

[0096] Figure 6 A structural schematic diagram of another image processing device is provided for an embodiment of the present application;

[0097] Figure 7 A structural schematic diagram of another image processing device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0099] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0100] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0101] The electronic devices involved in the embodiments of the present application may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, as well as various forms of user equipment (UE), mobile station (MS), terminal device, etc. For the convenience of description, the above-mentioned devices are collectively referred to as electronic devices.

[0102] In order to better understand the image processing method of the embodiment of the present application, the following first briefly introduces the existing samples that use sample sets to train the model. The existing samples usually have the following characteristics: First, the training data is too little or too one-sided, which cannot reflect the various situations in reality, often resulting in overfitting of the model and lack of generalization ability. Second, the actual collected training data often has an imbalance in the number of target objects, and the number of objects in one or several categories is far less than that in other categories, which will result in a very low detection accuracy of these scarce categories. The existing solution to the above problem is usually to manually collect more and more comprehensive data, but data collection and annotation require a lot of labor costs. Another method is data augmentation technology, which refers to transforming and synthesizing existing data through algorithms to achieve the purpose of expanding existing data sets. Existing data augmentation technology is mainly performed on the entire picture, such as horizontal flipping, contrast transformation, adding noise, etc. The disadvantages of these methods are that they can only simulate the overall changes in the environment to a certain extent, but cannot simulate changes in target posture, orientation, etc., and these methods cannot solve the problem of imbalance in the number of targets.

[0103] The following is a brief introduction to an image processing method provided in an embodiment of the present application. Figure 1 , Figure 1 A schematic diagram of an image processing method is provided for an embodiment of the present application. Figure 1 As shown, the image to be processed is segmented to obtain image blocks of the first area and image blocks of the second area, M reference pasting positions are determined in the image blocks of the first area, and the target pasting position of the preset image block is determined from the M reference pasting positions. The preset image block includes the target to be augmented in the image to be processed. The target to be augmented can be, for example, an image of a person, an image of a traffic light, etc. The preset image block is pasted to the target pasting position to obtain the target image. Therefore, the image processing method provided in the embodiment of the present application can be used to augment the training data by pasting the preset image block containing the scarce target (that is, the target to be augmented) in the image to be processed (that is, the training data) when the scarce targets in the actual training data are far less than other categories, so that the number of rare targets is balanced, thereby improving the richness of the training samples, and then improving the accuracy of rare target detection.

[0104] See also Figure 2 , Figure 2 A schematic diagram of a flow chart of an image processing method is provided for an embodiment of the present application. Figure 2 As shown, the image processing method includes steps 201-203, which are specifically as follows:

[0105] 201. Determine M reference pasting positions from the image to be processed, where M is a positive integer.

[0106] Optionally, M reference pasting positions may be obtained by randomly selecting from the image to be processed.

[0107] 202. Determine a target pasting position of a preset image block from the M reference pasting positions, where the preset image block includes a target to be augmented in the image to be processed.

[0108] The target to be augmented in the image to be processed is, for example, a person's image, a traffic light image, etc. The portion of the image block other than the target to be augmented is preset to be transparent.

[0109] It should be noted that when determining the target pasting position, if the part of the preset image block other than the target to be augmented is opaque, that is, the background part is opaque (the target to be augmented is the foreground), it is necessary to determine the target pasting position based on the area of ​​the preset image block; if the part of the preset image block other than the target to be augmented is transparent, that is, the background part is transparent, then the target pasting position can be directly determined based on the area of ​​the target to be augmented as the area of ​​the preset image block.

[0110] Optionally, in order to better fit the data augmentation, the size of the preset image block can be adjusted, so that preset image blocks of different sizes can be obtained, which can improve the richness of the objects to be augmented. A possible transformation method for obtaining the preset image block is:

[0111] The short side of the preset image block before transformation is L min , the long side is L max , adjust the size with a fixed aspect ratio, and note that the short side after adjustment is L' min , the long side is L' max . The preset image block category is also recorded as c. The transformation method is as follows:

[0112] 1) Randomly select the short side length L∈U[a c ,b c ];

[0113] 2) If but otherwise

[0114] where a c ,b c ,M c It is a hyperparameter related to the category. The hyperparameter can be understood as the data set before machine learning is performed on the data, and the category can be understood as the category of the image block, such as people, cars, obstacles, etc.

[0115] 203. Paste the preset image block to the target pasting position to obtain the target image.

[0116] Optionally, when pasting the preset image block to the target pasting position, the image of the image to be processed at the target pasting position may be cut out to make it a blank image block, and then the preset image block is pasted to the target pasting position.

[0117] In this example, M reference pasting positions are determined from the image to be processed, where M is a positive integer. A target pasting position of a preset image block is determined from the M reference pasting positions, and the preset image block includes the target to be augmented in the image to be processed. The preset image block is pasted to the target pasting position to obtain the target image. Therefore, when the scarce targets in the actual training data are far less than other categories, the image processing method provided by the embodiment of the present application can be used to augment the training data by pasting preset image blocks containing scarce targets (that is, targets to be augmented) in the image to be processed (that is, training data), so that the number of rare targets is balanced, thereby improving the richness of the training samples, and further improving the accuracy of rare target detection.

[0118] In a possible embodiment, before determining the M reference pasting positions, the image to be processed may be segmented. A possible method for segmenting the image to be processed includes: segmenting the image to be processed to obtain image blocks in a first area and image blocks in a second area, wherein the target to be augmented in the image to be processed appears in the first area but not in the second area.

[0119] Among them, the method for segmenting the image to be processed can be: segmenting by using the V-disparity map method, etc. Other segmentation methods can also be used, which are not specifically limited here. The image to be processed can be an image including a lane where a car is driving, etc. The image blocks in the first area can be image blocks in the road area, and the image blocks in the road area can be understood as image blocks in the area where the lane where the car is driving is located. The image blocks in the first area can be image blocks in the non-road area, and the image blocks in the non-road area can be understood as image blocks that are different from the image blocks in the road area. Of course, the image blocks in the first area and the image blocks in the second area can also be other image blocks. The first area is the area where the target to be augmented will appear. For example, in the user shopping scene, the image blocks in the first area are the shopping mall area in the shopping environment, and the image blocks in the second area are the non-shopping mall area. The target to be augmented can be, for example, a person; in the scene of element augmentation in the picture, the image blocks in the area where the element to be augmented will appear in the picture can be determined as the image blocks in the first area, and the image blocks in the area where the element to be augmented will not appear can be determined as the image blocks in the second area. Of course, it can also be other scenes. This is only an example and is not specifically limited.

[0120] Optionally, when determining M reference pasting positions from the image to be processed, the M reference pasting positions can be determined from the image blocks of the first area. When the image blocks of the first area are image blocks of the road area, the M reference pasting positions are determined from the image blocks of the road area.

[0121] In this example, by segmenting the image to be processed, image blocks in the first area and image blocks in the second area are obtained, and M reference pasting positions are determined from the image blocks in the first area (the target to be augmented appears in the first area and not in the second area), which can improve the effectiveness of image augmentation to a certain extent.

[0122] In a possible embodiment, a possible method for determining a target pasting position of a preset image block from M reference pasting positions includes: determining the target pasting position from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of N real targets, where the real target is an object in the image to be processed, and N is a positive integer.

[0123] Among them, the coverage rate can be determined according to the coverage area of ​​the reference pasting position in the image to be processed, and the target pasting position can be determined from the M reference pasting positions according to the coverage rate.

[0124] In this example, the target pasting position is determined by referring to the area of ​​the pasting position and the area of ​​the actual target. The pasting position can be determined from the perspective of area. The area can more realistically reflect the pasting effect, thereby improving the accuracy of determining the target pasting position to a certain extent.

[0125] In a possible embodiment, a possible method for determining a target pasting position from M reference pasting positions according to the area of ​​the reference pasting position and the areas of N real targets includes steps A1-A3, which are specifically as follows:

[0126] A1, obtaining the coverage area of ​​the image block covering the first area at each reference pasting position, and obtaining M target coverage areas;

[0127] A2. According to the area of ​​each reference pasting position and the M target coverage areas, determine the area coverage rate with each reference pasting position to obtain M first coverage rates;

[0128] A3. Determine a target pasting position from the M reference pasting positions according to the M first coverage rates.

[0129] The coverage area of ​​the image block of the road region covered by the reference pasting position may be understood as the overlapping area between the reference pasting position and the road.

[0130] The target pasting position may be determined from the M reference pasting positions according to the M first coverage ratios and the real coverage ratio between the area of ​​each reference pasting position and the real target.

[0131] Alternatively, a possible method for determining ground area coverage is:

[0132] The ground area coverage rate is calculated using the ground area coverage rate formula, as follows:

[0133]

[0134] Among them, α is the ground area coverage, area(B F ∩R) is the target coverage area, area(B F ) is the area of ​​the reference pasting position.

[0135] In this example, M first coverage ratios are obtained by referring to the coverage area of ​​the pasting position and the image block of the first region. According to the M first coverage ratios, the target pasting position is determined from the M reference pasting positions. Since the coverage ratio can truly reflect the degree of overlap between the reference pasting position and the image block of the first region, the accuracy of determining the target pasting position can be improved.

[0136] In a possible embodiment, a possible method for obtaining the coverage ratio (second coverage ratio) of the reference pasting position relative to the real target includes steps B1-B3, which are specifically as follows:

[0137] B1. Determine the coverage of the first reference pasting position to the N real targets according to the area of ​​the first reference pasting position and the areas of the N real targets, and obtain N third coverages, where the first reference pasting position is any one of the M reference pasting positions;

[0138] B2, determining the maximum value among the N third coverage rates as the true coverage rate corresponding to the first reference pasting position, and obtaining the second coverage rate corresponding to the first reference pasting position;

[0139] B3. Repeat the above step of obtaining the second coverage rate of the first reference pasting position until the real coverage rate corresponding to each reference pasting position is obtained, and M second coverage rates are obtained.

[0140] Optionally, the number of real targets can be represented by N, where N is a positive integer. A possible method for determining the coverage of real targets and obtaining the third coverage is:

[0141] The third coverage is calculated using the true target coverage formula, as follows:

[0142]

[0143] Among them, β is the third coverage rate, area(B i T ) is the area of ​​the ith real target, area(B F ∩B i T ) is the coverage area between the first reference pasting position and the i-th real target, To obtain the maximum value, the i-th true target can be understood as the i-th true target among N true targets, where N is a positive integer. It can be understood as the maximum value among the N real target coverage rates.

[0144] The above formula can be understood as obtaining the coverage rate of a reference pasting position and all real targets, and taking the maximum coverage rate as the real coverage rate of the reference pasting position.

[0145] Optionally, determining the target pasting position from M reference pasting positions according to the M first coverage ratios may specifically be: determining the target pasting position from M reference pasting positions according to the M first coverage ratios and the M second coverage ratios.

[0146] The reference pasting positions may be sorted by using the second coverage ratio, and the target pasting position may be determined from the M reference pasting positions according to whether the second coverage ratio corresponding to the reference pasting position is less than a preset coverage ratio threshold.

[0147] In this example, the third coverage rate can be accurately determined through the third coverage rate calculation formula, and the second coverage rate can be determined based on the third coverage rate, and the target pasting position can be determined based on the second coverage rate and the first coverage rate. Therefore, the accuracy of determining the target pasting position can be improved to a certain extent.

[0148] In a possible embodiment, a possible method for determining a target pasting position from M reference pasting positions according to M first coverage ratios and M second coverage ratios includes steps C1-C2, which are specifically as follows:

[0149] C1, sorting the M reference pasting positions according to the corresponding first coverage rates in accordance with a preset sorting method to obtain a target sequence;

[0150] C2. Compare the second coverage of the reference pasting position in the target sequence with the preset coverage threshold in the sequence order of the target sequence to determine the target second coverage, and use the reference pasting position corresponding to the target second coverage as the target pasting position. The target second coverage is the first second coverage that is less than the preset coverage threshold.

[0151] The preset sorting method may be to sort in descending order according to the first coverage rate. Alternatively, the sorting may be performed in descending order. If the M reference pasting positions are arranged in descending order according to the corresponding first coverage rate, then the comparison order is to compare them one by one from front to back with the preset coverage rate threshold; if the M reference pasting positions are arranged in descending order according to the corresponding first coverage rate, then the comparison order is to compare them one by one from back to front with the preset coverage rate threshold.

[0152] Optionally, the target sequence is obtained by sorting in descending order. In the target sequence, each reference pasting position has a corresponding sequence number, for example, K1 to KM, where K1 is the sequence number of the reference pasting position at the head of the sequence in the target sequence, and KM is the sequence number of the reference pasting position at the tail of the sequence.

[0153] Optionally, comparing the second coverage of the reference pasting position in the target sequence with the preset coverage threshold according to the sequence order of the target sequence can be understood as comparing the second coverage of the reference pasting positions with sequence numbers K1 to KM with the preset coverage threshold in sequence.

[0154] Optionally, in response to the second coverage of the reference pasting position in the target sequence being no less than a preset coverage threshold, it is determined that the target pasting position is empty. The target pasting position being empty can be understood as the absence of a pasting position of the preset image block in the image to be processed, in which case the preset image block can be discarded.

[0155] In this example, the target pasting position is determined based on the first coverage ratio and the second coverage ratio, M reference pasting positions are sorted by the first coverage ratio, and the target pasting position is determined based on the second coverage ratio, which can improve the efficiency of determining the target pasting position.

[0156] In a possible embodiment, a possible method for pasting a preset image block to a target pasting position to obtain a target image includes steps D1-D3, which are specifically as follows:

[0157] D1, determining a first mask image of an object to be augmented in a preset image block; the first mask image is a binary image;

[0158] D2, filtering the first mask image to blur the edge of the target to be augmented in the first mask image, thereby obtaining a second mask image;

[0159] D3. Paste the target to be augmented to the target pasting position according to the second mask image to obtain the target image.

[0160] The binary image is a foreground / background binary image, which can be understood as a foreground image of 1 and a background image of 0. The foreground image can be understood as an image region including an object to be augmented.

[0161] The method for filtering the first mask image may be mean filtering, Gaussian filtering, Poisson filtering, etc. After filtering the first mask image, the edge (i.e., the number between 0 and 1) of the target to be augmented will be blurred, so that after pasting the target to the target pasting position, the target can be better integrated with the image to be processed, thereby improving the pasting effect.

[0162] According to the pixel values ​​of the pixels of the second mask image and the pixel values ​​of the pixels of the target to be augmented, the target may be pasted to the image to be processed to obtain the target image.

[0163] In a possible embodiment, a possible method for pasting the target to be augmented to the target pasting position according to the second mask image to obtain the target image includes steps E1-E3, which are specifically as follows:

[0164] E1, obtaining the pixel value of each pixel of the target to be augmented in the preset image block, and the reference pixel value of each pixel at the target pasting position;

[0165] E2, determining a target pixel value for each pixel at the target pasting position based on the pixel value of each pixel of the target to be augmented, the reference pixel value of each pixel at the target pasting position, and the pixel value of each pixel in the second mask image;

[0166] E3. Replace the reference pixel value of each pixel at the target pasting position with the corresponding target pixel value to obtain the target image.

[0167] Optionally, the method for determining the target pixel value may be:

[0168] The target pixel value is determined by the pixel value determination formula. The pixel value determination formula is:

[0169]

[0170] Where i, j are pixel coordinates, is the pixel value of the image to be processed, fake i,j Represents the pixel value of the target that needs to be augmented, is the target pixel value, mask i,j is a mask with values ​​in the range [0,1].

[0171] In this example, the target pixel value of the target pasting position is determined by the pixel value determination formula. The pixel value determination formula can integrate the target to be augmented with the environment in the image to be processed well, thereby improving the pasting effect when pasting the preset image block to a certain extent, thereby improving the authenticity of the target image.

[0172] In a possible embodiment, the target image and the image to be processed can also be used as training samples in the training sample set to train the neural network to be trained. Therefore, by using the target image and the image to be processed as training samples to train the neural network to be trained, the target image can be added as a training sample, and the target image contains scarce targets (targets that need to be augmented), so that the number of scarce targets is balanced, the richness of the samples is improved, and thus the accuracy of the training of the neural network to be trained is improved, and then the accuracy of the trained neural network in detecting rare targets is improved.

[0173] See also Figure 3A , Figure 3A The present application provides a flowchart of an image processing method applied to a road image scene. Figure 3A As shown, the image processing method includes steps 301-304, which are specifically as follows:

[0174] 301. Segment the image to be processed to obtain road area image blocks and non-road area image blocks;

[0175] Optionally, a possible method for segmenting the image to be processed to obtain road area image blocks and non-road area image blocks includes steps F1-F4, which are specifically as follows:

[0176] F1. Obtain a reference disparity map of the image to be processed.

[0177] Optionally, the method for obtaining the disparity map of the image to be processed may be: obtaining the disparity map of the image to be processed by using a disparity map calculation model.

[0178] Among them, the disparity map calculation model can be established through a machine learning module. A possible way to establish the disparity map calculation model is: the disparity map calculation model is a detection model trained by a supervised learning model. The supervised learning model in the disparity map calculation model can be, for example, a weight model in an artificial neural network method, etc. A method for establishing the disparity map calculation model is: first, feature extraction is performed on the sample to obtain a feature set, and then the feature set is input into the training model. The training model learns according to the algorithm in the training model, and the algorithm can be, for example, a gradient descent method, Newton algorithm, conjugate gradient algorithm, etc., and finally the disparity map is output. In this way, by learning a large number of samples, the disparity map calculation model is finally obtained, wherein the samples are images to be processed and disparity maps.

[0179] F2. Use a preset conversion method to convert the reference disparity map into a V disparity map.

[0180] The V disparity map has the same height as the reference disparity map, and the width of the V disparity map is 256. The value on the width represents the disparity value. A possible example is to convert the reference disparity map into the V disparity map as follows:

[0181] like Figure 3B As shown, the left side is an example of a disparity map, with a width of 8 and a height of 5. The right side is the calculated V disparity map. Count the pixel values ​​of each row of the disparity map on the left. Taking the first row as an example, there is 1 pixel with a pixel value of 0, and the record value in the 0th column of the corresponding row of the V disparity map is 0; there are 6 pixels with a pixel value of 1, and the record value in the 1st column of the corresponding row of the V disparity map is 6; there are 0 pixels with a pixel value of 2, and the record value in the 2nd column of the corresponding row of the V disparity map is 0; there is 1 pixel with a pixel value of 3, and the record value in the 3rd column of the corresponding row of the V disparity map is 1; ... Because the range of pixel values ​​is 0-255, the width of the V disparity map is 255, and the height is the same as the disparity map.

[0182] F3. Perform straight line fitting on the V disparity map to obtain the region segmentation line.

[0183] Optionally, the straight line fitting method may be a RANSAC straight line fitting method. The region segmentation line may be expressed as: v=Ad+B, where v is the pixel row coordinate and d is the disparity value.

[0184] F4. Segment according to the region segmentation line to obtain road region image blocks and non-road region image blocks.

[0185] Optionally, according to the region segmentation curve, segmentation can be performed in the following manners, specifically:

[0186]

[0187] Among them, I road is the disparity value of the ground area image block, I notroad is the disparity value of the non-ground area image block, and thresh is a constant that can be set based on historical data or empirical values. Thresh is used to reduce noise in the image.

[0188] 302. Obtain M reference pasting positions from the road area image block and the non-road area image block, where M is a positive integer;

[0189] See for example Figure 2 The method for obtaining the reference pasting position provided by the method shown will not be repeated here.

[0190] 303. Determine a target pasting position of a preset image block from the M reference pasting positions;

[0191] See for example Figure 2 The method for obtaining the target pasting position provided by the method shown is not repeated here.

[0192] 304. Obtain a mask image of the preset image block, and paste the preset image block to a target pasting position according to the mask image to obtain a target image.

[0193] The method of pasting the preset image block to the target pasting position according to the mask image can refer to the method of pasting the preset image block to the target pasting position in the above embodiment, which will not be repeated here.

[0194] In this example, the target pixel value of the target pasting position is determined by the pixel value determination formula. The pixel value determination formula can better integrate the preset image block with the environment in the image to be processed, thereby improving the pasting effect of the preset image block to a certain extent, thereby improving the authenticity of the target image.

[0195] For the above embodiments, please refer to Figure 4 , Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application, as shown in the figure, includes a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the program includes instructions for executing the following steps;

[0196] Determine M reference pasting positions from the image to be processed, where M is a positive integer;

[0197] Determine a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed;

[0198] The preset image block is pasted to the target pasting position to obtain a target image.

[0199] In this example, M reference pasting positions are determined from the image to be processed, where M is a positive integer. A target pasting position of a preset image block is determined from the M reference pasting positions, and the preset image block includes the target to be augmented in the image to be processed. The preset image block is pasted to the target pasting position to obtain the target image. Therefore, when the scarce targets in the actual training data are far less than other categories, the image processing method provided by the embodiment of the present application can be used to augment the training data by pasting preset image blocks containing scarce targets (that is, targets to be augmented) in the image to be processed (that is, training data), so that the number of scarce targets is balanced, thereby improving the richness of the training samples, and further improving the accuracy of rare target detection.

[0200] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that in order to realize the above functions, the terminal includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0201] The embodiment of the present application can divide the terminal into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0202] In line with the above, see Figure 5 , Figure 5 The present invention provides a schematic diagram of the structure of an image processing device. The image processing device includes a first determination unit 501, a second determination unit 502 and a pasting unit 503, wherein:

[0203] A first determining unit 501 is used to determine M reference pasting positions from the image to be processed, where M is a positive integer;

[0204] The second determining unit 502 is used to determine a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed;

[0205] The pasting unit 503 is used to paste the preset image block to the target pasting position to obtain the target image.

[0206] Optional, such as Figure 6 As shown, the image processing apparatus further includes a segmentation unit 504, wherein the segmentation unit 504 is used to: before determining M reference pasting positions from the image to be processed,

[0207] Segmenting the image to be processed to obtain an image block in a first area and an image block in a second area, wherein an object to be augmented in the image to be processed appears in the first area but does not appear in the second area;

[0208] The first determining unit 501 is used for:

[0209] M reference pasting positions are determined from the image blocks of the first region.

[0210] Optionally, the image blocks in the first area are image blocks in a road area, and the image blocks in the second area are image blocks in a non-road area; the segmentation unit 504 is used to:

[0211] Segment the image to be processed to obtain image blocks of the road area and image blocks of the non-road area, wherein the target to be augmented in the image to be processed appears in the road area but not in the non-road area;

[0212] The first determining unit 501 is used for:

[0213] Determine M reference pasting locations from the image patches of the road area.

[0214] Optionally, the second determining unit 502 is configured to:

[0215] According to the area of ​​the reference pasting position and the areas of N real targets, the target pasting position is determined from the M reference pasting positions, the real target is the object in the image to be processed, and N is a positive integer.

[0216] Optionally, the second determining unit 502 is configured to:

[0217] Obtaining the coverage area of ​​the image block covering the first area at each reference pasting position to obtain M target coverage areas;

[0218] According to the area of ​​each reference pasting position and the M target coverage areas, determine the area coverage rate with each reference pasting position to obtain M first coverage rates;

[0219] According to the M first coverage ratios, a target pasting position is determined from the M reference pasting positions.

[0220] Optionally, the second determining unit 502 is further configured to: determine the coverage of the first reference pasting position to the N real targets according to the area of ​​the first reference pasting position and the areas of the N real targets, to obtain N third coverages, and the first reference pasting position is any one of the M reference pasting positions;

[0221] Determine the maximum value among the N third coverage rates as the true coverage rate corresponding to the first reference pasting position, and obtain the second coverage rate corresponding to the first reference pasting position;

[0222] Repeat the above step of obtaining the second coverage rate of the first reference pasting position until the real coverage rate corresponding to each reference pasting position is obtained, and M second coverage rates are obtained;

[0223] In determining the target pasting position from the M reference pasting positions according to the M first coverage ratios, the second determining unit 502 is used to:

[0224] According to the M first coverage ratios and the M second coverage ratios, a target pasting position is determined from the M reference pasting positions.

[0225] Optionally, in determining the target pasting position from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios, the second determining unit 502 is configured to:

[0226] Sorting the M reference pasting positions according to the corresponding first coverage rates in accordance with a preset sorting method to obtain a target sequence;

[0227] The second coverage rate of the reference pasting position in the target sequence is compared with the preset coverage rate threshold according to the sequence order of the target sequence to determine the target second coverage rate, and the reference pasting position corresponding to the target second coverage rate is used as the target pasting position. The target second coverage rate is the first second coverage rate that is less than the preset coverage rate threshold.

[0228] Optionally, the second determining unit 502 is further configured to:

[0229] In response to the second coverage rates of the reference pasting positions in the target sequence being no less than the preset coverage rate threshold, it is determined that the target pasting position is empty.

[0230] Optionally, the pasting unit 503 is used for:

[0231] Determine a first mask image of an object to be augmented in a preset image block; the first mask image is a binary image;

[0232] Filtering the first mask image so as to blur the edge of the object to be augmented in the first mask image, thereby obtaining a second mask image;

[0233] According to the second mask image, the target to be augmented is pasted to the target pasting position to obtain the target image.

[0234] Optionally, in terms of pasting the target to be augmented to the target pasting position according to the second mask image to obtain the target image, the pasting unit 503 is used to:

[0235] Obtaining the pixel value of each pixel of the target to be augmented in the preset image block and the reference pixel value of each pixel at the target pasting position;

[0236] Determine a target pixel value for each pixel at the target pasting position according to the pixel value of each pixel of the target to be augmented, the reference pixel value of each pixel at the target pasting position, and the pixel value of each pixel in the second mask image;

[0237] The reference pixel value of each pixel at the target pasting position is replaced with the corresponding target pixel value to obtain the target image.

[0238] Optionally, the portion of the image block other than the object to be augmented is preset to be transparent.

[0239] Optional, such as Figure 7 As shown, the image processing apparatus further includes a construction unit 505, and the construction unit 505 is used to:

[0240] The target image and the image to be processed are used as training samples in a training sample set to train the neural network to be trained.

[0241] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any image processing method recorded in the above method embodiments.

[0242] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any image processing method recorded in the above method embodiments.

[0243] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0244] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0245] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0246] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0247] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0248] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0249] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0250] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Determine M reference pasting positions from the image to be processed, where M is a positive integer; Determine a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed; Pasting the preset image block to the target pasting position to obtain a target image; Before determining M reference pasting positions from the image to be processed, the method further includes: Segmenting the image to be processed to obtain an image block in a first area and an image block in a second area, wherein an object to be augmented in the image to be processed appears in the first area but does not appear in the second area; The step of determining M reference pasting positions from the image to be processed includes: Determine M reference pasting positions from the image blocks in the first area; Determining a target pasting position of a preset image block from the M reference pasting positions includes: Determining the target pasting position from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of N real targets, wherein the real target is an object in the image to be processed, and N is a positive integer; The step of determining the target pasting position from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of the N real targets includes: Obtaining the coverage area of ​​the image block of each reference pasting position covering the first region, to obtain M target coverage areas; According to the area of ​​each reference pasting position and the M target coverage areas, determining the area coverage ratio with each reference pasting position to obtain M first coverage ratios; Determining the target pasting position from the M reference pasting positions according to the M first coverage rates; The method further includes: determining the coverage of the first reference pasting position to the N real targets according to the area of ​​the first reference pasting position and the areas of the N real targets, to obtain N third coverages, wherein the first reference pasting position is any one of the M reference pasting positions; Determine the maximum value among the N third coverage rates as the true coverage rate corresponding to the first reference pasting position, and obtain the second coverage rate corresponding to the first reference pasting position; Repeat the above step of obtaining the second coverage rate corresponding to the first reference pasting position until the real coverage rate corresponding to each reference pasting position is obtained, and M second coverage rates are obtained; The step of determining the target pasting position from the M reference pasting positions according to the M first coverage rates includes: The target pasting position is determined from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios.

2. The method according to claim 1, characterized in that The image blocks of the first area are image blocks of the road area, and the image blocks of the second area are image blocks of the non-road area; segmenting the image to be processed to obtain the image blocks of the first area and the image blocks of the second area includes: Segmenting the image to be processed to obtain image blocks of a road area and image blocks of a non-road area, wherein the object to be augmented in the image to be processed appears in the road area but does not appear in the non-road area; The determining M reference pasting positions from the image blocks of the first area includes: M reference pasting positions are determined from the image blocks of the road area.

3. The method according to claim 1, characterized in that The step of determining the target pasting position from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios includes: Sorting the M reference pasting positions according to the corresponding first coverage rates in accordance with a preset sorting method to obtain a target sequence; The second coverage rate of the reference pasting position in the target sequence is compared with the preset coverage rate threshold according to the sequence order of the target sequence to determine the target second coverage rate, and the reference pasting position corresponding to the target second coverage rate is used as the target pasting position. The target second coverage rate is the first second coverage rate that is smaller than the preset coverage rate threshold.

4. The method according to claim 3, characterized in that: The method further comprises: In response to the second coverage rates of the reference pasting positions in the target sequence being no less than the preset coverage rate threshold, it is determined that the target pasting position is empty.

5. The method according to any one of claims 1 to 4, characterized in that: Pasting the preset image block to the target pasting position to obtain the target image includes: Determine a first mask image of the target to be augmented in the preset image block; the first mask image is a binary image; filtering the first mask image so as to blur the edge of the object to be augmented in the first mask image, thereby obtaining a second mask image; According to the second mask image, the target to be augmented is pasted to the target pasting position to obtain the target image.

6. The method according to claim 5, characterized in that The step of pasting the target to be augmented to the target pasting position according to the second mask image to obtain the target image comprises: Acquire the pixel value of each pixel of the target to be augmented in the preset image block, and the reference pixel value of each pixel at the target pasting position; Determine a target pixel value for each pixel at the target pasting position according to the pixel value of each pixel of the target to be augmented, the reference pixel value of each pixel at the target pasting position, and the pixel value of each pixel in the second mask image; The reference pixel value of each pixel at the target pasting position is replaced with the corresponding target pixel value to obtain the target image.

7. The method according to any one of claims 1 to 4, characterized in that: Parts of the preset image block other than the object to be augmented are transparent.

8. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The target image and the image to be processed are used as training samples in a training sample set to train the neural network to be trained.

9. An image processing device, characterized in that: The device comprises a first determining unit, a second determining unit and a pasting unit, wherein: The first determining unit is used to determine M reference pasting positions from the image to be processed, where M is a positive integer; The second determining unit is used to determine a target pasting position of a preset image block from the M reference pasting positions; the preset image block includes a target to be augmented in the image to be processed; The pasting unit is used to paste the preset image block to the target pasting position to obtain a target image; The device further comprises a segmentation unit, which is used to: before determining M reference pasting positions from the image to be processed, Segmenting the image to be processed to obtain an image block in a first area and an image block in a second area, wherein an object to be augmented in the image to be processed appears in the first area but does not appear in the second area; The first determining unit is used for: Determine M reference pasting positions from the image blocks in the first area; The second determining unit is used for: Determining the target pasting position from the M reference pasting positions according to the area of ​​the reference pasting position and the areas of N real targets, wherein the real target is an object in the image to be processed, and N is a positive integer; The second determining unit is used for: Obtaining the coverage area of ​​the image block of each reference pasting position covering the first region, to obtain M target coverage areas; According to the area of ​​each reference pasting position and the M target coverage areas, determining the area coverage ratio with each reference pasting position to obtain M first coverage ratios; Determining the target pasting position from the M reference pasting positions according to the M first coverage rates; The second determining unit is further configured to: Determining the coverage of the first reference pasting position to the N real targets according to the area of ​​the first reference pasting position and the areas of the N real targets, to obtain N third coverages, wherein the first reference pasting position is any one of the M reference pasting positions; Determine the maximum value among the N third coverage rates as the true coverage rate corresponding to the first reference pasting position, and obtain the second coverage rate corresponding to the first reference pasting position; Repeat the above step of obtaining the second coverage rate corresponding to the first reference pasting position until the real coverage rate corresponding to each reference pasting position is obtained, and M second coverage rates are obtained; In the aspect of determining the target pasting position from the M reference pasting positions according to the M first coverage rates, the second determining unit is used to: The target pasting position is determined from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios.

10. The device according to claim 9, characterized in that The image blocks in the first area are image blocks in a road area, and the image blocks in the second area are image blocks in a non-road area; the segmentation unit is used for: Segmenting the image to be processed to obtain image blocks of a road area and image blocks of a non-road area, wherein the object to be augmented in the image to be processed appears in the road area but does not appear in the non-road area; The first determining unit is used for: M reference pasting positions are determined from the image blocks of the road area.

11. The device according to claim 9, characterized in that In the aspect of determining the target pasting position from the M reference pasting positions according to the M first coverage ratios and the M second coverage ratios, the second determining unit is used to: Sorting the M reference pasting positions according to the corresponding first coverage rates in accordance with a preset sorting method to obtain a target sequence; The second coverage rate of the reference pasting position in the target sequence is compared with the preset coverage rate threshold according to the sequence order of the target sequence to determine the target second coverage rate, and the reference pasting position corresponding to the target second coverage rate is used as the target pasting position. The target second coverage rate is the first second coverage rate that is smaller than the preset coverage rate threshold.

12. The device according to claim 11, characterized in that The second determining unit is further configured to: In response to the second coverage rates of the reference pasting positions in the target sequence being no less than the preset coverage rate threshold, it is determined that the target pasting position is empty.

13. The device according to any one of claims 9 to 12, characterized in that The pasting unit is used for: Determine a first mask image of the target to be augmented in the preset image block; the first mask image is a binary image; filtering the first mask image so as to blur the edge of the object to be augmented in the first mask image, thereby obtaining a second mask image; According to the second mask image, the target to be augmented is pasted to the target pasting position to obtain the target image.

14. The device according to claim 13, characterized in that In the aspect of pasting the target to be augmented to the target pasting position according to the second mask image to obtain the target image, the pasting unit is used to: Acquire the pixel value of each pixel of the target to be augmented in the preset image block, and the reference pixel value of each pixel at the target pasting position; Determine a target pixel value for each pixel at the target pasting position according to the pixel value of each pixel of the target to be augmented, the reference pixel value of each pixel at the target pasting position, and the pixel value of each pixel in the second mask image; The reference pixel value of each pixel at the target pasting position is replaced with the corresponding target pixel value to obtain the target image.

15. The device according to any one of claims 9 to 12, characterized in that Parts of the preset image block other than the object to be augmented are transparent.

16. The device according to any one of claims 9 to 12, characterized in that The device further comprises a construction unit, wherein the construction unit is used for: The target image and the image to be processed are used as training samples in a training sample set to train the neural network to be trained.

17. A terminal, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.

Images