A data processing method and apparatus for maintaining enhancement in combination with mosaic enhancement

Abstract

The application provides a data processing method and device for keeping enhancement combined with mosaic enhancement. A sample image is obtained, wherein the sample image is a plurality of pictures randomly selected from a data set; the sample image is scaled to a first preset pixel size in equal proportions to obtain a first feature map; the first feature map is intercepted at a second preset pixel size to form a second feature map of the same size; the importance score of the second feature map is calculated, and it is judged whether the importance score is greater than an importance score threshold; if the importance score is greater than the importance score threshold, a plurality of the second feature maps are spliced in a mosaic canvas to form a fusion image. The application helps to improve the image data richness and the important information retention degree.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and computer vision, and in particular to a data processing method and device combining retention enhancement and mosaic enhancement. BACKGROUND

[0002] In recent years, data enhancement is a crucial technology in solving many deep learning problems. For limited data in training models, data enhancement can produce equivalent value to more data without substantially increasing data, thereby optimizing the training effect.

[0003] Chinese patent CN113888513A discloses a steel bar detection and counting method based on a deep neural network model. The method includes expanding and enhancing the original picture data set of the steel bar by using a sliding window method and a mosaic enhancement method, adaptively determining the anchor box size by using a clustering algorithm, taking the steel bar picture as input, extracting image features by a deep neural network, outputting a prediction box through a detection head, calculating a loss function according to the prediction box and the real box, optimizing the model parameters by a back propagation algorithm, and finally performing soft non-maximum suppression processing on the prediction box to obtain the final steel bar quantity. The application uses the sliding window method and the mosaic enhancement method for data enhancement, which improves the counting speed of model recognition. However, the above-mentioned scheme enriches the data set by using mosaic data enhancement, which improves the robustness of the network. When the detection target is small and few, mosaic enhancement often appears the phenomenon that the detection target does not exist in the new image or small targets are missed, which not only slows down the training speed, but also leads to the problems of low data set richness, low important information retention, unstable effect, etc. Therefore, it is necessary to provide a data processing method and device combining retention enhancement and mosaic enhancement to improve the image data richness and important information retention. SUMMARY

[0004] Therefore, the present application provides a data processing method and device combining retention enhancement and mosaic enhancement to improve the image data richness and important information retention.

[0005] In a first aspect of the present application, a data processing method combining retention enhancement and mosaic enhancement is provided, which is applied to an image processing device, and the method comprises:

[0006] obtaining a sample image, wherein the sample image is a plurality of pictures randomly selected from a data set;

[0007] scaling the sample image to a first preset pixel size in a proportional manner to obtain a first feature map;

[0008] cropping the first feature map to a second preset pixel size to form a second feature map of the same size;

[0009] calculating an importance score of the second feature map and determining whether the importance score is greater than an importance score threshold;

[0010] if the importance score is greater than the importance score threshold, stitching multiple second feature maps in a mosaic canvas to form a fusion image.

[0011] On the basis of the above technical solutions, preferably, after the multiple second feature maps are stitched in the canvas to form the fusion image, the method further comprises:

[0012] determining whether the fusion image contains an original label, wherein the original label is a label in the sample image;

[0013] if the fusion image contains the original label, converting the original label into a new label.

[0014] On the basis of the above technical solutions, preferably, the determining whether the fusion image contains the original label comprises:

[0015] determining whether x 原始min , y 原始min , x 原始max and y 原始max satisfy the following formula,

[0016] x 原始min > x start + w1-5

[0017] y 原始min > y start + h1-5

[0018] x 原始max < x start + 5

[0019] y 原始max < x start + 5

[0020] wherein x 原始min , y 原始min , x 原始max and y 原始max represent horizontal and vertical coordinates of the upper left corner and the lower right corner of the original label, x start and y startThe horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image are respectively represented by w1, h1, and 5. The width of the second feature map is represented by w1, the height of the second feature map is represented by h1, and 5 represents the deletion of labels with a side length of less than 5 pixels in the fused image.

[0021] Preferably, determining whether the original label exists in the fused image specifically involves:

[0022] Determine x 原始min y 原始min x 原始max and y 原始max Does it satisfy the following formula?

[0023] x 原始min >x start +w1-5

[0024] y 原始min >y start +h1-5

[0025] x 原始max <x start +5

[0026] y 原始max <x start +5

[0027] Where, x 原始min y 原始min x 原始max and y 原始max The x and y coordinates represent the top-left and bottom-right corners of the original label, respectively. start and y start The horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image are respectively represented by w1, h1, and 5. The width of the second feature map is represented by w1, the height of the second feature map is represented by h1, and 5 represents the deletion of labels with a side length of less than 5 pixels in the fused image.

[0028] More preferably, the conversion of the original tag into a new tag specifically involves:

[0029] x 新min =ox-x start +x 原始min

[0030] y 新min =oy-y start +y 原始min

[0031] x 新max =ox-x start +x 原始max

[0032] y 新max=oy-y start +y 原始max

[0033] wherein, x 新min , y 新min , x 新max and y 新max respectively represent the horizontal and vertical coordinates of the upper left corner and the lower right corner of the new label, ox represents the horizontal coordinate of the center point, oy represents the vertical coordinate of the center point, x start and y start respectively represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image.

[0034] More preferably, the method further comprises:

[0035] generating a center point in the fusion image, wherein the center point is randomly generated in a rectangle with the upper left corner at coordinates (w / 4, h / 4) and the lower right corner at coordinates (3w / 4, 3h / 4), w being the width of the fusion image and h being the height of the fusion image;

[0036] dividing the fusion image into four rectangular images according to the center point.

[0037] More preferably, the calculating the importance score of the second feature map comprises:

[0038] performing a derivative operation on the label logical value corresponding to each pixel in the second feature map to obtain a single importance score;

[0039] summing the single importance scores to obtain the importance score.

[0040] More preferably, the area calculation formula of the mosaic canvas is:

[0041] w = w min × 4 / 3 + 1

[0042] h = h min × 4 / 3 + 1

[0043] wherein, w min is the minimum value of the width of the sample image, h min is the minimum value of the height of the sample image, and 1 represents a pixel with a side length of 1.

[0044] More preferably, the method further comprises:

[0045] randomly horizontally flipping and color space transforming the fusion image to obtain an enhanced fusion image.

[0046] The second aspect of the present application provides a data processing device for keeping the combination of enhancement and mosaic enhancement, comprising an image acquisition module and a data processing module, wherein,

[0047] The image acquisition module is used for acquiring a sample image, wherein the sample image is a plurality of pictures randomly selected from a data set;

[0048] The data processing module is used for scaling the sample image to a first preset pixel size in equal proportions to obtain a first feature map, intercepting the first feature map to a second preset pixel size to form a second feature map of the same size, calculating the importance score of the second feature map, and determining whether the importance score is greater than an importance score threshold value, if the importance score is greater than the importance score threshold value, then the plurality of second feature maps are spliced in a mosaic canvas to form a fusion image.

[0049] In the third aspect of the present application, an electronic device is provided, characterized by comprising a processor, a memory, a user interface and a network interface, the memory is used for storing instructions, the user interface and the network interface are used for communication with other devices, and the processor is used for executing the instructions stored in the memory to make the electronic device execute the above-mentioned method.

[0050] The present application provides a data processing method for keeping the combination of enhancement and mosaic enhancement, which has the following beneficial effects compared with the prior art:

[0051] (1) By intercepting the second feature map with an importance score greater than the importance score threshold value in the first feature map of the sample pattern after equivalent scaling, it is ensured that there are a large number of detection targets in the current intercepted second feature map that can be enhanced by mosaic enhancement, so as to avoid the absence of detection targets or the omission of detection targets when performing mosaic enhancement, thereby improving the data richness and retention degree of important information in the fusion image;

[0052] (2) The information with high importance in the sample image is converted into information in the fusion image, so that the fusion image formed by splicing saves as much information and detection targets as possible, which can improve the stability of picture training and improve the training efficiency;

[0053] (3) The 4 / 3 plus 1 pixel of the minimum height in the sample image and the 4 / 3 plus 1 pixel of the minimum width in the sample image are respectively taken as the height and width of the mosaic canvas, which ensures that the cutting area will not be larger than the sample image when cutting the area, and also helps the extraction of small targets in the sample image;

[0054] (4) The generation range of the center point can prevent the original label from being too small due to too small picture data, losing the detection significance, and avoid generating picture data with the center point in an extreme position, weakening the effect of the enhancement method and wasting time and resources. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0056] Figure 1 The flowchart of the data processing method provided by the embodiment of the present application combined with the enhancement and mosaic enhancement is shown.

[0057] Figure 2 The schematic diagram of the fusion image provided by the embodiment of the present application is shown.

[0058] Figure 3 The map curve comparison diagram provided by the embodiment of the present application is shown.

[0059] Figure 4 The number of non-detection target pictures comparison diagram provided by the embodiment of the present application is shown.

[0060] Figure 5 The structural schematic diagram of the data processing device provided by the embodiment of the present application is shown.

[0061] Figure 6 The structural schematic diagram of the electronic device provided by the embodiment of the present application is shown.

[0062] The figure mark explanation: 1, data processing device; 11, image acquisition module; 12, data processing module; 2, electronic device; 21, processor; 22, communication bus; 23, user interface; 24, network interface; 25, memory. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0064] Before introducing the embodiments of the present application, first, some terms involved in the embodiments of the present application are defined and explained.

[0065] In machine vision, saliency is a pattern of image segmentation, and a saliency map is an image that shows the uniqueness of each pixel. The goal of a saliency map is to simplify or change the representation of a general image into a style that is easier to analyze. For example, a pixel with a higher gray level in a color image will be displayed in a more obvious way in the saliency map. In the view of visual stimulation, if some features can be captured by attention, such characteristics are called saliency.

[0066] Mosaic enhancement is a data enhancement method that expands the number of data sets by randomly scaling, randomly cropping, and randomly arranging to increase the number of small samples while reducing the amount of data.

[0067] The following will be described in detail in combination with the accompanying drawings Figures 1-6 The application will be further described in detail.

[0068] The application discloses a data processing method combining retention enhancement and mosaic enhancement, as shown in the figure, the steps of the method include S1-S5. Figure 1

[0069] Step S1, acquiring a sample image, wherein the sample image is a plurality of pictures randomly selected from a data set.

[0070] In this step, the data set uses the CCTSB2021 public data set, and the sample image is four pictures randomly selected from the CCTSB2021 public data set. The above pictures are pictures of different sizes.

[0071] Step S2, scaling the sample image to a first preset pixel size to obtain a first feature map.

[0072] In this step, the four sample images are scaled at a constant ratio, and the width of the first feature map obtained after scaling is not greater than 1000 pixels and the height is not greater than 600 pixels.

[0073] Step S3, cutting the first feature map to a second preset pixel size to form a second feature map of the same size.

[0074] In this step, the second preset pixel size can be set to the minimum value of the width and height of the first feature map, and can also be set according to the size of the training image as needed.

[0075] Step S4, calculating the importance score of the second feature map and judging whether the importance score of the second feature map is greater than the importance score threshold, wherein the importance score of the second feature map is calculated using a gradient descent algorithm, and the lighter part of the second feature map represents a higher importance score.

[0076] ​In the embodiment, step S4 further comprises steps S41-S42.

[0077] In step S41, a derivative operation is performed on the label logic value corresponding to each pixel in the second feature map to obtain a single importance score.

[0078] In step S42, the single importance scores are summed to obtain an importance score.

[0079] In one example, the label logic value corresponding to each pixel point (x, y) in the second feature map is Ly(x), and the single importance score of the pixel point (x, y) can be the absolute value of the partial derivative of the label logic value with respect to the pixel point (x, y) |▽xLy(x)|. For an RGB image, the pixel value in each channel of the RGB channel needs to be calculated, and the maximum pixel value in the three channels is selected as the importance score. After obtaining the importance score of each pixel point (x, y), the importance score τ(S, x, y) of the actual cropped region S (the second feature map) is the sum of the single importance scores of each pixel in the above region, wherein the label logic value can also be obtained by training the classification layer of the neural network model, i.e. Ly(x)≈W T (x+b), x represents a pixel of a sample image, w is a width vector of the sample image, and b is a bias vector of the sample image.

[0080] In step S5, if the importance score of the second feature map is greater than the importance score threshold, the plurality of second feature maps are spliced in the mosaic canvas to form a fused image; if the importance score of the second feature map is less than the importance score threshold, the second feature map is reselected in the first feature map until the importance score of the second feature map is greater than the importance score threshold.

[0081] In this step, the setting of the importance score threshold is an adaptive process, and the threshold needs to be calculated by calculating the importance scores of all candidate second feature maps, and then selecting a suitable importance score threshold from the importance score set. The importance score threshold τ selected by the present application is 0.7-0.8.

[0082] Among them, the importance score of the cropped second feature map can be ensured to be greater than the importance score threshold according to the saliency map of the sample image, and the importance score threshold can be determined by judging the picture generation speed of the data set in advance. The importance score threshold is adjusted according to the actual situation.

[0083] In the embodiment, the center point is generated in the fusion image, wherein the center point is randomly generated in a rectangle with the upper left corner of coordinate (w / 4, h / 4) and the lower right corner of coordinate (3w / 4, 3h / 4), w is the width of the fusion image, and h is the height of the fusion image, and the fusion image is divided into four rectangular images according to the center point.

[0084] The expression formula of the second feature map is specifically:

[0085]

[0086] M(S) = M(S) + M(S) + M(S) + M(S) 1-4 ij ij is a binary mask of the selected second feature map, x1-x4 are the extracted second feature maps, and represents the exclusive or operation, represents a set of fusion image binary masks, and i and j each represent any one of 1 to 4.

[0087] In one example, in Figure 2 , second feature maps of different sizes are selected, and the splicing of the fusion image is implemented, the canvas is divided into four rectangles of upper left, lower left, upper right and lower right by the horizontal and vertical area boundary lines of the center point, and the upper left corner, the lower left corner, the upper right corner and the lower right corner of the corresponding orientation of the four second feature maps are placed in the center point, wherein the corresponding orientation refers to the upper left corner of the second feature map located in the upper left corner of the mosaic image is placed in the center point, the upper left corner of the second feature map located in the upper right corner of the mosaic image is placed in the center point, and the upper right corner of the second feature map located in the lower left corner of the mosaic image is placed in the center point, wherein the length and width of the upper left corner of the second feature map are ox and oy, the length and width of the lower left corner of the second feature map are ox and (h-oy), the length and width of the upper right corner of the second feature map are (w-ox) and oy, and the length and width of the lower right corner of the second feature map are (w-ox) and (h-oy), ox represents the horizontal coordinate of the center point, oy represents the vertical coordinate of the center point, w is the width of the fusion image, and h is the height of the fusion image.

[0088] The specified generation range of the center point can prevent the original label from being too small due to too small picture data, lose the detection significance, and avoid generating image data with the center point in an extreme position, so that the effect of the data processing method is weakened and time and resources are wasted.

[0089] The area calculation formula of the mosaic canvas is:

[0090] w = w min × 4 / 3 + 1

[0091] h = h min × 4 / 3 + 1​​

[0092] w min is the minimum value of the width of the sample image, h min is the minimum value of the height of the sample image, and 1 indicates that a pixel with a side length of 1 is located at the lower right corner of the mosaic image, and the upper left corner of the second feature map is placed at the center point.

[0093] The 4 / 3 of the minimum value of the height of the sample image plus 1 pixel and the 4 / 3 of the minimum value of the width of the sample image plus 1 pixel are respectively taken as the height and width of the mosaic canvas, which ensures that the cutting area will not be larger than the sample image when cutting the area, and also helps the extraction of small targets in the sample image.

[0094] In the embodiment, the method further includes steps S6-S7.

[0095] In step S6, it is judged whether the original label exists in the fusion image, wherein the original label is a label in the sample image.

[0096] In this step, not only it is needed to judge whether the fusion label exists in the original label, but also it is needed to judge whether the original label existing in the second feature map needs to be deleted. Since the original label existing in the second feature map may exceed the boundary of the fusion image, the redundant label needs to be deleted.

[0097] In one example, when the area is extracted, the left upper corner coordinates of the intercepted part are returned, and for each label in the extracted first feature map, the judgment condition is as follows:

[0098] x 原始min > x start + w1-5

[0099] y 原始min > y start + h1-5

[0100] x 原始max < x start + 5

[0101] y 原始max < x start + 5

[0102] wherein x 原始min , y 原始min , x 原始max and y 原始max respectively represent the horizontal and vertical coordinates of the upper left and lower right corners of the original label, x start and y start respectively represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image, w1 represents the width of the second feature map, h1 represents the height of the second feature map, and 5 represents the deletion of the label with a side length less than 5 pixels in the fusion image.

[0103] Step S7, if there is an original label in the fusion image, the original label is converted into a new label.

[0104] In one example, for the original label of the picture to be put in the lower right corner, it is needed to judge whether the label is complete, and the incomplete part will be replaced by a boundary line, the horizontal coordinate x 原始min If less than x start , it means that the left side of the original label is out of the boundary of the fusion picture, at this time x 新min becomes 0; on the contrary, it means that the left side of the original label exists in the new picture, that is, the original label is retained, and the original label is converted.

[0105] In this step, the original label is converted into a new label, specifically:

[0106] x 新min = ox-x start +x 原始min

[0107] y 新min = oy-y start +y 原始min

[0108] x 新max = ox-x start +x 原始max

[0109] y 新max = oy-y start +y 原始max

[0110] Wherein, x 新min , y 新min , x 新max and y 新max represent the horizontal and vertical coordinates of the upper left corner and the lower right corner of the new label, ox represents the horizontal coordinate of the center point, oy represents the vertical coordinate of the center point, x start and y start represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image.

[0111] In this embodiment, the fusion image is randomly horizontally flipped and color space transformed to obtain an enhanced fusion image.

[0112] By ensuring that the composed area of the mosaic picture contains information with higher importance, the phenomenon that the new picture has no detection target when performing a small target detection task can be reduced, the stability of training is improved to a certain extent, the effect of training is optimized, and the training efficiency is improved. In addition, the background of the detection picture is enriched, the robustness of the network is better, the GPU memory is reduced, and the limited data produces a value equivalent to more data.

[0113] As can be seen from Figure 3 , by comparing the original picture, 50% mosaic enhancement and 50% original picture, and 50% preserved mosaic enhancement and 50% original picture, the preserved mosaic enhancement converges faster and the training effect is also improved.

[0114] As can be seen from Figure 4 , compared with the existing mosaic enhancement, it can be seen that the preserved mosaic enhancement can ensure that more important information is retained in the enhanced picture during the training process.

[0115] Based on the above method, the embodiment of the application discloses a data processing device combining preserved enhancement and mosaic enhancement, referring to Figure 5 , the data processing device 1 comprises an image acquisition module 11 and a data processing module 12, wherein,

[0116] The image acquisition module 11 is used for acquiring a sample image, wherein the sample image is a plurality of pictures randomly selected from a data set.

[0117] The data processing module 12 is used for scaling the sample image to a first preset pixel size in equal proportions to obtain a first feature map, and cutting the first feature map to a second preset pixel size to form a second feature map of the same size, calculating an importance score of the second feature map, and judging whether the importance score is greater than an importance score threshold value, if the importance score of the second feature map is greater than the importance score threshold value, then splicing a plurality of second feature maps in a mosaic canvas to form a fusion image.

[0118] In one example, the data processing module 12 is used for judging whether there is an original label in the fusion image, wherein the original label is a label in the sample image, if there is an original label in the fusion image, the data processing module converts the original label into a new label.

[0119] In one example, the data processing module 12 is used for judging whether there is an original label in the fusion image, specifically:

[0120] judging whether x 原始min , y 原始min , x 原始max and y 原始max satisfy the following formula,

[0121] x 原始min > x start + w1-5

[0122] y 原始min > y start + h1-5

[0123] x 原始max < x start+5

[0124] y 原始max <x start +5

[0125] wherein, x 原始min , y 原始min , x 原始max and y 原始max represent the horizontal and vertical coordinates of the upper left corner and the lower right corner of the original label respectively, x start and y start represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image respectively, w1 represents the width of the second feature map, h1 represents the height of the second feature map, and 5 represents deleting the label in the fusion image with a side length less than 5 pixels.

[0126] In one example, the data processing module 12 converts the original label into a new label, specifically:

[0127] x 新min = ox - x start + x 原始min

[0128] y 新min = oy - y start + y 原始min

[0129] x 新max = ox - x start + x 原始max

[0130] y 新max = oy - y start + y 原始max

[0131] wherein, x 新min , y 新min , x 新max and y 新max represent the horizontal and vertical coordinates of the upper left corner and the lower right corner of the new label respectively, ox represents the horizontal coordinate of the center point, oy represents the vertical coordinate of the center point, x start and y start represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image respectively.

[0132] In one example, the data processing module 12 is configured to generate a center point in the fusion image, wherein the center point is randomly generated in a rectangle with the upper left corner at coordinates (w / 4, h / 4) and the lower right corner at coordinates (3w / 4, 3h / 4), w is the width of the fusion image, and h is the height of the fusion image, and the data processing module 12 is configured to divide the fusion image into four rectangular images according to the center point.

[0133] In one example, the data processing module 12 is configured to perform a derivation operation on the label logic value corresponding to each pixel in the second feature map to obtain a single importance score, and sum the single importance scores to obtain the importance score.

[0134] In one example, the area of the mosaic canvas is calculated according to the following formula:

[0135] w = w min x 4 / 3 + 1

[0136] h = h min x 4 / 3 + 1

[0137] wherein w min is the minimum value of the sample image width, h min is the minimum value of the sample image height, and 1 represents a pixel with a side length of 1.

[0138] In one example, the data processing module 12 performs random horizontal flipping and color space transformation on the fusion image to obtain an enhanced fusion image.

[0139] Referring to Figure 6 , a structural schematic diagram of an electronic device is provided in the embodiments of the present application. As shown in Figure 3 , the electronic device 2 can include at least one processor 21, at least one network interface 24, a user interface 23, a memory 25, and at least one communication bus 22.

[0140] The communication bus 22 is configured to realize the connection and communication between the components.

[0141] The user interface 23 can include a display screen (Display) and a camera (Camera). Optionally, the user interface 23 can further include a standard wired interface and a wireless interface.

[0142] The network interface 24 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0143] The processor 21 can include one or more processing cores. The processor 21 connects various parts within the server through various interfaces and lines, performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 25, and calling data stored in the memory 25. Alternatively, the processor 21 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 21, but can be realized by a separate chip.

[0144] The memory 25 can include a random access memory (RAM) and a read-only memory (ROM). Alternatively, the memory 25 includes a non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 25 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 25 can also be at least one storage device located away from the aforementioned processor 21. As shown, the memory 25 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an application program for keeping the data processing method of the enhancement and the mosaic enhancement. Figure 6 As shown, the memory 25 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an application program for keeping the data processing method of the enhancement and the mosaic enhancement.

[0145] In Figure 6In the electronic device 2 shown, the user interface 23 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 21 can be used to invoke an application program stored in the memory 25 and retaining a data processing method for enhancement and mosaic enhancement, which, when executed by one or more processors, causes the electronic device to perform one or more methods as described above.

[0146] A computer-readable storage medium stores instructions that, when executed by one or more processors, cause a computer to perform one or more methods as described above.

[0147] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0148] In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0149] In several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of units is only a logical function division. In actual implementation, additional division can be made, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some service interface, device or unit, and can be electrical or other forms.

[0150] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or distributed on a plurality of network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0151] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0152] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk and various program code storage media.

[0153] The above is only an exemplary embodiment of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the disclosure. The present application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional techniques in the art that are not described in the present disclosure.

[0154] The above is only a preferred embodiment of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method for maintaining enhancement in combination with mosaic enhancement, applied in an image processing device, characterized by, The method comprises: acquiring sample images, wherein the sample images are multiple pictures randomly selected from a data set; scaling the sample images to a first preset pixel size in equal proportions to obtain a first feature map; cutting the first feature map to a second preset pixel size to form a second feature map of the same size; calculating an importance score of the second feature map and determining whether the importance score is greater than an importance score threshold; the calculation of the importance score of the second feature map comprises: According to the label logic value corresponding to each pixel in the second feature map, a derivative operation is performed to obtain a single importance score, wherein the label logic value is obtained by training a classification layer of a neural network model, that is, Ly(x) = W T (x + b), x represents a pixel of the sample image, w represents a width vector of the sample image, and b represents a bias vector of the sample image. summing the single importance scores to obtain the importance score; if the importance score is greater than the importance score threshold, then multiple second feature maps are spliced in a mosaic canvas to form a fusion image; the area calculation formula of the mosaic canvas is: w = w min x = 4 / 3 + 1 h = h min x4 / 3 + 1 wherein w is the width of the fused image, h is the height of the fused image, w min is the minimum value of the sample image width, h min is the minimum value of the sample image height, and 1 represents a pixel with a side length of 1. after the multiple second feature maps are spliced in the canvas to form the fusion image, the method further comprises: determining whether there is an original label in the fusion image, wherein the original label is a label in the sample image; if the original label exists in the fusion image, then the original label is converted into a new label.

2. A method of maintaining enhanced data processing in combination with mosaic enhancement as recited in claim 1, wherein, the determination of whether there is an original label in the fusion image comprises: determine x 原始min , y 原始min , x 原始max and y 原始max whether the following formula is satisfied, x 原始min > x start + w1-5 y 原始min >y start +h1-5 x 原始max <x start +5 y 原始max <x start +5 wherein x 原始min , y 原始min , x 原始max and y 原始max represent the horizontal and vertical coordinates of the upper left and lower right corners of the original label, x start and y start represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image, w1 represents the width of the second feature map, h1 represents the height of the second feature map, and 5 represents deleting labels with a side length less than 5 pixels in the fused image.

3. A method of maintaining enhanced data processing in combination with mosaic enhancement as recited in claim 2, wherein, the conversion of the original label into a new label comprises: x 新min = ox - x start + x 原始min y 新min =oy-y start +y 原始min x 新max = ox - x start + x 原始max y 新max =oy-y start +y 原始max wherein x 新min , y 新min , x 新max and y 新max represent the horizontal and vertical coordinates of the upper left and lower right corners of the new label, ox represents the horizontal coordinate of the center point, oy represents the vertical coordinate of the center point, x start and y start represent the horizontal and vertical coordinates of the upper left corner of the second feature map on the sample image, respectively.

4. A method of maintaining enhanced data processing in combination with mosaic enhancement as recited in claim 1, wherein, the method further comprises: generating a center point in the fusion image, wherein the center point is randomly generated in a rectangle with the upper left corner at coordinates (w / 4, h / 4) and the lower right corner at coordinates (3w / 4, 3h / 4), w is the width of the fusion image, and h is the height of the fusion image; dividing the fusion image into four rectangular images according to the center point.

5. A method of maintaining enhanced data processing in combination with mosaic enhancement as recited in claim 1, wherein, the method further comprises: randomly horizontally flipping and color space transforming the fusion image to obtain an enhanced fusion image.

6. A data processing apparatus that maintains enhancement in combination with mosaic enhancement, characterized by, The data processing device (1) comprises an image acquisition module (11) and a data processing module (12), wherein the image acquisition module (11) is configured to acquire sample images, wherein the sample images are multiple pictures randomly selected from a data set; the data processing module (12) is configured to scale the sample images to a first preset pixel size in equal proportions to obtain a first feature map, cut the first feature map to a second preset pixel size to form a second feature map of the same size, calculate an importance score of the second feature map, and determine whether the importance score is greater than an importance score threshold, and if the importance score is greater than the importance score threshold, then multiple second feature maps are spliced in a mosaic canvas to form a fusion image; the calculation of the importance score of the second feature map comprises: According to the label logic value corresponding to each pixel in the second feature map, a derivative operation is performed to obtain a single importance score, wherein the label logic value is obtained by training a classification layer of a neural network model, that is, Ly(x) = W T (x + b), x represents a pixel of the sample image, w represents a width vector of the sample image, and b represents a bias vector of the sample image. summing the single importance scores to obtain the importance score; the area calculation formula of the mosaic canvas is: w = w min x4 / 3 + 1 h = h min x4 / 3 + 1 wherein w is the width of the fused image, h is the height of the fused image, w min is the minimum value of the sample image width, h min is the minimum value of the sample image height, and 1 represents a pixel with a side length of 1. after the multiple second feature maps are spliced in the canvas to form the fusion image, the method further comprises: determining whether there is an original label in the fusion image, wherein the original label is a label in the sample image; if the original label exists in the fusion image, then the original label is converted into a new label.

7. An electronic device, comprising: The electronic device (2) comprises a processor (21), a memory (25) for storing instructions, a user interface (23) and a network interface (24) for communicating with other devices, and the processor (21) is configured to execute the instructions stored in the memory (25) to make the electronic device (2) perform the method according to any one of claims 1-5.

Images