Image enhancement method and device, electronic equipment and storage medium

Abstract

The invention provides an image enhancement method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of initial images, and carrying out the resampling of the plurality of initial images, and obtaining a resampling image; performing channel decomposition on the resampling image to obtain a plurality of channel images; inputting the plurality of channel images into a target recognition model for target recognition to obtain a target object in the initial image; performing element decomposition on the target object to obtain sub-element image information of the target object; acquiring enhanced image information, and determining a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information; splicing the plurality of target enhanced sub-images to obtain a target enhanced image of the target object; according to the method, the target object is replaced by the target enhanced image, and the target object with lower resolution is replaced by the target enhanced image with higher resolution, so that the target object in the amplified initial image is clearer, and the near-to-eye display experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image enhancement method, device, electronic equipment and storage medium. Background Art

[0002] Near-eye display can be understood as a technology that creates virtual images in the field of view of one eye or two eyes. It renders light field information to the human eye through a display device placed within the non-clear vision distance of the human eye, and then reconstructs the virtual scene in front of the human eye.

[0003] During near-eye display, users can click on a portion of the image area in the video to zoom in on it; however, if the video resolution is low, the zoomed-in portion of the image area will become blurry, resulting in a poor near-eye display experience for the user. Therefore, how to improve the user's near-eye display experience is an urgent problem to be solved. Summary of the invention

[0004] The present invention provides an image enhancement method, device, electronic device and storage medium, which can solve the problem that a part of the image area becomes blurred after being enlarged, resulting in a poor near-eye display experience for the user.

[0005] According to a first aspect of the present invention, there is provided an image enhancement method, the method comprising:

[0006] Acquire multiple initial images, and resample the multiple initial images to obtain resampled images;

[0007] Performing channel decomposition on the resampled image to obtain multiple channel images;

[0008] Inputting the multiple channel images into a target recognition model for target recognition to obtain a target object in the initial image;

[0009] Decomposing the target object into elements to obtain sub-element image information of the target object;

[0010] Acquire enhanced image information, and determine a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information;

[0011] splicing the multiple target enhanced sub-images to obtain a target enhanced image of the target object;

[0012] The target object is replaced by the target enhanced image.

[0013] According to a second aspect of the present invention, there is provided an image enhancement device, the device comprising:

[0014] An image resampling module, used for acquiring a plurality of initial images, and resampling the plurality of initial images to obtain resampled images;

[0015] A channel decomposition module, used for performing channel decomposition on the resampled image to obtain multiple channel images;

[0016] A target recognition module, used for inputting the multiple channel images into a target recognition model for target recognition to obtain a target object in the initial image;

[0017] An element decomposition module, used to perform element decomposition on the target object to obtain sub-element image information of the target object;

[0018] an enhanced image determination module, used to obtain enhanced image information, and determine a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information;

[0019] A sub-image stitching module, used for stitching the multiple target enhanced sub-images to obtain a target enhanced image of the target object;

[0020] An image replacement module is used to replace the target object with the target enhanced image.

[0021] According to a third aspect of the present invention, there is provided an electronic device, comprising a processor and a memory.

[0022] The memory is used to store codes and related data;

[0023] The processor is used to execute the code in the memory to implement the image enhancement method as described in any one of the embodiments of the present invention.

[0024] According to a fourth aspect of the present invention, there is provided a storage medium on which a computer program is stored. When the program is executed by a processor, the image enhancement method as described in any one of the embodiments of the present invention is implemented.

[0025] In an embodiment of the present invention, a plurality of initial images are obtained, and the plurality of initial images are resampled to obtain resampled images; the resampled images are channel-decomposed to obtain a plurality of channel images; the plurality of channel images are input into a target recognition model for target recognition to obtain a target object in the initial image; the target object is element-decomposed to obtain sub-element image information of the target object; enhanced image information is obtained, and a plurality of target enhanced sub-images of the target object are determined based on the sub-element image information and the enhanced image information; the plurality of target enhanced sub-images are spliced ​​to obtain a target enhanced image of the target object; and the target object is replaced with the target enhanced image. That is, the present invention obtains multiple initial images (images corresponding to the enlarged partial image area), resamples and decomposes the multiple initial images into multiple channel images, and then inputs the multiple channel images into a target recognition model for target recognition to obtain a target object in the initial image; decomposes the target object into sub-elements to obtain sub-element image information of each area of ​​the target object; and determines multiple target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information, that is, according to the sub-element image information and the enhanced image information, determines a target enhanced sub-image with a higher resolution for each area of ​​the target object, splices the target enhanced sub-images with a higher resolution for each area to obtain a target enhanced image of the target object, and finally replaces the target object with the target enhanced image, that is, replaces the target object with a lower resolution with a target enhanced image with a higher resolution, thereby improving the resolution of the target object in the initial image, and then improving the resolution of the initial image, so that the target object in the enlarged initial image is clearer, bringing a better near-eye display experience to the user, and improving the user's near-eye display experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 is a flow chart of an image enhancement method provided by an embodiment of the present invention;

[0028] Figure 2 is another flowchart of the image enhancement method provided by an embodiment of the present invention;

[0029] Figure 3 is a structural schematic diagram of an image enhancement device provided by an embodiment of the present invention;

[0030] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0034] Figure 1 The following is a flow chart of an image enhancement method provided by an embodiment of the present invention. The method can be performed by an image enhancement device, which can be implemented in software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a computer, a server, etc. The following embodiments will be described by taking the device integrated into an electronic device as an example. Figure 1 , the method may specifically include the following steps:

[0035] Step 101: Acquire multiple initial images, and resample the multiple initial images to obtain resampled images.

[0036] Among them, the initial image can be understood as the image corresponding to the enlarged partial image area. Resampling can be understood as a data processing technology, which is mainly used to change the sampling rate or resolution of a data set. It can balance the data set by increasing or decreasing the number of data points, and is often used to deal with imbalance problems in data sets. In an embodiment of the present invention, since the video includes multiple initial images per second, it is necessary to restore the image features of each initial image through resampling and retain the image features of each initial image to the resampled image, so that the resampled image can contain the image features of multiple initial images. The more image features the resampled image contains, the more image features each channel image has, and the higher the accuracy of inputting multiple channel images into the target recognition model for target recognition, the higher the recognition accuracy of the target object.

[0037] Step 102: performing channel decomposition on the resampled image to obtain a plurality of channel images.

[0038] When the initial image is in RGB color mode, the image channels generally include an R channel, a G channel, and a B channel. The multiple channel images may include an R channel image, a G channel image, and a B channel image.

[0039] In an optional implementation, the resampled image may be split into multiple channel images by index splitting.

[0040] In another optional implementation, the resampled image may be split into multiple channel images by means of a function.

[0041] Step 103: input the multiple channel images into the target recognition model for target recognition to obtain the target object in the initial image.

[0042] Among them, the target recognition model can be understood as a model for identifying the position and category of objects in an image. In an embodiment of the present invention, the target recognition model can be YOLOv4-tiny. The target recognition model uses YOLOv4-tiny, which can achieve a faster target recognition speed and improve the recognition speed of the target object under the premise of ensuring the recognition accuracy of the target recognition, thereby improving the enhancement speed of image enhancement. The target recognition model may include a convolution module, a sampling module, and a fully connected module. The sampling module may include upsampling and downsampling. The target object can be understood as an object in the initial image, and the target object can be any object such as a person, a plant, an animal, a car, etc.

[0043] In an optional implementation, multiple channel images can be input into a convolution module, and the convolution module is used to extract features from the multiple channel images to obtain multiple feature maps, and the multiple feature maps are convolved to obtain multiple convolution feature blocks; the multiple convolution feature blocks are input into a sampling module using the convolution module, and the multiple convolution feature blocks are sampled using the sampling module to obtain multiple sampled feature maps; the multiple sampled feature maps are input into a full connection module through the sampling module, and the multiple sampled feature maps are fully connected using the full connection module to obtain the target object in the initial image. The convolution operation can include 1×1 point convolution and 3×3 standard convolution.

[0044] Specifically, using a convolution module to perform convolution operations on multiple feature maps to obtain multiple convolution feature blocks can include: simultaneously loading sub-feature maps of preset sizes and preset weights from each feature map, and then performing convolution calculations based on the sub-feature maps and preset weight blocks to obtain multiple convolution feature blocks of preset sizes corresponding to each feature map. This is equivalent to using a cyclic blocking strategy to achieve the purpose of performing parallel convolution operations on multiple feature maps at the same time, thereby improving the convolution calculation speed of the convolution operation in the target recognition process, thereby improving the recognition speed of the target recognition model, and thus improving the enhancement speed of image enhancement.

[0045] Step 104: Decompose the target object into elements to obtain sub-element image information of the target object.

[0046] The sub-element image information may be understood as image information of different regions of the target object.

[0047] In an optional implementation, the target object may be input into an image segmentation network for element decomposition to obtain sub-element information of the target object.

[0048] Step 105 , acquiring enhanced image information, and determining a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information.

[0049] Among them, the enhanced image information can be understood as the image information obtained after image enhancement of images of different areas of the target object. The enhanced image information may include multiple enhanced sub-labels and multiple enhanced sub-images, and one enhanced sub-label corresponds to at least one enhanced sub-image. The enhanced sub-image can be understood as a high-definition image of a partial area of ​​the target object. The enhanced sub-label can be understood as the object category contained in the enhanced sub-image. The sub-element image information may include multiple sub-labels and multiple sub-element images, and one sub-label corresponds to a sub-element image. The sub-element image can be understood as an image of a partial area of ​​the target object. The sub-label can be understood as the object category contained in the sub-image.

[0050] In an optional implementation, a target enhancement subtag corresponding to each subtag can be determined based on multiple enhancement subtags and multiple subtags; a target enhancement subimage corresponding to each subtag can be determined based on the sub-element image corresponding to each subtag and the multiple enhancement subimages corresponding to the target enhancement subtag; and the target enhancement subimage corresponding to each subtag can be determined as multiple target enhancement subimages of the target object. In this way, based on the multiple sub-element images of the target object and the subtags corresponding to each sub-element image, a target enhancement subimage with a higher resolution corresponding to each sub-element image can be determined from the enhanced image information, and then a target enhancement image with a higher resolution of the target object can be obtained by splicing the multiple target enhancement subimages, thereby improving the resolution of the target object and improving the user's near-eye display experience.

[0051] Specifically, according to multiple enhancer tags and multiple sub-tags, a target enhancer tag corresponding to each sub-tag is determined, including: tag matching multiple enhancer tags and multiple sub-tags to obtain a first matching result; when the first matching result is a successful match, determining the successfully matched enhancer tag as the target enhancer tag corresponding to the sub-tag.

[0052] For example, the target object is a drone in the initial image, and the target object is decomposed into elements to obtain sub-element image information of the target object, including: blade image, sub-label of blade image (blade), fuselage image, sub-label corresponding to fuselage image (body), camera image, sub-label corresponding to camera image (camera). The enhanced image information includes: multiple high-definition blade images, enhanced sub-labels of multiple high-definition blade images (blade), multiple high-definition fuselage images, enhanced sub-labels corresponding to multiple high-definition fuselage images (body), multiple high-definition camera images, enhanced sub-labels corresponding to multiple high-definition camera images (camera), multiple high-definition blessing characters images, and enhanced sub-labels corresponding to multiple high-definition blessing characters (blessing characters). Multiple enhanced sub-tags (blades, fuselage, cameras) and multiple sub-tags (blades, fuselage, cameras, blessing characters) can be matched. If the enhanced sub-tags (blades, fuselage, cameras) and the sub-tags (blades, fuselage, cameras) are the same, the first matching result can be determined as a successful match; then the successfully matched enhanced sub-tags (blades, fuselage, cameras) can be determined as the target enhanced sub-tags corresponding to the sub-tags; then, based on the sub-element images corresponding to each sub-tag and the multiple enhanced sub-images corresponding to the target enhanced sub-tag, the target enhanced sub-images corresponding to each sub-tag are determined; and the target enhanced sub-images corresponding to each sub-tag are determined as multiple target enhanced sub-images of the target object.

[0053] Step 106: stitching multiple target enhanced sub-images to obtain a target enhanced image of the target object.

[0054] Among them, the target enhanced image can be understood as a high-definition image of the target object.

[0055] In an optional implementation, pixel coordinate points of multiple target enhanced sub-images are obtained; based on the pixel coordinate points, splicing coordinates of the multiple target enhanced sub-images are determined; and the multiple target enhanced sub-images are spliced ​​based on the splicing coordinates. The pixel coordinate points can be understood as the coordinates of the pixel points adjacent to the splicing coordinates, and the number of pixel coordinate points can be 4 or 16.

[0056] Specifically, according to the pixel coordinate points, the splicing coordinates of the multiple target enhanced sub-images are determined, and a bilinear interpolation method or a bicubic interpolation method can be used. Bilinear interpolation and bicubic interpolation are the most commonly used interpolation methods in two-dimensional space.

[0057] When the bilinear interpolation method is used to determine the splicing coordinates of the multiple target enhanced sub-images according to the pixel coordinates of the interpolation points of the multiple target enhanced sub-images, the number of pixel coordinates is 4. When the bicubic interpolation method is used to determine the splicing coordinates of the multiple target enhanced sub-images according to the pixel coordinates of the interpolation points of the multiple target enhanced sub-images, the number of pixel coordinates is 16.

[0058] More specifically, pixel coordinate points adjacent to the stitching coordinates may be obtained, and then a cubic polynomial may be applied to perform interpolation in the first preset axis direction and the second preset axis direction respectively, and finally the interpolation result may be calculated to obtain the stitching coordinates.

[0059] In the embodiment of the present invention, the bicubic interpolation method uses more pixel points in the neighborhood range than the bilinear interpolation method, and considering the change rate of the pixel values ​​between each neighboring point, the accuracy of the interpolated stitching coordinates is higher. Therefore, in order to improve the accuracy of the stitching coordinates, the bicubic difference method is preferably used to determine the stitching coordinates of multiple target enhanced sub-images according to the pixel coordinate points of the to-be-interpolated points of the multiple target enhanced sub-images.

[0060] Step 107: Replace the target object with the target enhanced image.

[0061] In an embodiment of the present invention, multiple initial images (images corresponding to the enlarged partial image area) are acquired, the multiple initial images are resampled and channel-decomposed to obtain multiple channel images, and then the multiple channel images are input into a target recognition model for target recognition to obtain a target object in the initial image; the target object is decomposed into sub-elements to obtain sub-element image information of each area of ​​the target object; and multiple target enhanced sub-images of the target object are determined according to the sub-element image information and the enhanced image information, that is, target enhanced sub-images with higher resolutions of each area of ​​the target object are determined according to the sub-element image information and the enhanced image information, and the target enhanced sub-images with higher resolutions of each area are spliced ​​to obtain a target enhanced image of the target object, and finally the target object is replaced with the target enhanced image, that is, the target object with lower resolution is replaced with the target enhanced image with higher resolution, thereby improving the resolution of the target object in the initial image, and then improving the resolution of the initial image, so that the target object in the enlarged initial image is clearer, bringing a better near-eye display experience to the user, and improving the user's near-eye display experience.

[0062] In some embodiments, after acquiring the enhanced image information, it also includes: acquiring the magnification of the initial image; when the magnification is greater than a preset threshold, triggering the execution of the step of determining multiple target enhanced sub-images of the target object based on the sub-element image information and the enhanced image information; when the magnification is less than or equal to the preset threshold, acquiring the target label of the target object; and determining the target enhanced image of the target object based on the target label and the enhanced image information.

[0063] The preset threshold value can be understood as a preset critical value of the magnification of the initial image. The target label can be understood as a category of the target object.

[0064] Specifically, the enhanced image information may also include multiple enhanced tags and multiple enhanced images, one enhanced tag corresponds to one enhanced image, and the target enhanced image of the target object is determined according to the target tag and the enhanced image information, including: matching the target tag with the multiple enhanced tags to obtain a second matching result; when the second matching result is a successful match, determining the enhanced image corresponding to the enhanced tag as the target enhanced image of the target object. The enhanced image can be understood as a high-definition image of the object contained in the image. The enhanced tag can be understood as the object category contained in the enhanced image.

[0065] For example, the target object is a drone in the initial image, and the enhanced image information includes: a high-definition drone image, an enhanced sub-label (drone) of the high-definition drone image, and a high-definition blessing character image (high-definition fuselage image). The magnification is 5, and the preset threshold is 6. It can be determined that the magnification is less than the preset threshold, and the target label (drone) and multiple enhanced labels (drone, blessing character) are matched to obtain a second matching result; when the second matching result is a successful match, the enhanced image (high-definition drone image) corresponding to the successfully matched enhanced label (drone) is determined as the target enhanced image of the target object.

[0066] In an embodiment of the present invention, when the magnification is greater than a preset threshold, that is, when the magnification is large, when viewing the initial image, the user needs to observe more detailed features of the target object in the initial image, and because the magnification is too large, the detailed features of the target object are more blurred, then multiple target enhanced sub-images of the target object can be determined according to the sub-element image information and the enhanced image information; multiple target enhanced sub-images are spliced ​​to obtain a target enhanced image of the target object, and the target object is replaced with the target enhanced image, so that the target enhanced image includes a higher resolution image of each region of the target object, so that the image resolution at the detailed features of the target object is higher, so as to meet the user's need to observe more detailed features of the target object in the initial image. When the magnification is less than or equal to the preset threshold, that is, when the magnification is small, the resolution of the detailed features of the target object can meet the user's need to observe more detailed features of the target object in the initial image. At this time, there is no need to determine multiple target sub-images of the target object and splice multiple target sub-images to obtain a target enhanced image. It is only necessary to determine the target enhanced image corresponding to the entire target object according to the target label and enhanced image information of the target object, thereby improving the speed and efficiency of image enhancement.

[0067] The image enhancement method provided by the embodiment of the present invention is further described below. Figure 2 As shown, Figure 2 FIG. 4 is another flow chart of the image enhancement method provided by an embodiment of the present invention, which may specifically include the following steps:

[0068] Step 201 , obtaining a plurality of initial images, and resampling the plurality of initial images to obtain resampled images.

[0069] Step 202: performing channel decomposition on the resampled image to obtain a plurality of channel images.

[0070] Step 203: input the multiple channel images into the target recognition model for target recognition to obtain the target object in the initial image.

[0071] Step 204: Decompose the target object into elements to obtain sub-element image information of the target object.

[0072] Step 205: Obtain enhanced image information.

[0073] Step 206, obtaining the magnification of the initial image.

[0074] Step 207, determine whether the magnification is greater than a preset threshold, if so, execute step 208; if not, execute step 212.

[0075] Step 208: Determine a target enhancer subtag corresponding to each subtag according to the multiple enhancer subtags and the multiple subtags.

[0076] Step 209 : determining the target enhanced sub-image corresponding to each sub-label according to the sub-element images corresponding to each sub-label and the multiple enhanced sub-images corresponding to the target enhanced sub-label.

[0077] Since there are multiple enhanced sub-images, the enhanced sub-image with the highest similarity to the sub-element image can be determined from the multiple enhanced sub-images based on the pixel values ​​of the sub-element image and the pixel values ​​of the enhanced sub-image. Therefore, in an optional implementation, the first pixel value of the sub-element image corresponding to each sub-label can be determined; multiple second pixel values ​​of multiple enhanced sub-images corresponding to the target enhanced sub-label can be determined; based on the first pixel value and multiple second pixel values, the target enhanced sub-image corresponding to each sub-label can be determined, so that the enhanced sub-image with the highest similarity to each sub-element image can be determined from the multiple enhanced sub-images.

[0078] Specifically, if it is desired to determine the enhanced sub-image with the highest similarity to the sub-element image from multiple enhanced sub-images, the minimum difference between the pixel value of the sub-element image and the pixel value of the enhanced sub-image can be set, and the enhanced sub-image corresponding to the second pixel value that is less than the minimum difference can be determined as the target enhanced sub-image corresponding to the sub-element image. Therefore, the pixel difference between the first pixel value and multiple second pixel values ​​can be determined; when the pixel difference is less than the preset difference, the second pixel value is determined as the target pixel value; and the enhanced sub-image corresponding to the target pixel value is determined as the target enhanced sub-image corresponding to the sub-label. The preset difference can be understood as the minimum difference between the preset first pixel value and the second pixel value.

[0079] For example, the preset difference is E. The sub-element image includes: a body image and a camera image, and the sub-tags of the sub-element image include: a sub-tag of the body image: body, and a sub-tag of the camera image: camera. The target enhancement sub-tags corresponding to the sub-tags include: body and camera. The multiple enhanced sub-images corresponding to the target enhancement sub-tag include: multiple high-definition body images and multiple high-definition camera images, and the multiple high-definition body images include: P1, P2 and P3. The multiple high-definition camera images include: P4, P5 and P6. It can be determined that the first pixel value of the sub-element image (body image) is X, the second pixel value of P1 is Y1, the second pixel value of P2 is Y2, and the second pixel value of P3 is Y3. The determined first pixel value of the sub-element image includes: the first pixel value X1 of the body image and the first pixel value X2 of the camera image. The multiple second pixel values ​​of the multiple enhanced sub-images determined include: the second pixel value of P1 is Y1, the second pixel value of P2 is Y2, the second pixel value of P3 is Y3, the second pixel value of P4 is Y4, the second pixel value of P5 is Y5, and the second pixel value of P6 is Y6. Among them, the difference between Y1 and X is D1, the difference between Y2 and X is D2, the difference between Y3 and X is D3, and D3<E<D2<D1. It can be determined that D3 is less than the preset difference E, then the second pixel value Y2 corresponding to D3 less than E can be determined as the target pixel value, and the enhanced sub-image P2 corresponding to the target pixel value Y2 is determined as the target enhanced sub-image corresponding to the sub-label (body). The determination process of the target enhanced sub-image corresponding to the sub-label (camera) is the same as that of the target enhanced sub-image corresponding to the sub-label (body), and will not be repeated.

[0080] Step 210: determine the target enhanced sub-images corresponding to the sub-labels as multiple target enhanced sub-images of the target object.

[0081] Step 211, stitching multiple target enhanced sub-images to obtain a target enhanced image of the target object.

[0082] After executing step 211 , execute step 213 .

[0083] Step 212: Determine a target enhanced image of the target object according to the target label and the enhanced image information.

[0084] Step 213: Replace the target object with the target enhanced image.

[0085] In an embodiment of the present invention, multiple initial images (images corresponding to the enlarged partial image area) are acquired, the multiple initial images are resampled and channel-decomposed to obtain multiple channel images, and then the multiple channel images are input into a target recognition model for target recognition to obtain a target object in the initial image; the target object is decomposed into sub-elements to obtain sub-element image information of each area of ​​the target object; and multiple target enhanced sub-images of the target object are determined according to the sub-element image information and the enhanced image information, that is, target enhanced sub-images with higher resolutions of each area of ​​the target object are determined according to the sub-element image information and the enhanced image information, and the target enhanced sub-images with higher resolutions of each area are spliced ​​to obtain a target enhanced image of the target object, and finally the target object is replaced with the target enhanced image, that is, the target object with lower resolution is replaced with the target enhanced image with higher resolution, thereby improving the resolution of the target object in the initial image, and then improving the resolution of the initial image, so that the target object in the enlarged initial image is clearer, bringing a better near-eye display experience to the user, and improving the user's near-eye display experience.

[0086] Figure 3 FIG. 1 is a schematic diagram of a structure of an image enhancement device provided by an embodiment of the present invention. The device is suitable for executing the image enhancement method provided by an embodiment of the present invention. Figure 3 As shown, the device may specifically include:

[0087] The image resampling module 301 is used to obtain a plurality of initial images and resample the plurality of initial images to obtain resampled images;

[0088] A channel decomposition module 302, configured to perform channel decomposition on the resampled image to obtain a plurality of channel images;

[0089] A target recognition module 303 is used to input the multiple channel images into a target recognition model for target recognition to obtain a target object in the initial image;

[0090] An element decomposition module 304 is used to perform element decomposition on the target object to obtain sub-element image information of the target object;

[0091] An enhanced image determination module 305 is used to obtain enhanced image information and determine a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information;

[0092] A sub-image stitching module 306 is used to stitch the multiple target enhanced sub-images to obtain a target enhanced image of the target object;

[0093] Image replacement module 307, used to replace the target object with the target enhanced image

[0094] Optionally, the enhanced image information includes multiple enhanced sub-labels and multiple enhanced sub-images, one enhanced sub-label corresponds to at least one enhanced sub-image, the sub-element image information includes multiple sub-labels and multiple sub-element images, one sub-label corresponds to one sub-element image, and the enhanced image determination module 305 is specifically used to:

[0095] Determining a target enhancer subtag corresponding to each subtag according to the multiple enhancer subtags and the multiple subtags;

[0096] Determine the target enhancement sub-image corresponding to each sub-label according to the sub-element images corresponding to each sub-label and the multiple enhancement sub-images corresponding to the target enhancement sub-label;

[0097] The target enhanced sub-images corresponding to the sub-labels are determined as a plurality of target enhanced sub-images of the target object.

[0098] Optionally, the enhanced image determination module 305 determines, according to the multiple enhanced sub-tags and the multiple sub-tags, a target enhanced sub-tag corresponding to each sub-tag, including:

[0099] Performing tag matching on the multiple enhancement subtags and the multiple subtags to obtain a first matching result;

[0100] When the first matching result is a successful match, the successfully matched enhancer subtag is determined as a target enhancer subtag corresponding to the subtag.

[0101] Optionally, the enhanced image determination module 305 determines the target enhanced sub-image corresponding to each sub-label according to the sub-element images corresponding to each sub-label and the multiple enhanced sub-images corresponding to the target enhanced sub-label, including:

[0102] Determine a first pixel value of a sub-element image corresponding to each sub-tag;

[0103] Determining a plurality of second pixel values ​​of a plurality of enhancer images corresponding to the target enhancer label;

[0104] A target enhanced sub-image corresponding to each of the sub-labels is determined according to the first pixel value and the plurality of second pixel values.

[0105] Optionally, the enhanced image determination module 305 determines the target enhanced sub-image corresponding to each sub-label according to the first pixel value and the plurality of second pixel values, including:

[0106] determining a pixel difference between the first pixel value and the plurality of second pixel values;

[0107] When the pixel difference is less than a preset difference, determining the second pixel value as the target pixel value;

[0108] The enhanced sub-image corresponding to the target pixel value is determined as the target enhanced sub-image corresponding to the sub-label.

[0109] Optionally, the sub-image stitching module 306 is specifically configured to:

[0110] Obtaining pixel coordinate points of the multiple target enhanced sub-images;

[0111] Determining the splicing coordinates of the multiple target enhanced sub-images according to the pixel coordinate points;

[0112] The multiple target enhanced sub-images are stitched together according to the stitching coordinates.

[0113] Furthermore, the enhanced image determination module 305 is further specifically configured to:

[0114] Acquiring the magnification of the initial image;

[0115] When the magnification is greater than a preset threshold, triggering the step of determining a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information;

[0116] When the magnification is less than or equal to the preset threshold, acquiring a target label of the target object;

[0117] A target enhanced image of the target object is determined according to the target tag and the enhanced image information.

[0118] Optionally, the enhanced image information further includes a plurality of enhanced labels and a plurality of enhanced images, one enhanced label corresponds to one enhanced image, and the enhanced image determination module 305 determines the target enhanced image of the target object according to the target label and the enhanced image information, including:

[0119] Performing label matching on the target label and the plurality of enhanced labels to obtain a second matching result;

[0120] When the second matching result is a successful match, the enhanced image corresponding to the successfully matched enhanced tag is determined as the target enhanced image of the target object.

[0121] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0122] The image enhancement device provided by the embodiment of the present invention can obtain multiple initial images (images corresponding to the enlarged partial image area), resample and channel decompose the multiple initial images to obtain multiple channel images, and then input the multiple channel images into the target recognition model for target recognition to obtain the target object in the initial image; perform sub-element decomposition on the target object to obtain sub-element image information of each area of ​​the target object; and determine multiple target enhanced sub-images of the target object based on the sub-element image information and the enhanced image information, that is, determine target enhanced sub-images with higher resolutions for each area of ​​the target object based on the sub-element image information and the enhanced image information, splice the target enhanced sub-images with higher resolutions for each area to obtain the target enhanced image of the target object, and finally replace the target object with the target enhanced image, that is, replace the target object with a lower resolution with a target enhanced image with a higher resolution, thereby improving the resolution of the target object in the initial image, and then improving the resolution of the initial image, so that the target object in the enlarged initial image is clearer, bringing a better near-eye display experience to the user, and improving the user's near-eye display experience.

[0123] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0124] Please refer to Figure 4 , an electronic device 50 is provided, comprising:

[0125] a processor 51; and

[0126] A memory 52, used to store executable instructions of the processor;

[0127] The processor 51 is configured to execute the above-mentioned method by executing the executable instructions.

[0128] The processor 51 can communicate with the memory 52 via a bus 53 .

[0129] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned method is implemented.

[0130] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image enhancement method, characterized in that: The method comprises: Acquire multiple initial images, and resample the multiple initial images to obtain resampled images; Performing channel decomposition on the resampled image to obtain multiple channel images; Inputting the multiple channel images into a target recognition model for target recognition to obtain a target object in the initial image; Decomposing the target object into elements to obtain sub-element image information of the target object; Acquire enhanced image information, and determine a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information; splicing the multiple target enhanced sub-images to obtain a target enhanced image of the target object; The target object is replaced by the target enhanced image.

2. The method according to claim 1, characterized in that The enhanced image information includes a plurality of enhanced sub-labels and a plurality of enhanced sub-images, one enhanced sub-label corresponds to at least one enhanced sub-image, the sub-element image information includes a plurality of sub-labels and a plurality of sub-element images, one sub-label corresponds to a sub-element image, and determining a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information includes: Determining a target enhancer subtag corresponding to each subtag according to the multiple enhancer subtags and the multiple subtags; Determine the target enhancement sub-image corresponding to each sub-label according to the sub-element images corresponding to each sub-label and the multiple enhancement sub-images corresponding to the target enhancement sub-label; The target enhanced sub-images corresponding to the sub-labels are determined as a plurality of target enhanced sub-images of the target object.

3. The method according to claim 2, characterized in that The step of determining, according to the plurality of enhancer subtags and the plurality of subtags, a target enhancer subtag corresponding to each subtag comprises: Performing tag matching on the multiple enhancement subtags and the multiple subtags to obtain a first matching result; When the first matching result is a successful match, the successfully matched enhancer subtag is determined as a target enhancer subtag corresponding to the subtag.

4. The method according to claim 2, characterized in that: The determining, according to the sub-element images corresponding to the sub-labels and the plurality of enhanced sub-images corresponding to the target enhanced sub-labels, the target enhanced sub-image corresponding to the sub-labels comprises: Determine a first pixel value of a sub-element image corresponding to each sub-tag; Determining a plurality of second pixel values ​​of a plurality of enhancer images corresponding to the target enhancer label; A target enhanced sub-image corresponding to each of the sub-labels is determined according to the first pixel value and the plurality of second pixel values.

5. The method according to claim 4, characterized in that The step of determining the target enhanced sub-image corresponding to each sub-label according to the first pixel value and the plurality of second pixel values ​​includes: determining a pixel difference between the first pixel value and the plurality of second pixel values; When the pixel difference is less than a preset difference, determining the second pixel value as the target pixel value; The enhanced sub-image corresponding to the target pixel value is determined as the target enhanced sub-image corresponding to the sub-label.

6. The method according to claim 1, characterized in that The step of stitching the plurality of target enhanced sub-images to obtain a target enhanced image of the target object includes: Obtaining pixel coordinate points of the multiple target enhanced sub-images; Determining the splicing coordinates of the multiple target enhanced sub-images according to the pixel coordinate points; The multiple target enhanced sub-images are stitched together according to the stitching coordinates.

7. The method according to claim 1, characterized in that After obtaining enhanced image information, it also includes: Acquiring the magnification of the initial image; When the magnification is greater than a preset threshold, triggering the step of determining a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information; When the magnification is less than or equal to the preset threshold, acquiring a target label of the target object; A target enhanced image of the target object is determined according to the target tag and the enhanced image information.

8. The method according to claim 7, characterized in that The enhanced image information further includes a plurality of enhanced labels and a plurality of enhanced images, one enhanced label corresponds to one enhanced image, and determining the target enhanced image of the target object according to the target label and the enhanced image information includes: Performing label matching on the target label and the plurality of enhanced labels to obtain a second matching result; When the second matching result is a successful match, the enhanced image corresponding to the successfully matched enhanced tag is determined as the target enhanced image of the target object.

9. An image enhancement device, characterized in that: The device comprises: An image resampling module, used for acquiring a plurality of initial images, and resampling the plurality of initial images to obtain resampled images; A channel decomposition module, used for performing channel decomposition on the resampled image to obtain multiple channel images; A target recognition module is used to input the multiple channel images into a target recognition model for target recognition to obtain a target object in the initial image; An element decomposition module, used to perform element decomposition on the target object to obtain sub-element image information of the target object; an enhanced image determination module, used to obtain enhanced image information, and determine a plurality of target enhanced sub-images of the target object according to the sub-element image information and the enhanced image information; A sub-image stitching module, used for stitching the multiple target enhanced sub-images to obtain a target enhanced image of the target object; An image replacement module is used to replace the target object with the target enhanced image.

10. An electronic device, characterized in that: Including processor and memory, The memory is used to store codes and related data; The processor is used to execute the code in the memory to implement the image enhancement method according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the image enhancement method according to any one of claims 1 to 8 is implemented.

Images