Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method. The method comprises the following steps: acquiring a plurality of first to-be-processed images and fusion parameters set by a user; performing fusion processing on the plurality of first to-be-processed images based on the fusion parameters to obtain fusion data; and performing image processing based on the fusion data to obtain a target image corresponding to the first to-be-processed image. According to the embodiment of the invention, the fusion processing is performed on the plurality of first to-be-processed images through the fusion parameters, the fusion data more conforming to the user intention can be obtained, and the image processing is performed according to the fusion data, so that the more accurate target image can be obtained. Due to the use of the fusion data, the relationship between different first to-be-processed images can be identified more flexibly, so that the dimension of image processing is expanded, the purpose of improving the accuracy of image processing is achieved, and the user experience can be further improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0002] With the continuous development of information technology, image processing technologies such as image search and image generation have been widely used in various fields. Traditional image search methods usually rely on similarity search based on image content, that is, image search, and traditional image generation methods usually rely on generative models based on image features, that is, image generation. With the continuous advancement of computer vision technology, especially the widespread application of deep learning methods, automatic recognition of image content and feature extraction have become the key to achieving efficient image processing. Among the existing image processing methods, traditional methods mostly rely on single features of images, such as color, texture, shape and other information. These methods often need to describe the image content by extracting the feature vector of the image, and perform image retrieval by calculating the distance between the feature vectors, or generate images by feature vectors. Although these methods can effectively process images, they still have some limitations. For example, the processing ability of complex scenes is insufficient and the diversified features are ignored, which leads to reduced image processing accuracy. Therefore, how to provide an image processing method that can improve the accuracy of image processing has become an urgent problem to be solved. Summary of the invention

[0003] The embodiment of the present invention provides an image processing method, aiming to solve the problem of low image processing accuracy of existing image processing methods. By fusing multiple first images to be processed through fusion parameters, fusion data that is more in line with the user's intention can be obtained, and image processing is performed based on the fusion data to obtain a more accurate target image. Due to the use of fusion data, the relationship between different first images to be processed can be more flexibly identified, thereby expanding the retrieval dimension to achieve the purpose of improving the accuracy of image processing, and can further improve the user experience. The image processing method provided by the embodiment of the present invention can not only meet the needs in complex scenes, but also provide new perspectives and solutions for image recognition in related fields.

[0004] In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising the following steps:

[0005] Acquire multiple first images to be processed and fusion parameters set by a user;

[0006] Performing fusion processing on the plurality of first to-be-processed images based on the fusion parameters to obtain fusion data;

[0007] Image processing is performed based on the fused data to obtain a target image corresponding to the first image to be processed.

[0008] Optionally, before acquiring a plurality of first images to be processed, the method further includes:

[0009] Acquire multiple second images to be processed and scaling instruction data selected by a user;

[0010] determining the image content of each of the second images to be processed;

[0011] Based on the scaling instruction data or the image content, image scaling processing is performed on the plurality of second images to be processed to obtain the plurality of first images to be processed.

[0012] Optionally, the acquiring the fusion parameters set by the user includes:

[0013] Providing a preset image fusion method to a user, so that the user can select a target image fusion method from the preset image fusion methods;

[0014] Based on the target image fusion mode, the fusion parameter is determined.

[0015] Optionally, determining the fusion parameter based on the target image fusion mode includes:

[0016] prompting the user to set an image priority for each of the first images to be processed;

[0017] The fusion parameter is determined based on the target image fusion mode and the image priority.

[0018] Optionally, the preset image fusion mode includes at least hierarchical splicing and grid layout, and the determining the fusion parameter based on the target fusion mode and the image priority includes:

[0019] When the target fusion mode is the hierarchical stitching, prompting the user to adjust the transparent data for each of the first to-be-processed images, and determining the fusion parameters based on the transparent data and the image priority;

[0020] When the target fusion mode is the grid layout, the user is prompted to set the number of grid rows and columns, and the fusion parameters are determined based on the number of grid rows and columns and the image priority.

[0021] Optionally, the fusing the plurality of first images to be processed based on the fusion parameter to obtain fused data includes:

[0022] Based on the fusion parameters, performing image fusion processing on the plurality of first images to be processed to obtain image fusion data;

[0023] Performing feature extraction processing on the plurality of first images to be processed based on a preset feature extraction model to obtain image features corresponding to each of the first images to be processed;

[0024] Performing feature fusion processing on image features corresponding to each of the first images to be processed based on the fusion parameters to obtain feature fusion data;

[0025] The fusion data is determined based on the image fusion data and the feature fusion data.

[0026] Optionally, performing feature fusion processing on image features corresponding to each of the first to-be-processed images based on the fusion parameters to obtain the feature fusion data includes:

[0027] Obtaining fusion weight data of each of the image features from a user, and determining the image content of each of the first images to be processed;

[0028] Based on the fusion weight data and the fusion parameters, performing feature fusion processing on image features corresponding to each of the first to-be-processed images to obtain feature fusion data to be optimized;

[0029] The feature fusion data is determined based on the image content and the feature fusion data to be optimized.

[0030] In a second aspect, an embodiment of the present invention further provides an image processing device, the image processing device comprising:

[0031] An acquisition module, used for acquiring a plurality of first images to be processed and fusion parameters set by a user;

[0032] A fusion module, used for performing fusion processing on the plurality of first images to be processed based on the fusion parameters to obtain fusion data;

[0033] A processing module is used to perform image processing based on the fused data to obtain a target image corresponding to the first image to be processed.

[0034] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method provided in the embodiment of the present invention when executing the computer program.

[0035] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method provided in the embodiment of the invention are implemented.

[0036] In an embodiment of the present invention, multiple first images to be processed and fusion parameters set by the user are obtained; based on the fusion parameters, the multiple first images to be processed are fused to obtain fusion data; based on the fusion data, image processing is performed to obtain a target image corresponding to the first image to be processed. By fusing multiple first images to be processed using fusion parameters, fusion data that is more in line with the user's intention can be obtained, and image processing is performed based on the fusion data to obtain a more accurate target image. Due to the use of fusion data, the relationship between different first images to be processed can be identified more flexibly, thereby expanding the dimension of image processing to achieve the purpose of improving the accuracy of image processing, and can further improve the user experience. The image processing method provided in the embodiment of the present invention can not only meet the needs in complex scenes, but also provide new perspectives and solutions for image recognition in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flow chart of an image processing method provided by an embodiment of the present invention;

[0039] Figure 2 is a structural schematic diagram of an image processing device provided in an embodiment of the present invention;

[0040] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, Figure 1is a flowchart of an image processing method provided by an embodiment of the present invention, comprising:

[0043] 101. Acquire multiple first images to be processed and fusion parameters set by a user.

[0044] In an embodiment of the present invention, the above-mentioned image processing method can be applied to the user's intelligent terminal or image processing platform. The above-mentioned intelligent terminal can be a camera, a smart phone, a smart tablet computer and other terminals. The above-mentioned image processing platform can be constructed by a server or a server cluster. The above-mentioned server or server cluster can be any electronic device with functions such as image processing, image analysis, data storage, and data transmission. An image search engine can be deployed on the above-mentioned image processing platform, and the image search function can be realized through the above-mentioned image search engine. Alternatively, when the computing resources of the above-mentioned intelligent terminal can meet the computing resources required for deploying the image search engine, the above-mentioned image search engine can also be deployed on the above-mentioned intelligent terminal to realize the image search function. The above-mentioned image processing platform can also be deployed with an image generation model. The above-mentioned image generation model can be any image generation model based on deep learning. The image generation function can be realized through the above-mentioned image generation model, or when the computing resources of the above-mentioned intelligent terminal can meet the deployment of the above-mentioned image generation model, the above-mentioned image generation model can also be deployed on the above-mentioned intelligent terminal to realize the image generation function.

[0045] The first image to be processed may be selected by the user and uploaded to the image processing platform. Specifically, any image may be captured by the smart terminal or acquired through the network through the data transmission function. The user may click on any image to determine the complete image corresponding to the click as the first image to be processed. Alternatively, the user may click on the coordinates of any image to determine the corresponding object, generate an identification frame corresponding to the object, extract the identification frame from any image, and use each extracted identification frame as a first image to be processed.

[0046] For example, when any of the above images is a close-up image of an object or a person, the image only includes the object or the person and the corresponding background. At this time, any of the above images can be directly determined as the first image to be processed.

[0047] Alternatively, when any of the above images includes a complex image with multiple elements (or can be understood as a panoramic image), the corresponding identification frame can be determined by the user clicking on the coordinates in the above complex image, wherein each click can correspondingly capture an identification frame (which can be simply understood as a point-to-image frame) and serve as the above-mentioned first image to be processed.

[0048] In summary, the first image to be processed may be obtained by the user selecting multiple images or multiple objects (or elements) in one image at the same time.

[0049] The above-mentioned fusion parameters can be understood as the way in which the user sets the fusion of multiple first images to be processed in a customized manner, as well as the specific implementation parameters corresponding to the fusion method. That is, the above-mentioned fusion parameters may include the target image fusion method and the specific implementation parameters corresponding to the target image fusion method. For example, when the above-mentioned target fusion method is hierarchical splicing, the above-mentioned specific implementation parameters may include the image transparency and image priority of each first image to be processed during hierarchical splicing. When the above-mentioned target fusion method is splicing, the above-mentioned specific implementation parameters may include image priority, and the above-mentioned image priority can be used to determine the fusion order of each first image to be processed during the image fusion process.

[0050] 102. Perform fusion processing on a plurality of first to-be-processed images based on fusion parameters to obtain fusion data.

[0051] In an embodiment of the present invention, the fusion processing may include image fusion processing and feature fusion processing. The image fusion processing may be to obtain image fusion data by splicing, superimposing or other methods of multiple first images to be processed. The feature fusion processing may be to extract corresponding image features from each of the first images to be processed, and to fuse the image features by weighted averaging, feature splicing, fusion based on deep learning, etc. to obtain feature fusion data. At least one of the image fusion data and the feature fusion data is used as the fusion data.

[0052] When the above-mentioned fusion processing only includes the above-mentioned image fusion processing, the above-mentioned fusion data may only include image fusion data; when the above-mentioned fusion processing only includes the above-mentioned feature fusion processing, the above-mentioned fusion data may only include feature fusion data; or the above-mentioned image fusion processing and the above-mentioned feature fusion processing may be performed simultaneously to obtain corresponding image fusion data and feature fusion data, and the image fusion data and the feature fusion data are combined as the above-mentioned fusion data.

[0053] 103. Perform image processing based on the fused data to obtain a target image corresponding to the first image to be processed.

[0054] In an embodiment of the present invention, the above-mentioned image processing can be implemented by an image search engine or an image generation model deployed on the above-mentioned image processing platform or the above-mentioned intelligent terminal. The above-mentioned image search engine can be constructed by any image processing algorithm, for example, it can be an algorithm such as VGGNet, ResNet or Inception that uses a feature extraction method to implement image processing, or it can be an algorithm such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features) or BRIEF (Binary Robust Independent Elementary Features) that uses a feature descriptor method to implement image processing, or it can be an algorithm such as kd tree (k-dimensional tree), LSH (Locality-Sensitive Hashing) that uses an approximate nearest neighbor search (ANN) method to implement image processing.

[0055] It should be noted that since the fused data is obtained by fusion processing based on multiple first images to be processed selected by the user, when the fused data is used as a search target or a generation target for image processing, it can well expand the dimension of image processing and further improve the accuracy of image processing.

[0056] For example, when the above-mentioned image processing is specifically an image search, the above-mentioned fusion data can be input into the above-mentioned image search engine to perform feature extraction to obtain feature data, and then find related images or target objects to be associated by comparing the similarity of the feature data, and then retrieve the target content (i.e., the target image). During the image search process, users can be supported to set and adjust the search conditions and the screening conditions for the results, so as to more accurately screen out the best matching image (i.e., the target image).

[0057] After the target image is determined, the images with different similarities determined during the search process and the target image can be graphically displayed to the user, providing a more intuitive and accurate interactive experience.

[0058] More specifically, if the fusion data only includes the feature fusion data or the image fusion data, only the feature fusion data or the image fusion data can be provided to the image search engine, and the target image can be determined by the image search engine. If the fusion data includes both the feature fusion data and the image fusion data, the feature fusion data and the image fusion data can be provided to the image search engine at the same time, and the image search engine can determine the first target image according to the image fusion data, and determine the second target image according to the feature fusion data. The first target image and the second target image can be provided to the user at the same time, or the first target image and the second target image can be used for mutual verification. If the image similarity between the first target image and the second target image is greater than a preset similarity threshold, the first target image and the second target image can be provided to the user at the same time. On the contrary, if the image similarity is less than the preset similarity threshold, it means that the image information provided by the first target image and the second target image is greatly different, and the user can be prompted about the difference in the image information.

[0059] Alternatively, when the above-mentioned image processing is specifically the above-mentioned image generation, the above-mentioned fusion data can be provided to the above-mentioned image generation model, so that the above-mentioned image generation model generates a target image related to the above-mentioned fusion data.

[0060] In an embodiment of the present invention, multiple first images to be processed and fusion parameters set by the user are obtained; based on the fusion parameters, the multiple first images to be processed are fused to obtain fusion data; based on the fusion data, image processing is performed to obtain a target image corresponding to the first image to be processed. By fusing multiple first images to be processed using fusion parameters, fusion data that is more in line with the user's intention can be obtained, and image processing is performed based on the fusion data to obtain a more accurate target image. Due to the use of fusion data, the relationship between different first images to be processed can be identified more flexibly, thereby expanding the dimension of image processing to achieve the purpose of improving the accuracy of image processing, and can further improve the user experience. The image processing method provided in the embodiment of the present invention can not only meet the needs in complex scenes, but also provide new perspectives and solutions for image recognition in related fields.

[0061] It can be understood that in the specific implementation of the present application, related data such as the second image to be processed, the first image to be processed, the target image, fusion parameters, fusion weight data, etc., when the embodiments in the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data and the construction, training and use of related models such as feature extraction models need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0062] It should be noted that the image processing method provided in the embodiment of the present invention can be applied to devices such as computers and servers that can perform image processing.

[0063] Optionally, before the step of obtaining multiple first images to be processed, multiple second images to be processed and scaling instruction data selected by the user can also be obtained; the image content of each second image to be processed is determined; and based on the scaling instruction data or the image content, image scaling processing is performed on the multiple second images to be processed to obtain multiple first images to be processed.

[0064] In the embodiment of the present invention, when the user selects the second image to be processed, it can be a complete image or an image in a recognition frame obtained by intercepting the above-mentioned point image into a frame function as the above-mentioned second image to be processed. The size of the recognition frame usually depends on the image size of the object to be identified, so the second image to be processed can be scaled so that different second images to be processed have the same or similar resolutions, so as to better perform image fusion processing.

[0065] The above-mentioned scaling instruction data can be obtained by the user selecting a uniform image size, and the above-mentioned second image to be processed can be scaled according to the above-mentioned scaling instruction data to obtain the first image to be processed that meets the user's needs.

[0066] The above-mentioned image content can be realized by any content recognition algorithm, such as an image classification algorithm, a semantic segmentation algorithm, an instance segmentation algorithm, a key point detection algorithm, etc. The above-mentioned image content may include key objects, key figures, or regions of interest in the image, etc. Specifically, the scaling ratio can be calculated to ensure that the size of the key objects, key figures, or regions of interest in each second image to be processed remains consistent after image scaling. Finally, the above-mentioned scaling ratio can be used to perform image scaling on multiple second images to be processed to obtain multiple first images to be processed.

[0067] Optionally, in the step of obtaining the fusion parameters set by the user, a preset image fusion method may be provided to the user, so that the user can select a target image fusion method from the preset image fusion methods; and determine the fusion parameters based on the target image fusion method.

[0068] In an embodiment of the present invention, the preset image fusion mode may specifically include horizontal stitching, vertical stitching, grid layout and hierarchical stitching. The fusion parameters may include a target fusion mode, and the target fusion mode may be selected by the user from the horizontal stitching, vertical stitching, grid layout and hierarchical stitching.

[0069] Specifically, when the target fusion mode is horizontal splicing, the first images to be processed can be spliced ​​in the horizontal direction in sequence to obtain the image fusion data. When the target fusion mode is vertical splicing, the first images to be processed can be spliced ​​in the vertical direction in sequence to obtain the image fusion data. When the target fusion mode is grid layout, the first images to be processed can be arranged in rows and columns to obtain the image fusion data. When the target fusion mode is hierarchical splicing, the first images to be processed can be superimposed and spliced ​​to obtain the image fusion data.

[0070] In a possible embodiment, a background canvas of a specific size (e.g., a white or transparent background) may be created. When performing the above-mentioned image fusion processing, the above-mentioned horizontal stitching, vertical stitching, grid layout, and hierarchical stitching and other processing may be performed in the above-mentioned background canvas. The above-mentioned background canvas may be set according to the number and resolution of the above-mentioned first images to be processed. When the above-mentioned number is larger and the resolution is higher, a larger background canvas may be set accordingly. Conversely, when the above-mentioned number is smaller and the resolution is lower, a smaller background canvas may be set accordingly.

[0071] Optionally, in the step of determining fusion parameters based on the target image fusion mode, the user may be prompted to set an image priority for each first image to be processed; and the fusion parameters are determined based on the target image fusion mode and the image priority.

[0072] In an embodiment of the present invention, the user can set an image priority for each first image to be processed (i.e., images with high priority are images that the user attaches more importance to and can be placed in a higher priority position during the fusion process), and combine the above-mentioned image priority and the above-mentioned target fusion method into the above-mentioned fusion parameters.

[0073] It should be noted that when the fusion parameters include image priorities, when image fusion processing is performed according to the fusion parameters, the image priorities can provide a basis for the fusion order of the first image to be processed, so as to achieve a more personalized layout.

[0074] For example, when the target fusion mode is horizontal stitching, the first images to be processed may be sequentially extracted in the order of the image priorities from high to low for stitching in the horizontal direction (the stitching direction may be from left to right or from right to left). When the target fusion mode is vertical stitching, the first images to be processed may be sequentially extracted in the order of the image priorities from high to low for stitching in the vertical direction (the stitching direction may be from top to bottom or from bottom to top). When the target fusion mode is grid layout, the first images to be processed may be sequentially extracted in the order of the image priorities from high to low to fill each grid in turn. When the target fusion mode is hierarchical stitching, the first images to be processed may be sequentially extracted in the order of the image priorities from low to high or from high to low for superposition.

[0075] Optionally, in the step of determining fusion parameters based on the target fusion mode and image priority, when the target fusion mode is hierarchical stitching, the user can be prompted to adjust the transparent data for each first image to be processed, and determine the fusion parameters based on the transparent data and the image priority; when the target fusion mode is grid layout, the user can be prompted to set the number of grid rows and columns, and determine the fusion parameters based on the number of grid rows and columns and the image priority.

[0076] In an embodiment of the present invention, the preset image fusion method at least includes hierarchical stitching and grid layout, the transparent data may be transparency, and the number of grid rows and columns may be the number of each row and the number of each column.

[0077] Specifically, when the target fusion method is hierarchical stitching, if it is directly superimposed, only the first image to be processed superimposed on the top layer can be displayed, and the first image to be processed on the lower layer will be blocked. Therefore, the user can be prompted to adjust the transparent data for each first image to be processed, and the transparent data and image priority can be combined into the above fusion parameters. For example, the transparency can be adjusted through the Alpha channel to achieve the purpose of partially displaying the first image to be processed superimposed on the bottom layer after superposition. It should be noted that when prompting the user to adjust the transparent data for each first image to be processed, if the transparent data adjusted by the user is not received, the transparent data is empty. When the image fusion processing is subsequently performed based on the above fusion parameters, if the transparent data is empty, the background in each first image to be processed can be determined by algorithms such as key point detection algorithms and edge detection algorithms, and the above background can be made transparent before hierarchical superposition.

[0078] Alternatively, when the target fusion mode is a grid layout, the user is prompted to set the number of grid rows and columns, and the number of grid rows and columns and the image priority are combined as fusion parameters. When image fusion processing is subsequently performed based on the above fusion parameters, if the number of grid rows and columns is empty (i.e., the user has not set it according to the prompt), the above image fusion processing can be performed according to the random number of rows and columns.

[0079] It can be understood that by obtaining the fusion parameters selected by the user for image fusion processing, the user can customize the specific position of each picture and construct a stitching effect with a specific layout. The user can also arrange the first images to be processed in the required order to show priority or importance, and can also achieve the superposition of the first images to be processed through hierarchical settings to support transparency and complex visual effects.

[0080] Optionally, in the step of fusing multiple first images to be processed based on fusion parameters to obtain fusion data, image fusion processing can also be performed on the multiple first images to be processed based on the fusion parameters to obtain image fusion data; feature extraction processing can be performed on the multiple first images to be processed based on a preset feature extraction model to obtain image features corresponding to each first image to be processed; feature fusion processing can be performed on the image features corresponding to each first image to be processed based on the fusion parameters to obtain feature fusion data; and fusion data can be determined based on the image fusion data and the feature fusion data.

[0081] In an embodiment of the present invention, the image fusion processing may be to fuse multiple first images to be processed by splicing, superposition, hierarchical splicing or grid layout according to the fusion parameters to obtain image fusion data. The preset feature extraction model may be a deep learning model, such as a convolutional neural network, etc. Specifically, each first image to be processed may be input into the preset feature extraction model for feature extraction processing to obtain the image features. The image features may be feature vectors of multiple layers, and the specific number of layers may depend on the number of convolutional layers of the feature extraction model.

[0082] The above fusion parameters may include image priority, and the above feature fusion processing may be implemented by weighted averaging, feature splicing or deep fusion. For example, the above feature vectors (i.e., image features of all first images to be processed) may be weighted according to the above image priority (wherein the feature vector corresponding to the first image to be processed with a higher priority has a higher weight when weighted, and vice versa, the feature vector corresponding to the first image to be processed with a lower priority has a lower weight when weighted) to reflect the contribution of each first image to be processed in the overall feature (i.e., weighted average). After the weighting is completed, the above feature fusion data may be obtained. Alternatively, all feature vectors (i.e., image features of all first images to be processed) may be directly arranged and spliced ​​according to the above image priority to form a long vector (wherein, the feature vector corresponding to the first image to be processed with the highest image priority may also be arranged to the leftmost side of the long vector, and the feature vector corresponding to the first image to be processed with the lowest image priority may also be arranged to the rightmost side of the long vector, and the same is true for other positions, or the feature vector with the highest priority may also be the rightmost side, and so on), and the above long vector may be used as the above feature fusion data. Alternatively, a neural network can be used to perform deep learning fusion on the above image features to generate more expressive feature fusion data.

[0083] In a possible embodiment, after extracting the above-mentioned image features, the extracted image features can also be stored in a database for subsequent image processing. During the storage process, a corresponding relationship can be established between each image feature and the corresponding first image to be processed to facilitate accurate matching.

[0084] Furthermore, the stored image features may be indexed to improve the search efficiency of subsequent image processing. Specifically, local sensitive hashing, inverted indexing or other efficient data results may be used to speed up the search efficiency of subsequent image processing.

[0085] Optionally, in the step of performing feature fusion processing on image features corresponding to each first image to be processed based on fusion parameters to obtain feature fusion data, the user's fusion weight data for each image feature can also be obtained, and the image content of each first image to be processed can be determined; based on the fusion weight data and the fusion parameters, feature fusion processing is performed on the image features corresponding to each first image to be processed to obtain feature fusion data to be optimized; based on the image content and the feature fusion data to be optimized, the feature fusion data is determined.

[0086] In the embodiment of the present invention, during the feature fusion process, the user can also be allowed to customize the selection or adjustment of feature weights, so that the feature fusion is more personalized and can adapt to different application scenarios and user needs. Therefore, after the image features are extracted, when performing feature fusion, the fusion weight data set by the user for each image feature can be obtained, and the image content of each first image to be processed can be determined at the same time. The above image content can also be realized through algorithms such as key point detection, image classification, and convolutional neural network.

[0087] It is understandable that if the above fusion weight data is obtained, the above fusion weight data is not empty. At this time, the above feature fusion processing can be performed according to the fusion weight data and the above fusion parameters to obtain the above feature fusion data to be optimized. Alternatively, if the above fusion weight data cannot be obtained, it means that the user has not made corresponding settings. At this time, an adaptive weighting mechanism can be used to dynamically adjust the weight of the image features corresponding to each first image to be processed in the feature fusion processing to obtain the above feature fusion data to be optimized, or according to the image priority corresponding to the above fusion parameters, the weight of the image features corresponding to each first image to be processed in the feature fusion processing can be adjusted to obtain the above feature fusion data to be optimized.

[0088] After obtaining the feature fusion data to be optimized, the feature fusion data to be optimized can be optimized according to the image content to obtain optimized feature fusion data. For example, the fusion weight data can be adjusted in real time according to changes in image content and different context information.

[0089] After obtaining the optimized feature fusion data, the optimized feature fusion data can be directly used as the feature fusion data. Alternatively, the optimized feature fusion data can be further processed.

[0090] For example, since the above-mentioned image features are multi-layer feature vectors, after fusion, the optimized feature fusion data can also have multiple layers, which may include high-level features and low-level features. Due to the characteristics of the low-level convolutional layer and the high-level convolutional layer in the convolutional neural network, the low-level features can provide detail information, while the high-level features can provide global context and semantic information. Therefore, the high-level features and low-level features in the optimized feature fusion data can be combined to obtain the combined feature fusion data. At this time, the combined feature fusion data can be used as the above-mentioned feature fusion data. It can be understood that when performing image processing through the combined feature fusion data, their respective advantages can be utilized to form a more comprehensive feature representation, thereby improving the generalization ability and being able to better handle various complex scenes and diverse input data.

[0091] Alternatively, the combined feature fusion data may be further processed.

[0092] For example, the combined feature fusion data is subjected to dimensionality reduction or regularization processing to obtain target feature fusion data, and the target feature fusion data is used as the feature fusion data to adapt to subsequent image processing.

[0093] It is understandable that the embodiments of the present invention can achieve more efficient and accurate processing results (i.e., target images) through image processing of feature fusion data or image fusion data. Moreover, by obtaining the user's fusion parameters, custom stitching can be achieved to improve the flexibility of image layout and the accuracy and richness of image processing. The use of feature fusion data will allow the system to more flexibly identify the relationship between different objects, thereby expanding the image processing dimension, improving user experience, and meeting the needs of complex scenarios. It not only enriches the image processing scenarios and improves the accuracy of feature extraction, but also continuously improves the intelligence level of related systems through user interaction and feedback mechanisms, providing strong support for various application scenarios.

[0094] like Figure 2 As shown, an embodiment of the present invention further provides an image processing device, comprising:

[0095] An acquisition module 201 is used to acquire a plurality of first images to be processed and fusion parameters set by a user;

[0096] A fusion module 202, configured to perform fusion processing on the plurality of first images to be processed based on the fusion parameters to obtain fusion data;

[0097] The processing module 203 is used to perform image processing based on the fused data to obtain a target image corresponding to the first image to be processed.

[0098] Optionally, the image processing device further includes:

[0099] A second acquisition module, used to acquire a plurality of second images to be processed and scaling instruction data selected by a user;

[0100] A first determining module, used to determine the image content of each of the second images to be processed;

[0101] The first scaling module is used to perform image scaling processing on the plurality of second images to be processed based on the scaling instruction data or the image content to obtain the plurality of first images to be processed.

[0102] Optionally, the acquisition module 201 includes:

[0103] A first providing submodule is used to provide a preset image fusion method to a user, so that the user can select a target image fusion method from the preset image fusion methods;

[0104] The first determination submodule is used to determine the fusion parameters based on the target image fusion mode.

[0105] Optionally, the first determining submodule includes:

[0106] a first prompting unit, configured to prompt the user to set an image priority for each of the first images to be processed;

[0107] The first determining unit is used to determine the fusion parameter based on the target image fusion mode and the image priority.

[0108] Optionally, the preset image fusion mode includes at least hierarchical splicing and grid layout, and the first determining unit includes:

[0109] A first determining subunit is configured to, when the target fusion mode is the hierarchical splicing, prompt the user to adjust the transparent data for each of the first to-be-processed images, and determine the fusion parameter based on the transparent data and the image priority;

[0110] The second determining subunit is used to prompt the user to set the number of grid rows and columns when the target fusion mode is the grid layout, and determine the fusion parameter based on the number of grid rows and columns and the image priority.

[0111] Optionally, the fusion module 202 includes:

[0112] An image fusion submodule, configured to perform image fusion processing on the plurality of first images to be processed based on the fusion parameters to obtain image fusion data;

[0113] a feature extraction submodule, configured to perform feature extraction processing on the plurality of first images to be processed based on a preset feature extraction model, so as to obtain image features corresponding to each of the first images to be processed;

[0114] A feature fusion submodule, used for performing feature fusion processing on the image features corresponding to each of the first to-be-processed images based on the fusion parameters to obtain feature fusion data;

[0115] The second determining submodule is used to determine the fusion data based on the image fusion data and the feature fusion data.

[0116] Optionally, the feature fusion submodule includes:

[0117] A first acquisition unit, used to acquire fusion weight data of each of the image features by a user, and determine the image content of each of the first images to be processed;

[0118] a feature fusion unit, configured to perform feature fusion processing on the image features corresponding to each of the first images to be processed based on the fusion weight data and the fusion parameters, so as to obtain feature fusion data to be optimized;

[0119] The second determining unit is used to determine the feature fusion data based on the image content and the feature fusion data to be optimized.

[0120] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, characterized in that it includes a processor, and the processor can execute any one of the above-mentioned image processing methods.

[0121] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing the image processing method which is stored in the memory 302 and can be run on the processor 301, wherein:

[0122] The processor 301 runs the computer program of the image processing method stored in the memory 302 to perform the following steps:

[0123] Acquire multiple first images to be processed and fusion parameters set by a user;

[0124] Performing fusion processing on the plurality of first to-be-processed images based on the fusion parameters to obtain fusion data;

[0125] Image processing is performed based on the fused data to obtain a target image corresponding to the first image to be processed.

[0126] Optionally, before acquiring the plurality of first images to be processed, the method executed by the processor 301 further includes:

[0127] Acquire multiple second images to be processed and scaling instruction data selected by a user;

[0128] determining the image content of each of the second images to be processed;

[0129] Based on the scaling instruction data or the image content, image scaling processing is performed on the plurality of second images to be processed to obtain the plurality of first images to be processed.

[0130] Optionally, the acquiring of the fusion parameters set by the user performed by the processor 301 includes:

[0131] Providing a preset image fusion method to a user, so that the user can select a target image fusion method from the preset image fusion methods;

[0132] Based on the target image fusion mode, the fusion parameter is determined.

[0133] Optionally, the determining of the fusion parameter based on the target image fusion mode performed by the processor 301 includes:

[0134] prompting the user to set an image priority for each of the first images to be processed;

[0135] The fusion parameter is determined based on the target image fusion mode and the image priority.

[0136] Optionally, the preset image fusion mode includes at least hierarchical splicing and grid layout, and the processor 301 determines the fusion parameter based on the target fusion mode and the image priority, including:

[0137] When the target fusion mode is the hierarchical stitching, prompting the user to adjust the transparent data for each of the first to-be-processed images, and determining the fusion parameters based on the transparent data and the image priority;

[0138] When the target fusion mode is the grid layout, the user is prompted to set the number of grid rows and columns, and the fusion parameters are determined based on the number of grid rows and columns and the image priority.

[0139] Optionally, the processor 301 performs fusion processing on the plurality of first to-be-processed images based on the fusion parameter to obtain fusion data, including:

[0140] Based on the fusion parameters, performing image fusion processing on the plurality of first images to be processed to obtain image fusion data;

[0141] Performing feature extraction processing on the plurality of first images to be processed based on a preset feature extraction model to obtain image features corresponding to each of the first images to be processed;

[0142] Performing feature fusion processing on image features corresponding to each of the first images to be processed based on the fusion parameters to obtain feature fusion data;

[0143] The fusion data is determined based on the image fusion data and the feature fusion data.

[0144] Optionally, the processor 301 performs feature fusion processing on the image features corresponding to each of the first to-be-processed images based on the fusion parameters to obtain the feature fusion data, including:

[0145] Obtaining fusion weight data of each of the image features from a user, and determining the image content of each of the first images to be processed;

[0146] Based on the fusion weight data and the fusion parameters, performing feature fusion processing on image features corresponding to each of the first to-be-processed images to obtain feature fusion data to be optimized;

[0147] The feature fusion data is determined based on the image content and the feature fusion data to be optimized.

[0148] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the image processing method or the application-side image processing method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the above-mentioned computer program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the above-mentioned computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0150] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises the following steps: Acquire multiple first images to be processed and fusion parameters set by a user; Performing fusion processing on the plurality of first to-be-processed images based on the fusion parameters to obtain fusion data; Image processing is performed based on the fused data to obtain a target image corresponding to the first image to be processed.

2. The image processing method according to claim 1, characterized in that: Before acquiring the plurality of first images to be processed, the method further includes: Acquire multiple second images to be processed and scaling instruction data selected by a user; determining the image content of each of the second images to be processed; Based on the scaling instruction data or the image content, image scaling processing is performed on the plurality of second images to be processed to obtain the plurality of first images to be processed.

3. The image processing method according to claim 1, characterized in that: The obtaining of the fusion parameters set by the user includes: Providing a preset image fusion method to a user, so that the user can select a target image fusion method from the preset image fusion methods; Based on the target image fusion mode, the fusion parameter is determined.

4. The image processing method according to claim 3, characterized in that: The determining the fusion parameter based on the target image fusion mode includes: prompting the user to set an image priority for each of the first images to be processed; The fusion parameter is determined based on the target image fusion mode and the image priority.

5. The image processing method according to claim 4, characterized in that: The preset image fusion mode includes at least hierarchical splicing and grid layout, and the determining of the fusion parameters based on the target fusion mode and the image priority includes: When the target fusion mode is the hierarchical stitching, prompting the user to adjust the transparent data for each of the first to-be-processed images, and determining the fusion parameters based on the transparent data and the image priority; When the target fusion mode is the grid layout, the user is prompted to set the number of grid rows and columns, and the fusion parameters are determined based on the number of grid rows and columns and the image priority.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The fusing the plurality of first images to be processed based on the fusion parameters to obtain fusion data includes: Based on the fusion parameters, performing image fusion processing on the plurality of first images to be processed to obtain image fusion data; Performing feature extraction processing on the plurality of first images to be processed based on a preset feature extraction model to obtain image features corresponding to each of the first images to be processed; Performing feature fusion processing on image features corresponding to each of the first images to be processed based on the fusion parameters to obtain feature fusion data; The fusion data is determined based on the image fusion data and the feature fusion data.

7. The image processing method according to claim 6, characterized in that: The performing feature fusion processing on the image features corresponding to each of the first to-be-processed images based on the fusion parameters to obtain the feature fusion data includes: Obtaining fusion weight data of each of the image features from a user, and determining the image content of each of the first images to be processed; Based on the fusion weight data and the fusion parameters, performing feature fusion processing on image features corresponding to each of the first to-be-processed images to obtain feature fusion data to be optimized; The feature fusion data is determined based on the image content and the feature fusion data to be optimized.

8. An image processing device, characterized in that: The image processing device comprises: An acquisition module, used for acquiring a plurality of first images to be processed and fusion parameters set by a user; A fusion module, used for performing fusion processing on the plurality of first images to be processed based on the fusion parameters to obtain fusion data; A processing module is used to perform image processing based on the fused data to obtain a target image corresponding to the first image to be processed.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the image processing method according to any one of claims 1 to 7 are implemented.

Images