Image processing method and device
Through the combination of image feature processing model and overlapping comparison model, the problem of inaccurate image differences in the prior art is solved, and image feature extraction and comparison with higher accuracy is achieved, which is suitable for image comparison of fitness effects.
Patent Information
- Application Number
- CN202310542593.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
The method of determining the difference between multiple images in the prior art is inaccurate, especially in the comparison of fitness effects. The existing edge detection algorithm loses the advanced features of the image and cannot accurately extract the contour features, resulting in inaccurate comparison results.
The image feature processing model is used for feature extraction and fusion processing, combined with the image overlap comparison model to obtain overlapping information, and through multiple feature extraction and fusion, an accurate target feature map is obtained, and the overlapping comparison module is used for comparison, to improve image differences accuracy.
It improves the accuracy of the differences between multiple images, can more accurately extract and compare the outline changes of images, and is suitable for application scenarios such as image comparison of fitness effects.
Smart Images

Figure CN118941808A_ABST
Abstract
Description
Technical Field
[0001] The present application mainly relates to the field of artificial intelligence technology, and specifically to an image processing method and device. Background Art
[0002] In the current era, there is a craze for home fitness. For fitness effects, people are more concerned about visual changes in body shape than subtle changes in stereotyped digital dimensions. At present, we can only put the photos before and after fitness together for observation and comparison. Due to differences in picture size, target position coordinates and background environment, it is difficult to directly compare two images. In addition, the focus of fitness effect comparison is on the changes in the contours of the human body before and after, and only a small number of pixels in the image are edge points. If the existing edge detection algorithm is used, many high-level features in the image will be lost, and contour features cannot be accurately extracted, and contour changes cannot be accurately compared.
[0003] That is, the prior art method for determining the difference between multiple images is inaccurate. Summary of the invention
[0004] The present application provides an image processing method and device, aiming to solve the problem of inaccurate methods for determining differences between multiple images in the prior art.
[0005] In a first aspect, the present application provides an image processing method, the image processing method comprising:
[0006] Acquire image information to be processed, wherein the image information to be processed includes at least 2 images to be processed;
[0007] Using an image feature processing model to perform feature extraction and fusion processing on the image information to be processed to obtain target image information;
[0008] The target image information is overlapped and compared using an image overlap comparison model to obtain overlap degree information.
[0009] Optionally, the image feature processing model includes a feature extraction model and a feature fusion module, and the image feature processing model is used to perform feature extraction and fusion processing on the image information to be processed to obtain target image information, including:
[0010] Using a feature extraction model to perform multiple feature extractions on the image to be processed to obtain image feature information;
[0011] The feature fusion module is used to fuse the image feature information to obtain a target feature map of the image to be processed.
[0012] Optionally, the image feature information includes a global feature map and an edge feature map, the feature extraction model includes a global feature extraction module and an edge feature extraction module, and the feature extraction model is used to perform multiple feature extractions on the image to be processed to obtain the image feature information, including:
[0013] Using the global feature extraction module to extract global features of the image to be processed, to obtain a global feature map of the image to be processed;
[0014] The edge feature extraction module is used to extract edge features from the global feature map to obtain an edge feature map of the image to be processed.
[0015] Optionally, the global feature extraction module includes at least two global convolution blocks connected in series, each global convolution block includes at least two global convolution layers, and the number of channels of the at least two global convolution blocks increases sequentially. The global feature extraction module is used to perform global feature extraction on the image to be processed to obtain a global feature map of the image to be processed, including:
[0016] The image to be processed is sequentially input into the at least two global convolution blocks to obtain the global feature map.
[0017] Optionally, the edge feature extraction module includes an edge feature fusion convolution block and at least two edge convolution blocks corresponding to the at least two global convolution blocks, each edge convolution block includes at least two edge convolution layers, and the number of channels of the at least two global convolution blocks increases sequentially, and the edge feature extraction module is used to extract edge features from the global feature map to obtain the edge feature map of the image to be processed, including:
[0018] Input the feature maps output by the at least two global convolution blocks into the corresponding edge convolution blocks respectively to obtain at least two edge feature sub-maps;
[0019] The at least two edge feature sub-graphs are input into the edge feature fusion convolution block for fusion to obtain the edge feature graph of the image to be processed.
[0020] Optionally, the adopting the feature fusion module to fuse the image feature information to obtain a target feature map of the image to be processed includes:
[0021] Multiplying the global feature map and the edge feature map to obtain a multiplied feature map;
[0022] Obtaining preset edge feature weight coefficients and global feature weight coefficients;
[0023] The global feature map and the multiplied feature map are weighted and summed based on the edge feature weight coefficient and the global feature weight coefficient to obtain a target feature map of the image to be processed.
[0024] Optionally, the target image information includes a first target feature map and a second target feature map, the image overlapping contrast model includes an overlapping contrast module, the overlapping contrast module includes at least two overlapping contrast convolution blocks and a fully connected layer connected in series, and the number of channels of the at least two overlapping contrast convolution blocks decreases sequentially;
[0025] The method of performing overlap comparison on the target image information by using the image overlap comparison model to obtain overlap degree information includes:
[0026] Perform feature fusion based on the first target feature image and the second target feature image to obtain a fused image;
[0027] The fused image is sequentially input into the at least two overlapping contrast convolution blocks and the fully connected layer to obtain the overlapping contrast parameter.
[0028] Optionally, the image overlapping contrast model includes a dimensionality reduction module, the dimensionality reduction module includes at least two dimensionality reduction convolution blocks connected in series, the dimensionality reduction convolution block includes a convolution layer, an activation function layer and a maximum pooling layer, and the number of channels in the at least two dimensionality reduction convolution blocks decreases sequentially;
[0029] The step of performing feature fusion based on the first target feature image and the second target feature image to obtain a fused image includes:
[0030] Inputting the first target feature image and the second target feature image into the dimension reduction module respectively to obtain a third target feature image after dimension reduction and a fourth target feature image after dimension reduction;
[0031] The third target feature image and the fourth target feature image are fused to obtain the fused image.
[0032] Optionally, the image processing method comprises:
[0033] Obtaining a preset neural network model, a plurality of sample image pairs, annotated overlap comparison parameters of the plurality of sample image pairs, and annotated classification labels of each pixel point of each sample image in the plurality of sample image pairs, wherein each of the sample image pairs includes a first sample image before fitness and a second sample image after fitness, and the annotated classification label indicates whether the pixel point is an edge point;
[0034] Inputting the plurality of sample image pairs into the image feature processing model to obtain predicted classification labels of each pixel point of each sample image in the plurality of sample image pairs and predicted overlap contrast parameters of the plurality of sample image pairs;
[0035] Input the predicted classification label and the annotated classification label of each pixel point of each sample image in the multiple sample image pairs into a preset binary classification cross entropy loss function to obtain a contour extraction loss;
[0036] Determining the similarity between the transformation of the annotated overlap contrast parameter and the predicted overlap contrast parameter as a transformation matrix loss;
[0037] determining a total loss based on the contour extraction loss and the transformation matrix loss;
[0038] The preset neural network model is iteratively trained based on the total loss to obtain the image overlapping contrast model.
[0039] In a second aspect, the present application provides an image processing device, the image processing device comprising:
[0040] An acquisition unit, used for acquiring image information to be processed, wherein the image information to be processed includes at least two images to be processed;
[0041] An image processing unit, used for performing feature extraction and fusion processing on the image information to be processed by using an image feature processing model to obtain target image information;
[0042] The overlapping comparison unit is used to perform overlapping comparison on the target image information by using an image overlapping comparison model to obtain overlapping degree information.
[0043] Optionally, the image feature processing model includes a feature extraction model and a feature fusion module, and the image feature processing model is used to perform feature extraction and fusion processing on the image information to be processed to obtain target image information, including:
[0044] Using a feature extraction model to perform multiple feature extractions on the image to be processed to obtain image feature information;
[0045] The feature fusion module is used to fuse the image feature information to obtain a target feature map of the image to be processed.
[0046] Optionally, the image feature information includes a global feature map and an edge feature map, the feature extraction model includes a global feature extraction module and an edge feature extraction module, and the feature extraction model is used to perform multiple feature extractions on the image to be processed to obtain the image feature information, including:
[0047] Using the global feature extraction module to extract global features of the image to be processed, to obtain a global feature map of the image to be processed;
[0048] The edge feature extraction module is used to extract edge features from the global feature map to obtain an edge feature map of the image to be processed.
[0049] Optionally, the global feature extraction module includes at least two global convolution blocks connected in series, each global convolution block includes at least two global convolution layers, and the number of channels of the at least two global convolution blocks increases sequentially. The global feature extraction module is used to perform global feature extraction on the image to be processed to obtain a global feature map of the image to be processed, including:
[0050] The image to be processed is sequentially input into the at least two global convolution blocks to obtain the global feature map.
[0051] Optionally, the edge feature extraction module includes an edge feature fusion convolution block and at least two edge convolution blocks corresponding to the at least two global convolution blocks, each edge convolution block includes at least two edge convolution layers, and the number of channels of the at least two global convolution blocks increases sequentially, and the edge feature extraction module is used to extract edge features from the global feature map to obtain the edge feature map of the image to be processed, including:
[0052] Input the feature maps output by the at least two global convolution blocks into the corresponding edge convolution blocks respectively to obtain at least two edge feature sub-maps;
[0053] The at least two edge feature sub-graphs are input into the edge feature fusion convolution block for fusion to obtain the edge feature graph of the image to be processed.
[0054] Optionally, the adopting the feature fusion module to fuse the image feature information to obtain a target feature map of the image to be processed includes:
[0055] Multiplying the global feature map and the edge feature map to obtain a multiplied feature map;
[0056] Obtaining preset edge feature weight coefficients and global feature weight coefficients;
[0057] The global feature map and the multiplied feature map are weighted and summed based on the edge feature weight coefficient and the global feature weight coefficient to obtain a target feature map of the image to be processed.
[0058] Optionally, the target image information includes a first target feature map and a second target feature map, the image overlapping contrast model includes an overlapping contrast module, the overlapping contrast module includes at least two overlapping contrast convolution blocks and a fully connected layer connected in series, and the number of channels of the at least two overlapping contrast convolution blocks decreases sequentially;
[0059] The method of performing overlap comparison on the target image information by using the image overlap comparison model to obtain overlap degree information includes:
[0060] Perform feature fusion based on the first target feature image and the second target feature image to obtain a fused image;
[0061] The fused image is sequentially input into the at least two overlapping contrast convolution blocks and the fully connected layer to obtain the overlapping contrast parameter.
[0062] Optionally, the image overlapping contrast model includes a dimensionality reduction module, the dimensionality reduction module includes at least two dimensionality reduction convolution blocks connected in series, the dimensionality reduction convolution block includes a convolution layer, an activation function layer and a maximum pooling layer, and the number of channels in the at least two dimensionality reduction convolution blocks decreases sequentially;
[0063] The step of performing feature fusion based on the first target feature image and the second target feature image to obtain a fused image includes:
[0064] Inputting the first target feature image and the second target feature image into the dimension reduction module respectively to obtain a third target feature image after dimension reduction and a fourth target feature image after dimension reduction;
[0065] The third target feature image and the fourth target feature image are fused to obtain the fused image.
[0066] Optionally, the image processing method comprises:
[0067] Obtaining a preset neural network model, a plurality of sample image pairs, annotated overlap comparison parameters of the plurality of sample image pairs, and annotated classification labels of each pixel point of each sample image in the plurality of sample image pairs, wherein each of the sample image pairs includes a first sample image before fitness and a second sample image after fitness, and the annotated classification label indicates whether the pixel point is an edge point;
[0068] Inputting the plurality of sample image pairs into the image feature processing model to obtain predicted classification labels of each pixel point of each sample image in the plurality of sample image pairs and predicted overlap contrast parameters of the plurality of sample image pairs;
[0069] Input the predicted classification label and the annotated classification label of each pixel point of each sample image in the multiple sample image pairs into a preset binary classification cross entropy loss function to obtain a contour extraction loss;
[0070] Determining the similarity between the transformation of the annotated overlap contrast parameter and the predicted overlap contrast parameter as a transformation matrix loss;
[0071] determining a total loss based on the contour extraction loss and the transformation matrix loss;
[0072] The preset neural network model is iteratively trained based on the total loss to obtain the image overlapping contrast model.
[0073] In a third aspect, the present application provides an electronic device, the electronic device comprising:
[0074] one or more processors;
[0075] Memory; and
[0076] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the image processing method according to any one of the first aspects.
[0077] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for loading by a processor to execute the steps in the image processing method described in any one of the first aspects.
[0078] The present application provides an image processing method and device, which includes: obtaining image information to be processed, the image information to be processed includes at least two images to be processed; using an image feature processing model to perform feature extraction and fusion processing on the image information to be processed to obtain target image information; using an image overlap comparison model to perform overlap comparison on the target image information to obtain overlap degree information. The present application first extracts and fuses features of multiple images to be processed, and the accuracy of the extracted target feature map is higher than that of the prior art, thereby improving the accuracy of determining the differences between multiple images. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0080] Figure 1 is a schematic diagram of a scene of an image processing system provided in an embodiment of the present application;
[0081] Figure 2 It is a schematic diagram of an embodiment of the image processing method provided in the embodiments of the present application;
[0082] Figure 3 It is a structural schematic diagram of an image feature processing model and an image overlap comparison model in one embodiment of the image processing method provided in the embodiments of the present application;
[0083] Figure 4 It is a structural schematic diagram of a global feature extraction module in one embodiment of the image processing method provided in the embodiments of the present application;
[0084] Figure 5 It is a structural schematic diagram of an edge feature extraction module in one embodiment of the image processing method provided in the embodiments of the present application;
[0085] Figure 6 It is a flow chart of an embodiment of the image processing method provided in the embodiments of the present application, in which a feature extraction model is used to perform multiple feature extractions on a to-be-processed image to obtain image feature information;
[0086] Figure 7 is a structural schematic diagram of a dimensionality reduction module and an overlapping comparison module in one embodiment of an image processing method provided in an embodiment of the present application;
[0087] Figure 8 is a flowchart of another embodiment of the image processing method provided in the embodiments of the present application;
[0088] Fig. 9 is a schematic diagram of the structure of an embodiment of an image processing device provided in an embodiment of the present application;
[0089] Fig.10 It is a schematic diagram of the structure of an embodiment of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION
[0090] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0091] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0092] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. In order to enable any technician in the field to implement and adopt the present application, the following description is given. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without adopting these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.
[0093] The embodiments of the present application provide an image processing method and device, which are described in detail below.
[0094] See also Figure 1 , Figure 1 This is a scene schematic diagram of an image processing system provided in an embodiment of the present application. The image processing system may include an electronic device 100, in which an image processing device is integrated.
[0095] In the embodiment of the present application, the electronic device 100 can be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the electronic device 100 can be a desktop computer, a portable computer, a network server, a PDA (Personal Digital Assistant), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, a smart TV, etc. This embodiment does not limit the type of the electronic device 100.
[0096] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer electronic devices as shown in, e.g. Figure 1 Only one electronic device is shown in the figure. It can be understood that the image processing system can also include one or more other electronic devices that can process data, which are not specifically limited here.
[0097] In addition, if Figure 1 As shown, the image processing system may further include a memory 200 for storing data.
[0098] It should be noted that Figure 1 The scene diagram of the image processing system shown is only an example. The image processing system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the image processing system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.
[0099] First, an image processing method is provided in an embodiment of the present application, and the image processing method includes: obtaining image information to be processed, wherein the image information to be processed includes at least two images to be processed; performing feature extraction and fusion processing on the image information to be processed using an image feature processing model to obtain target image information; and performing overlapping comparison on the target image information using an image overlapping comparison model to obtain overlapping degree information.
[0100] like Figure 2 As shown, Figure 2 : is a flow chart of an embodiment of the image processing method provided in the embodiment of the present application, and the image processing method comprises the following steps S201 to S203:
[0101] S201: Obtain image information to be processed.
[0102] In the embodiment of the present application, the image information to be processed includes at least 2 images to be processed.
[0103] The image to be processed may be any image uploaded by a user, for example, a fitness image uploaded by a user. The fitness image uploaded by the user is obtained, and the fitness image is grayed to obtain the image to be processed. Specifically, the size of the image to be processed is 128*128*1.
[0104] S202, using the image feature processing model to perform feature extraction and fusion processing on the image information to be processed to obtain target image information.
[0105] In the embodiment of the present application, an image feature processing model is obtained, wherein the image feature processing model can be CNN, RNN, etc., which is set according to the specific situation.
[0106] In a specific embodiment, the image feature processing model includes a feature extraction model and a feature fusion module. The image feature processing model is used to extract features and fuse the image information to be processed to obtain target image information, including: using the feature extraction model to extract features of the image to be processed multiple times to obtain image feature information; using the feature fusion module to fuse the image feature information to obtain a target feature map of the image to be processed.
[0107] Furthermore, the image feature information includes a global feature map and an edge feature map, and the feature extraction model includes a global feature extraction module and an edge feature extraction module.
[0108] like Figure 3 and Figure 4 As shown, in a specific embodiment, the global feature extraction module includes at least two global convolution blocks connected in series, each global convolution block includes at least two global convolution layers, and the number of channels of the at least two global convolution blocks increases successively. The number of global convolution blocks can be 2, 3 or more, and the number of global convolution layers in the global convolution block can be 2, 3 or more.
[0109] Specifically, the global feature extraction module includes three global convolutional layers, and the three global convolutional blocks are convolutional block conv1, convolutional block conv2, and convolutional block conv3. Among them, convolutional block conv1 contains two convolutional layers and ReLU layers. Each convolutional layer uses a 3*3 convolutional kernel with a step size of 1. It performs convolution operations on grayscale images and uses the ReLU activation function. The number of channels of convolutional block conv1 is 32, and it outputs a 32-channel feature map. Convolutional block conv2 contains three convolutional layers and ReLU layers. Each convolutional layer uses a 3*3 convolutional kernel with a step size of 1. It performs convolution operations on grayscale images and uses the ReLU activation function. The number of channels of convolutional block conv2 is 64, and it outputs a 64-channel feature map. The convolution block conv3 contains three convolution layers and a ReLU layer. Each convolution layer uses a 3*3 convolution kernel with a step size of 1. It performs convolution operations on grayscale images and uses a ReLU activation function. The number of channels of the convolution block conv3 is 128, and it outputs a 128-channel feature map. This application uses the powerful feature extraction capability of CNN to map the input single-channel image to a high-dimensional feature space. In addition, in order to retain the global feature information of the image and facilitate the combination with subsequent feature edge mapping, no pooling operation is performed in this module.
[0110] like Figure 3 and Figure 5As shown, in a specific embodiment, the edge feature extraction module includes an edge feature fusion convolution block and at least two edge convolution blocks corresponding to at least two global convolution blocks, each edge convolution block includes at least two edge convolution layers, and the number of channels of the at least two edge convolution layers increases sequentially. The number of edge convolution blocks can be 2, 3 or more, and the number of edge convolution layers in the edge convolution block can be 2, 3 or more.
[0111] Specifically, the number of edge convolution blocks is 3, and each edge convolution block is the same. Among them, the edge convolution block includes 3 edge convolution layers connected in series, and the 3 edge convolution layers connected in series are convolution layer conv4, convolution layer conv5 and convolution layer conv6. Among them, convolution layer conv4 uses a 1*1 convolution kernel, the number of channels is 32, and outputs a feature map of 32 channels; convolution layer conv5 uses a 3*3 convolution kernel, the number of channels is 32, and outputs a feature map of 32 channels; convolution layer conv6 uses a 1*1 convolution kernel, the number of channels is 128, and outputs a feature map of 128 channels.
[0112] like Figure 5 As shown, in a specific embodiment, the edge feature fusion convolution block conv7 adopts a 1*1 convolution kernel, the number of channels is 128, and outputs a feature map of 128 channels.
[0113] See also Figure 6 In the embodiment of the present application, a feature extraction model is used to perform multiple feature extractions on the image to be processed to obtain image feature information, including S2021-S2022:
[0114] S2021. Use a global feature extraction module to perform global feature extraction on the image to be processed to obtain a global feature map of the image to be processed.
[0115] In the embodiment of the present application, the image to be processed is input into the global feature extraction module to obtain the global feature map of the image to be processed. For example, the size of the image to be processed is 128*128*1, and the image to be processed of 128*128*1 is input into the global feature extraction module to obtain a global feature map of 128*128*128.
[0116] Specifically, the image to be processed is input into three global convolution blocks in the global feature extraction module, and the three global convolution blocks output three feature maps respectively, and the last feature map among the three feature maps is the global feature map.
[0117] S2022. Use an edge feature extraction module to extract edge features from the global feature map to obtain an edge feature map of the image to be processed.
[0118] In a specific embodiment, the edge feature extraction module includes only one edge convolution block, and the global feature map is input into the edge feature extraction module for edge feature extraction to obtain the edge feature map of the image to be processed. For example, the size of the image to be processed is 128*128*1, and the image to be processed of 128*128*1 is input into the global feature extraction module to obtain a global feature map of 128*128*128, and the global feature map of 128*128*128 is input into the edge feature extraction module to obtain an edge feature map of 128*128*128.
[0119] In another specific embodiment, the edge feature extraction module includes at least two edge convolution blocks. The edge feature extraction module is used to extract edge features from the global feature map to obtain an edge feature map of the image to be processed, including:
[0120] First, the feature maps output by at least two global convolution blocks are respectively input into corresponding edge convolution blocks to obtain at least two edge feature sub-maps.
[0121] Specifically, the edge feature extraction module includes three edge convolution blocks, and the three feature maps output by the three global convolution blocks in the global feature extraction module are respectively input into the three edge convolution blocks of the edge feature extraction module for edge feature extraction, thereby obtaining three edge feature sub-maps output by the three edge convolution blocks. For example, three edge feature sub-maps of 128*128*128 are output by the three edge convolution blocks.
[0122] Then, at least two edge feature sub-maps are input into the edge feature fusion convolution block for fusion to obtain an edge feature map of the image to be processed.
[0123] For example, at least two edge feature sub-maps are input into the edge feature fusion convolution block for fusion to obtain a 128*128*128 edge feature map.
[0124] In the embodiment of the present application, a feature fusion module is used to fuse image feature information to obtain a target feature map of the image to be processed, including:
[0125] (1) Multiply the global feature map and the edge feature map to obtain the multiplied feature map.
[0126] (2) Obtain the preset edge feature weight coefficient and global feature weight coefficient.
[0127] In a specific embodiment, the edge feature weight coefficient is α, and the global feature weight coefficient is 1-α.
[0128] (3) Based on the edge feature weight coefficient and the global feature weight coefficient, the global feature map and the multiplied feature map are weighted and summed to obtain the target feature map of the image to be processed.
[0129] Specifically, the calculation formula of the target feature graph G1 is as follows:
[0130] G1=αM1+(1-α)F1M1
[0131] Among them, F1 is the global feature map and M1 is the edge feature map.
[0132] First, the global feature F1 and the edge feature M1 are multiplied by matrix elements to obtain the convolution feature with weight attributes to achieve the function of feature filtering. α is a hyperparameter that controls the weights of edge features and global features in the image. The target feature map G1 achieves the effect of fusing global image features on the basis of extracting the edge feature map M1 of the human body.
[0133] In a specific embodiment, the first image is input into the image feature processing model to obtain the first target feature image G1. The second image is input into the image feature processing model to obtain the second target feature image G2.
[0134] S203: Use an image overlap comparison model to perform overlap comparison on target image information to obtain overlap degree information.
[0135] In the embodiment of the present application, the image overlap contrast model includes an overlap contrast module and an image feature processing model in any of the above embodiments. The image feature processing model and the overlap contrast module are connected in series. The structure of the overlap contrast module is as follows: Figure 7 The overlap information is the overlap comparison parameter.
[0136] In the embodiment of the present application, the image overlap contrast model is used to perform overlap contrast on the target image information to obtain overlap degree information, which includes:
[0137] (1) Obtain a preset neural network model, multiple sample image pairs, annotation overlap comparison parameters of the multiple sample image pairs, and annotated classification labels of each pixel point of each sample image in the multiple sample image pairs.
[0138] Each sample image pair includes a first sample image before fitness and a second sample image after fitness. The preset neural network model is an untrained image overlap comparison model, which can be CNN, RNN, etc. The labeled overlap comparison parameters of multiple sample image pairs and the labeled classification labels of each sample image in the multiple sample image pairs can be manually labeled.
[0139] Specifically, the label class indicates whether the pixel is an edge point. If the pixel is an edge point, the label class is recorded as 1; if the pixel is a non-edge point, the label class is recorded as 0.
[0140] Specifically, each sample image in a plurality of sample image pairs constitutes a set T, T = {(X i ,Yi ),i=1,2,…,N}, where represents each sample image in multiple sample image pairs, represents the corresponding actual binary label image, It is the annotation classification label of each pixel in the sample image.
[0141] (2) Inputting the multiple sample image pairs into the image feature processing model, obtaining the predicted classification label of each pixel point of each sample image in the multiple sample image pairs and the predicted overlapping contrast parameters of the multiple sample image pairs.
[0142] Specifically, the predicted classification label P(x i ). The predicted value of each pixel on the edge feature map is the predicted classification label P(x) of each pixel on each sample image. i ).
[0143] (3) The predicted classification labels and annotated classification labels of each pixel point of each sample image in the multiple sample image pairs are input into a preset binary classification cross entropy loss function to obtain the contour extraction loss.
[0144] In a specific embodiment, the preset binary cross entropy loss function is as follows:
[0145] Among them, S1 is the contour extraction loss, P(x i ) is the predicted classification label of each pixel on each sample image, y i Annotate the classification labels for each pixel on each sample image.
[0146] (4) The similarity between the labeled overlap contrast parameters and the predicted overlap contrast parameters is determined as the transformation matrix loss.
[0147] In a specific embodiment, the similarity between the labeled overlap contrast parameter and the predicted overlap contrast parameter is calculated and can be calculated using a cosine similarity formula to obtain a transformation matrix loss.
[0148] In another specific embodiment, the annotation overlap contrast parameters are converted into a 3*3 annotation transformation matrix, and the predicted overlap contrast parameters are converted into a 3*3 predicted transformation matrix. The first 8 elements of the annotation transformation matrix are the 8 parameters of the annotation overlap contrast parameters, and the last element of the annotation transformation matrix is 1; the first 8 elements of the predicted transformation matrix are the 8 parameters of the predicted overlap contrast parameters, and the last element of the predicted transformation matrix is 1. The similarity between the annotation transformation matrix and the predicted transformation matrix is determined as the transformation matrix loss. The 6 parameters of the annotation transformation matrix are rotation parameters, and the 3 parameters of the annotation transformation matrix are translation parameters. Usually, one of the translation parameters is directly set to 1 for freedom constraint.
[0149] Among them, the parameters on the 8 channels of the predicted overlapping contrast parameters are the first 8 elements of the predicted transformation matrix H', and the predicted transformation matrix H' is as follows:
[0150]
[0151] (5) Determine the total loss based on the contour extraction loss and the transformation matrix loss.
[0152] In a specific embodiment, a weighted sum is performed on the contour extraction loss and the transformation matrix loss to obtain the total loss.
[0153] In another specific embodiment, weighted summing of the contour extraction loss and the transformation matrix loss to obtain a total loss includes: transforming the second sample image using the predicted overlap contrast parameter to obtain a transformed image. Determining the similarity between the transformed image and the first sample image as the human image loss. Determining the total loss based on the contour extraction loss, the human image loss, and the transformation matrix loss.
[0154] Specifically, the predicted overlap contrast parameter is converted into a 3*3 prediction transformation matrix, wherein the first 8 elements of the prediction transformation matrix are 8 parameters of the predicted overlap contrast parameter, the last element of the prediction transformation matrix is 1, and the second sample image is multiplied by the prediction transformation matrix to obtain a transformation image.
[0155] Specifically, the calculation formulas for human image loss S2 and transformation matrix loss S3 are as follows:
[0156]
[0157]
[0158] Among them, H ij To mark the parameters in the transformation matrix, H ij ' is the parameters in the prediction transformation matrix, n is the number of samples, P1 is the first sample image, P2 1 To transform the image.
[0159] Specifically, the contour extraction loss, the human image loss, and the transformation matrix loss are weighted and summed based on a preset loss weight coefficient to obtain a total loss S.
[0160] Among them, the total loss S satisfies the following formula:
[0161] S=a*S1+b*S2+c*S3
[0162] Among them, a, b, and c are the loss weight coefficients of contour extraction loss, human image loss, and transformation matrix loss, respectively.
[0163] (6) The preset neural network model is iteratively trained based on the total loss to obtain an image overlapping contrast model.
[0164] Specifically, determine whether the total loss exceeds the preset value. If the total loss exceeds the preset value, update the model weight parameters of the preset neural network model, repeat the total loss calculation steps, and obtain a new total loss. When the new total loss does not exceed the preset value, output the preset neural network model at this time to obtain an image overlapping comparison model.
[0165] In the embodiment of the present application, the target image information includes a first target feature map and a second target feature map, the image overlapping contrast model includes an overlapping contrast module, the overlapping contrast module includes at least two overlapping contrast convolution blocks and a fully connected layer connected in series, and the number of channels of the at least two overlapping contrast convolution blocks decreases successively. The target image information is overlapped and contrasted using the image overlapping contrast model to obtain overlapping degree information, including:
[0166] (1) Feature fusion is performed based on the first target feature image and the second target feature image to obtain a fused image.
[0167] like Figure 7 As shown, in an embodiment of the present application, the overlapping contrast module includes at least two overlapping contrast convolution blocks and a fully connected layer connected in series, and the number of channels of at least two overlapping contrast convolution blocks decreases successively. Specifically, the overlapping contrast module includes 2 overlapping contrast convolution blocks connected in series, the first overlapping contrast convolution block includes an overlapping contrast convolution layer conv10, the overlapping contrast convolution layer conv10 uses a 7*7 convolution kernel, the number of channels is 32, and the overlapping contrast convolution layer conv10 outputs a feature map of 32 channels. The second overlapping contrast convolution block includes an overlapping contrast convolution layer conv11, the overlapping contrast convolution layer conv11 uses a 5*5 convolution kernel, the number of channels is 16, and the overlapping contrast convolution layer conv8 outputs a feature map of 16 channels. The number of channels of the fully connected layer fc is 8, and the overlapping contrast parameters include parameters on 8 channels. For example, the predicted transformation matrix H' is determined based on the 8 channels of the overlapping contrast parameters.
[0168] In a specific embodiment, feature fusion is performed based on the first target feature image and the second target feature map to obtain a fused image, including: splicing the first target feature image and the second target feature map in the channel dimension to obtain a fused image.
[0169] In order to ensure the one-to-one correspondence of elements when generating the target feature image, the feature space size is not reduced. If the above target feature image is directly used for matching relationship calculation, the number of parameters will explode, reducing the efficiency of adoption.
[0170] In order to improve efficiency, in another specific embodiment, the image overlap contrast model includes a dimension reduction module. The structure of the dimension reduction module is as follows: Figure 7 As shown. The dimensionality reduction module includes at least two dimensionality reduction convolution blocks connected in series. The dimensionality reduction convolution block includes a convolution layer, a ReLU layer and a MaxPoo1 layer. The number of channels in at least two dimensionality reduction convolution blocks decreases successively. Specifically, the dimensionality reduction module includes 2 dimensionality reduction convolution blocks connected in series. The first dimensionality reduction convolution block includes a dimensionality reduction convolution layer conv8, a ReLU layer and a MaxPoo1 layer. The dimensionality reduction convolution layer conv8 uses a 3*3 convolution kernel, the number of channels is 128, and the ReLU activation function is used. The MaxPoo1 layer uses a 2*2 convolution kernel, and the dimensionality reduction convolution layer conv8 outputs a feature map of 128 channels. The second dimensionality reduction convolution block includes a dimensionality reduction convolution layer conv9, a ReLU layer and a MaxPoo1 layer. The dimensionality reduction convolution layer conv9 uses a 3*3 convolution kernel, the number of channels is 64, and the ReLU activation function is used. The MaxPoo1 layer uses a 2*2 convolution kernel, and the dimensionality reduction convolution layer conv9 outputs a feature map of 64 channels. The overlapping contrast module and the dimensionality reduction module are shown in FIG. Figure 7 shown.
[0171] Specifically, the first target feature image and the second target feature image are subjected to feature fusion to obtain a fused image, including: inputting the first target feature image and the second target feature image into a dimension reduction module respectively to obtain a third target feature image after dimension reduction and a fourth target feature image after dimension reduction. Specifically, the first target feature image G1 is input into the dimension reduction module to obtain a third target feature image of 32*32*64; the second target feature image G2 is input into the dimension reduction module to obtain a fourth target feature image of 32*32*64.
[0172] The third target feature image and the fourth target feature image are fused to obtain a fused image. Specifically, the third target feature image and the fourth target feature image are spliced into a 32*32*128 fused image in the channel dimension.
[0173] (2) The fused image is sequentially input into at least two overlapping contrast convolution blocks and a fully connected layer to obtain overlapping contrast parameters.
[0174] Further, such as Figure 8 As shown, Figure 8 : is another embodiment flow chart of the image processing method provided in the embodiment of the present application, the image processing method comprises the following steps S301 to S306:
[0175] S301, obtaining a first image of a user before and after exercising.
[0176] In a specific embodiment, a first image P1 before fitness and a second image P2 after fitness uploaded by a user are obtained.
[0177] In another specific embodiment, a photo-taking prompt is issued to the user, and the photo-taking prompt may include: prompting the user to wear requirements (such as underwear) and reference postures (such as a human body standing naturally with arms naturally open at 45 degrees). The camera is activated to capture and record a first image before fitness. After a long period of fitness, the user enters the system again, and a photo-taking prompt is issued to the user. The user maintains the same posture, and the camera is activated to capture and record a second image after fitness.
[0178] S302: Obtain a pre-trained image overlap comparison model and an image feature processing model.
[0179] S303: Input the first image into an image feature processing model to obtain a first target feature image.
[0180] Specifically, the first image is input into the image feature processing model to obtain a first target feature image G1.
[0181] S304: Input the second image into the image feature processing model to obtain a second target feature image.
[0182] Specifically, the second image is input into the image feature processing model to obtain a second target feature image G2.
[0183] S305 , using an image overlap contrast model to perform overlap contrast on the first target feature image and the second target feature image to obtain overlap contrast parameters.
[0184] S306: Determine a fitness evaluation result of the user based on the overlapping comparison parameters.
[0185] In a specific embodiment, the second image is transformed using the overlap contrast parameter to obtain a target transformed image. Specifically, the overlap contrast parameter is converted into a 3*3 transformation matrix, and the second image is transformed using the 3*3 transformation matrix to obtain the target transformed image. Among them, the first 8 elements of the overlap contrast parameter are 8 parameters of the overlap contrast parameter, and the last element of the transformation matrix is 1. The similarity between the target transformed image and the first sample image is determined as a fitness evaluation parameter, and the fitness evaluation result is determined according to the fitness evaluation parameter and a preset evaluation mapping relationship.
[0186] Further, a preset evaluation mapping relationship is obtained. For example, the preset mapping evaluation relationship is: the fitness evaluation parameter is e. If e is in [x, y], it can be judged that the fitness has a certain effect. Among them, x and y are both preset hyperparameters. If e is less than x, it is interference caused by noise, etc.; if e is greater than y, it may be due to inconsistent user posture or wearing problems that cause image comparison failure. Therefore, e is constrained to the range of [x, y] to determine the fitness evaluation result.
[0187] Furthermore, the overlapping comparison diagram of the first target feature image G1 and the second target feature image G2 and the fitness evaluation result are output, and the user can intuitively feel the changes in body shape.
[0188] In order to better implement the image processing method in the embodiment of the present application, based on the image processing method, an image processing device is also provided in the embodiment of the present application. The image processing device is integrated in an electronic device, such as Fig. 9 As shown, the image processing device includes:
[0189] An acquisition unit 401 is used to acquire image information to be processed, wherein the image information to be processed includes at least two images to be processed;
[0190] The image processing unit 402 is used to perform feature extraction and fusion processing on the image information to be processed by using an image feature processing model to obtain target image information;
[0191] The overlapping comparison unit 403 is used to perform overlapping comparison on the target image information by using the image overlapping comparison model to obtain overlapping degree information.
[0192] Optionally, the image feature processing model includes a feature extraction model and a feature fusion module, and the image feature processing model is used to perform feature extraction and fusion processing on the image information to be processed to obtain target image information, including:
[0193] Use the feature extraction model to perform multiple feature extractions on the image to be processed to obtain image feature information;
[0194] The feature fusion module is used to fuse the image feature information to obtain the target feature map of the image to be processed.
[0195] Optionally, the image feature information includes a global feature map and an edge feature map, the feature extraction model includes a global feature extraction module and an edge feature extraction module, and the feature extraction model is used to perform multiple feature extractions on the image to be processed to obtain image feature information, including:
[0196] A global feature extraction module is used to extract global features of the image to be processed, so as to obtain a global feature map of the image to be processed;
[0197] The edge feature extraction module is used to extract edge features from the global feature map to obtain the edge feature map of the image to be processed.
[0198] Optionally, the global feature extraction module includes at least two global convolution blocks connected in series, each global convolution block includes at least two global convolution layers, the number of channels of the at least two global convolution blocks increases sequentially, and the global feature extraction module is used to perform global feature extraction on the image to be processed to obtain a global feature map of the image to be processed, including:
[0199] The image to be processed is sequentially input into at least two global convolution blocks to obtain a global feature map.
[0200] Optionally, the edge feature extraction module includes an edge feature fusion convolution block and at least two edge convolution blocks corresponding to the at least two global convolution blocks, each edge convolution block includes at least two edge convolution layers, the number of channels of the at least two global convolution blocks increases sequentially, and the edge feature extraction module is used to extract edge features from the global feature map to obtain an edge feature map of the image to be processed, including:
[0201] Input the feature maps output by at least two global convolution blocks into corresponding edge convolution blocks respectively to obtain at least two edge feature sub-maps;
[0202] At least two edge feature sub-maps are input into the edge feature fusion convolution block for fusion to obtain an edge feature map of the image to be processed.
[0203] Optionally, a feature fusion module is used to fuse image feature information to obtain a target feature map of the image to be processed, including:
[0204] Multiply the global feature map and the edge feature map to obtain a multiplied feature map;
[0205] Obtaining preset edge feature weight coefficients and global feature weight coefficients;
[0206] The global feature map and the multiplied feature map are weighted and summed based on the edge feature weight coefficient and the global feature weight coefficient to obtain the target feature map of the image to be processed.
[0207] Optionally, the target image information includes a first target feature map and a second target feature map, the image overlapping contrast model includes an overlapping contrast module, the overlapping contrast module includes at least two overlapping contrast convolution blocks and a fully connected layer connected in series, and the number of channels of the at least two overlapping contrast convolution blocks decreases sequentially;
[0208] The image overlap contrast model is used to perform overlap contrast on the target image information to obtain overlap degree information, including:
[0209] Perform feature fusion based on the first target feature image and the second target feature image to obtain a fused image;
[0210] The fused image is sequentially input into at least two overlapping contrast convolution blocks and a fully connected layer to obtain overlapping contrast parameters.
[0211] Optionally, the image overlapping contrast model includes a dimensionality reduction module, the dimensionality reduction module includes at least two dimensionality reduction convolution blocks connected in series, the dimensionality reduction convolution block includes a convolution layer, an activation function layer and a maximum pooling layer, and the number of channels in the at least two dimensionality reduction convolution blocks decreases sequentially;
[0212] Performing feature fusion based on the first target feature image and the second target feature image to obtain a fused image includes:
[0213] Inputting the first target feature image and the second target feature image into a dimension reduction module respectively to obtain a third target feature image after dimension reduction and a fourth target feature image after dimension reduction;
[0214] The third target feature image and the fourth target feature image are fused to obtain a fused image.
[0215] Optionally, the image processing method comprises:
[0216] Obtaining a preset neural network model, a plurality of sample image pairs, annotated overlap comparison parameters of the plurality of sample image pairs, and annotated classification labels of each pixel point of each sample image in the plurality of sample image pairs, wherein each sample image pair includes a first sample image before fitness and a second sample image after fitness, and the annotated classification label indicates whether the pixel point is an edge point;
[0217] Inputting a plurality of sample image pairs into an image feature processing model to obtain a predicted classification label of each pixel point of each sample image in the plurality of sample image pairs and a predicted overlap contrast parameter of the plurality of sample image pairs;
[0218] Input the predicted classification label and the annotated classification label of each pixel point of each sample image in the multiple sample image pairs into a preset binary classification cross entropy loss function to obtain a contour extraction loss;
[0219] The similarity between the transformation of the annotated overlap contrast parameter and the predicted overlap contrast parameter is determined as the transformation matrix loss;
[0220] Determine the total loss based on the contour extraction loss and the transformation matrix loss;
[0221] The preset neural network model is iteratively trained based on the total loss to obtain the image overlap comparison model.
[0222] The embodiment of the present application further provides an electronic device, which integrates any image processing device provided in the embodiment of the present application, and the electronic device includes:
[0223] one or more processors;
[0224] Memory; and
[0225] One or more applications, wherein the one or more applications are stored in the memory and are configured to execute the steps of the image processing method in any of the above-mentioned image processing method embodiments by the processor.
[0226] like Fig.10 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:
[0227] The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:
[0228] The processor 601 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 602, and using data stored in the memory 602, the processor 601 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 601 may include one or more processing cores; the processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, physical interface and application programs, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.
[0229] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.
[0230] The electronic device also includes a power supply 603 for supplying power to each component. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.
[0231] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to physical settings and function controls.
[0232] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602, thereby realizing various functions, as follows:
[0233] Obtain image information to be processed, which includes at least two images to be processed; use an image feature processing model to perform feature extraction and fusion processing on the image information to be processed to obtain target image information; use an image overlap comparison model to perform overlap comparison on the target image information to obtain overlap degree information.
[0234] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0235] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any image processing method provided in the embodiment of the present application. For example, the computer program can be loaded by a processor to execute the following steps:
[0236] Obtain image information to be processed, which includes at least two images to be processed; use an image feature processing model to perform feature extraction and fusion processing on the image information to be processed to obtain target image information; use an image overlap comparison model to perform overlap comparison on the target image information to obtain overlap degree information.
[0237] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.
[0238] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.
[0239] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0240] The above is a detailed introduction to an image processing method and device provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. An image processing method, characterized in that: The image processing method comprises: Acquire image information to be processed, wherein the image information to be processed includes at least 2 images to be processed; Using an image feature processing model to perform feature extraction and fusion processing on the image information to be processed to obtain target image information; The target image information is overlapped and compared using an image overlap comparison model to obtain overlap degree information.
2. The image processing method according to claim 1, characterized in that: The image feature processing model includes a feature extraction model and a feature fusion module. The image feature processing model is used to perform feature extraction and fusion processing on the image information to be processed to obtain target image information, including: Using a feature extraction model to perform multiple feature extractions on the image to be processed to obtain image feature information; The feature fusion module is used to fuse the image feature information to obtain a target feature map of the image to be processed.
3. The image processing method according to claim 2, characterized in that: The image feature information includes a global feature map and an edge feature map, the feature extraction model includes a global feature extraction module and an edge feature extraction module, and the feature extraction model is used to perform multiple feature extractions on the image to be processed to obtain image feature information, including: Using the global feature extraction module to extract global features of the image to be processed, to obtain a global feature map of the image to be processed; The edge feature extraction module is used to extract edge features from the global feature map to obtain an edge feature map of the image to be processed.
4. The image processing method according to claim 3, characterized in that: The global feature extraction module includes at least two global convolution blocks connected in series, each global convolution block includes at least two global convolution layers, and the number of channels of the at least two global convolution blocks increases sequentially. The global feature extraction module is used to perform global feature extraction on the image to be processed to obtain a global feature map of the image to be processed, including: The image to be processed is sequentially input into the at least two global convolution blocks to obtain the global feature map.
5. The image processing method according to claim 3, characterized in that: The edge feature extraction module includes an edge feature fusion convolution block and at least two edge convolution blocks corresponding to the at least two global convolution blocks, each edge convolution block includes at least two edge convolution layers, and the number of channels of the at least two global convolution blocks increases sequentially. The edge feature extraction module is used to extract edge features from the global feature map to obtain the edge feature map of the image to be processed, including: Input the feature maps output by the at least two global convolution blocks into the corresponding edge convolution blocks respectively to obtain at least two edge feature sub-maps; The at least two edge feature sub-graphs are input into the edge feature fusion convolution block for fusion to obtain the edge feature graph of the image to be processed.
6. The image processing method according to claim 3, characterized in that: The adopting the feature fusion module to fuse the image feature information to obtain the target feature map of the image to be processed includes: Multiplying the global feature map and the edge feature map to obtain a multiplied feature map; Obtaining preset edge feature weight coefficients and global feature weight coefficients; The global feature map and the multiplied feature map are weighted and summed based on the edge feature weight coefficient and the global feature weight coefficient to obtain a target feature map of the image to be processed.
7. The image processing method according to claim 1, characterized in that: The target image information includes a first target feature map and a second target feature map, the image overlapping contrast model includes an overlapping contrast module, the overlapping contrast module includes at least two overlapping contrast convolution blocks and a fully connected layer connected in series, and the number of channels of the at least two overlapping contrast convolution blocks decreases sequentially; The method of performing overlap comparison on the target image information by using the image overlap comparison model to obtain overlap degree information includes: Perform feature fusion based on the first target feature image and the second target feature image to obtain a fused image; The fused image is sequentially input into the at least two overlapping contrast convolution blocks and the fully connected layer to obtain the overlapping contrast parameter.
8. The image processing method according to claim 7, characterized in that: The image overlapping contrast model includes a dimensionality reduction module, the dimensionality reduction module includes at least two dimensionality reduction convolution blocks connected in series, the dimensionality reduction convolution block includes a convolution layer, an activation function layer and a maximum pooling layer, and the number of channels in the at least two dimensionality reduction convolution blocks decreases sequentially; The step of performing feature fusion based on the first target feature image and the second target feature image to obtain a fused image includes: Inputting the first target feature image and the second target feature image into the dimension reduction module respectively to obtain a third target feature image after dimension reduction and a fourth target feature image after dimension reduction; The third target feature image and the fourth target feature image are fused to obtain the fused image.
9. The image processing method according to claim 8, characterized in that: The image processing method comprises: Obtaining a preset neural network model, a plurality of sample image pairs, annotated overlap comparison parameters of the plurality of sample image pairs, and annotated classification labels of each pixel point of each sample image in the plurality of sample image pairs, wherein each of the sample image pairs includes a first sample image before fitness and a second sample image after fitness, and the annotated classification label indicates whether the pixel point is an edge point; Inputting the plurality of sample image pairs into the image feature processing model to obtain predicted classification labels of each pixel point of each sample image in the plurality of sample image pairs and predicted overlap contrast parameters of the plurality of sample image pairs; Input the predicted classification label and the annotated classification label of each pixel point of each sample image in the multiple sample image pairs into a preset binary classification cross entropy loss function to obtain a contour extraction loss; Determining the similarity between the transformation of the annotated overlap contrast parameter and the predicted overlap contrast parameter as a transformation matrix loss; determining a total loss based on the contour extraction loss and the transformation matrix loss; The preset neural network model is iteratively trained based on the total loss to obtain the image overlapping contrast model.
10. An image processing device, characterized in that: The image processing device comprises: An acquisition unit, used for acquiring image information to be processed, wherein the image information to be processed includes at least two images to be processed; An image processing unit, used for performing feature extraction and fusion processing on the image information to be processed by using an image feature processing model to obtain target image information; The overlapping comparison unit is used to perform overlapping comparison on the target image information by using an image overlapping comparison model to obtain overlapping degree information.
11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of the image processing method according to any one of claims 1 to 9.
