Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of electronic information. The image processing method comprises the following steps: image preprocessing: carrying out noise removal, contrast enhancement and image smoothing operations on an image before the image is analyzed; image segmentation: dividing the image into a plurality of parts or regions for further processing; feature extraction and matching: extracting useful feature points from the image for subsequent analysis; image classification and identification: classifying and identifying the images by using a machine learning algorithm; image transformation: carrying out geometric transformation on the image through methods of rotation, zooming and translation; and the image restoration is used for restoring damaged, missing or incomplete image areas. According to the method, the visual effect of the image can be remarkably improved through contrast enhancement and color correction technologies, and features in the image can be automatically learned by means of methods such as a CNN (convolutional neural network) in deep learning.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and in particular to an image processing method and device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of digital technology, images, as an important information carrier, have been widely used in many fields. From the early simple optical imaging to the current high-resolution, multi-spectral digital images, the scale and complexity have changed dramatically. For example, in the medical field, medical imaging technologies such as CT (computed tomography) and MRI (magnetic resonance imaging) can generate a large number of complex images of the internal structure of the human body, which are essential for the diagnosis and treatment of diseases. In the security field, surveillance cameras can obtain a large number of video images in real time, and these images need to be processed to extract useful information, such as the identification of people and vehicles. In addition, the popularity of smartphones has led to people taking a large number of photos and videos every day, and beautifying and editing these images has become a common demand.

[0003] In image recognition applications, effective features need to be extracted from images. However, due to the complexity of image content, accurately extracting representative features has always been a difficult problem. Early image processing devices had limited computing power. When processing large-scale, high-resolution images, the processing speed would be very slow. The development of storage media also has an impact on image processing. In the past, storage media had small capacity and slow read and write speeds, which limited the storage and subsequent processing of images.

[0004] Therefore, those skilled in the art provide an image processing method and apparatus, an electronic device, and a storage medium to solve the problems raised in the above background technology. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides an image processing method and device, an electronic device and a storage medium. Various noises in the image can be effectively removed through advanced image processing methods. The visual effect of the image can be significantly improved through contrast enhancement and color correction techniques. With the help of methods such as convolutional neural network (CNN) in deep learning, the features in the image can be automatically learned, which solves the problem that the computing power of early image processing devices was limited. When processing large-scale, high-resolution images, the processing speed would be very slow. The development of storage media also has an impact on image processing. In the past, storage media had small capacity and slow reading and writing speeds, which limited the storage and subsequent processing of images.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] An image processing method, the method comprising:

[0010] Image preprocessing, which involves noise removal, contrast enhancement, and image smoothing before image analysis;

[0011] Image segmentation, dividing an image into parts or regions for further processing;

[0012] Feature extraction and matching: extracting useful feature points from images for subsequent analysis, such as image recognition and object detection;

[0013] Image classification and recognition, using machine learning algorithms to classify and identify images;

[0014] Image transformation, which performs geometric transformations on images by rotation, scaling, and translation;

[0015] Image restoration is used to repair damaged, missing or incomplete image areas so that the restored image looks natural and seamless. Commonly used techniques include image interpolation and image generative adversarial networks.

[0016] Furthermore, the image preprocessing process includes:

[0017] De-noising: use mean filtering, median filtering, and Gaussian filtering methods to eliminate noise in the image;

[0018] Edge detection, extracting edge features in images through gradient and Laplacian operator methods;

[0019] Image enhancement, improving the quality and readability of images through image enhancement processing, such as histogram equalization.

[0020] Furthermore, the image segmentation method specifically includes:

[0021] Threshold-based segmentation divides the pixels of an image into two categories by setting a threshold. If the pixel value is greater than a certain threshold, the pixel belongs to one category, otherwise it belongs to another category. The global threshold method or the local threshold method is usually used for segmentation. The global threshold method uses the same threshold for segmentation of the entire image, and the local threshold method uses different thresholds for segmentation in different areas of the image. It is suitable for images with large changes in illumination.

[0022] Edge-based segmentation is to segment an image by extracting edge information from the image. The edge of an image is the boundary between objects. Segmentation can use the edge to distinguish different areas. The Sobel operator or Canny edge detection is usually used for segmentation. The Sobel operator calculates the gradient of the image grayscale image to find the place where the intensity changes in the image, that is, the edge. The Canny edge detection is a more advanced edge detection algorithm with good noise resistance and accuracy.

[0023] Region-based segmentation, which segments regions based on the similarity of pixels in an image, includes region growing and split-merge methods. The region growing method starts from a seed point and gradually adds adjacent pixels to the region until a certain similarity standard is met. The split-merge method first divides the image into several small regions and then gradually merges similar regions until a preset condition is met.

[0024] Deep learning-based segmentation can achieve more accurate and robust image segmentation by training deep neural networks, especially in complex images, including:

[0025] U-Net, a typical convolutional neural network structure for medical image segmentation, adopts an encoder-decoder structure and retains detailed information through skip connections;

[0026] FCN (Fully Convolutional Network), a convolutional neural network for semantic segmentation, which performs segmentation by applying convolution operations to the entire image instead of a single local region;

[0027] Mask R-CNN, a network for instance segmentation, is not only able to identify objects in an image but also generate accurate segmentation masks for each object.

[0028] Furthermore, the feature extraction and matching specifically include:

[0029] Determine the corresponding area of ​​the measured feature in the image to be measured;

[0030] Extracting a representative region representing the corresponding region from the corresponding region, wherein the grayscale values ​​of the pixels distributed in the representative region are within a preset range;

[0031] The regional features corresponding to the representative regions are used to determine whether the measured features are corresponding features.

[0032] Further, determining the corresponding area of ​​the measured feature in the image to be measured includes:

[0033] Determine the region of interest in the image to be tested by using a template image for the feature to be tested and the position of the region of interest in the template image;

[0034] The corresponding area in the image to be tested is extracted using the grayscale value of each pixel in the area of ​​interest in the image to be tested.

[0035] Further, extracting a representative area representing the corresponding area from the corresponding area includes:

[0036] The corresponding area is fitted by using the contour fitting method to determine the characteristic contour of the feature to be measured and the center point of the characteristic contour;

[0037] Using different distances from the center point, multiple sub-region outlines of the same shape as the corresponding outline are generated, thereby generating a sub-region defined by the multiple sub-region outlines, wherein the pixel values ​​of the pixels distributed in each sub-region are in different intervals;

[0038] A subregion corresponding to the detection requirement is selected from the plurality of subregions as a representative region.

[0039] Furthermore, the representative region is a region that has the same shape as the corresponding region and is scaled in the same proportion.

[0040] Furthermore, the image classification and recognition utilizes a convolutional neural network (CNN) to classify and recognize images, which processes images through multiple convolutional layers and pooling layers, automatically extracting low-level features (such as edges, corners) and high-level features (such as (objects, scenes)) in the image, thereby recognizing objects in the image.

[0041] Further, an image processing device includes an image processing chip, an image sensor and an embedded system;

[0042] The image processing chip is used to process image data, and may use a GPU (graphics processing unit) or an FPGA (field programmable gate array);

[0043] The image sensor is used to capture image data, and a CMOS sensor or a CCD sensor may be used;

[0044] The embedded system is a small computing device designed for image processing applications and is often used in smart devices in embedded systems, such as smart monitoring and autonomous driving.

[0045] Furthermore, an electronic device and a storage medium include:

[0046] processor;

[0047] A memory, used to store a computer program executable by a processor, wherein the computer program executes the image processing method according to any one of claims 1 to 8;

[0048] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor.

[0049] (III) Beneficial effects

[0050] The present invention provides an image processing method and device, electronic equipment and storage medium. The invention has the following beneficial effects:

[0051] 1. The present invention provides an image processing method and device, an electronic device and a storage medium. Various noises in an image, such as Gaussian noise, salt and pepper noise, etc., can be effectively removed through advanced image processing methods. The visual effect of the image can be significantly improved through contrast enhancement and color correction technology. With the help of methods such as convolutional neural network (CNN) in deep learning, the features in the image can be automatically learned.

[0052] 2. The present invention provides an image processing method and device, an electronic device and a storage medium. The image processing method, device, electronic device and storage medium together constitute a complete image processing system, which supports the entire process from image acquisition, processing to storage and transmission. Through the application of deep learning, the accuracy and efficiency of image processing are continuously improved, which has promoted the development of many industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the image processing method of the present invention;

[0054] Figure 2 It is a schematic diagram of the image preprocessing process of the present invention;

[0055] Figure 3 The figure is a flow chart of the image segmentation method of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the specific embodiments of the present invention to clearly and completely describe the technical solutions in the specific embodiments of the present invention. Obviously, the specific embodiments described are only part of the specific embodiments of the present invention, not all of the specific embodiments. Based on the specific embodiments of the present invention, all other specific embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Specific implementation method:

[0058] like Figure 1-3 As shown, a specific embodiment of the present invention provides an image processing method, the method comprising:

[0059] Image preprocessing, which involves noise removal, contrast enhancement, and image smoothing before image analysis;

[0060] The image preprocessing process includes:

[0061] De-noising: use mean filtering, median filtering, and Gaussian filtering methods to eliminate noise in the image;

[0062] Edge detection, extracting edge features in images through gradient and Laplacian operator methods;

[0063] Image enhancement, improving the quality and readability of images through image enhancement processing, such as histogram equalization.

[0064] Image segmentation, dividing an image into parts or regions for further processing;

[0065] The image segmentation method specifically includes:

[0066] Threshold-based segmentation divides the pixels of an image into two categories by setting a threshold. If the pixel value is greater than a certain threshold, the pixel belongs to one category, otherwise it belongs to another category. The global threshold method or the local threshold method is usually used for segmentation. The global threshold method uses the same threshold for segmentation of the entire image, and the local threshold method uses different thresholds for segmentation in different areas of the image. It is suitable for images with large changes in illumination.

[0067] Edge-based segmentation is to segment an image by extracting edge information from the image. The edge of an image is the boundary between objects. Segmentation can use the edge to distinguish different areas. The Sobel operator or Canny edge detection is usually used for segmentation. The Sobel operator calculates the gradient of the image grayscale image to find the place where the intensity changes in the image, that is, the edge. The Canny edge detection is a more advanced edge detection algorithm with good noise resistance and accuracy.

[0068] Region-based segmentation, which segments regions based on the similarity of pixels in an image, includes region growing and split-merge methods. The region growing method starts from a seed point and gradually adds adjacent pixels to the region until a certain similarity standard is met. The split-merge method first divides the image into several small regions and then gradually merges similar regions until a preset condition is met.

[0069] Deep learning-based segmentation can achieve more accurate and robust image segmentation by training deep neural networks, especially in complex images, including:

[0070] U-Net, a typical convolutional neural network structure for medical image segmentation, adopts an encoder-decoder structure and retains detailed information through skip connections;

[0071] FCN (Fully Convolutional Network), a convolutional neural network for semantic segmentation, which performs segmentation by applying convolution operations to the entire image instead of a single local region;

[0072] Mask R-CNN, a network for instance segmentation, is not only able to identify objects in an image but also generate accurate segmentation masks for each object.

[0073] Feature extraction and matching: extracting useful feature points from images for subsequent analysis, such as image recognition and object detection;

[0074] The feature extraction and matching specifically include:

[0075] Determine the corresponding area of ​​the measured feature in the image to be measured;

[0076] Extracting a representative region representing the corresponding region from the corresponding region, wherein the grayscale values ​​of the pixels distributed in the representative region are within a preset range;

[0077] The regional features corresponding to the representative regions are used to determine whether the measured features are corresponding features.

[0078] Image classification and recognition, using machine learning algorithms to classify and identify images;

[0079] Determining the corresponding area of ​​the measured feature in the image to be measured includes:

[0080] Determine the region of interest in the image to be tested by using a template image for the feature to be tested and the position of the region of interest in the template image;

[0081] The corresponding area in the image to be tested is extracted using the grayscale value of each pixel in the area of ​​interest in the image to be tested.

[0082] Extracting representative regions representing corresponding regions from corresponding regions includes:

[0083] The corresponding area is fitted by using the contour fitting method to determine the characteristic contour of the feature to be measured and the center point of the characteristic contour;

[0084] Using different distances from the center point, multiple sub-region outlines of the same shape as the corresponding outline are generated, thereby generating a sub-region defined by the multiple sub-region outlines, wherein the pixel values ​​of the pixels distributed in each sub-region are in different intervals;

[0085] Selecting a sub-region corresponding to the detection requirement from the multiple sub-regions as a representative region;

[0086] The representative region is a region that has the same shape as the corresponding region and is scaled in the same proportion.

[0087] The image classification and recognition utilizes a convolutional neural network (CNN) to classify and recognize images, which processes images through multiple convolutional layers and pooling layers, automatically extracts low-level features (such as edges, corners) and high-level features (such as objects, scenes) in the image, thereby recognizing objects in the image.

[0088] Image transformation, which performs geometric transformations on images by rotation, scaling, and translation;

[0089] Image restoration is used to repair damaged, missing or incomplete image areas so that the restored image looks natural and seamless. Commonly used techniques include image interpolation and image generative adversarial networks.

[0090] The image processing device comprises an image processing chip, an image sensor and an embedded system;

[0091] The image processing chip is used to process image data, and may use a GPU (graphics processing unit) or an FPGA (field programmable gate array);

[0092] The image sensor is used to capture image data, and a CMOS sensor or a CCD sensor may be used;

[0093] The embedded system is a small computing device designed for image processing applications and is often used in smart devices in embedded systems, such as smart monitoring and autonomous driving.

[0094] The electronic device and storage medium include:

[0095] processor;

[0096] A memory, used to store a computer program executable by a processor, wherein the computer program executes the image processing method according to any one of claims 1 to 8;

[0097] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor.

[0098] In the present invention, various noises in the image, such as Gaussian noise, salt and pepper noise, etc., can be effectively removed through advanced image processing methods. The visual effect of the image can be significantly improved through contrast enhancement and color correction technology. With the help of methods such as convolutional neural network (CNN) in deep learning, the features in the image can be automatically learned.

[0099] The image processing method, device, electronic device and storage medium of the present invention together constitute a complete image processing system, which supports the entire process from image acquisition, processing to storage and transmission. Through the application of deep learning, the accuracy and efficiency of image processing are continuously improved, which has promoted the development of many industries.

[0100] Although specific embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the specific embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image processing method, characterized in that: The method comprises: Image preprocessing, which involves noise removal, contrast enhancement, and image smoothing before image analysis; Image segmentation, dividing an image into parts or regions for further processing; Feature extraction and matching: extracting useful feature points from images for subsequent analysis, such as image recognition and object detection; Image classification and recognition, using machine learning algorithms to classify and identify images; Image transformation, which performs geometric transformations on images by rotation, scaling, and translation; Image restoration is used to repair damaged, missing or incomplete image areas so that the restored image looks natural and seamless. Commonly used techniques include image interpolation and image generative adversarial networks.

2. An image processing method according to claim 1, characterized in that: The image preprocessing process includes: De-noising: use mean filtering, median filtering, and Gaussian filtering methods to eliminate noise in the image; Edge detection, extracting edge features in images through gradient and Laplacian operator methods; Image enhancement, improving the quality and readability of images through image enhancement processing, such as histogram equalization.

3. An image processing method according to claim 1, characterized in that: The image segmentation method specifically includes: Threshold-based segmentation divides the pixels of an image into two categories by setting a threshold. If the pixel value is greater than a certain threshold, the pixel belongs to one category, otherwise it belongs to another category. The global threshold method or the local threshold method is usually used for segmentation. The global threshold method uses the same threshold for segmentation of the entire image, and the local threshold method uses different thresholds for segmentation in different areas of the image. It is suitable for images with large changes in illumination. Edge-based segmentation is to segment an image by extracting edge information from the image. The edge of an image is the boundary between objects. Segmentation can use the edge to distinguish different areas. The Sobel operator or Canny edge detection is usually used for segmentation. The Sobel operator calculates the gradient of the image grayscale image to find the place where the intensity changes in the image, that is, the edge. The Canny edge detection is a more advanced edge detection algorithm with good noise resistance and accuracy. Region-based segmentation, which segments regions based on the similarity of pixels in an image, includes region growing and split-merge methods. The region growing method starts from a seed point and gradually adds adjacent pixels to the region until a certain similarity standard is met. The split-merge method first divides the image into several small regions and then gradually merges similar regions until a preset condition is met. Deep learning-based segmentation can achieve more accurate and robust image segmentation by training deep neural networks, especially in complex images, including: U-Net, a typical convolutional neural network structure for medical image segmentation, adopts an encoder-decoder structure and retains detailed information through skip connections; FCN, a convolutional neural network for semantic segmentation, which performs segmentation by applying convolution operations to the entire image instead of a single local region; Mask R-CNN, a network for instance segmentation, is not only able to identify objects in an image but also generate accurate segmentation masks for each object.

4. The image processing method according to claim 1, characterized in that: The feature extraction and matching specifically include: Determine the corresponding area of ​​the measured feature in the image to be measured; Extracting a representative region representing the corresponding region from the corresponding region, wherein the grayscale values ​​of the pixels distributed in the representative region are within a preset range; The regional features corresponding to the representative regions are used to determine whether the measured features are corresponding features.

5. An image processing method according to claim 4, characterized in that: Determining the corresponding area of ​​the measured feature in the image to be measured includes: Determine the region of interest in the image to be tested by using a template image for the feature to be tested and the position of the region of interest in the template image; The corresponding area in the image to be tested is extracted using the grayscale value of each pixel in the area of ​​interest in the image to be tested.

6. An image processing method according to claim 4, characterized in that: The extracting a representative area representing the corresponding area from the corresponding area comprises: The corresponding area is fitted by using the contour fitting method to determine the characteristic contour of the feature to be measured and the center point of the characteristic contour; Using different distances from the center point, multiple sub-region outlines of the same shape as the corresponding outline are generated, thereby generating a sub-region defined by the multiple sub-region outlines, wherein the pixel values ​​of the pixels distributed in each sub-region are in different intervals; A subregion corresponding to the detection requirement is selected from the plurality of subregions as a representative region.

7. An image processing method according to claim 4, characterized in that: The representative region is a region that has the same shape as the corresponding region and is scaled in the same proportion.

8. The image processing method according to claim 1, characterized in that: The image classification and recognition utilizes a convolutional neural network (CNN) to classify and recognize images, which processes images through multiple convolutional layers and pooling layers, automatically extracts low-level features and high-level features in the image, and thus recognizes objects in the image.

9. An image processing device, characterized in that: Including image processing chips, image sensors and embedded systems; The image processing chip is used to process image data, and a GPU or FPGA may be used; The image sensor is used to capture image data, and a CMOS sensor or a CCD sensor may be used; The embedded system is a small computing device designed for image processing applications and is often used in smart devices in embedded systems, such as smart monitoring and autonomous driving.

10. An electronic device and a storage medium, characterized in that: include: processor; A memory, used to store a computer program executable by a processor, wherein the computer program executes the image processing method according to any one of claims 1 to 8; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor.

Images