Image processing method, device, equipment and storage medium thereof

Abstract

The present application provides an image processing method, apparatus, device, and storage medium thereof. The method comprises: obtaining a first mask image and a second mask image; performing a projection change process on the second mask image based on the shooting angle of the first mask image to obtain a first comparison mask image; deleting a first target pixel in the first mask image based on the first comparison mask image to obtain a first target mask image, where the first target pixel is a pixel in the first mask image that represents a foreign object in the barrel to be detected; extracting a first local mask image from the first target mask image, where the first local mask image is determined based on the distribution of pixels in the first target mask image and represents a local mask image of the non-contour portion of the target area; and obtaining a first barrel body image based on the first local mask image and the first image. The present method improves image quality and the effectiveness of subsequent foreign object detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and storage medium thereof. Background Art

[0002] Foreign matter inspection refers to the process of quality testing for foreign matter in liquid products. Liquid products include, but are not limited to, water, beverages, cosmetics, pharmaceuticals, and petrochemicals. The purpose of foreign matter inspection is to ensure that products comply with relevant standards and regulations, safeguarding the health and safety of consumers.

[0003] Currently, when conducting quality inspection on bottled liquids, the original image of the bottled liquid is generally captured by a camera first, and then the original image of the bottled liquid is processed by target recognition and other methods to obtain the image to be inspected. Finally, the image to be inspected is inspected through machine vision inspection or manual inspection to obtain the inspection result of the bottled liquid.

[0004] However, existing image processing methods have problems such as poor processing effect and affecting the subsequent foreign matter detection effect. Summary of the Invention

[0005] The present application provides an image processing method, apparatus, device and storage medium thereof, which are used to solve the problem that existing image processing methods have poor processing effects and affect the subsequent foreign matter detection effect.

[0006] In a first aspect, the present application provides an image processing method, the method comprising:

[0007] Obtain a first mask image and a second mask image, wherein the first mask image is a mask image of the first image, and the second mask image is a mask image of the second image, and the first image and the second image are images taken at different shooting angles of the target area on the barrel to be tested;

[0008] Performing projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image, wherein the shooting angle of the first comparison mask image is different from that of the first mask image;

[0009] According to the first comparison mask image, a first target pixel point in the first mask image is deleted to obtain a first target mask image, where the first target pixel point is a pixel in the first mask image that represents a foreign object in the barrel to be detected;

[0010] Extracting a first local mask image from the first target mask image, where the first local mask image is determined based on the distribution of pixels in the first target mask image, and the first local mask image represents a local mask image of a non-contour line portion in the target area;

[0011] A first barrel body image is obtained according to the first local mask image and the first image.

[0012] In this application, obtaining a first mask image and a second mask image includes:

[0013] Acquire a first initial image of the barrel to be measured taken at a first shooting angle of view, and a second initial image of the barrel to be measured taken at a second shooting angle of view;

[0014] Performing target detection processing on the target area in the first initial image to obtain a first image;

[0015] performing target detection processing on the target area in the second initial image to obtain a second image;

[0016] Performing semantic segmentation on the first image to obtain a first mask image;

[0017] Perform semantic segmentation on the second image to obtain a second mask image.

[0018] In the present application, according to the shooting angle of the first mask image, the second mask image is subjected to projection change processing to obtain a first comparison mask image, including:

[0019] Determining a first world coordinate system, a second world coordinate system, and a transformation relationship between the first world coordinate system and the second world coordinate system, wherein the first world coordinate system is a world coordinate system constructed with the position of the first camera as the origin, the first camera being the camera that captures the first image; the second world coordinate system is a world coordinate system constructed with the position of the second camera as the origin, the second camera being the camera that captures the second image; and the transformation relationship between the first world coordinate system and the second world coordinate system is determined based on extrinsic parameters between the first camera and the second camera;

[0020] According to the transformation relationship between the first world coordinate system and the second world coordinate system, projection change processing is performed on the second mask image to obtain a first comparison mask image.

[0021] In the present application, according to the transformation relationship between the first world coordinate system and the second world coordinate system, the second mask image is subjected to projection change processing to obtain a first comparison mask image, including:

[0022] Obtain a second coordinate set of each pixel point on the second mask image in the second world coordinate system;

[0023] According to the second coordinate set and the transformation relationship between the first world coordinate system and the second world coordinate system, a projection change process is performed on the second mask image to obtain a first comparison mask image.

[0024] In this application, obtaining a second coordinate set of each pixel point on the second mask image in the second world coordinate system includes:

[0025] Determine a second intrinsic parameter of the second camera and second position information of the barrel to be tested relative to the second camera;

[0026] A second coordinate set is determined according to the second intrinsic parameter and the second position information.

[0027] In the present application, according to the first comparison mask image, the first target pixel point in the first mask image is deleted to obtain the first target mask image, including:

[0028] Comparing the position correspondence between the pixel points on the first comparison mask image and the pixel points on the first mask image;

[0029] If there is a first target pixel point with no corresponding position, the first target pixel point in the first mask image is deleted to obtain a first target mask image.

[0030] In this application, extracting a first local mask image from a first target mask image includes:

[0031] Determine the position of each pixel in the first target mask image;

[0032] Determining a contour line portion in the first target mask image according to positions of each pixel point in the first target mask image;

[0033] The contour line portion is deleted to obtain a first local mask image.

[0034] In the present application, determining the contour line portion in the first target mask image according to the position of each pixel point in the first target mask image includes:

[0035] Determining the number of pixels along the y-axis in the first target mask image according to the positions of the pixels in the first target mask image;

[0036] According to the number of pixels along the y-axis direction in the first target mask image, a contour line portion in the first target mask image is determined.

[0037] In the present application, determining the contour line portion in the first target mask image according to the number of pixels along the y-axis in the first target mask image includes:

[0038] Determine the pixel points in the barrel surface sub-region according to the number of pixel points along the y-axis in the first target mask image;

[0039] Clustering is performed on the pixel points in the barrel body surface sub-region to determine the contour line portion in the first target mask image.

[0040] In this application, determining the first barrel body image according to the first local mask image and the first image includes:

[0041] Extracting a first initial barrel body image from the first image according to the first local mask image;

[0042] Performing shape transformation processing on the first initial barrel body image to obtain a first barrel body image.

[0043] In this application, the method further comprises:

[0044] Performing projection change processing on the first target mask image according to the shooting angle of the second mask image to obtain a second comparison mask image;

[0045] Deleting a second target pixel in the second mask image according to the second comparison mask image to obtain a second target mask image, where the second target pixel is a pixel of the foreign object in the second mask image;

[0046] Extracting a second local mask image from the second target mask image, where the second local mask image is a local mask image representing a non-contour line portion of the target area in the second target mask image;

[0047] Obtain a second barrel body image according to the second local mask image and the second image;

[0048] A target barrel body image is obtained according to the first barrel body image and the second barrel body image.

[0049] In this application, obtaining a target barrel body image according to the first barrel body image and the second barrel body image includes:

[0050] Determining a correspondence between pixels in the first mask image and pixels in the second mask image;

[0051] According to the corresponding relationship, the first barrel body image and the second barrel body image are merged to obtain a target barrel body image.

[0052] In a second aspect, the present application provides an image processing device, comprising:

[0053] An acquisition module is used to acquire a first mask image and a second mask image, wherein the first mask image is a mask image of the first image, and the second mask image is a mask image of the second image, and the first image and the second image are images taken at different shooting angles of the target area on the barrel to be tested;

[0054] a projection change processing module, configured to perform projection change processing on the second mask image according to the shooting angle of the first mask image, to obtain a first comparison mask image, wherein the shooting angle of the first comparison mask image is different from that of the first mask image;

[0055] a deletion processing module, configured to delete a first target pixel in the first mask image according to the first comparison mask image to obtain a first target mask image, wherein the first target pixel is a pixel in the first mask image that represents a foreign object in the barrel to be detected;

[0056] An extraction module is used to extract a first local mask image from the first target mask image, where the first local mask image is determined based on the distribution of pixels in the first target mask image, and the first local mask image represents a local mask image of a non-contour line portion in the target area;

[0057] The obtaining module is used to obtain a first barrel body image according to the first local mask image and the first image.

[0058] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0059] Memory stores computer-executable instructions;

[0060] The processor executes the computer-executable instructions stored in the memory to implement the method of the present application.

[0061] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method of the present application.

[0062] The image processing method, apparatus, device and storage medium provided by the present application obtain a first mask image and a second mask image, wherein the first mask image is a mask image of a first image, and the second mask image is a mask image of a second image, and the first image and the second image are images of a target area on a barrel to be tested photographed at different shooting angles; according to the shooting angle of the first mask image, the second mask image is subjected to projection change processing to obtain a first comparison mask image, and the shooting angle of the first comparison mask image is different from that of the first mask image; according to the first comparison mask image, the first target pixel point in the first mask image is deleted to obtain a first target mask image, and the first target pixel point is a pixel in the first mask image that represents a foreign body in the barrel to be tested; the first target pixel point is extracted from the first target mask image. A local mask map, the first local mask map is determined according to the distribution of pixels in the first target mask map, and the first local mask map represents a local mask map of the non-contour line part in the target area; according to the first local mask map and the first image, a means of obtaining a first barrel body image can be achieved by obtaining images from other perspectives to determine the pixels in the first mask map that represent foreign matter in the barrel to be tested, and after deleting the pixels that represent foreign matter in the barrel to be tested, the obtained image can better reflect the non-contour line part in the target area on the barrel to be tested. Therefore, by comparing the first mask map and the first image, a clearer first barrel body image can be obtained, so as to improve the effect of foreign matter detection when using the first barrel body image to detect foreign matter in the barrel to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0064] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0065] Figure 2 A flowchart of another image processing method provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0068] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0070] In existing technologies, foreign matter detection of bottled liquids is particularly important in canning production lines for foods, pharmaceuticals, and other products. Currently, existing methods typically capture images of the bottled liquid and perform recognition and detection on them to obtain foreign matter detection results. However, this method is typically designed for smooth bottles, such as liquor, beer, and oral liquid bottles. For barrel-shaped containers like cooking oil drums, the numerous contour lines on their surfaces make it easy to misjudge contour lines when capturing images of the bottled liquid for recognition and detection. This results in inaccurate foreign matter detection results, reducing the effectiveness of foreign matter detection.

[0071] Currently, most companies have developed automated, intelligent equipment that can replace humans in online foreign body detection. However, these devices and methods are mostly designed for smooth bottles, such as those for liquor, beer, and oral liquids. Beverage and edible oil bottles often have complex, uneven designs to accommodate the effects of liquid pressure and increase tensile and compressive rigidity. This results in numerous shadow areas in the images captured by cameras.

[0072] In order to solve the above problems, the present application provides an image processing method, which can obtain a first mask image and a second mask image with different shooting angles, and then perform projection change processing on the second mask image with the shooting angle of the first mask image to obtain a first comparison mask image, so as to first delete the foreign matter part in the first mask image through the first comparison mask image to obtain a first target mask image, and then extract a first local mask image based on the distribution of pixels in the first target mask image, and finally modify the first image based on the first local mask image to obtain a first barrel body image that can be used for foreign matter detection.

[0073] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0074] The execution subject of the image processing method provided in the embodiment of the present application can be a server. The server can be a mobile phone, tablet, computer and other devices. This embodiment does not impose any special restrictions on the implementation method of the execution subject, as long as the execution subject can obtain the first mask image and the second mask image, the first mask image is the mask image of the first image, the second mask image is the mask image of the second image, and the first image and the second image are images of the target area on the barrel to be tested taken at different shooting angles; according to the shooting angle of the first mask image, the second mask image is subjected to projection change processing to obtain a first comparison mask image, and the shooting angle of the first comparison mask image is different from that of the first mask image; according to the first comparison mask image, the first target pixel point in the first mask image is deleted to obtain a first target mask image, and the first target pixel point is the pixel in the first mask image that represents the foreign matter in the barrel to be tested; the first local mask image is extracted from the first target mask image, the first local mask image is determined according to the distribution of the pixels in the first target mask image, and the first local mask image represents the local mask image of the non-contour line part in the target area; according to the first local mask image and the first image, the first barrel body image is obtained.

[0075] Among them, a mask is an image used in image processing and computer vision. A mask is a binary image in which pixel values are 0 or 1, indicating which pixels should be retained or ignored.

[0076] Figure 1 This is a flow chart of the image processing method provided in the embodiment of the present application. The execution subject of the method can be a server or other server, and this embodiment is not particularly limited here. Figure 1 As shown, the method may include:

[0077] S101. Obtain a first mask image and a second mask image, where the first mask image is a mask image of a first image, and the second mask image is a mask image of a second image. The first image and the second image are images taken of a target area on the barrel to be measured at different shooting angles.

[0078] The first mask image may be a mask image of a first image, and the first image may be an image captured by a first camera at a first shooting angle of view of a target area on the barrel to be tested.

[0079] The second mask image may be a mask image of a second image, and the second image may be an image captured by a second camera at a second shooting angle of view on a target area on the barrel to be tested.

[0080] In the embodiment of the present application, the first shooting perspective and the second shooting perspective being different may mean that the shooting positions of the first shooting perspective and the second shooting perspective are different, or the shooting angles of the first shooting perspective and the second shooting perspective are different.

[0081] The target area may refer to the area of the barrel body to be tested that is photographed. For example, the target area may be the front of the barrel body to be tested.

[0082] The barrel to be tested may refer to a cylindrical container with an uneven contour line on the barrel surface. For example, the barrel to be tested may be an edible oil barrel.

[0083] In the embodiment of the present application, the method of obtaining the first mask image and the second mask image may include:

[0084] Acquire a first initial image of the barrel to be measured taken at a first shooting angle of view, and a second initial image of the barrel to be measured taken at a second shooting angle of view;

[0085] Performing target detection processing on the target area in the first initial image to obtain a first image;

[0086] performing target detection processing on the target area in the second initial image to obtain a second image;

[0087] Performing semantic segmentation on the first image to obtain a first mask image;

[0088] Perform semantic segmentation on the second image to obtain a second mask image.

[0089] Object detection processing may refer to a method for determining an image of the barrel body to be tested from an initial image. In this embodiment of the present application, the object detection processing method may be a network detection method such as YOLO (You Only Look Once) or Fast-RCNN (Region-based Convolutional Neural Network, a two-stage object detection model). In this embodiment of the present application, the YOLO algorithm can be used to determine and extract a first image of the barrel body to be tested from the first initial image, and to determine and extract a second image of the barrel body to be tested from the second initial image.

[0090] Semantic segmentation is a computer vision task that aims to assign each pixel in an image to its corresponding semantic category. In this embodiment, semantic segmentation can be used to determine a first mask image and a second mask image, each of which represents pixels in the barrel body area to be tested as 1 and pixels in other areas as 0.

[0091] S102 : Perform projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image, where the shooting angle of the first comparison mask image is different from that of the first mask image.

[0092] Perspective Transformation is an image processing technology that can project a planar image onto a new perspective or plane.

[0093] Performing projection change processing on the second mask image according to the shooting angle of the first mask image may refer to a process of projecting the second mask image at the shooting angle of the first mask image to obtain the first comparison mask image.

[0094] In the embodiment of the present application, the method of performing projection change processing on the second mask image according to the shooting angle of the first mask image to obtain the first comparison mask image may include:

[0095] Determining a first world coordinate system, a second world coordinate system, and a transformation relationship between the first world coordinate system and the second world coordinate system, wherein the first world coordinate system is a world coordinate system constructed with the position of the first camera as the origin, the first camera being the camera that captures the first image; the second world coordinate system is a world coordinate system constructed with the position of the second camera as the origin, the second camera being the camera that captures the second image; and the transformation relationship between the first world coordinate system and the second world coordinate system is determined based on extrinsic parameters between the first camera and the second camera;

[0096] According to the transformation relationship between the first world coordinate system and the second world coordinate system, projection change processing is performed on the second mask image to obtain a first comparison mask image.

[0097] The first world coordinate system may be a coordinate system established with the origin of the first camera, and the x-axis of the first world coordinate system may be a position of the first camera facing the center of the barrel to be measured.

[0098] The second world coordinate system may be a coordinate system established with the origin of the second camera, and the x-axis of the first world coordinate system may be a position of the second camera facing the center of the barrel to be measured.

[0099] Extrinsic parameters refer to the position and orientation of a camera in the world coordinate system, also known as the camera's pose. They can be determined using rotation matrices and translation vectors.

[0100] The transformation relationship may refer to a transformation relationship that causes the first world coordinate system to coincide with the second world coordinate system through translation, rotation, etc. In some embodiments, the transformation relationship may be obtained through extrinsic parameters.

[0101] In the embodiment of the present application, the method of performing projection change processing on the second mask image according to the transformation relationship between the first world coordinate system and the second world coordinate system to obtain the first comparison mask image may include:

[0102] Obtain a second coordinate set of each pixel point on the second mask image in the second world coordinate system;

[0103] According to the second coordinate set and the transformation relationship between the first world coordinate system and the second world coordinate system, a projection change process is performed on the second mask image to obtain a first comparison mask image.

[0104] The second coordinate set may refer to a set of coordinates of the barrel body position represented by each pixel point on the second mask image within the second world coordinate system. In some embodiments, the coordinates of the pixel points within the second world coordinate system may be determined based on the positional relationship between the camera and the barrel body, as well as the barrel body data of the barrel body.

[0105] After the second coordinate set is determined, the second coordinate set can be projected into the first world coordinate system according to a transformation relationship between the first world coordinate system and the second world coordinate system, thereby obtaining a first comparison mask image.

[0106] In this embodiment of the present application, the method for obtaining the second coordinate set of each pixel point on the second mask image in the second world coordinate system may include:

[0107] Determine a second intrinsic parameter of the second camera and second position information of the barrel to be tested relative to the second camera;

[0108] A second coordinate set is determined according to the second intrinsic parameter and the second position information.

[0109] Intrinsic parameters refer to the inherent properties of the camera itself, and may include focal length, principal point position, and distortion. Focal length refers to the distance from the camera lens to the imaging plane; principal point position refers to the optical center on the imaging plane, or the center point of the image; distortion parameters refer to the image distortion caused by factors such as the shape and position of the optical lens during imaging, and may include radial distortion and tangential distortion; and pixel size refers to the distance between two adjacent pixels on the imaging plane.

[0110] The position information may include the distance information from the optical center of the camera to the center of the barrel to be measured, and the height information from the optical center of the camera to the bottom plane of the barrel to be measured, wherein the optical center of the camera may be the optical center of the camera, that is, the imaging center of the camera, wherein the optical center of the camera is located on the optical axis of the camera, that is, the main optical axis of the camera.

[0111] Among them, according to the internal parameters and position information, the method for determining the coordinate set can be to use the optical center of the camera as the origin, and determine the coordinates of the point to be measured according to the radius of the barrel to be measured, the depth of the contour line, and the distance between the point to be measured and the optical center of the camera. Thus, by repeatedly determining the coordinates of the point to be measured, a coordinate set can be obtained.

[0112] S103 . Deleting a first target pixel in the first mask image according to the first comparison mask image to obtain a first target mask image. The first target pixel is a pixel in the first mask image that represents a foreign object in the barrel to be detected.

[0113] The deletion process may be a process of deleting the first target pixel point in the first mask image.

[0114] The first target pixel is a pixel in the first mask image that is characterized as a foreign object on the surface of the barrel to be tested. For example, when there is a foreign object in the barrel to be tested, the first target pixel may be a pixel of the foreign object in the first mask image.

[0115] The first target mask image may be a first mask image from which pixels of foreign matter in the barrel to be detected are deleted.

[0116] In an embodiment of the present application, since the shooting angles of the first mask image and the second mask image are different, the positions of the foreign matter in the barrel to be tested in the first mask image and the second mask image are also different. Therefore, after the second mask image is projected onto the first mask image, the positions of the pixels represented by the foreign matter in the first world coordinate system and the second world coordinate system are different. Therefore, the first target pixel point representing the foreign matter in the first mask image can be determined through the first comparison mask image, and deleted to obtain the first target mask image.

[0117] In the embodiment of the present application, the method of deleting the first target pixel in the first mask image according to the first comparison mask image to obtain the first target mask image may include:

[0118] Comparing the position correspondence between the pixel points on the first comparison mask image and the pixel points on the first mask image;

[0119] If there is a first target pixel point with no corresponding position, the first target pixel point in the first mask image is deleted to obtain a first target mask image.

[0120] The position correspondence relationship may refer to whether the positions of the pixels on the first comparison mask image and the pixels on the first mask image correspond in the same world coordinate system. The position correspondence relationship may include a position correspondence relationship that characterizes the correspondence of the pixels, or a position correspondence relationship that characterizes the non-correspondence of the pixels, wherein the position correspondence relationship that characterizes the correspondence of the pixels may refer to that the correspondence relationship of the pixels in the first mask image and the second mask image conforms to the conversion relationship between the first world coordinate system and the second world coordinate system, and the position correspondence relationship that characterizes the non-correspondence of the pixels may refer to that the correspondence relationship of the pixels in the first mask image and the second mask image does not conform to the conversion relationship between the first world coordinate system and the second world coordinate system.

[0121] S104 , extracting a first local mask image from the first target mask image, where the first local mask image is determined based on the distribution of pixels in the first target mask image, and the first local mask image represents a local mask image of a non-contour line portion in the target area.

[0122] Among them, the first local mask image may refer to a local mask image that represents the non-contour line portion of the target area in the first target mask image. In some embodiments, the barrel body to be tested in the first target mask image may include a contour line portion and a non-contour line portion, wherein the contour line portion may refer to the raised or concave line portion on the surface of the barrel body to be tested, and the non-contour line portion may refer to the smooth surface portion of the barrel body to be tested.

[0123] In this embodiment of the present application, the method of extracting the first local mask image from the first target mask image may include:

[0124] Determine the position of each pixel in the first target mask image;

[0125] Determining a contour line portion in the first target mask image according to positions of each pixel point in the first target mask image;

[0126] The contour line portion is deleted to obtain a first local mask image.

[0127] The positions of the pixels in the first target mask image may refer to the coordinate positions of the pixels in the first target mask image. By determining the coordinate positions of the pixels in the first target mask image, the distribution of the pixels can be determined, and the contour line portion in the first target mask image can be determined based on the distribution.

[0128] In the embodiment of the present application, the method for determining the contour line portion in the first target mask image according to the position of each pixel point in the first target mask image may include:

[0129] Determining the number of pixels along the y-axis in the first target mask image according to the positions of the pixels in the first target mask image;

[0130] According to the number of pixels along the y-axis direction in the first target mask image, a contour line portion in the first target mask image is determined.

[0131] Among them, by determining the number of pixels along the y-axis direction in the first target mask image, the edge part and the contour line part of the barrel body in the first target mask image can be determined. For example, when there are more pixels distributed along the y-axis direction, it can be determined that the part is the edge part of the barrel body, and by setting the edge line, the contour line part can be determined.

[0132] In the embodiment of the present application, the method for determining the contour line portion in the first target mask image according to the number of pixels along the y-axis direction in the first target mask image may include:

[0133] Determine the pixel points in the barrel surface sub-region according to the number of pixel points along the y-axis in the first target mask image;

[0134] Clustering is performed on the pixel points in the barrel body surface sub-region to determine the contour line portion in the first target mask image.

[0135] Among them, determining the pixel points in the sub-area of the barrel body surface may refer to determining the pixel points in the barrel body edge area and the pixel points in the barrel body surface sub-area according to the distribution of each pixel point.

[0136] Clustering can refer to dividing a set of pixels into different classes or clusters according to a specific standard, so that the similarity of data objects in the same cluster is as large as possible.

[0137] In some embodiments, the first target mask image can be split in half, and the number and distribution of pixels along the y-axis of the image can be counted. Based on the number and distribution of pixels, two histogram curves are generated. After obtaining the histogram curves, the vertex with the maximum value on the curve can be determined to locate the surface subregion in the barrel to be tested. Finally, by setting a threshold, the y coordinates of all pixels exceeding this threshold can be clustered to obtain the contour line portion.

[0138] S105: Determine a first barrel body image according to the first local mask image and the first image.

[0139] The first barrel body image may refer to an image extracted from the first image based on the first local mask image.

[0140] In the embodiment of the present application, determining the first barrel body image according to the first local mask image and the first image includes:

[0141] Extracting a first initial barrel body image from the first image according to the first local mask image;

[0142] Performing shape transformation processing on the first initial barrel body image to obtain a first barrel body image.

[0143] The first initial barrel body image may refer to extracting the first initial barrel body image from the first image according to the first local mask image.

[0144] Shape transformation processing can refer to transforming a distorted image into a regular first barrel image. In this embodiment of the present application, the first image may be tilted. To facilitate subsequent image quality inspection, the first initial barrel image can be subjected to shape transformation processing to obtain a regular first barrel image. In this embodiment of the present application, after obtaining the first barrel image, the barrel under test can be directly inspected using the first barrel image to obtain an inspection result.

[0145] The image processing method provided in the embodiment of the present application can determine the pixels in the first mask image that are characterized as foreign matter in the barrel to be tested by acquiring images from other perspectives, and after deleting the pixels that are characterized as foreign matter in the barrel to be tested, the obtained image can better reflect the non-contour line part in the target area on the barrel to be tested. Therefore, by comparing the first mask image and the first image, a clearer first barrel body image can be obtained, so as to improve the effect of foreign matter detection when using the first barrel body image to detect foreign matter in the barrel to be tested.

[0146] In this embodiment of the present application, the image processing method may further include:

[0147] Performing projection change processing on the first target mask image according to the shooting angle of the second mask image to obtain a second comparison mask image;

[0148] Deleting a second target pixel in the second mask image according to the second comparison mask image to obtain a second target mask image, where the second target pixel is a pixel of the foreign object in the second mask image;

[0149] Extracting a second local mask image from the second target mask image, where the second local mask image is a local mask image representing a non-contour line portion of the target area in the second target mask image;

[0150] Obtain a second barrel body image according to the second local mask image and the second image;

[0151] A target barrel body image is obtained according to the first barrel body image and the second barrel body image.

[0152] Among them, in order to avoid the contour area occluding foreign matter in the barrel to be tested, the target barrel body image can be determined by splicing the first image and the second image taken from different shooting angles, thereby reducing the occlusion blind area of the contour area as much as possible. In this way, the quality of the target barrel body image can be improved to perform foreign matter inspection on the barrel to be tested.

[0153] The target barrel body image may be a multi-view image of the barrel to be tested.

[0154] In this embodiment of the present application, the method for obtaining the target barrel body image according to the first barrel body image and the second barrel body image may include:

[0155] Determining a correspondence between pixels in the first mask image and pixels in the second mask image;

[0156] According to the corresponding relationship, the first barrel body image and the second barrel body image are merged to obtain a target barrel body image.

[0157] The correspondence relationship may refer to the correspondence relationship between pixels in the first mask image and pixels in the second mask image having the same position. The correspondence relationship can be used to determine the common parts of the first barrel body image and the second barrel body image, and merge them based on the common parts to obtain the target barrel body image.

[0158] The merging process may refer to a process of merging the first barrel body image and the second barrel body image into a target barrel body image.

[0159] The image processing method provided in the embodiment of the present application can obtain the target barrel body image of the barrel to be tested from more perspectives by obtaining the first barrel body image and the second barrel body image. In this way, the area of the barrel body blocked by the contour line part on the barrel to be tested can be avoided, thereby improving the effect of subsequent foreign matter detection.

[0160] Figure 2 This is a flow chart of another image processing method provided in an embodiment of the present application. The execution subject of this method can be a server or other server, and this embodiment does not make any special restrictions here, such as Figure 2 As shown, the method may include:

[0161] S201: Acquire a first-perspective image, a second-perspective image, and a third-perspective image of a cooking oil barrel.

[0162] The first perspective image, the second perspective image and the third perspective image may be images taken of the cooking oil barrel by three cameras at different positions, wherein the three cameras at different positions may be the first camera, the second camera and the third camera respectively.

[0163] S202 : Perform target detection on the first perspective image, the second perspective image, and the third perspective image respectively to obtain a first target image, a second target image, and a third target image.

[0164] Among them, the image rectangular area of the edible oil bottle in the first perspective image, the second perspective image and the third perspective image is detected through image target detection networks, such as YOLO, Fast-RCNN and other networks, so as to obtain the first target image, the second target image and the third target image.

[0165] S203 , performing semantic segmentation on the first target image, the second target image, and the third target image respectively to obtain a first mask image, a second mask image, and a third mask image.

[0166] Among them, the segmentation model can be trained through the SegFormer (Transformer-based semantic segmentation model) and SegNet (deep convolutional neural network model for semantic segmentation) segmentation networks, and then the first target image, the second target image and the third target image are respectively input into the segmentation model to determine the connected areas of the edible oil barrel body in the target image to obtain the first barrel body target image, the second barrel body target image and the third barrel body target image. According to the threshold segmentation algorithm, a pixel threshold is set to binarize the first barrel body target image, the second barrel body target image and the third barrel body target image respectively to obtain the first mask image, the second mask image and the third mask image, wherein the pixel value representing the edible oil barrel body in the first mask image, the second mask image and the third mask image is 1, and the pixel value of other positions is 0.

[0167] S204 , respectively deleting pixels representing foreign matter inside the oil barrel from the first mask image, the second mask image, and the third mask image to obtain a first corrected mask image, a second corrected mask image, and a third corrected mask image.

[0168] The method for deleting the pixels representing the foreign matter inside the oil barrel in the first mask image, the second mask image, and the third mask image to obtain the first corrected mask image, the second corrected mask image, and the third corrected mask image may be:

[0169] 1. The first intrinsic parameter M1 of the first camera, the second intrinsic parameter M2 of the second camera, the third intrinsic parameter M3 of the third camera, the first extrinsic parameter Q1 between the first and second cameras, and the second extrinsic parameter Q2 between the second and third cameras are obtained through the camera calibration method using a 2D plane standard target.

[0170] 2. Use the 3D standard target camera calibration method to obtain the distance L between the optical center of the second camera and the center of the cooking oil barrel, as well as the height H between the optical center of the second camera and the bottom plane of the cooking oil barrel.

[0171] 3. Obtain the 3D coordinate value of the cooking oil barrel in the second mask image in the second world coordinate system of the second camera.

[0172] Among them, we can first assume that the image coordinates of a point on the edible oil barrel in the second mask image are (u0, v0), and then assume that the optical center of the first camera is the origin O, P is a point on the edible oil barrel, r is the radius of the edible oil barrel, D is the depth of the undulating surface of the barrel, and the X coordinate value of point P on the camera is u0. Through the formula: X0=(u0-cx) / fx、Y0=(v0-cy) / fy、Z0=L-sqrt((rD) 2 -X0 2 ) The spatial coordinates (X0, Y0, Z0) of the point (u0, v0) in the second world coordinate system can be determined, where cx and fx are the principal point pixel and focal length on the x-axis in the second intrinsic parameter M2, and cy and fy are the principal point pixel and focal length on the y-axis in the second intrinsic parameter M2.

[0173] 4. Obtain the 2D projection coordinate value of the pixel point in the second mask image in the first coordinate system of the first camera and the 2D projection coordinate value in the third coordinate system of the third camera.

[0174] According to the first intrinsic parameter M1 and the third intrinsic parameter M3, as well as the first extrinsic parameter Q1 and the second extrinsic parameter Q2, the coordinate information [ui, vi, Xi, Yi, Zi] of the pixel point in the second mask image is mapped to the first coordinate system and the third coordinate system respectively, and the first comparison projection image and the third comparison projection image are obtained. In the embodiment of the present application, the formula can be used:

[0175] λ[ui, vi, 1] =M1Q1[Xi, Yi, Zi, 1];

[0176] λ[ui,vi,1] =M3Q2[Xi,Yi,Zi,1];

[0177] A first comparison projection image and a third comparison projection image are obtained, where λ is a preset coefficient.

[0178] 5. Determine the pixels representing the foreign matter inside the oil barrel in the first mask image, the second mask image, and the third mask image.

[0179] Among them, by comparing the intersection of the pixel points in the first comparison projection image and the first mask image, and comparing the intersection of the pixel points in the third comparison projection image and the third mask image, the pixel points in the first mask image that are not repeated and represent the foreign matter pixels inside the oil barrel, as well as the pixel points in the third mask image that are not repeated and represent the foreign matter pixels inside the oil barrel can be determined.

[0180] At the same time, the first and third comparison projection images can be reflected and projected into the second world coordinate system through an inverse transformation to obtain a second comparison projection image. By comparing the intersection of the pixels in the second comparison projection image and the second mask image, the pixels in the second mask image that are not repeated and represent the foreign matter inside the oil drum can be determined.

[0181] 6. Pixels representing foreign matter inside the oil drum are deleted from the first mask image, the second mask image, and the third mask image to obtain a first corrected mask image, a second corrected mask image, and a third corrected mask image.

[0182] S205, extracting a first partial mask image representing the smooth surface of the oil drum from the first corrected mask image, a second partial mask image representing the smooth surface of the oil drum from the second corrected mask image, and a third partial mask image representing the smooth surface of the oil drum from the third corrected mask image;

[0183] S206. Extract a first smooth barrel body image from the first target image according to the first local mask image, extract a second smooth barrel body image from the second target image according to the second local mask image, and extract a third smooth barrel body image from the third target image according to the third local mask image.

[0184] When extracting the first local mask representing the smooth surface of the oil drum from the first corrected mask, the first corrected mask can be split in half, and the number of pixels along the y-axis can be counted. This yields two histogram curves. The regions of the oil drum can be determined based on the vertices of the maximum values on the curves. A threshold t is set, and the y-axis coordinates of all pixels exceeding the threshold t can be clustered, thereby obtaining the upper and lower y-axis boundaries of N regions.

[0185] Among them, the threshold t1=[y1min i , y1max i ]; threshold t2=[y2min i , 2ymax i ], i∈[1,N].

[0186] According to t1 and t2, the quadrilateral area B to be deleted in the first modified mask image is determined i = [(0, y1min i ), (w, y2min i ), (w, y2max i ), (0,y1max i )], i∈[1, N], w is the width of the first modified mask image, by dividing the quadrilateral area B i By cutting out, a first local mask image can be obtained, and a second local mask image and a third local mask image can also be obtained from this.

[0187] Since the first partial mask image left after the removal is a plurality of irregular trapezoids A i , so for the convenience of subsequent synthesis, multiple irregular quadrilaterals A i They are adjusted to highly uniform rectangles through trilinear interpolation. Among them, the quadrilateral area B can be calculated first i Height information S ih =[t1.y1max i -t1,y1min i , t2.y1max i -t2.y1min i ], i∈[1, N]; then calculate the irregular quadrilateral A i A 1h =[t1.y1min1,t2.y1min1];A ih =[t1.y1min i -t1.y1max i -1, t2.y1min i -t2.y1max i -1], i∈[2,N]; A N+1h=[h-t1.y1max N , h-t2.y1max N ].

[0188] Among them, h can be the original image height, A i The maximum value of the height is used as the generated rectangle C i , i∈[1, N+1], and keep the width w consistent, then we can get the height information C of the synthesized image ih =max(A ih ), i∈[1,N+1].

[0189] Thus, the modified first local mask map can be obtained through the image affine transformation operation, and based on this principle, the modified second local mask map and the third local mask map can be obtained.

[0190] By obtaining the corrected first local mask image, second local mask image and third local mask image, the first smooth barrel body image, the second smooth barrel body image and the third smooth barrel body image can be extracted from the first target image, the second target image and the third target image respectively.

[0191] S207: Merge the first smooth barrel body image, the second smooth barrel body image, and the third smooth barrel body image to obtain a barrel body composite image for foreign body detection.

[0192] Among them, the first smooth barrel body image, the second smooth barrel body image, and the third smooth barrel body image can be merged in a positional relationship by determining the overlapping pixel points on the first local mask image, the second local mask image, and the third local mask image, thereby obtaining a barrel body composite image for foreign object detection. For example, the first smooth barrel body image is composed of three first smooth barrel body sub-images, the second smooth barrel body image is composed of three second smooth barrel body images, and the third smooth barrel body image is composed of three third smooth barrel body images. By determining the positional relationship between the three first smooth barrel body images, the three second smooth barrel body images, and the three third smooth barrel body images, a barrel body composite image for foreign object detection is synthesized.

[0193] After obtaining the composite image of the barrel body, foreign objects can be extracted from the composite image of the barrel body through methods such as Yolo, Fast-RCNN (Fast-Region-based Convolutional Neural Network, target detection algorithm), VGG (Visual Geometry Group, deep convolutional neural network model), Segformer, PatchCore, CFLOW-AD, etc., thereby completing foreign object detection.

[0194] Another image processing method provided in an embodiment of the present application can use three cameras to photograph the oil bottle from different angles, thereby forming three images of the bottle body photographed from three positions and angles. Based on the geometric constraint relationship between the three cameras, the shadow area on the oil barrel body is filtered out, and the effective detection area inside the oil barrel body is retained to a large extent. The effective detection area can be the concave and convex area of the bottle wall or the shadow caused by the scratch area, thereby improving the detection effect of the oil barrel body.

[0195] Figure 3 This is a schematic diagram of the structure of the image processing device provided in the embodiment of the present application. Figure 3 As shown, the image processing device 30 includes: an acquisition module 301, a projection change processing module 302, a deletion processing module 303, an extraction module 304 and a obtaining module 305.

[0196] An acquisition module 301 is configured to acquire a first mask image and a second mask image, wherein the first mask image is a mask image of the first image, and the second mask image is a mask image of the second image, wherein the first image and the second image are images of the target area on the barrel to be measured taken at different shooting angles;

[0197] The projection change processing module 302 is configured to perform projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image, where the shooting angle of the first comparison mask image is different from that of the first mask image.

[0198] The deletion processing module 303 is configured to delete the first target pixel in the first mask image according to the first comparison mask image to obtain a first target mask image, where the first target pixel is a pixel in the first mask image that indicates a foreign object in the barrel to be detected;

[0199] An extraction module 304 is configured to extract a first local mask image from the first target mask image, where the first local mask image is determined based on the distribution of pixels in the first target mask image, and the first local mask image represents a local mask image of a non-contour portion of the target area.

[0200] The obtaining module 305 is used to obtain a first barrel body image according to the first local mask image and the first image.

[0201] In the embodiment of the present application, the acquisition module 301 may also be specifically used to:

[0202] Acquire a first initial image of the barrel to be measured taken at a first shooting angle of view, and a second initial image of the barrel to be measured taken at a second shooting angle of view;

[0203] Performing target detection processing on the target area in the first initial image to obtain a first image;

[0204] performing target detection processing on the target area in the second initial image to obtain a second image;

[0205] Performing semantic segmentation on the first image to obtain a first mask image;

[0206] Perform semantic segmentation on the second image to obtain a second mask image.

[0207] In the embodiment of the present application, the projection change processing module 302 may also be specifically configured to:

[0208] Determining a first world coordinate system, a second world coordinate system, and a transformation relationship between the first world coordinate system and the second world coordinate system, wherein the first world coordinate system is a world coordinate system constructed with the position of the first camera as the origin, the first camera being the camera that captures the first image; the second world coordinate system is a world coordinate system constructed with the position of the second camera as the origin, the second camera being the camera that captures the second image; and the transformation relationship between the first world coordinate system and the second world coordinate system is determined based on extrinsic parameters between the first camera and the second camera;

[0209] According to the transformation relationship between the first world coordinate system and the second world coordinate system, projection change processing is performed on the second mask image to obtain a first comparison mask image.

[0210] In the embodiment of the present application, the projection change processing module 302 may also be specifically configured to:

[0211] Obtain a second coordinate set of each pixel point on the second mask image in the second world coordinate system;

[0212] According to the second coordinate set and the transformation relationship between the first world coordinate system and the second world coordinate system, a projection change process is performed on the second mask image to obtain a first comparison mask image.

[0213] In the embodiment of the present application, the projection change processing module 302 may also be specifically configured to:

[0214] Determine a second intrinsic parameter of the second camera and second position information of the barrel to be tested relative to the second camera;

[0215] A second coordinate set is determined according to the second intrinsic parameter and the second position information.

[0216] In the embodiment of the present application, the deletion processing module 303 may also be specifically used to:

[0217] Comparing the position correspondence between the pixel points on the first comparison mask image and the pixel points on the first mask image;

[0218] If there is a first target pixel point with no corresponding position, the first target pixel point in the first mask image is deleted to obtain a first target mask image.

[0219] In the embodiment of the present application, the extraction module 304 may also be specifically used to:

[0220] Determine the position of each pixel in the first target mask image;

[0221] Determining a contour line portion in the first target mask image according to positions of each pixel point in the first target mask image;

[0222] The contour line portion is deleted to obtain a first local mask image.

[0223] In the embodiment of the present application, the extraction module 304 may also be specifically used to:

[0224] Determining the number of pixels along the y-axis in the first target mask image according to the positions of the pixels in the first target mask image;

[0225] According to the number of pixels along the y-axis direction in the first target mask image, a contour line portion in the first target mask image is determined.

[0226] In the embodiment of the present application, the extraction module 304 may also be specifically used to:

[0227] Determine the pixel points in the barrel surface sub-region according to the number of pixel points along the y-axis in the first target mask image;

[0228] Clustering is performed on the pixel points in the barrel body surface sub-region to determine the contour line portion in the first target mask image.

[0229] In the embodiment of the present application, the obtaining module 305 may also be specifically used for:

[0230] Extracting a first initial barrel body image from the first image according to the first local mask image;

[0231] Performing shape transformation processing on the first initial barrel body image to obtain a first barrel body image.

[0232] In the embodiment of the present application, the obtaining module 305 may also be specifically used for:

[0233] Performing projection change processing on the first target mask image according to the shooting angle of the second mask image to obtain a second comparison mask image;

[0234] Deleting a second target pixel in the second mask image according to the second comparison mask image to obtain a second target mask image, where the second target pixel is a pixel of the foreign object in the second mask image;

[0235] Extracting a second local mask image from the second target mask image, where the second local mask image is a local mask image representing a non-contour line portion of the target area in the second target mask image;

[0236] Obtain a second barrel body image according to the second local mask image and the second image;

[0237] A target barrel body image is obtained according to the first barrel body image and the second barrel body image.

[0238] In the embodiment of the present application, the obtaining module 305 may also be specifically used for:

[0239] Determining a correspondence between pixels in the first mask image and pixels in the second mask image;

[0240] According to the corresponding relationship, the first barrel body image and the second barrel body image are merged to obtain a target barrel body image.

[0241] The image processing method provided by the present application comprises an acquisition module 301 for acquiring a first mask image and a second mask image, wherein the first mask image is a mask image of a first image, and the second mask image is a mask image of a second image, and the first image and the second image are images of a target area on a barrel to be tested photographed at different shooting angles; a projection change processing module 302 for performing projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image, wherein the shooting angle of the first comparison mask image is different from that of the first mask image; and a deletion processing module 303 for Based on the first comparison mask image, the first target pixel in the first mask image is deleted to obtain a first target mask image. The first target pixel is the pixel in the first mask image that represents the foreign object in the barrel to be detected. An extraction module 304 is used to extract a first local mask image from the first target mask image. The first local mask image is determined based on the distribution of pixels in the first target mask image and represents the local mask image of the non-contour portion of the target area. A obtaining module 305 is used to obtain a first barrel body image based on the first local mask image and the first image. This improves the effectiveness of foreign object detection.

[0242] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 4 As shown, the electronic device 40 includes:

[0243] The electronic device 40 may include one or more processors 401 , one or more computer-readable storage media memories 402 , a communication component 403 , and other components. The processor 401 , the memory 402 , and the communication component 403 are connected via a bus 404 .

[0244] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402 , so that the at least one processor 401 performs the above image processing method.

[0245] The specific implementation process of the processor 401 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0246] In the above Figure 4 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0247] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0248] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0249] In some embodiments, a computer program product is further provided, comprising a computer program or instructions, which implement the steps in any one of the above-mentioned image processing methods when executed by a processor.

[0250] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0251] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0252] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions. The instructions can be loaded by a processor to execute the steps of any image processing method provided in the embodiment of the present application.

[0253] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0254] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0255] Since the instructions stored in the storage medium can execute the steps in any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0256] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0257] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Obtaining a first mask image and a second mask image, wherein the first mask image is a mask image of the first image, and the second mask image is a mask image of the second image, and the first image and the second image are images captured at different shooting angles of view of the target area on the barrel to be measured; performing projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image, wherein the shooting angle of the first comparison mask image is different from that of the first mask image; Deleting a first target pixel in the first mask image according to the first comparison mask image to obtain a first target mask image, where the first target pixel is a pixel in the first mask image that indicates a foreign object in the barrel to be detected; Extracting a first local mask image from the first target mask image, where the first local mask image is determined based on a distribution of pixels in the first target mask image, and the first local mask image represents a local mask image of a non-contour line portion in the target area; A first barrel body image is obtained according to the first local mask image and the first image.

2. The method according to claim 1, characterized in that The obtaining of the first mask image and the second mask image includes: Acquire a first initial image of the barrel to be tested taken at a first shooting angle of view, and a second initial image of the barrel to be tested taken at a second shooting angle of view; performing target detection processing on the target area in the first initial image to obtain the first image; performing target detection processing on the target area in the second initial image to obtain the second image; Performing semantic segmentation on the first image to obtain a first mask image; Perform semantic segmentation on the second image to obtain a second mask image.

3. The method according to claim 1, characterized in that The step of performing projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image includes: Determining a first world coordinate system, a second world coordinate system, and a transformation relationship between the first world coordinate system and the second world coordinate system, wherein the first world coordinate system is a world coordinate system constructed with a position of a first camera as an origin, the first camera is a camera that captures the first image, the second world coordinate system is a world coordinate system constructed with a position of a second camera as an origin, the second camera is a camera that captures the second image, and the transformation relationship between the first world coordinate system and the second world coordinate system is determined based on extrinsic parameters between the first camera and the second camera; According to the transformation relationship between the first world coordinate system and the second world coordinate system, projection change processing is performed on the second mask image to obtain a first comparison mask image.

4. The method according to claim 3, characterized in that The step of performing projection change processing on the second mask image according to the transformation relationship between the first world coordinate system and the second world coordinate system to obtain a first comparison mask image includes: Obtaining a second coordinate set of each pixel point on the second mask image in the second world coordinate system; According to the second coordinate set and the transformation relationship between the first world coordinate system and the second world coordinate system, projection change processing is performed on the second mask image to obtain a first comparison mask image.

5. The method according to claim 4, characterized in that The obtaining of a second coordinate set of each pixel point on the second mask image in the second world coordinate system includes: Determining second intrinsic parameters of the second camera and second position information of the barrel to be tested relative to the second camera; The second coordinate set is determined according to the second intrinsic parameter and the second position information.

6. The method according to claim 1, characterized in that The deleting the first target pixel in the first mask image according to the first comparison mask image to obtain the first target mask image includes: Comparing the position correspondence between the pixel points on the first comparison mask image and the pixel points on the first mask image; If there is a first target pixel point with no corresponding position, the first target pixel point in the first mask image is deleted to obtain a first target mask image.

7. The method according to claim 1, characterized in that The extracting a first local mask image from the first target mask image includes: Determining the position of each pixel in the first target mask image; determining a contour line portion in the first target mask image according to positions of respective pixel points in the first target mask image; The contour line portion is deleted to obtain a first local mask image.

8. The method according to claim 7, characterized in that The step of determining the contour line portion in the first target mask image according to the position of each pixel point in the first target mask image includes: Determining the number of pixels along the y-axis in the first target mask image according to the position of each pixel in the first target mask image; A contour line portion in the first target mask image is determined according to the number of pixels along the y-axis direction in the first target mask image.

9. The method according to claim 8, characterized in that The step of determining the contour line portion in the first target mask image according to the number of pixels along the y-axis in the first target mask image includes: Determine the pixel points in the barrel surface sub-region according to the number of pixel points along the y-axis in the first target mask image; Clustering is performed on the pixel points in the barrel body surface sub-region to determine the contour line portion in the first target mask image.

10. The method according to claim 1, characterized in that The determining of the first barrel body image according to the first local mask image and the first image includes: Extracting a first initial barrel body image from the first image according to the first local mask image; Performing shape transformation processing on the first initial barrel body image to obtain a first barrel body image.

11. The method according to claim 1, wherein The method further comprises: performing projection change processing on the first target mask image according to the shooting angle of the second mask image to obtain a second comparison mask image; Deleting a second target pixel in the second mask image according to the second comparison mask image to obtain a second target mask image, where the second target pixel is a pixel of the foreign object in the second mask image; Extracting a second local mask image from the second target mask image, where the second local mask image is a local mask image representing a non-contour line portion of the target area in the second target mask image; Obtaining a second barrel body image according to the second local mask image and the second image; A target barrel body image is obtained according to the first barrel body image and the second barrel body image.

12. The method according to claim 11, characterized in that Obtaining a target barrel body image according to the first barrel body image and the second barrel body image includes: Determining a correspondence between pixels in the first mask image and pixels in the second mask image; According to the corresponding relationship, the first barrel body image and the second barrel body image are merged to obtain a target barrel body image.

13. An image processing device, characterized in that: include: An acquisition module is configured to acquire a first mask image and a second mask image, wherein the first mask image is a mask image of a first image, and the second mask image is a mask image of a second image, wherein the first image and the second image are images captured at different shooting angles of view of a target area on the barrel to be measured; a projection change processing module, configured to perform projection change processing on the second mask image according to the shooting angle of the first mask image to obtain a first comparison mask image, wherein the shooting angle of the first comparison mask image is different from that of the first mask image; a deletion processing module, configured to delete a first target pixel in the first mask image according to the first comparison mask image to obtain a first target mask image, wherein the first target pixel is a pixel in the first mask image that represents a foreign object in the barrel to be detected; an extraction module, configured to extract a first local mask image from the first target mask image, wherein the first local mask image is determined based on a distribution of pixels in the first target mask image, and the first local mask image represents a local mask image of a non-contour portion of the target area; An obtaining module is used to obtain a first barrel body image based on the first local mask image and the first image.

14. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 12 when executed by a processor.

Images