A method and apparatus for detecting a target feature of an arc region

Abstract

A target feature detection method of an arc-shaped region comprises the following steps: acquiring an image of an arc-shaped region as a region to be detected and converting the image into a rectangular region, and performing target feature detection based on the image of the rectangular region; the target feature detection comprises the following steps: acquiring a two-dimensional image and a three-dimensional image of a target to be detected, wherein the two-dimensional image comprises texture information of the target to be detected containing grayscale information, and the three-dimensional image comprises topographic information of the target to be detected containing height information; performing feature coding on the two-dimensional image and the three-dimensional image respectively; performing multi-scale feature extraction processing on the feature coding to obtain corresponding feature maps; performing feature fusion operation on the feature maps as input feature maps to obtain a final feature map; and obtaining a final result through convolution operation and an activation function on the final feature map. Both the texture information and the topographic information are considered, and the height information of the periphery is considered for comprehensive judgment and analysis in defect detection, so that the false detection rate and the missed detection rate are reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and device for detecting target features in an arc-shaped region within the field of image processing. Background Technology

[0002] With the popularization and promotion of new energy vehicles, users' demands for vehicle quality and performance are constantly increasing. Lithium batteries, as a crucial component of new energy vehicles, play a vital role in safety and range, among other aspects. With the rapid development of machine vision technology, intelligent and unmanned inspection has become the goal pursued by many automated production lines and factories. During the production process of automotive lithium batteries, even minor defects can affect the overall performance and quality of the battery; therefore, quality control during lithium battery production is particularly critical.

[0003] After the sealing nails for lithium batteries are welded, the weld quality of the annular area of ​​the sealing nail needs to be inspected. Traditional inspection methods generally involve visual inspection or manual touch inspection. This method cannot achieve completely standardized results, and the inspection time and cost are relatively high. During the inspection process, visual fatigue can lead to false detections, and manual touch inspection can leave dirt on the sealing nail surface, potentially increasing the risk of corrosion. Furthermore, with increasingly stringent process requirements, it is necessary to detect even smaller defects, such as pits and weld bursts smaller than 0.1mm. These defects cannot be detected by visual inspection or manual touch inspection alone. Therefore, reducing the false detection rate, improving inspection efficiency, and enhancing defect detection capabilities have become urgent technical problems to be solved in the current sealing nail manufacturing process.

[0004] Using artificial eyes and / or touch to inspect the weld quality of the annular area of ​​a sealing nail often results in a high error rate, and different people have different perceptions of defects, which can lead to biased results. Currently, more and more people are using 2D or 3D cameras to capture images and perform defect detection based on the image information. However, defect detection using 2D camera images is based on texture information, which cannot obtain height difference information about the defects. To improve detection capabilities, 3D cameras are being used to obtain the object's shape information for defect detection. This method only utilizes the object's shape information and does not utilize its texture information. If 3D height images are used, plane fitting is mainly performed using point cloud information. The distance from points within the detection area of ​​the sealing nail to the fitted plane is calculated. If the distance from a point to the plane is greater than a certain threshold, it is considered a candidate defect region. Connectivity analysis is performed on the candidate defect regions, and information such as roundness, area, needle-likeness, length, and width are used to filter and obtain the final defect region. This method is highly sensitive to noise, only considers local information about the shape, and does not take into account the surrounding height information for comprehensive judgment and analysis. Therefore, it has a high false detection rate and false negative rate. Summary of the Invention

[0005] This application provides a target feature detection method and detection device for arc-shaped regions, which has the characteristics of low false detection rate and low false negative rate when detecting target features in arc-shaped regions.

[0006] According to the first aspect, one embodiment provides a target feature detection method for an arc-shaped region, including...

[0007] Obtain the arc-shaped region to be detected;

[0008] Convert the image of the arc-shaped region to be detected into an image of a rectangular region; and,

[0009] Target feature detection based on images with rectangular regions;

[0010] The target feature detection based on the image of the rectangular region includes:

[0011] Acquire two-dimensional and three-dimensional images of the object to be detected. The two-dimensional image includes texture information containing grayscale information of the image to be detected, and the three-dimensional image includes shape information containing height information of the image to be detected.

[0012] Feature mapping α1 is obtained by feature encoding of a two-dimensional image, and feature mapping α2 is obtained by feature encoding of a three-dimensional image.

[0013] After performing multi-scale feature extraction on feature map α1, feature map η1 is obtained; after performing multi-scale feature extraction on feature map α2, feature map η2 is obtained.

[0014] The final feature map is obtained by performing feature fusion operations on feature maps η1 and η2 as input feature maps; and...

[0015] The final feature map is obtained by performing a 1×1 convolution operation and a sigmoid activation function.

[0016] In one embodiment, target feature detection based on an image of a rectangular region further includes:

[0017] Position enhancement coding is performed on feature maps η1 and η2. The position enhancement coding formula can be expressed as:

[0018]

[0019] Where x and y are the coordinate information parallel to the X-axis and Y-axis, respectively, D refers to the number of channels in the feature map, p is the channel index information of feature map η1, and q is the channel index information of feature map η2. According to this formula, feature map η1 and feature map η2 are sequentially subjected to position enhancement encoding to obtain feature map γ with the same size and number of channels as the input feature map.

[0020] Feature map μ1 is obtained by adding feature map η1 and feature map γ through channels, and feature map μ2 is obtained by adding feature map η2 and feature map γ through channels.

[0021] The final feature map is obtained by performing feature fusion operations on feature maps μ1 and μ2 as input feature maps.

[0022] In one embodiment, the feature encoding includes:

[0023] Acquire the input image, wherein the input image is a two-dimensional image or a three-dimensional image;

[0024] The first feature map is obtained by performing two convolution operations and a non-linear activation operation on the input image.

[0025] The first feature map is processed by pooling operations at different scales to form m second feature maps of different scales, where m is the number of scales;

[0026] After convolving and upsampling the m second feature maps respectively, m third feature maps of the same size as the first feature maps are obtained;

[0027] After concatenating all m third feature maps with the first feature map through channels, a multi-scale fused feature map is obtained.

[0028] A new first feature map is obtained by performing a convolution operation on the multi-scale fused feature map;

[0029] The process of repeating the pooling operation at different scales several times to obtain a new first feature map completes the feature encoding.

[0030] In one embodiment, performing feature fusion operations on the input feature map to obtain the final feature map includes:

[0031] After concatenating the input feature maps by channels, a 1×1 convolution operation is performed to obtain the first fused feature map;

[0032] The first fused feature map is divided into channels, and is divided into four equal parts, namely I1, I2, I3, and I4. I1 is directly output as feature map J1. J1 and I2 are concatenated and then subjected to a 3×3 convolution operation to output feature map J2. J2 and I3 are concatenated and then subjected to a 3×3 convolution operation to output feature map J3. J3 and I4 are concatenated and then subjected to a 3×3 convolution operation to output feature map J4. J1, J2, J3, and J4 are concatenated and then subjected to a 1×1 convolution operation to obtain the second fused feature map.

[0033] The third fusion feature map is obtained by adding the second fusion feature map to the input feature map.

[0034] Using the third fused feature map as input, repeat the process of dividing the first feature map into channels and obtaining the third fused feature map several times to obtain the final feature map.

[0035] In one embodiment, the image to be detected of the object to be detected is a texture image or a shape image, including: acquiring a two-dimensional image and a three-dimensional image containing the object to be detected output by a vision sensor.

[0036] In one embodiment, the arcuate region is an annular region or a portion thereof.

[0037] In one embodiment, converting an image of an arc-shaped region to be detected into an image of a rectangular region includes:

[0038] Obtain the center coordinates of the concentric arcs in the arc-shaped detection area, the radius of the inner circle containing the inner arc, and the radius of the outer circle containing the outer arc;

[0039] Based on the preset radius step size and angle step size, the arc-shaped detection area is divided into regions to obtain continuous fan-shaped sub-regions;

[0040] Convert the polar coordinates of the sector sub-region relative to the center of the concentric arc to coordinates in a rectangular two-dimensional coordinate system, and convert the sector sub-region to a rectangular sub-region at the corresponding coordinate position in the rectangular two-dimensional coordinate system;

[0041] Generate an image of a rectangular sub-region based on the grayscale values ​​of each pixel in the sector sub-region and image interpolation;

[0042] The images of the rectangular sub-regions that correspond one-to-one with all the sector sub-regions are combined to form the image of the rectangular region.

[0043] In one embodiment, the angle step size is:

[0044]

[0045] in, l represents the preset discretized distance precision, and r1 and r2 are the radii of the inner and outer circles, respectively.

[0046] In one embodiment, generating an image of a rectangular sub-region based on the grayscale values ​​of each pixel in the fan-shaped sub-region and image interpolation includes:

[0047] Obtain the discretized radius d of the pixels in the rectangular sub-region to be generated. k and discretization angle θ k Where k is the pixel number;

[0048] Based on the center coordinates (x0, y0) of the concentric circular arc and the discretized radius d k and discretization angle θ k Obtain the coordinates (r) of the pixel in the sector sub-region. k c k ):

[0049] r k =x0-d k sinθ k ,c k =y0+d k cosθ k

[0050] Based on the gray values ​​and coordinates of the corresponding positions in the sector sub-region, nearest neighbor, bilinear, or bicubic interpolation is performed to obtain the image of the rectangular sub-region.

[0051] According to a second aspect, one embodiment provides a detection device, comprising:

[0052] The visual sensor 11 is used to acquire and output a two-dimensional image and a three-dimensional image containing the object to be detected. The two-dimensional image includes texture information containing grayscale information of the image to be detected, and the three-dimensional image includes shape information containing height information of the image to be detected.

[0053] Memory 12 is used to store programs;

[0054] The processor 13 is configured to implement any of the above methods by executing a program stored in the memory, wherein the image to be detected is a texture image or a shape image.

[0055] According to a third aspect, one embodiment provides a computer-readable storage medium storing a program that can be executed by a processor to implement the method described in any of the preceding claims.

[0056] In the target feature detection method for arc-shaped regions, the false detection and false negative rates can be reduced by converting the image of the arc-shaped region to be detected into an image of a rectangular region. By performing feature encoding extraction based on the acquired two-dimensional image containing texture information (including grayscale information) and three-dimensional image containing shape information (including height information), both texture and shape information are considered. Furthermore, the surrounding height information is taken into account during defect detection for comprehensive judgment and analysis, thereby reducing the false detection and false negative rates. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a target feature detection method for an arc-shaped region in one embodiment of this application;

[0058] Figure 2 This is a schematic flowchart of a concentric arc region detection method according to an embodiment of this application;

[0059] Figure 3 This is an image to be detected containing the object to be detected, obtained according to one embodiment of this application;

[0060] Figure 4 To obtain based on Figure 2 The embodiment shows a contour information map with the inner and outer boundaries of the region surrounding the object to be detected;

[0061] Figure 5 This is a schematic diagram of the concentric circle fitting method according to an embodiment of this application;

[0062] Figure 6 for Figure 5 A schematic diagram of a specific method flow for one embodiment of step 0031;

[0063] Figure 7 In one embodiment based on Figure 4 A schematic diagram of the initial inner boundary of the region to be detected is generated from the contour information;

[0064] Figure 8 In one embodiment, a sliding window and its width and step size are defined for the inner or outer boundary detection area or the inner and outer boundary detection area.

[0065] Figure 9 This is a flowchart illustrating a method for detecting edge points within each detection sub-region in one embodiment;

[0066] Figure 10for Figure 8 A schematic diagram of a detection sub-region formed within a sliding window is shown.

[0067] Figure 11 for Figure 9 A schematic diagram of a specific method flow for one embodiment of step 203;

[0068] Figure 12 for Figure 5 A schematic diagram of the specific method flow of another embodiment of step 0031;

[0069] Figure 13 This is a schematic diagram of a method for clustering edge points detected within a detection sub-region, as shown in one embodiment.

[0070] Figure 14 for Figure 13 A schematic diagram of the specific method flow of one embodiment of step 01;

[0071] Figure 15 for Figure 14 A schematic diagram of the graphic transformation of one embodiment of step 012;

[0072] Figure 16 for Figure 14 A schematic diagram of a specific method flow for one embodiment of step 014;

[0073] Figure 17 for Figure 14 A schematic diagram of the graphic transformation of an embodiment of step 014;

[0074] Figure 18 for Figure 1 A schematic diagram of the overall network structure of one embodiment of step 02;

[0075] Figure 19 for Figure 1 A schematic diagram of a specific method flow for one embodiment of step 022;

[0076] Figure 20 for Figure 1 A schematic diagram of the network structure for feature encoding in one embodiment of step 022;

[0077] Figure 21 for Figure 1 A schematic diagram of the specific method flow of another embodiment of step 02;

[0078] Figure 22 This is a flowchart illustrating the method for performing position enhancement encoding on the result of step 023;

[0079] Figure 23This is a flowchart illustrating an embodiment of performing feature fusion operations on the input feature map in step 024 to obtain the final feature map;

[0080] Figure 24 This is a structural block diagram of a detection device according to one embodiment. Detailed Implementation

[0081] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments. Similar elements in different embodiments are referred to by related similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the present application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present application are not shown or described in the specification. This is to avoid obscuring the core parts of the present application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0082] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0083] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0084] In existing technologies, image feature detection of objects, including image information captured by 2D cameras and defect detection based on image texture information, fails to obtain height difference information of the defects. To improve detection capabilities, 3D cameras have been used to obtain the object's shape information for defect detection. However, this method only utilizes the object's shape information and does not take into account its texture information. If a 3D height image is used, plane fitting is mainly performed using point cloud information to calculate the distance from points within the detection area of ​​the sealing nail to the fitted plane. If the distance from a point to the plane is greater than a certain threshold, it is considered a candidate defect region. Connectivity analysis is performed on the candidate defect regions, and information such as roundness, area, needle-likeness, length, and width are used for screening to obtain the final defect region. This method is highly sensitive to noise and only considers local shape information, without considering the surrounding height information for comprehensive judgment and analysis. Therefore, this application provides a target feature detection method for arc-shaped regions, which uses 2D and 3D images to achieve heterogeneous data fusion, and performs target feature detection based on the fused shape and texture information. Understandably, this target feature detection method for arc-shaped regions is not only suitable for detecting the circular area of ​​lithium battery sealing nails, but also applicable to the target feature detection of circular end faces of bearings and arc-shaped regions with circular or segmental circular areas, such as optical discs.

[0085] The following explanation uses the annular area of ​​the lithium battery sealing pin as an example. Please refer to [link / reference]. Figure 1 The target feature detection method for this arc-shaped region may specifically include:

[0086] Step 00: Obtain the arc-shaped region to be detected.

[0087] The arc-shaped region can be a ring-shaped region or a part of a ring-shaped region (a fan-shaped region). In the detection of ring-shaped or fan-shaped regions, when fitting concentric circles containing the inner and outer arc-shaped boundaries based on the captured image, the existing technology performs circle fitting for the inner and outer circles separately, obtaining the center and radius of the inner and outer circles respectively. The average of the centers of the inner and outer circles is taken as the final center of the concentric circle that forms the ring, and the radius is the radius of each fitted radius. This method of fitting concentric circles of the ring only considers the local optimum and does not consider the global optimum. The independently estimated center does not coincide with the center of the actual captured image of the object. The concentric arcs fitted by this method have a high false detection rate and a high false negative rate. In one embodiment of this application, a new arc-shaped region detection method is provided to overcome the above-mentioned problems. This arc-shaped region detection method is used to determine the arc-shaped region of the object to be detected. The following uses a lithium battery sealing nail as an example to specifically introduce this arc-shaped region detection method. It is understandable that this arc-shaped area detection method is not only suitable for detecting the circular area of ​​lithium battery sealing nails, but also applicable to the detection of circular end faces of bearings and annular or fan-shaped areas with circular areas, such as optical discs.

[0088] Please refer to Figure 2 The method for detecting arc-shaped regions includes:

[0089] Step 001: Obtain the image containing the object to be detected. Taking a sealing nail as an example, an image containing the sealing nail is obtained through an image sensor. This image can be a two-dimensional image or a three-dimensional image; please refer to [reference needed]. Figure 3 This is an image to be acquired that includes the location of the lithium battery sealing nail.

[0090] Step 002: Perform edge detection based on the acquired image to be detected, based on the edge detection obtained in step 001. Figure 3 The image of the sealing nail is used to segment the annular region of the sealing nail to be detected, thereby obtaining the contour information of the inner and outer boundaries of the region surrounding the object to be detected.

[0091] For obtaining contour information, in some embodiments, deep learning methods can be used, employing a segmentation network model to obtain the binary region of the detected sealing nail. Morphological processing is then performed on the binary region to remove burrs and other information, obtaining the contour information of the inner and outer boundaries of the binary region for subsequent concentric circle fitting. Please refer to [reference needed]. Figure 4 , Figure 4 The two circles in the figure are the inner and outer circles to be fitted.

[0092] Step 003: Based on the principle of minimizing the sum of the squares of the distances from all edge points in the contour information to the center of the concentric arc to be fitted, perform concentric arc fitting on the inner and outer boundaries.

[0093] Since the concentric arc region is fitted with concentric arcs based on the principle of minimizing the sum of the squares of the distances from all edge points in the contour information to the center of the concentric arc to be fitted, the concentric arc region obtained after fitting the concentric arc region can achieve global optimization. By comprehensively considering the estimation of the center and radius of the inner and outer arcs, the estimated center has a higher degree of overlap with the real concentric arc and is more consistent with the real results. Target feature detection on the concentric arc region determined by this method can reduce the false detection rate and the false negative rate.

[0094] Step 004: Take the arc-shaped region between the fitted inner and outer arcs as the region to be detected.

[0095] The above-mentioned step 003, which uses the principle of minimizing the sum of the squares of the distances from all edge points in the contour information to the center of the concentric arc to be fitted, performs concentric arc fitting on the inner and outer boundaries. This can be achieved by combining the inventive concept of this application with other methods known to those skilled in the art. In one embodiment of this application, based on the given inventive concept of concentric arc fitting, a new and specific concentric arc fitting method is provided. Please refer to [reference needed]. Figure 5 Specifically, it includes:

[0096] Step 0031: Based on the contour information, perform edge point recognition to obtain the set of edge points of the inner boundary and the set of edge points of the outer boundary in the contour information.

[0097] The aforementioned method for edge point recognition based on contour information to obtain the set of edge points for the inner boundary and the set of edge points for the outer boundary in the contour information can be obtained using existing techniques. This application also provides a new specific method, which you can refer to. Figure 6 ,include:

[0098] Step 003111: Receive the region of interest input by the user, and generate the initial inner boundary region to be detected and the initial outer boundary region to be detected on the image to be detected.

[0099] refer to Figure 3 The inner and outer boundaries perceived by the human eye are used to generate initial inner boundary detection regions and initial outer boundary detection regions, respectively. The region containing the inner boundary is the initial inner boundary detection region, and the region containing the outer boundary is the initial outer boundary detection region. In one embodiment, reference... Figure 7For the initial inner boundary detection region, an inner circle and an outer circle are set. The radius of the inner circle is the lower limit of the inner boundary radius, and the radius of the outer circle is the upper limit of the inner boundary radius. The area enclosed by the inner and outer circles is the initial inner boundary detection region. Similarly, for the initial outer boundary detection region, an inner circle and an outer circle are set. This time, the radius of the inner circle is the lower limit of the outer boundary radius, and the radius of the outer circle is the upper limit of the outer boundary radius. The area enclosed by the inner and outer circles is the initial outer boundary detection region.

[0100] Step 003112: Receive the width and displacement step of the inner boundary sliding window input by the user, cover the initial inner boundary detection area with the inner boundary sliding window, and move along the initial inner boundary detection area according to the displacement step of the inner boundary sliding window. After each displacement, the inner boundary sliding window forms an inner boundary detection sub-region.

[0101] Please refer to Figure 7 Set the ROI (sliding window), obtain the ROI information, and the area between the two dashed circles can be the inner and outer dashed circles formed by the inner boundary sliding window sliding along the initial inner boundary of the area to be detected. At this time, the area between the two dashed circles includes... Figure 3 The inner boundary is defined by w, which is the width of the inner boundary sliding window, and s, which is the displacement step size of the inner boundary sliding window (the displacement step size is the angular displacement step size). After each displacement, the inner boundary sliding window forms an inner boundary detection sub-region.

[0102] Step 003113: Receive the width and displacement step of the outer boundary sliding window input by the user, cover the initial outer boundary detection area with the outer boundary sliding window, and move along the initial outer boundary detection area according to the displacement step of the outer boundary sliding window. After each displacement, the outer boundary sliding window forms an outer boundary detection sub-region.

[0103] Please refer to Figure 8 The area between the two dashed circles can be defined by the inner and outer dashed circles formed by the sliding window along the initial outer boundary of the region to be detected. In this case, the area between the two dashed circles includes... Figure 3 The outer boundary is defined by w, which is the width of the outer boundary sliding window, and s, which is the displacement step size of the outer boundary sliding window. After each displacement, the outer boundary sliding window forms an outer boundary detection sub-region.

[0104] Understandably, the order of steps 003112 and 003113 can be interchanged or not in any particular order.

[0105] Step 003114: Detect an inner boundary edge point within each inner boundary detection sub-region, and detect an outer boundary edge point within each outer boundary detection sub-region.

[0106] Edge point detection is performed on each inner boundary detection sub-region to obtain an inner boundary edge point, and edge point detection is performed on each outer boundary detection sub-region to obtain an outer boundary edge point. In this way, the individual inner boundary edge points can form the inner boundary (…). Figure 4 The inner circle, and the individual edge points of the outer boundary can form the outer boundary (the inner circle). Figure 4 (Outer circle in the middle).

[0107] In some embodiments, the method for detecting edge points within each detection sub-region can employ existing techniques. One embodiment of this application employs a novel edge point detection method; please refer to [reference needed]. Figure 9 ,include:

[0108] Step 201: Construct a grayscale projection curve f using the grayscale values ​​of each pixel in the area to be detected.

[0109] For each sub-region in the sequence of inner or outer boundary detection sub-regions, construct a grayscale projection curve f using the grayscale values ​​of each pixel within the sub-region. Please refer to [reference needed]. Figure 10 , Figure 10 The dashed box in the middle is Figure 8 One of the detection sub-regions, along the width direction (corresponding to) Figure 10 (The five points connected by the dashed box form a row) Calculate the grayscale value interpolation for each pixel. Specifically, neighbor interpolation or bilinear interpolation can be used. If the interpolated grayscale value is g... ab , where 'a' represents the 'a'th column pixel in the projection direction ( Figure 10 There are 5 columns of pixels, and each column has 3 data points. The projection direction is... Figure 8 The direction of the sliding window width w is perpendicular to the direction of the sliding window width w in the sub-region, and b represents the b-th data point in the pixel column in the projection direction. Therefore, the average gray value of the pixel in the a-th column in the sub-region can be obtained.

[0110]

[0111] Where r and c represent the row and column coordinates of the center pixel of the sub-region, respectively, and t represents half the width of the sub-region and is equal to 0.5w.

[0112] s a The set {s a The array is represented as a grayscale projection curve, denoted as} Where k is the projection length (which can be...) Figure 10 The number of pixel columns in the projection direction within each sub-region.

[0113] Step 202: Calculate the gradient of the grayscale projection curve f to obtain the gradient projection curve f′.

[0114] After obtaining the grayscale projection curve f, its gradient is calculated. If the gradient is calculated using the Gauss kernel function, the gradient value for each pixel can be expressed as:

[0115]

[0116] Where x is the offset of the pixel coordinates relative to the origin, and σ is the filter kernel strength. The filter kernel strength is obtained based on the set filter kernel size n′, and can be obtained using the formula...

[0117]

[0118] The calculation yielded, where This indicates rounding down to the nearest integer.

[0119] In some embodiments, filtering operations can also be incorporated into the calculation process. In some embodiments, filtering and gradient interpolation can be combined into a single process, thereby saving computation time. In this case, the value of σ can be calculated based on the width of the set filter kernel. Then, after obtaining the gradient value, the gradient projection curve f′ can be constructed, and it can be represented as f′=f×h, where × represents a one-dimensional convolution operation.

[0120] Step 203: Filter edge points based on gradient projection curve f′.

[0121] In some embodiments, the edge point selection method based on the gradient projection curve f′ can be implemented using existing techniques. In one embodiment of this application, a novel edge point selection method is provided; please refer to [the relevant documentation]. Figure 11 Specifically, it includes:

[0122] Step 301: Perform threshold filtering on the points on the gradient projection curve f′ to filter out peak points that are greater than the preset threshold.

[0123] Based on screening criteria

[0124] |f′(x)|>|f′(x-1)|, |f′(x)|>|f′(x+1)|, |f′(x)|>t abs

[0125] Perform threshold filtering that simultaneously satisfies the above three relationships, where t abs Let f′(x) be the amplitude of a point on the gradient projection curve f′, and f′(x-1) and f′(x+1) be the amplitudes of adjacent points of f′(x). Based on the above threshold filtering, peak points on the projection curve f′ that meet the threshold filtering are selected. Through the above filtering conditions, both the preset gradient threshold filtering and the gradient filtering of adjacent pixels are performed, thus obtaining more accurate edge points.

[0126] Step 302: Perform polarity filtering on the peak points to select peak points with the same preset polarity.

[0127] Based on the points selected in step 301, further polarity filtering is performed. Polarity filtering can be done from white to black, from black to white, or by any polarity change (including from black to white and from white to black). It should be noted that if a polarity filtering method is used, all edge points must be filtered using the same polarity filtering method.

[0128] Step 303: After polarity filtering, perform position filtering on the peak points and select edge points by choosing one of the following methods: start point, end point, nearest center, and maximum intensity.

[0129] Location filtering can be performed by starting point (the first point of the location), ending point (the last point of the location), nearest center (the center of the location), or maximum intensity. In essence, it searches along the search direction to obtain several data points that meet the criteria. The starting point is the first feature point found along the search direction, the ending point is the last feature point found along the search direction, and the nearest center is the closest point to the nearest location. Figure 10 The coordinates of the center of rotation of the rectangle shown are used to determine the strongest gradient; the feature point with the largest gradient magnitude is selected. It should be noted that if a location filtering method is used, all edge points must be filtered using the same location filtering method.

[0130] Step 003115: Based on the inner boundary edge points detected in all inner boundary detection sub-regions, obtain the set of edge points of the inner boundary; based on the outer boundary edge points detected in all outer boundary detection sub-regions, obtain the set of edge points of the outer boundary.

[0131] In one embodiment of this application, another novel method for edge point recognition based on contour information to obtain the set of edge points for the inner boundary and the set of edge points for the outer boundary in the contour information is also provided. Please refer to [reference needed]. Figure 12 ,include:

[0132] Step 003121: Receive the region of interest input by the user and generate an initial arc-shaped region of interest containing inner and outer boundaries on the image to be detected.

[0133] refer to Figure 3The system generates an initial arc-shaped detection region containing the inner and outer boundaries perceived by the human eye. In one embodiment, for the initial arc-shaped detection region, an inner circle and an outer circle are set, wherein the radius of the inner circle is the lower limit of the radius of the inner boundary, and the radius of the outer circle is the upper limit of the radius of the outer boundary. The area enclosed between the inner and outer circles is the initial arc-shaped detection region.

[0134] Step 003122: Receive the width and displacement step of the sliding window input by the user, cover the inner and outer boundaries of the initial annular detection area with the sliding window, and move it along the initial annular detection area according to the displacement step. After each displacement, the sliding window forms a detection sub-region.

[0135] Please refer to Figure 8 The region between the two dashed circles can be defined by the inner and outer dashed circles formed by a sliding window that slides along the initial annular region to be detected, spanning the inner and outer boundaries. In this case, the area between the two dashed circles includes... Figure 3 The inner and outer boundaries are defined in the diagram, w is the width of the sliding window that spans the inner and outer boundaries, and s is the displacement step size (angular displacement step size) of the sliding window that spans the inner and outer boundaries. The sliding window after each displacement forms a detection sub-region.

[0136] Step 003123: Detect two edge points in each detection sub-region.

[0137] Edge point detection is performed on each detection sub-region to obtain two edge points, including an inner boundary edge point and an outer boundary edge point.

[0138] The method for detecting edge points within each detection sub-region can employ existing techniques. One embodiment of this application uses a novel edge point detection method; please refer to [reference needed]. Figure 9 and Figure 11 and the above references Figure 9 and Figure 11 The edge point detection method described herein will not be repeated here.

[0139] Step 003124: Cluster the edge points detected in all detection sub-regions to obtain the edge point set of the inner boundary and the edge point set of the outer boundary.

[0140] Here, the clustering method can be any existing clustering method. One embodiment of this application provides a novel clustering method; please refer to [reference needed]. Figure 13 ,include:

[0141] Step 401: Obtain the center coordinates of all edge points.

[0142] Obtaining the center coordinates of all edge points involves calculating the average value of all edge points on the X-axis and the average value on the Y-axis.

[0143] Step 402: Calculate the distance from all edge points to the center coordinate point.

[0144] Step 403: Vote on the above distances and obtain distance statistics.

[0145] Step 404: Discretize the above distances according to a preset step size to obtain the index of the above statistical information.

[0146] Step 405: Obtain the distance d that has the highest probability of being an inner boundary edge point, starting from the minimum distance. min The distance d that has the highest probability of being the outer boundary edge point is found by starting from the maximum distance. max Take the average value Edge points whose distance from the center coordinate point is less than the average value are inner boundary edge points, and edge points whose distance from the center coordinate point is greater than the average value are outer boundary edge points.

[0147] Step 0032: Based on the principle of minimizing the sum of the squares of the distances from all edge points in the inner boundary edge point set and the outer boundary edge point set to the center of the concentric arc to be fitted, perform concentric arc fitting on the inner and outer boundaries, calculate the center coordinates of the concentric arc to be fitted, the radius of the inner circle containing the inner arc, and the radius of the outer circle containing the outer arc, thereby obtaining the inner and outer arcs of the concentric arc.

[0148] Since they are concentric circular arcs, it is necessary to estimate the center, the radius of the inner circle containing the inner arc, and the radius of the outer circle containing the outer arc. Based on the principle of minimizing the sum of the squares of the distances from the inner and outer arcs to the concentric circular arc to be fitted, as an embodiment of this application, the objective function for fitting the concentric circular arc can be expressed as:

[0149]

[0150] Where (x0, y0) is the center of the concentric circular arc to be fitted, r1 is the radius of the inner circle containing the inner arc, and r2 is the radius of the outer circle containing the outer arc. i y i ) represents the coordinates of the edge points of the inner boundary, i is the index of the edge points of the inner boundary, and (u j v j ) represents the coordinates of the edge points of the outer boundary, j represents the index of the edge points of the outer boundary, n1 represents the number of edge points of the inner boundary, and n2 represents the number of edge points of the outer boundary.

[0151] For the center (x0, y0) of the concentric circular arc to be fitted, it can be obtained using the formula

[0152]

[0153] Calculated; where,

[0154]

[0155]

[0156]

[0157]

[0158]

[0159] Based on the center (x0, y0) of the concentric circular arc to be fitted, according to the formula

[0160]

[0161] Calculate the radius r1 of the inner circle containing the inner arc of the concentric circular arc to be fitted, according to the formula.

[0162]

[0163] Calculate the radius r2 of the outer circle containing the outer arc to be fitted.

[0164] Based on the arc-shaped region detection method described above, the arc-shaped region obtained can reduce the false detection and false negative rates when used to detect target features in arc-shaped regions.

[0165] Step 01: Convert the image of the arc-shaped region to be detected into an image of a rectangular region.

[0166] The segmented sealing nail region is divided into several regions, and these regions are then converted from fan-shaped to rectangular regions. By converting the image of the arc-shaped detection area into a rectangular region before target feature detection, the influence of external pixel grayscale values ​​at the edges on detection is effectively reduced, improving detection accuracy and stability at edges. Since only grayscale values ​​within the detection range are used, and grayscale values ​​outside the sealing nail detection area are not utilized, the influence of external grayscale values ​​and height values ​​is effectively avoided, thus reducing the probability of false detections at the boundaries in target feature detection based on arc-shaped regions.

[0167] The method for converting an image of an arc-shaped detection region into an image of a rectangular region can employ existing techniques. One embodiment of this application provides a novel method, please refer to [reference needed]. Figure 14 ,include:

[0168] Step 011: Obtain the center coordinates of the concentric arcs in the arc-shaped detection area, the radius of the inner circle containing the inner arc, and the radius of the outer circle containing the outer arc.

[0169] Step 012: Divide the arc-shaped area to be detected into regions according to the preset radius step size and angle step size to obtain continuous fan-shaped sub-regions.

[0170] For region division, some embodiments may divide the region according to the area of ​​the arc-shaped region; in one embodiment of this application, the division is based on a radius step size and an angle step size. Please refer to... Figure 15 Based on the radius step size, a portion of the arc-shaped area to be detected within an angular range can be divided into n³ consecutive sector sub-regions along the radius direction. Figure 15 In this process, a portion of the arc-shaped area to be detected within an angular range is divided into three consecutive sector sub-regions. Based on the angular step size, the arc-shaped area to be detected can be further divided into n4 sector sub-regions, resulting in a total of n3 x n4 consecutive sector sub-regions.

[0171] The angle step size can be set to 45 degrees. For one embodiment, please refer to... Figure 15 Divide the entire ring into 16 parts according to the angle step size, then the angle step size is 22.5 degrees.

[0172] In some embodiments, the annulus is divided into several blocks according to the radius step size and the angle step size. Therefore, the converted rectangular image data can be processed equally, so that defects can be detected and analyzed independently for each rectangular sub-region, ensuring that detection is performed in parallel and improving the overall detection efficiency.

[0173] Step 013: Convert the polar coordinates of the sector sub-region relative to the center of the concentric arc into coordinates in a rectangular two-dimensional coordinate system, and convert the sector sub-region into a rectangular sub-region at the corresponding coordinate position in the rectangular two-dimensional coordinate system.

[0174] Step 014: Generate an image of a rectangular sub-region based on the grayscale values ​​of each pixel in the sector sub-region and image interpolation.

[0175] In some embodiments, the method for generating images of rectangular sub-regions can employ existing techniques. In one embodiment of this application, a novel method is used, as described above. Figure 16 and Figure 17 Specifically, it includes:

[0176] Step 0141: Obtain the discretized radius d of the pixels in the rectangular sub-region to be generated. k and discretization angle θ k Where k is the pixel number.

[0177] Step 0142: Discretize the center coordinates (x0, y0) of the concentric circular arc and the radius d. k and discretization angle θ k Get the coordinates (r) of the pixel in the sector sub-region. k c k ):

[0178] r k =x0-d k sinθ k ,c k =y0+d k cosθ k

[0179] The radius of the circle containing the pixel being converted is d. k The angle is θ k Therefore, let the x-coordinate of the transformed rectangular image be the angle and the y-coordinate be the radius. The transformed coordinates are known to be (θ). k d k If the coordinates of the given information are r, then the coordinates of the information in the original sector image are (r). k c k ).

[0180] For discretization, the radius d k The discretization step size is 1, and the angle θ i The discretization step size is crucial. If the discretization step size is too small, the computational load is high, but the resolution of the converted image is high. If the discretization step size is too large, the converted image will be severely distorted. Therefore, an adaptive conversion parameter needs to be determined based on the obtained parameters of the inner and outer circular arcs. In some embodiments, the angle step size can be set using existing techniques. One embodiment of this application provides a new method for setting the angle step size, whereby the angle step size is set as follows:

[0181]

[0182] in, l represents the preset discretized distance precision, and r1 and r2 are the radii of the inner circle containing the inner arc and the outer circle containing the outer arc, respectively. This angle step size setting method can improve the quality of the converted image while ensuring precision, achieving a balanced processing, and is an adaptive angle discretization step size method.

[0183] Because (r k c k Since ) is a floating-point number, it cannot correspond to the grayscale value in the original sector image. Therefore, interpolation is required to obtain the converted rectangular image, so proceed to step 0143.

[0184] Step 0143: Based on the grayscale values ​​and coordinates within the neighborhood of the corresponding position in the sector-shaped sub-region, perform nearest neighbor, bilinear, or bicubic interpolation to obtain the image of the rectangular sub-region. In some embodiments, if nearest neighbor interpolation is used within the neighborhood, then r... k and c k Round down; if bilinear interpolation is used, take (r) k c k The four pixels surrounding ); if bicubic interpolation is used, then take (r) k c k The sixteen pixels surrounding it.

[0185] Step 015: Combine the images of the rectangular sub-regions that correspond one-to-one with all the sector sub-regions into a rectangular region image.

[0186] Step 02: Target feature detection based on the rectangular region of the image. Target feature detection based on the rectangular region of the image can be achieved using existing techniques. In one embodiment of this application, a novel deep learning-based method is employed. (See reference...) Figure 1 Specific methods include:

[0187] Step 021: Obtain the two-dimensional and three-dimensional images of the object to be detected. The two-dimensional image includes the texture information containing grayscale information of the image to be detected, and the three-dimensional image includes the shape information containing height information of the image to be detected.

[0188] Please refer to the overall network architecture. Figure 18 Acquire the two-dimensional image data I of the object to be detected. RGB and 3D image data I D Two-dimensional image data represents the texture information of an object, while 3D image data represents the height information of an object. Feature encoding is performed on the two-dimensional image to obtain feature map α1, and feature encoding is performed on the three-dimensional image to obtain feature map α2.

[0189] In this embodiment of the application, fusion of heterogeneous data is achieved using 2D and 3D images, and target feature detection is performed based on the fused shape and texture information.

[0190] Step 022: Perform feature encoding on the two-dimensional image to obtain feature map α1, and perform feature encoding on the three-dimensional image to obtain feature map α2.

[0191] The aforementioned feature encoding can be implemented using existing techniques. However, in one embodiment of this application, a novel feature encoding method is employed. Please refer to [reference needed]. Figure 19 Specific methods include:

[0192] Step 0221: Obtain the input image, which can be a two-dimensional or three-dimensional image.

[0193] Step 0222: The first feature map is obtained by performing two convolution operations and a non-linear activation operation on the input image.

[0194] In one embodiment of this application, please refer to Figure 20 In the feature encoding network structure, for the first layer, the input two-dimensional or three-dimensional image is processed by two convolutions and non-linear activation operations to obtain the first feature map α0.

[0195] Step 0223: The first feature map is processed by pooling operations at different scales to form m second feature maps of different scales, where m is the number of scales.

[0196] The obtained first feature map α0 is subjected to pooling operations of different scales to form second feature maps of different scales. like Figure 20 In the example shown, m = 4.

[0197] Step 0224: After convolving and upsampling the m second feature maps respectively, m third feature maps of the same size as the first feature maps are obtained.

[0198] Obtain feature maps of m different scales After convolution and upsampling, they are unified into a third feature map of the same size as the first feature map α0.

[0199] Step 0225: After concatenating all m third feature maps with the first feature map through channels, a multi-scale fused feature map is obtained.

[0200] refer to Figure 20 The m third feature maps are concatenated with the first feature map α0 through channel concatenation calculation to obtain a multi-scale fused feature map.

[0201] Step 0226: Perform convolution operations on the multi-scale fused feature maps to obtain a new first feature map.

[0202] Step 0227: Repeat the above process of pooling operations at different scales several times to obtain a new first feature map to complete feature encoding.

[0203] After the new first feature map is processed by pooling, it is used as the input of the next-level network. The process of steps 0223 to 0226 above is repeated multiple times to achieve feature encoding of the two-dimensional image to obtain feature map α1 and feature encoding of the three-dimensional image to obtain feature map α2.

[0204] In the above operations, the convolution in step 0222 and the pooling in step 0223 prevent the feature distribution from shifting, while the nonlinear activation (ReLU) operation in step 0222 enhances the nonlinearity of the detection.

[0205] Step 023: After performing multi-scale feature extraction on feature map α1, feature map η1 is obtained. After performing multi-scale feature extraction on feature map α2, feature map η2 is obtained.

[0206] Step 024: Use feature map η1 and feature map η2 as input feature maps to perform feature fusion operation to obtain the final feature map.

[0207] Generally, feature maps η1 and η2 obtained in step 024 can be used as input feature maps for feature fusion operations to obtain the final feature map. However, considering the case of weak texture images, such as the texture image of a sealing nail, in order to obtain better detection results, it is necessary to improve the overall feature encoding capability. Therefore, in one embodiment of this application, please refer to... Figure 21 Step 024 includes:

[0208] Step 0241: Perform position enhancement coding on the obtained feature map η1 to obtain feature map μ1, and perform position enhancement coding on the feature map η2 to obtain feature map μ2;

[0209] Step 0242: Use feature map μ1 and feature map μ2 as input feature maps to perform feature fusion operation to obtain the final feature map.

[0210] Whether the feature map η1 and feature map η2 are used as input feature maps for feature fusion to obtain the final feature map, or the feature map μ1 and feature map μ2 are used as input feature maps for feature fusion to obtain the final feature map, the feature map obtained from grayscale (feature map η1 or μ1) and the feature map obtained from height extraction (feature map η2 or μ2) are fused. This takes into account both texture information and shape information, and considers the surrounding height information for comprehensive judgment and analysis during defect detection, thereby reducing the false detection and false detection rates.

[0211] In some embodiments, the location enhancement coding process can be implemented using existing techniques. This application provides a novel location enhancement coding process; please refer to [reference needed]. Figure 22 Specifically, it includes:

[0212] Step 501: Perform position enhancement coding on feature map η1 and feature map η2. The position enhancement coding formula can be expressed as:

[0213]

[0214] Where x and y are the coordinate information parallel to the X-axis and Y-axis, respectively, D refers to the number of channels in the feature map, p is the channel index information of feature map η1, and q is the channel index information of feature map η2. According to this formula, feature map η1 and feature map η2 are sequentially subjected to position enhancement encoding to obtain feature map γ with the same size and number of channels as the input feature map.

[0215] Step 502: Perform channel addition operation on feature map η1 and feature map γ to obtain feature map μ1, and perform channel addition operation on feature map η2 and feature map γ to obtain feature map μ2.

[0216] Step 503: Use feature map μ1 and feature map μ2 as input feature maps to perform feature fusion operation to obtain the final feature map, which is used for feature decoding.

[0217] In some embodiments, the input feature map is fused to obtain the final feature map, which can be achieved using existing techniques. One embodiment of this application employs a novel implementation method; please refer to [reference needed]. Figure 23 Specifically, it includes:

[0218] Step 601: After concatenating the channels of the input feature map, perform a 1×1 convolution operation to obtain the first fused feature map.

[0219] Taking the feature map μ1 and feature map μ2 as input feature maps to perform feature fusion operation to obtain the final feature map as an example, after concatenating the grayscale feature map μ1 and the height-extracted feature map μ2 through channel concatenation, a 1×1 convolution operation is performed to obtain the first fused feature map.

[0220] Based on the fused shape and texture information, the feature map obtained from grayscale and the feature map obtained from height are concatenated by channels and then subjected to 1×1 convolution operation to obtain the first fused feature map. Since the height information around the pixel is taken into account, the problem of false detection at the boundary is reduced, and the false detection rate is also reduced.

[0221] Step 602: Divide the first fused feature map into channels, dividing it into four equal parts, denoted as I1, I2, I3, and I4. I1 is directly output as feature map J1. J1 and I2 are concatenated and then subjected to a 3×3 convolution operation to output feature map J2. J2 and I3 are concatenated and then subjected to a 3×3 convolution operation to output feature map J3. J3 and I4 are concatenated and then subjected to a 3×3 convolution operation to output feature map J4. After concatenating the channels of J1, J2, J3, and J4, a 1×1 convolution operation is performed to obtain the second fused feature map.

[0222] Step 603: Add the second fused feature map to the input feature map to obtain the third fused feature map.

[0223] Step 604: Using the third fused feature map as input, repeat steps 602 to 603 several times to obtain the final feature map.

[0224] Step 025: Obtain the final result by performing a 1×1 convolution operation and a sigmoid activation function on the final feature map.

[0225] The method for target feature detection based on images with rectangular regions is implemented using a neural network model trained using deep learning. The loss function of this neural network model can be expressed as:

[0226]

[0227] Where α is the class number smoothing factor, and γ is the difficulty-simplification balancing factor. To predict the confidence level for the z-th category, p z The confidence level of the z-th category of the label.

[0228] One embodiment of this application provides a detection device; please refer to [reference needed]. Figure 24 ,include:

[0229] The vision sensor 11 is used to acquire and output two-dimensional and three-dimensional images containing the object to be detected. The two-dimensional image includes texture information containing grayscale information of the image to be detected, and the three-dimensional image includes shape information containing height information of the image to be detected.

[0230] In some embodiments, the visual sensor 11 includes a 2D camera and a 3D camera to acquire texture information containing grayscale information and shape information containing height information, thereby enabling the fusion of texture information and shape information, and comprehensively considering grayscale information and height information to achieve higher accuracy detection.

[0231] Memory 12 is used to store programs.

[0232] The processor 13 is configured to implement the method for detecting target features in an arc-shaped region according to any of the above embodiments by executing a program stored in the memory 12, wherein the image to be detected is a texture image or a shape image.

[0233] One embodiment of this application also provides a computer-readable storage medium storing a program that can be executed by a processor to implement the ring region detection method and / or the method for detecting target features of a ring region in any of the above embodiments.

[0234] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0235] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A method of detecting a target feature of an arc-shaped region, characterized by, The method comprises the steps of: obtaining an arc-shaped region as a region to be detected; dividing the arc-shaped region to be detected into continuous fan-shaped sub-regions, converting the fan-shaped sub-regions from polar coordinates into a rectangular two-dimensional coordinate system, and converting the fan-shaped sub-regions into rectangular sub-regions at corresponding coordinate positions, generating an image of the rectangular sub-regions according to the gray values of the pixels of the fan-shaped sub-regions and image interpolation, and combining the images of the rectangular sub-regions corresponding to all the fan-shaped sub-regions into an image of the rectangular region; and detecting a target feature based on the image of the rectangular region; The step of detecting a target feature based on the image of the rectangular region comprises the steps of: obtaining a two-dimensional image and a three-dimensional image of a target object, wherein the two-dimensional image comprises texture information of the target image containing gray information, and the three-dimensional image comprises topographic information of the target image containing height information; Feature encoding of a two-dimensional image to obtain a feature map Feature encoding of a three-dimensional image to obtain a feature map ; to the feature map obtained after multi-scale feature extraction processing , to the feature map obtained after multi-scale feature extraction processing ; The feature map and the feature map The feature fusion operation is performed on the input feature map to obtain a final feature map; and obtaining a final result through 1x1 convolution operation and sigmoid activation function of the final feature map.

2. The method of claim 1, wherein, The step of detecting a target feature based on the image of the rectangular region further comprises the steps of: to the feature map and the feature map The position-enhanced encoding is performed, and the position-enhanced encoding formula can be expressed as: ， Wherein, x, y are coordinate information parallel to X-axis and Y-axis respectively, D refers to the channel number of the feature map, p is the channel index information of the feature map , q is the channel index information of the feature map ; according to the formula, the position enhancement coding is sequentially performed on the feature map and the feature map to obtain a feature map γ which is equal in size and same in channel number as the input feature map. obtaining a feature map obtaining a feature map obtaining a feature map obtaining a feature map ; The feature map and the feature map The feature fusion operation is performed on the input feature map to obtain a final feature map.

3. The method of claim 1 or 2, wherein, The feature encoding comprises the steps of: obtaining an input image, wherein the input image is the two-dimensional image or the three-dimensional image; obtaining a first feature map through twice convolution operation and nonlinear activation operation of the input image; forming m second feature maps of different scales through different-scale pooling operation of the first feature map, wherein m is the number of scales; obtaining m third feature maps of the same size as the first feature map through convolution and upsampling of the m second feature maps respectively; obtaining a multi-scale fusion feature map through channel splicing processing of all the m third feature maps and the first feature map; obtaining a new first feature map through convolution operation of the multi-scale fusion feature map; completing the feature encoding by repeating the process of the new first feature map through different-scale pooling operation of the new first feature map for several times.

4. The method of claim 1 or 2, wherein, The feature fusion operation of the input feature map to obtain the final feature map comprises the steps of: obtaining a first fusion feature map through 1x1 convolution operation of the input feature map after channel splicing; The first fused feature map is divided into channels, and then divided into four equal parts, which are represented as follows: , , , ,in, Directly output as feature map , and After concatenation, the output is a feature map after 3×3 convolution. , and After concatenating the channels, a 3×3 convolution operation is performed to output the feature map. , and After concatenating the channels, a 3×3 convolution operation is performed to output the feature map. ,Will , , , After channel concatenation, a 1×1 convolution operation is performed to obtain the second fused feature map; obtaining a third fusion feature map through addition operation of the second fusion feature map and the input feature map; obtaining the final feature map by repeating the process of obtaining the third fusion feature map through channel division of the first fusion feature map for several times.

5. The method of claim 1, wherein, The obtained target image of the target object is a texture image or a topographic image, comprising the steps of: obtaining a two-dimensional image and a three-dimensional image containing the target object output by a visual sensor.

6. The method of claim 1, wherein, The arc-shaped region is a ring-shaped region or a part of a ring-shaped region.

7. The method of claim 6, wherein, The step of dividing the arc-shaped region to be detected into continuous fan-shaped sub-regions, converting the fan-shaped sub-regions from polar coordinates into a rectangular two-dimensional coordinate system, and converting the fan-shaped sub-regions into rectangular sub-regions at corresponding coordinate positions, generating an image of the rectangular sub-regions according to the gray values of the pixels of the fan-shaped sub-regions and image interpolation, and combining the images of the rectangular sub-regions corresponding to all the fan-shaped sub-regions into an image of the rectangular region comprises the steps of: obtaining the center coordinates of the concentric circular arcs of the arc-shaped region to be detected, the radius of the inner circle where the inner circular arc is located, and the radius of the outer circle where the outer circular arc is located; According to a preset radius step and an angle step, the arc-shaped to-be-detected region is divided into regions to obtain continuous fan-shaped sub-regions; The polar coordinates of the fan-shaped sub-regions relative to the center of the concentric arc are converted into coordinates in a rectangular two-dimensional coordinate system, and the fan-shaped sub-regions are converted into rectangular sub-regions at corresponding coordinate positions in the rectangular two-dimensional coordinate system; An image of the rectangular sub-region is generated according to the gray values of the pixels of the fan-shaped sub-region and image interpolation; The images of the rectangular sub-regions corresponding to all the fan-shaped sub-regions are grouped to form an image of the rectangular region.

8. The method of claim 7, wherein, The angle step is: ； wherein, , is a preset discretized distance accuracy, and are the radii of the inner and outer circles, respectively.

9. The method of claim 7, wherein, Generating the image of the rectangular sub-region according to the gray values of the pixels of the fan-shaped sub-region and image interpolation includes: acquiring a discretization radius of a pixel point of a rectangular sub-region to be generated and a discretization angle ; wherein k is a serial number of the pixel point; based on the center coordinates (x0, y0), the discretized radius and the discretized angle of the concentric circle arc , obtain the coordinates of the pixel point in the fan-shaped sub-region , : ； The gray values and coordinates within the neighborhood range of the corresponding positions in the fan-shaped sub-region are subjected to nearest neighbor or bilinear or bicubic interpolation to obtain the image of the rectangular sub-region.

10. A detection device, characterized by It includes: a visual sensor (11) for acquiring and outputting a two-dimensional image and a three-dimensional image containing a to-be-detected object, the two-dimensional image including texture information of the to-be-detected image containing gray information, and the three-dimensional image including topographic information of the to-be-detected image containing height information; a memory (12) for storing a program; a processor (13) for executing the program stored in the memory to implement the method of any one of claims 1-9, the to-be-detected image being a texture image or a topographic image.

11. A computer readable storage medium, characterized in that, The medium has a program stored thereon, and the program can be executed by a processor to implement the method of any one of claims 1-9.

Images