Abnormal region detection method and device, electronic equipment and storage medium

Abstract

The invention provides an abnormal region detection method and device, electronic equipment and a storage medium. The method comprises the steps of performing block processing on a target image, and determining an abnormal pixel value interval of a target sub-image in a plurality of sub-images based on the plurality of sub-images after block processing; determining a continuous area formed by pixels in an abnormal pixel value interval from the target sub-image; and on the basis of the size of the continuous region, determining an abnormal region with abnormality in the target sub-image.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to an abnormal area detection method, device, electronic device and storage medium. Background Art

[0002] Surface defects of industrial products and raw materials often have adverse effects on their aesthetics, comfort and performance, so enterprises need to conduct strict quality inspections on industrial products and raw materials to deal with the defects as necessary. In a related technology, some enterprises use manual sampling to detect surface defects of industrial products and raw materials. The traditional manual sampling method has problems such as poor quality inspection effect and low coverage. In another related technology, some enterprises use machine learning / machine vision to detect surface defects of industrial products and raw materials. The machine learning model needs to collect a large number of sample images for training. It is difficult to collect negative samples (samples with defects), and the machine learning model is sensitive to lighting, color difference, and noise. The brightness of the light, the reasonable difference in color of different batches of products, and various noises during shooting have a great impact on the pixel value of the image. It is difficult for the machine learning model to adapt to these environmental differences. Summary of the invention

[0003] The embodiments of the present application provide an abnormal area detection method, device, electronic device and storage medium to solve the problems existing in the related art.

[0004] In a first aspect, an embodiment of the present application provides an abnormal region detection method, comprising: performing block processing on a target image, and determining an abnormal pixel value interval of a target sub-image among the multiple sub-images based on the multiple sub-images after the block processing; determining a continuous region composed of pixels in the abnormal pixel value interval from the target sub-image; and determining an abnormal region in the target sub-image where an abnormality exists based on the size of the continuous region.

[0005] In a second aspect, an embodiment of the present application provides an abnormal area detection device, comprising: an abnormal pixel value interval determination unit, configured to perform block processing on a target image, and determine the abnormal pixel value interval of a target sub-image among the multiple sub-images after the block processing; a continuous area determination unit, configured to determine a continuous area composed of pixels in the abnormal pixel value interval from the target sub-image; and an abnormal area determination unit, configured to determine an abnormal area in the target sub-image where an abnormality exists based on the size of the continuous area.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0008] Compared with the prior art, this application has the following advantages:

[0009] According to an embodiment of the present application, by dividing a target image into multiple sub-images and combining the spatial information and statistical information of the pixels in the sub-images, the abnormal pixel value interval in each sub-image can be detected without a large number of positive and negative samples. Dividing the target image into sub-images of smaller sizes not only enhances the adaptability of the anomaly detection algorithm to different illumination and reflectivity on the surface of the object, but also effectively reduces the amount of calculation. According to the distribution of pixels in the abnormal pixel value interval, a continuous area can be determined, and according to the size of the continuous area, the abnormal area where the abnormality exists in the target sub-image can be determined. The discovery of abnormal pixels and the splicing of abnormal areas are decoupled from each other. While supporting the efficient discovery of simple defects, the discovery and splicing of complex abnormal pixels can also be increased as needed.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0012] Figure 1 is a flow chart showing an abnormal area detection method according to an embodiment of the present application;

[0013] Figure 2 is a structural block diagram showing an abnormal region detection device according to an embodiment of the present application; and

[0014] Figure 3 is a block diagram showing an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0016] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0017] In a related technology prior to this application, some companies used machine learning / machine vision to detect surface defects of industrial products and raw materials. The training cost of machine learning models is high, and a large number of sample images need to be collected for training, and negative samples (samples with defects) are difficult to collect. The relationship between machine learning models and effects is uncertain, and model tuning is difficult. Machine learning models are sensitive to lighting, color difference, and noise. The brightness of light, reasonable differences in color of different batches of products, and various noises during shooting have a great impact on the pixel value of the image. It is difficult for machine learning models to adapt to these environmental differences. Machine vision algorithms rely heavily on edge geometric element detection algorithms, such as straight line detection and circle detection algorithms, and abnormal areas are usually irregular, so it is difficult for machine vision algorithms to detect irregularly shaped defects.

[0018] An embodiment of the present application provides an abnormal area detection method 100, which is applied to an electronic device, which may be an electronic device such as a server, a computer, a mobile phone, or a smart watch. Figure 1 The abnormal region detection method 100 of the present application is described. Figure 1 FIG. 1 is a flow chart showing an abnormal region detection method according to an embodiment of the present application. Figure 1 As shown, the abnormal region detection method 100 may include the following steps S101 to S103. Steps S101 to S103 will be described in detail below in conjunction with a specific embodiment.

[0019] First, proceed to step S101. In step S101, the target image is divided into blocks, and based on the multiple sub-images after the block processing, the abnormal pixel value interval of the target sub-image in the multiple sub-images is determined.

[0020] The target image is an image used for abnormal area detection, and can be a surface image of a flat material such as a pavement, a tile, a corrugated paper, a textile, a variety of plates and the like. The target image is divided into blocks to obtain multiple sub-images. For example, if the target image is a 2000*2000 image, and the target image is cut into blocks of 200*200, a total of 100 sub-images can be obtained. Taking the python code as an example, the statement blockImg=uniImg[i*200:(i+1)*200,j*200:(j+1)*200] can be used to obtain multiple sub-images after cutting, and the size of the sub-image is 200*200. The target image is divided into blocks to obtain sub-images of smaller size, so that the light brightness and color differences of different batches in the same sub-image are small enough, which not only enhances the adaptability of the abnormal area detection method to different illumination and reflectivity of the object surface, but also effectively reduces the amount of calculation. After obtaining multiple sub-images, the abnormal pixel value interval of the target sub-image can be determined in the multiple sub-images.

[0021] In one embodiment, before step S101, the abnormal region detection method 100 further includes: acquiring an RGB format image, and converting the RGB format image into an HSV format image; and converting the HSV format image into the target image, where the target image is a single-channel image.

[0022] The camera device takes a picture of the surface of the product or raw material that needs to be detected for abnormal areas, and generates an RGB format image. The RGB format is convenient for the machine to display the image, and then the electronic device needs to obtain the RGB format image generated by the camera device. In one embodiment, the electronic device is connected to the camera device for communication, and the camera device can transmit the generated RGB format image to the electronic device. In another embodiment, the owner of the camera device can import the RGB format image into the storage device through a mobile storage device. The present application does not limit the way in which the electronic device obtains the RGB format image.

[0023] After the electronic device obtains the RGB format image, the electronic device can also convert the RGB format image into an HSV format image. In one embodiment, the electronic device first loads the RGB format image into the memory. Taking the python code as an example, the orgImg=cv2.imread(picPath,cv2.IMREAD_COLOR) statement can be used to load the RGB format image stored on the electronic device disk picPath path into the orgImg variable. After the RGB format image is loaded into the memory, the electronic device converts the RGB format image into an HSV image. Also taking the python code as an example, the hsvImg=cv2.cvtColor(orgImg,cv2.COLOR_BGR2HSV) statement can be used to convert the RGB format image stored in orgImg into an HSV format image and store it in the hsvImg variable. The HSV format makes it easier for humans to understand the differences in images. The embodiment of the present application simulates human vision to detect abnormal areas on the surface of products or raw materials, so it is necessary to convert the RGB format image taken by the camera device into an HSV format image so that the electronic device can detect abnormal areas.

[0024] After converting the RGB format image to the HSV format image, it is also necessary to convert the HSV format image to the target image, which is a single-channel image. The HSV format image can be represented by a three-dimensional array, and the number of elements in the three dimensions are h, w, and 3, that is, a three-dimensional array with a height of h, a length of w, and a depth of 3. The third dimension of the three-dimensional array is the channel dimension, which represents the color channel of the image. In the HSV format image, the three color channels are H (Hue), S (color saturation), and V (brightness). Converting the HSV format image to a single-channel image refers to performing dimensionality reduction processing on the image to obtain an image containing only one color channel. For example, the channel dimension can select the V channel in the HSV color space, or it can take the H channel or the S channel. This application does not limit the dimension selected for the single-channel image. For example, the electronic device converts an HSV format image into a target image, and the target image is a single-channel image with a V channel. Taking the Python code as an example, using chan=2 (taking the channel with index 2, that is, the V channel), uniImg=hsvImg[:,:,chan] statement can convert the HSV format image into a single-channel image with a V channel.

[0025] In one embodiment, the method of determining the abnormal pixel value interval of a target sub-image among the multiple sub-images based on the multiple sub-images after block processing includes: dividing the pixel values ​​of all pixels in the target sub-image into multiple intervals; counting the number of pixels corresponding to each interval of the multiple intervals, and determining the interval with the smallest pixel statistical number from the multiple intervals based on the statistically obtained number of pixel statistical numbers corresponding to each interval; and determining the abnormal pixel value interval of the target sub-image based on the pixel statistical numbers of adjacent intervals of the interval with the smallest pixel statistical number.

[0026] The target sub-image can be represented by an array. If the target sub-image is a single-channel image, the target sub-image can be represented by a two-dimensional array. Each element in the two-dimensional array is a pixel, and each pixel has a pixel value. For example, if the size of the target sub-image is 200*200, then the target sub-image corresponds to a two-dimensional array with a length of 200 and a height of 200, that is, there are 40,000 pixels in total, and each pixel has its corresponding pixel value. After obtaining the pixel values ​​of all pixels in the target sub-image, these pixel values ​​can be divided into multiple intervals.

[0027] In one embodiment, before dividing the pixel values ​​of all pixels in the target sub-image into a plurality of intervals, the method further comprises: performing smoothing processing on the target sub-image.

[0028] The role of smoothing is to eliminate small noise points in the image. In one embodiment, the electronic device can use a convolution window to calculate the average value of the target sub-image for smoothing. The size of the convolution window can be 7*7, 8*8, 9*9, etc. If the product for abnormal area detection is a planar material with periodic texture such as a textile, the texture will repeat after a certain interval period, and this period is called the warp and weft period. The size of the convolution window can be set to a multiple of the warp and weft period of this type of planar material (for example, 1 times or 2 times). If the product for abnormal area detection is a planar material without periodic texture, the size of the convolution window can be set to a multiple of the minimum defect size of this type of planar material (for example, 1 times or 2 times). This application does not limit the size of the convolution window. For example, taking Python code as an example, using the statement ker2d=np.ones((win,win)) / (win*win), the convolution window required for smoothing can be generated. If the size of the convolution window is 9*9, then the ker2d (that is, the convolution window) is a two-dimensional array with a length of 9 and a height of 9, and all elements are 1 / (9*9); then using the statement smoothedBlockImg=ndi.convolve(blockImg,ker2d), the target sub-image is convolved with the convolution window to generate a smoothed target sub-image.

[0029] In one embodiment, dividing the pixel values ​​of all pixels in the target sub-image into multiple intervals includes: obtaining the maximum pixel value and the minimum pixel value of all pixels in the target sub-image; and dividing the interval formed by the minimum pixel value and the maximum pixel value equally to obtain the multiple intervals.

[0030] The target sub-image can be represented by a two-dimensional array, and each element in the two-dimensional array is a pixel. Then, the pixel values ​​of all pixels in the two-dimensional array can be obtained, and the maximum pixel value and the minimum pixel value can be obtained among all the pixel values. Then, the interval formed by the minimum pixel value and the maximum pixel value is divided equally to obtain multiple intervals. For example, the minimum pixel value in the target sub-image is 0, the maximum pixel value is 100, and the interval formed by the minimum pixel value and the maximum pixel value is [0,100]. If the interval is divided into 10 equal parts, 10 intervals are obtained, each interval is [0,10), [10,20), and so on, until [90,100]; if the interval is divided into 5 equal parts, 5 intervals are obtained, each interval is [0,20), [20,40), and so on, until [80,100]. The above description of the number of equal divisions of the interval formed by the minimum pixel value and the maximum pixel value is only exemplary, and those skilled in the art can select the number of equal divisions according to actual conditions.

[0031] After obtaining multiple intervals, the number of pixels corresponding to each of the multiple intervals may be counted, and based on the statistical number of pixels corresponding to each interval obtained by counting, an interval with the smallest statistical number of pixels may be determined from the multiple intervals.

[0032] Table 1 below shows an example of a statistical table for counting the number of pixels corresponding to each interval.

[0033] Index Number 0 1 2 3 4 5 6 7 8 9 quantity 1783 17723 7271 5150 4039 2838 972 171 48 5 boundary 35 41.8 48.6 55.4 62.2 69 75.8 82.6 89.4 96.2 103

[0034] Table 1

[0035] Taking Table 1 as an example, the determination of the interval with the smallest number of pixel statistics from multiple intervals is described. The target sub-image counted in Table 1 is a 200*200 image with a total of 40,000 pixel values. Among these 40,000 pixel values, the minimum pixel value is 35 and the maximum pixel value is 103. The interval formed by the minimum pixel value and the maximum pixel value is [35,103]. Dividing the interval into 10 equal parts, 10 intervals can be obtained, each of which is [35,41.8), [41.8,40), and so on, until [96.2,103]. The interval with the smallest number of pixel statistics is [96.2,103], and the number of this interval is 5. After obtaining the interval with the smallest number of pixel statistics, the abnormal pixel value interval of the target sub-image can be determined based on the pixel statistics of the adjacent intervals of this interval.

[0036] In one embodiment, determining the abnormal pixel value interval of the target sub-image based on the pixel statistical number of the adjacent interval of the interval with the smallest pixel statistical number includes: judging whether the pixel statistical number of the adjacent interval is less than a predetermined threshold, wherein if the pixel statistical number of the adjacent interval is less than the predetermined threshold, the adjacent interval and the interval with the smallest pixel statistical number are determined as the abnormal pixel value interval of the target sub-image; if the pixel statistical number of the adjacent interval is not less than the predetermined threshold, the interval with the smallest statistical number is determined as the abnormal pixel value interval.

[0037] Taking Table 1 as an example, the interval with the smallest number of pixel statistics is [96.2, 103], and the number of pixel statistics in this interval is 5. In the target sub-image, the area of ​​the abnormal area is smaller than the area of ​​the normal area, that is, the number of pixels in the abnormal area is less than the number of pixels in the normal area. This is a reliable priori knowledge. Therefore, the interval with the smallest number of pixel statistics [96.2, 103] is the abnormal pixel value interval. The adjacent interval of the interval with the smallest number of pixel statistics is [89.4, 96.2), and the number of pixel statistics in this interval is 48. After obtaining the number of pixel statistics of the adjacent interval, it is necessary to determine whether the number of pixel statistics of the adjacent interval is less than a predetermined threshold. The predetermined threshold can be a multiple of the number of pixel statistics corresponding to the interval with the smallest number of pixel statistics, for example, 10 times, 15 times, 20 times, etc. The predetermined threshold can also be set according to the experience of those skilled in the art, and this application does not limit this. Taking the predetermined threshold value of 10 times the number of pixel statistics corresponding to the interval with the smallest number of pixel statistics as an example, it is necessary to determine whether the number of pixel statistics corresponding to the adjacent interval [89.4, 96.2) is less than 10 times the number of pixel statistics corresponding to the interval with the smallest number of pixel statistics [96.2, 103]. In this example, the number of pixel statistics corresponding to the adjacent interval [89.4, 96.2) is 48, which is less than 10 times (i.e., 50) the number of pixel statistics corresponding to the interval with the smallest number of pixel statistics [96.2, 103]. Therefore, it is necessary to determine the adjacent interval and the interval with the smallest number of pixel statistics (i.e., [89.4, 103]) as the abnormal pixel value interval of the target sub-image. If the number of pixels in the adjacent interval is not less than the predetermined threshold, only the interval with the smallest number of pixel statistics is determined as the abnormal pixel value interval of the target sub-image.

[0038] In one embodiment, the method of determining the abnormal pixel value interval of a target sub-image among the multiple sub-images based on the multiple sub-images after block processing includes: dividing the target sub-image to obtain a plurality of sub-image units, each of the plurality of sub-image units including a plurality of pixels; counting the number of sub-image units corresponding to each interval of the multiple intervals, and determining the abnormal pixel value interval of the target sub-image among the multiple sub-images based on the statistically obtained number of first units corresponding to each interval.

[0039] The electronic device can further divide the target sub-image to obtain a plurality of sub-image units, each of which includes a plurality of pixels. For example, the size of the target sub-image is 200*200, and the target sub-image can be further divided into 100 20*20 sub-image units, each of which has 400 pixels. Then, the number of sub-image units corresponding to each of the multiple intervals can be counted, that is, the pixels in each sub-image unit belong to which interval, and 1 is added to the statistical number corresponding to the interval. For example, there are a total of 100 sub-image units in the target sub-image (which can be numbered as sub-image unit 1 to sub-image unit 100). Taking the intervals shown in Table 1 as an example, the initial value of the number corresponding to these 10 intervals is 0. There are 400 pixels in sub-image unit 1, of which 50 pixels belong to the interval [35,41.8), and the number corresponding to the interval is increased by 1; 200 pixels belong to the interval [41.8,48.6), and the number corresponding to the interval is increased by 1; 1 pixel belongs to [96.2,103], and the number corresponding to the interval is increased by 1. And so on, until the intervals to which all pixels in sub-image units 1 to sub-image units 100 belong are counted. Then, based on the first unit statistical number corresponding to each interval obtained by statistics, the abnormal pixel value interval of the target sub-image can be determined. The first unit statistical number can be the number obtained by counting all sub-image units in the target sub-image, or the number obtained by counting sub-image units in the row and / or column to which a certain sub-image unit belongs.

[0040] In one embodiment, the determining of the abnormal pixel value interval of the target sub-image among the multiple sub-images based on the first unit statistical quantity corresponding to each interval obtained by statistics includes: performing statistics on all sub-image units in the target sub-image in each interval of the multiple intervals, and determining the abnormal pixel value interval of the target sub-image among the multiple sub-images based on the second unit statistical quantity corresponding to each interval; or, selecting any one sub-image unit from the multiple sub-image units, performing statistics on the sub-image units in the row and / or column to which the any one sub-image unit belongs in each interval of the multiple intervals, and determining the abnormal pixel value interval of the target sub-image among the multiple sub-images based on the third unit statistical quantity corresponding to each interval.

[0041] The electronic device can count all sub-image units in the target sub-image in each interval to obtain a second unit statistical number. For example, if the target sub-image includes 100 sub-image units, then the intervals to which the pixels in the 100 sub-image units belong can be counted. Table 2 below shows an example of a statistical table for counting all sub-image units in the target sub-image in each interval.

[0042] Index Number 0 1 2 3 4 5 6 7 8 9 quantity 55 95 80 70 65 58 53 52 20 2 boundary 35 41.8 48.6 55.4 62.2 69 75.8 82.6 89.4 96.2 103

[0043] Table 2

[0044] For example, taking the interval [35,41.8) in Table 2 as an example, it can be known that there are pixels in 55 of the 100 sub-image units of the target sub-image that belong to the interval [35,41.8). Similarly, the second unit statistical numbers corresponding to other intervals can be known. According to the prior knowledge that the area of ​​the abnormal area is smaller than the area of ​​the normal area, the interval corresponding to the second unit statistical number less than half of the number of all sub-image units in the target sub-image can be determined as an abnormal pixel value interval. Taking Table 2 as an example, half of the number of all sub-image units in the target sub-image is 50, and the second unit statistical numbers corresponding to the intervals [89.4,96.2) and [96.2,103] are less than 50, then these two intervals (i.e. [89.4,103]) can be determined as abnormal pixel value intervals. The interval corresponding to the second unit statistical number less than 1 / 3 or 1 / 4 of the number of all sub-image units in the target sub-image can also be determined as an abnormal pixel value interval, and this application does not impose any restrictions on this.

[0045] The electronic device may also select any one sub-image unit from the plurality of sub-image units, and count the sub-image units in the row and / or column to which the arbitrary sub-image unit belongs in each interval to obtain a third unit statistical quantity. For example, the target sub-image includes 100 sub-image units (i.e., an image with a length of 10 and a height of 10), and any one sub-image unit (e.g., a sub-image unit with coordinates (2,3)) may be selected from the 100 sub-image units, and the interval to which the pixels in the sub-image unit in the row and / or column to which the optional sub-image unit belongs is counted. Table 3 below shows an example of a statistical table for counting the sub-image units in the row to which any one sub-image unit belongs in each interval.

[0046] Index Number 0 1 2 3 4 5 6 7 8 9 quantity 5 9 8 7 6 6 5 5 2 1 boundary 35 41.8 48.6 55.4 62.2 69 75.8 82.6 89.4 96.2 103

[0047] Table 3

[0048] For example, taking the interval [35,41.8) in Table 3 as an example, it can be known that there are pixels in 5 sub-image units in the row (i.e., 10 sub-image units) to which any sub-image unit belongs, and by analogy, the third unit statistics corresponding to other intervals can be known. According to the prior knowledge that the area of ​​the abnormal area is smaller than the area of ​​the normal area, the interval corresponding to the third unit statistics less than half of the number of sub-image units in the row to which any sub-image unit belongs can be determined as an abnormal pixel value interval. Taking Table 3 as an example, half of the number of sub-image units in the row to which any sub-image unit belongs is 5, and the third unit statistics corresponding to the intervals [89.4,96.2) and [96.2,103] are less than 5, then these two intervals (i.e., [89.4,103]) can be determined as abnormal pixel value intervals. The interval corresponding to the third unit statistics less than 1 / 3 or 1 / 4 of the number of sub-image units in the row to which any sub-image unit belongs can also be determined as an abnormal pixel value interval, and this application does not limit this.

[0049] According to the embodiment of the present application, the abnormal pixel value interval is determined by an algorithm combining spatial information and statistical information in the target sub-image, has high adaptability, does not rely on a large number of positive and negative samples for training, and reduces the implementation cost of the present application.

[0050] Next, the process proceeds to step S102. In step S102, a continuous region consisting of pixels in the abnormal pixel value interval is determined from the target sub-image.

[0051] In one embodiment, the step of determining the continuous region formed by pixels in the abnormal pixel value interval from the target sub-image comprises: obtaining at least one pixel in the abnormal pixel value interval from the target sub-image; and determining the continuous region formed by pixels with adjacent coordinates from the at least one pixel. After obtaining the abnormal pixel value interval of the target sub-image, the electronic device can obtain all pixels in the target sub-image whose pixel values ​​are in the abnormal pixel value interval, and obtain pixels with adjacent coordinates, and then can splice these pixels with adjacent coordinates through a splicing algorithm to determine the continuous region formed by these adjacent pixels. For example, the pixels in the target sub-image whose pixel values ​​are in the abnormal pixel value interval are A, B, C, and D, and the coordinates corresponding to these four pixels are (1,1), (1,2), (2,1), and (3,2), respectively. From the coordinates of the above pixels, it can be known that the pixels with adjacent coordinates are A, B, and C. The pixels A, B, and C are spliced ​​to determine the continuous region formed by the pixels A, B, and C.

[0052] Next, the process proceeds to step S103. In step S103, based on the size of the continuous region, an abnormal region in the target sub-image is determined.

[0053] After determining the continuous area in the target sub-image, the electronic device can determine an abnormal area in the target sub-image where an abnormality exists based on the size of the continuous area. The abnormal area is an area where a defect or flaw exists in the product or raw material.

[0054] In one embodiment, based on the size of the continuous region, determining the abnormal region where the abnormality exists in the target sub-image includes: obtaining a size threshold, the size threshold including at least one of the following: an area threshold, a height threshold, and a length threshold; and selecting a continuous region that meets the size threshold from the continuous region as the abnormal region where the abnormality exists in the target sub-image.

[0055] The size threshold can be set by the user according to the user's preference. For example, the user can set the area threshold to 20, and the continuous area with an area greater than 20 is determined to be an abnormal area. For another example, the user can set the area threshold to 20, the height threshold to 5, and the length threshold to 4, and the continuous area with an area greater than 20, a height greater than 5, and a length greater than 4 is determined to be an abnormal area. For another example, the user hopes to obtain a long and narrow area with a large span as an abnormal area. The user can set the height threshold or the length threshold to 20, and the continuous area with a height or length greater than 20 is determined to be an abnormal area. The user can also set the maximum interval of non-continuous areas to obtain abnormal areas with complex distribution. The above description of the size threshold is only exemplary, and those skilled in the art can select the specific content included in the size threshold according to actual conditions.

[0056] In one embodiment, after step S103, the abnormal region detection method 100 further includes: determining whether the abnormal region in the target sub-image overlaps with the boundary of the target sub-image and is continuous with the abnormal region of the adjacent sub-image of the target sub-image, wherein, if so, the abnormal region in the target sub-image and the abnormal region of the adjacent sub-image of the target sub-image are merged, and the merged abnormal region is reported.

[0057] After determining the abnormal area, the electronic device needs to summarize and report the abnormal area in the target sub-image. If the abnormal area does not overlap with the boundary of the target sub-image, the abnormal area is directly summarized and reported. If the abnormal area overlaps with the boundary of the target sub-image, it is necessary to further check whether the area is continuous with the abnormal area of ​​the adjacent target sub-image. If continuous, the continuous abnormal areas are merged and summarized and reported; if not continuous, the abnormal area is directly summarized and reported.

[0058] In one embodiment, after the abnormal areas in the target sub-image are summarized, the electronic device can report all abnormal areas. In another embodiment, the user can set the number of abnormal areas reported according to his or her preferences, and the electronic device reports the abnormal areas that meet the number set by the user. The present application does not limit the number of abnormal areas reported. After obtaining the reported abnormal areas, the user can use the display software to display these abnormal areas, and further confirm and execute corresponding strategies, such as pausing the machine, manually repairing, marking, etc.

[0059] According to the embodiments of the present application, easy-to-understand user preference parameters are used to control the number of abnormal areas, which better adapts to user needs in different scenarios. The discovery of abnormal pixels and the splicing of abnormal areas are decoupled from each other. While supporting the efficient discovery of simple defects, the discovery and splicing of complex abnormal pixels can also be increased as needed.

[0060] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, an embodiment of the present application also provides an abnormal area detection device, which is deployed in an electronic device, and the electronic device can be an electronic device such as a server, a computer, a mobile phone, or a smart watch. Figure 2 The abnormal region detection device 200 of the present application is described. Figure 2 FIG. 1 is a block diagram showing a structure of an abnormal region detection device according to an embodiment of the present application. Figure 2 As shown, the abnormal region detection device 200 may include: an abnormal pixel value interval determination unit 201, a continuous region determination unit 202 and an abnormal region determination unit 203.

[0061] The abnormal pixel value interval determining unit 201 is configured to perform block processing on the target image, and determine the abnormal pixel value interval of the target sub-image among the multiple sub-images based on the multiple sub-images after the block processing.

[0062] The abnormal pixel value interval determining unit 201 may be a component in an electronic device, and the electronic device determines the abnormal pixel value interval of the target sub-image through the component.

[0063] The continuous region determining unit 202 is configured to determine a continuous region consisting of pixels in the abnormal pixel value interval from the target sub-image.

[0064] The continuous region determining unit 202 may be a component in an electronic device, and the electronic device uses the component to determine a continuous region formed by pixels in an abnormal pixel value interval.

[0065] The abnormal region determining unit 203 is configured to determine an abnormal region where an abnormality exists in the target sub-image based on the size of the continuous region.

[0066] In one embodiment, the abnormal pixel value interval determination unit 201 is further configured to divide the pixel values ​​of all pixels in the target sub-image into multiple intervals; count the number of pixels corresponding to each of the multiple intervals, and determine the interval with the smallest pixel statistical number from the multiple intervals based on the statistically obtained pixel statistical number corresponding to each interval; determine the abnormal pixel value interval of the target sub-image based on the pixel statistical number of adjacent intervals of the interval with the smallest pixel statistical number.

[0067] In one embodiment, the abnormal pixel value interval determining unit 201 includes a smoothing processing module configured to perform smoothing processing on the target sub-image.

[0068] In one embodiment, the abnormal pixel value interval determination unit 201 includes an interval division module, which is configured to obtain the maximum pixel value and the minimum pixel value of all pixels in the target sub-image; and divide the interval formed by the minimum pixel value and the maximum pixel value into equal parts to obtain the multiple intervals.

[0069] In one embodiment, the abnormal pixel value interval determination unit 201 is further configured to determine whether the pixel statistical number of the adjacent interval is less than a predetermined threshold, wherein, if the pixel statistical number of the adjacent interval is less than the predetermined threshold, the adjacent interval and the interval with the smallest pixel statistical number are determined as the abnormal pixel value interval of the target sub-image; if the pixel statistical number of the adjacent interval is not less than the predetermined threshold, the interval with the smallest pixel statistical number is determined as the abnormal pixel value interval.

[0070] In one embodiment, the abnormal pixel value interval determination unit 201 is further configured to divide the target sub-image to obtain a plurality of sub-image units, each of the plurality of sub-image units including a plurality of pixels; count the number of sub-image units corresponding to each interval of the plurality of intervals, and determine the abnormal pixel value interval of the target sub-image in the plurality of sub-images based on the statistically obtained number of first units corresponding to each interval.

[0071] In one embodiment, the abnormal pixel value interval determination unit 201 is further configured to perform statistics on all sub-image units in the target sub-image in each interval of the multiple intervals, and determine the abnormal pixel value interval of the target sub-image in the multiple sub-images based on the second unit statistical quantity corresponding to each interval; or, select any one sub-image unit from the multiple sub-image units, perform statistics on the sub-image units in the row and / or column to which the any one sub-image unit belongs in each interval of the multiple intervals, and determine the abnormal pixel value interval of the target sub-image in the multiple sub-images based on the third unit statistical quantity corresponding to each interval.

[0072] In one embodiment, the continuous region determining unit 202 is further configured to obtain at least one pixel in the abnormal pixel value interval from the target sub-image; and determine a continuous region consisting of pixels with adjacent coordinates from the at least one pixel.

[0073] In one embodiment, the abnormal region determination unit 203 is further configured to obtain a size threshold, wherein the size threshold includes at least one of the following: an area threshold, a height threshold, and a length threshold; and a continuous region satisfying the size threshold is selected from the continuous regions as the abnormal region where an abnormality exists in the target sub-image.

[0074] In one embodiment, the abnormal area detection device 200 also includes an abnormal area reporting unit, which is configured to determine whether the abnormal area in the target sub-image overlaps with the boundary of the target sub-image and is continuous with the abnormal area of ​​the adjacent sub-image of the target sub-image, wherein, if so, the abnormal area in the target sub-image and the abnormal area of ​​the adjacent sub-image of the target sub-image are merged, and the merged abnormal area is reported.

[0075] The abnormal area reporting unit may be a component in the electronic device, and the electronic device reports the abnormal area through the component.

[0076] In one embodiment, the abnormal region detection device 200 further includes an image conversion unit configured to acquire an RGB format image and convert the RGB format image into an HSV format image; and convert the HSV format image into the target image, which is a single-channel image.

[0077] The image conversion unit may be a component in an electronic device, and the electronic device converts the acquired image into a single-channel target image through the component.

[0078] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0079] According to the embodiment of the present application, by dividing the target image into multiple sub-images and combining the spatial information and statistical information of the pixels in the sub-images, the abnormal pixel value interval in each sub-image can be detected, and then the abnormal area in each sub-image can be detected. Dividing the target image into sub-images of smaller sizes not only enhances the adaptability of the abnormality detection algorithm to different illumination and reflectivity of the object surface, but also effectively reduces the amount of calculation.

[0080] According to the embodiment of the present application, the abnormal pixel value interval is determined by an algorithm combining spatial information and statistical information in the target sub-image, has high adaptability, does not rely on a large number of positive and negative samples for training, and reduces the implementation cost of the present application.

[0081] According to the embodiments of the present application, easy-to-understand user preference parameters are used to control the number of abnormal areas, which better adapts to user needs in different scenarios. The discovery of abnormal pixels and the splicing of abnormal areas are decoupled from each other. While supporting the efficient discovery of simple defects, the discovery and splicing of complex abnormal pixels can also be increased as needed.

[0082] Figure 3 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 3 As shown, the electronic device includes: a memory 301 and a processor 302. The memory 301 stores a computer program that can be run on the processor 302. When the processor 302 executes the computer program, the method in the above embodiment is implemented. The number of the memory 301 and the processor 302 can be one or more.

[0083] The electronic device also includes:

[0084] The communication interface 303 is used to communicate with external devices and perform data exchange transmission.

[0085] If the memory 301, the processor 302 and the communication interface 303 are implemented independently, the memory 301, the processor 302 and the communication interface 303 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0086] Optionally, in a specific implementation, if the memory 301, the processor 302 and the communication interface 303 are integrated on a chip, the memory 301, the processor 302 and the communication interface 303 can communicate with each other through an internal interface.

[0087] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0088] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0089] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0090] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0091] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0092] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0093] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0094] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0095] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0096] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0097] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0098] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0099] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting an abnormal region, comprising: Performing block processing on the target image, and determining an abnormal pixel value interval of the target sub-image among the multiple sub-images based on the multiple sub-images after the block processing; Determine a continuous region consisting of pixels in the abnormal pixel value interval from the target sub-image; Based on the size of the continuous region, an abnormal region in the target sub-image where an abnormality exists is determined.

2. The method according to claim 1, wherein: The determining, based on the multiple sub-images after the block processing, an abnormal pixel value interval of a target sub-image among the multiple sub-images comprises: Dividing the pixel values ​​of all pixels in the target sub-image into a plurality of intervals; Counting the number of pixels corresponding to each of the multiple intervals, and determining an interval with a minimum number of pixel statistics from the multiple intervals based on the statistically obtained number of pixel statistics corresponding to each interval; Based on the pixel statistics of the adjacent intervals of the interval with the smallest pixel statistics, the abnormal pixel value interval of the target sub-image is determined.

3. The method according to claim 2, before dividing the pixel values ​​of all pixels in the target sub-image into a plurality of intervals, the method further comprises: The target sub-image is smoothed.

4. The method according to claim 2, wherein: The step of dividing the pixel values ​​of all pixels in the target sub-image into a plurality of intervals comprises: Obtaining the maximum pixel value and the minimum pixel value of all pixels in the target sub-image; The interval formed by the minimum pixel value and the maximum pixel value is divided into equal parts to obtain the multiple intervals.

5. The method according to any one of claims 2 to 4, wherein: The determining of the abnormal pixel value interval of the target sub-image based on the pixel statistical quantity of the adjacent interval of the interval with the smallest pixel statistical quantity includes: It is determined whether the pixel statistics of the adjacent interval is less than a predetermined threshold, wherein: If the pixel statistics of the adjacent interval is less than a predetermined threshold, the adjacent interval and the interval with the smallest pixel statistics are determined as the abnormal pixel value interval of the target sub-image; If the pixel statistics of the adjacent intervals are not less than a predetermined threshold, the interval with the smallest pixel statistics is determined as an abnormal pixel value interval.

6. The method according to any one of claims 2 to 4, wherein: The determining, based on the multiple sub-images after the block processing, an abnormal pixel value interval of a target sub-image among the multiple sub-images comprises: Performing a division process on the target sub-image to obtain a plurality of sub-image units, each of the plurality of sub-image units comprising a plurality of pixels; The number of sub-image units corresponding to each of the multiple intervals is counted, and based on the statistically obtained number of first unit statistics corresponding to each of the multiple intervals, an abnormal pixel value interval of a target sub-image among the multiple sub-images is determined.

7. The method according to claim 6, wherein: The determining, based on the first unit statistical quantity corresponding to each interval obtained by statistics, of an abnormal pixel value interval of a target sub-image among the multiple sub-images comprises: In each of the plurality of intervals, statistics are performed on all sub-image units in the target sub-image, and based on the second unit statistics corresponding to each interval, an abnormal pixel value interval of the target sub-image in the plurality of sub-images is determined; or, Select any one sub-image unit from the multiple sub-image units, count the sub-image units in the row and / or column to which the any one sub-image unit belongs in each interval of the multiple intervals, and determine the abnormal pixel value interval of the target sub-image in the multiple sub-images based on the statistical number of the third unit corresponding to each interval.

8. The method according to claim 1 or 2, wherein: The step of determining, from the target sub-image, a continuous region consisting of pixels in the abnormal pixel value interval comprises: Acquire at least one pixel in the abnormal pixel value interval from the target sub-image; A continuous region consisting of pixels having adjacent coordinates is determined from the at least one pixel.

9. The method according to claim 1 or 2, wherein: Determining an abnormal region in the target sub-image where an abnormality exists based on the size of the continuous region includes: Obtaining a size threshold, where the size threshold includes at least one of the following: an area threshold, a height threshold, and a length threshold; A continuous region that meets the size threshold is selected from the continuous regions as an abnormal region where an abnormality exists in the target sub-image.

10. The method according to claim 1 or 2, further comprising: Determine whether the abnormal region in the target sub-image overlaps with the boundary of the target sub-image and is continuous with the abnormal region of the adjacent sub-image of the target sub-image, wherein: If yes, the abnormal region in the target sub-image and the abnormal region of the adjacent sub-image of the target sub-image are merged, and the merged abnormal region is reported.

11. The method according to claim 1 or 2, further comprising: Acquire an RGB format image, and convert the RGB format image into an HSV format image; The HSV format image is converted into the target image, where the target image is a single-channel image.

12. An abnormal area detection device, comprising: an abnormal pixel value interval determination unit configured to perform block processing on the target image and determine the abnormal pixel value interval of the target sub-image among the multiple sub-images based on the multiple sub-images after the block processing; a continuous region determining unit configured to determine, from the target sub-image, a continuous region consisting of pixels in the abnormal pixel value interval; The abnormal region determining unit is configured to determine an abnormal region where an abnormality exists in the target sub-image based on the size of the continuous region.

13. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images