Dust identification method, device and terminal equipment

Abstract

The present application is applicable to the field of image processing technology, and provides a dust identification method, device and terminal equipment. The dust identification method comprises: acquiring a video image of a target area; processing the video image to obtain a foreground binary image; performing object contour detection on the foreground binary image to obtain an object contour set containing multiple object contours; based on at least one of the color feature, shape feature and diffusion feature of the dust, detecting each object contour in the object contour set to determine whether each object contour contains dust. The present application can save the step of densely arranging a large number of dust detectors, avoid the impact of the installation process on production activities, reduce the complexity of installation, and reduce the cost of commissioning and device maintenance.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a dust identification method, device and terminal equipment. Background Art

[0002] Metallurgy, building materials, metal mines, quarries, construction waste treatment, building blasting, ports, industrial yards and other places will generate dust during the transportation of raw materials, yard construction, material transfer, production operations, etc., which is a common unorganized dust emission. The above-mentioned unorganized dust emission has the characteristics of unconcentrated emission, irregularity, uncertain diffusion, and a large diffusion range, which can easily cause non-point source pollution. Therefore, it is extremely difficult to effectively monitor and control it, which also makes it difficult for traditional single-point dust removal technologies such as bag / cartridge dust collectors, electrostatic precipitators, water spray / fog and other dust removal equipment to effectively play an ideal role in the face of unorganized dust emission.

[0003] At present, some industries and enterprises are promoting the use of dust concentration monitors for dust monitoring. However, this method can only detect the dust concentration, PM2.5, PM10, temperature, humidity, atmospheric pressure, wind direction, and wind force at some local points where the dust concentration online monitor is installed, and cannot detect the dust concentration at any location in the area. If accurate monitoring of regional dust is to be achieved, a large number of dust concentration monitors must be densely arranged throughout the entire yard, operation and other places, which will have high investment costs, complex installation, and high maintenance costs. The installation bracket will seriously affect the movement of vehicles and personnel operations in the yard, operation and other places. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the embodiments of the present application provide a dust identification method, an apparatus and a terminal device.

[0005] This application is implemented through the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a dust identification method, comprising: acquiring a video image of a target area; processing the video image to obtain a foreground binary image; performing object contour detection on the foreground binary image to obtain an object contour set including multiple object contours; based on at least one of the color characteristics, shape characteristics, and diffusion characteristics of the dust, detecting each object contour in the object contour set to determine whether each object contour contains dust.

[0007] The above dust identification method obtains a video image of the target area, processes the video image to obtain a foreground binary image, and then performs object contour detection on the foreground binary image to obtain an object contour set containing multiple object contours. Finally, based on the characteristics of dust, each object contour in the object contour set is detected to see whether it contains dust. Therefore, the above dust identification method can save the step of densely arranging a large number of dust detectors, avoid the impact of the installation process on production activities, reduce the complexity of installation, and reduce the cost of commissioning and device maintenance.

[0008] Based on the first aspect, in some possible implementations, the method detects each object contour in the object contour set based on at least one of the color features, shape features and diffusion features of the dust, and determines whether each object contour contains a dust contour, including: extracting the area contour of each motion area in the object contour, determining the minimum circumscribed rectangle of the vertical boundary of the area contour, and the pixel area and centroid coordinates of the area contour; identifying the dust in the area contour based on the aspect ratio and rectangularity of the minimum circumscribed rectangle to obtain a suspected dust area, wherein the rectangularity is the ratio of the contour area to the area of ​​the minimum circumscribed rectangle; calculating the pixel grayscale value range corresponding to the dust in the dust color feature detection rule, and identifying the suspected dust area based on the pixel grayscale value range.

[0009] Based on the first aspect, in some possible implementations, the suspected dust area is identified based on the pixel grayscale value range to obtain a first dust area. After the suspected dust area is identified based on the pixel grayscale value range, it also includes: determining whether the first dust area is a real dust area based on the rate of change of the area of ​​the same area contour corresponding to adjacent video frames.

[0010] Based on the first aspect, in some possible implementations, the determining whether the first dust area is a real dust area based on the rate of change of the area of ​​the contour of the same area corresponding to adjacent video frames includes: for any two objects, if the centroid coordinates of the motion contours satisfy a preset relationship, determining that the two objects are the same object and counting the two objects; otherwise, adding the two objects to a preset suspected dust list, and counting and timing the two objects; if the number of second objects in the suspected dust list exceeds a number threshold, and the growth rate of the area of ​​the second object reaches a growth rate threshold, determining that the second object is dust; if the existence time of the first object in the suspected dust list exceeds a time threshold, eliminating the first object from the suspected dust list, the first object is any object in the suspected dust list.

[0011] Based on the first aspect, in some possible implementations, the processing of the video image to obtain a foreground binary image includes: defogging the video image to obtain a grayscale image; performing background modeling on the grayscale image, extracting foreground moving objects by a background subtraction method, and obtaining the foreground binary image.

[0012] Based on the first aspect, in some possible implementations, the defogging process is performed on the video image to obtain a grayscale image, including: calculating the grayscale histogram of the video image by channel to obtain a maximum grayscale value and a minimum grayscale value, wherein the maximum grayscale value and the minimum grayscale value constitute an extreme value range; dividing the video image into multiple image blocks; for a first image block whose grayscale value is outside the extreme value range, assigning the grayscale of the first image block to 0 or 255; for a second image block whose grayscale value is within the extreme value range, linearly transforming the grayscale of the second image block to between 0 and 255 to obtain the grayscale image; wherein the coefficient of the linear transformation is k′ is a constant, is the internal grayscale variance of the second image block, is the image noise variance of the second image block.

[0013] Based on the first aspect, in some possible implementations, before performing object contour detection on the foreground binary image, the method includes: performing erosion and dilation processing on the foreground binary image to remove image noise in the foreground binary image.

[0014] In a second aspect, an embodiment of the present application provides a dust identification device, comprising: an acquisition module, used to acquire a video image of a target area; a processing module, used to process the video image to obtain a foreground binary image; an object contour recognition module, used to perform object contour detection on the foreground binary image to obtain an object contour set containing multiple object contours; a dust identification module, used to detect each object contour in the object contour set based on at least one of the color characteristics, shape characteristics and diffusion characteristics of the dust, and determine whether the object contour set contains a dust contour.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the dust identification method as described in any one of the first aspects when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the dust identification method as described in any one of the first aspects is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the dust identification method described in any one of the first aspects above.

[0018] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 This is a schematic diagram of an application scenario of a dust identification method provided by an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of the process of a dust identification method provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a video image defogging process provided by an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of a process of background modeling of a grayscale image by using a Gaussian mixture model provided in an embodiment of the present application;

[0024] Figure 5 is a foreground binary image processed by a mixed Gaussian model provided in an embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of a process for determining a set of object contours of a foreground binary image provided by an embodiment of the present application;

[0026] Figure 7 It is a schematic diagram of a process for determining whether an image contains dust based on the diffusion characteristics of dust provided by an embodiment of the present application;

[0027] Figure 8 It is an image determined as dust by diffusion characteristics provided in the embodiment of the present application, and has an experimental visual calibration;

[0028] Fig. 9 is a schematic diagram of the structure of a dust identification device provided in an embodiment of the present application;

[0029] Fig.10is a schematic diagram of the structure of a dust identification device provided in an embodiment of the present application;

[0030] Fig.11 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0032] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0033] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0035] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0037] In places such as metallurgy, building materials, metal mines, quarries, construction waste treatment, building blasting, ports and industrial yards, dust is generated during raw material transportation, yard construction, material transfer, production operations, etc., which is common unorganized dust emissions. The above-mentioned unorganized dust emissions have the characteristics of unconcentrated and irregular emissions, uncertain diffusion, and a large diffusion range, which can easily cause non-point source pollution. Therefore, it is extremely difficult to effectively monitor and control them, which also leads to traditional single-point dust removal technologies such as bag / cartridge dust collectors, electrostatic precipitators, water spray / fog and other dust removal equipment. It is difficult to effectively play an ideal role when facing unorganized dust emissions.

[0038] At present, some industries and enterprises are promoting the use of dust concentration monitors for dust monitoring. However, this method can only detect the dust concentration, PM2.5, PM10, temperature, humidity, atmospheric pressure, wind direction, and wind force at some local points where the dust concentration online monitor is installed, and cannot detect the dust concentration at any location in the area. If accurate monitoring of regional dust is to be achieved, a large number of dust concentration monitors must be densely arranged throughout the entire yard, operation and other places, which will have high investment costs, complex installation, and high maintenance costs. The installation bracket will seriously affect the movement of vehicles and personnel operations in the yard, operation and other places.

[0039] Based on the above problems, the embodiment of the present application provides a dust recognition method, which obtains a video image of a target area, processes the video image to obtain a foreground binary image, and then performs object contour detection on the foreground binary image to obtain an object contour set containing multiple object contours. Finally, based on the characteristics of dust, it is detected whether each object contour in the object contour set contains dust.

[0040] For example, the embodiments of the present application can be applied to Figure 1In the exemplary scenario shown. The scenario includes a video image acquisition device 10 and a terminal device 20. The video image acquisition device 10 is used to acquire a video image of a target area and send the video image to the terminal device 20. The video image may contain dust. The terminal device 20 is used to identify whether the video image contains dust.

[0041] For example, the terminal device 20 processes the video image to obtain a foreground binary image, then performs object contour detection on the foreground binary image to obtain an object contour set including multiple object contours, and then detects whether each object contour in the object contour set contains dust based on the characteristics of dust.

[0042] In this embodiment, the terminal device 20 can be an industrial robot, a mobile phone, a computer, a tablet computer, a laptop computer, a netbook, a personal digital assistant (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device 20.

[0043] In order to make those skilled in the art better understand the present invention, Figure 1 , the technical solutions in the embodiments of the present application are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Figure 2 is a schematic flow chart of a dust identification method provided in an embodiment of the present application, referring to Figure 2 , the dust identification method is described in detail as follows:

[0045] In step 101, a video image of a target area is acquired.

[0046] The image acquired by the camera in real time is a video stream. Exemplarily, different video images of the target scene at various time points can be acquired by the camera. For example, the image of the target scene in the video frame can be saved by the OpenCv algorithm library. The cap.read() algorithm in the OpenCv algorithm library is called to save the video stream acquired by the camera frame by frame. Of course, video images at different time points in the video stream can also be acquired by other algorithms.

[0047] In one scenario, the terminal device can send a video image acquisition instruction to the video image acquisition device once every preset time. The video image acquisition device acquires the video image of the target area based on the instruction and returns the video image within a certain time period to the terminal device, so that the terminal device executes subsequent steps to identify dust in the video image.

[0048] In another scenario, the video image acquisition device may also return the real-time acquired video images to the terminal device, so that the terminal device executes subsequent steps to identify dust in the video image, which is not limited in this embodiment of the present application.

[0049] In step 102, the video image is processed to obtain a foreground binary image.

[0050] Exemplarily, step 102 may include: performing defogging, size reduction and other processing on the video image sequence to obtain a relatively clear grayscale image sequence. Thereafter, background modeling is performed on the grayscale image, and foreground moving objects are extracted by background difference method to obtain a foreground binary image.

[0051] The adaptive histogram equalization method is used to perform dehazing on the video image sequence, specifically: the grayscale histogram of the video image is calculated by channel, and the maximum grayscale value and the minimum grayscale value are obtained, and the maximum grayscale value and the minimum grayscale value constitute the extreme value range; the video image is divided into multiple image blocks; for the first image block whose grayscale value is outside the extreme value range, the grayscale of the first image block is assigned to 0 or 255; for the second image block whose grayscale value is within the extreme value range, the grayscale of the second image block is linearly transformed to between 0 and 255 to obtain a grayscale image. The coefficients of the linear transformation are k′ is a constant, is the internal grayscale variance of the second image block, is the image noise variance of the second image block.

[0052] See also Figure 3 , the defogging process of video image A is taken as an example for explanation. Wherein, video image A is a video image obtained by converting a video stream acquired by a camera, and video image A is a single frame image.

[0053] Specifically, after acquiring the video image A, the grayscale levels of the pixels in the video image A are counted, and the grayscale histogram of the video image A is established according to the pixel grayscale levels obtained by the statistics. After that, the cumulative histogram of the grayscale histogram is calculated, and the mapping relationship between the grayscale levels of the grayscale histogram and the grayscale levels of the cumulative histogram is determined. According to the mapping relationship, the grayscale level of each pixel of the image is cyclically output to obtain a new histogram A', that is, the foreground binary image of the video image A.

[0054] In this embodiment, the background modeling of the grayscale image is performed by using the Gaussian mixture model algorithm, and the parameters of the kNN background subtractor based on OpenCV are set according to the experimental data to reduce the influence of similar targets such as the water mist sprayed by the fog cannon.

[0055] For example, each pixel of the background image can be modeled by a mixed Gaussian model composed of K Gaussian distributions, that is:

[0056]

[0057] Among them, x j represents the value of pixel j at time t. If the pixel has three channels, RGB, then x j is a vector, x j =[x jR x jG x jB ], represents the estimated value of the weight coefficient of the i-th Gaussian distribution in the mixed Gaussian model at time t, and Respectively represent the mean vector and covariance matrix of the i-th Gaussian distribution in the mixed Gaussian model at time t (here it is assumed that the red, green, and blue components of the pixel are independent of each other); η represents the Gaussian distribution probability density function. Where:

[0058]

[0059]

[0060]

[0061] In the first frame of the image, the first Gaussian distribution corresponding to each pixel is initialized, the mean is assigned to the value of the current pixel, the weight is assigned to 1, and the means and weights of other Gaussian distribution functions except the first Gaussian distribution are initialized to zero.

[0062] At time t, for each pixel X in the image frame t Match it with the corresponding Gaussian model, the matching rule is: if the pixel value X t If the distance from the mean of the i-th Gaussian distribution Gi in the mixed Gaussian model is less than 2.5 times its standard deviation, then the Gaussian distribution Gi is defined as t match.

[0063] If it is found that there is at least one Gaussian distribution in the mixed Gaussian model of the pixel that matches the pixel value X t If the Gaussian mixture model is matched, the parameter update rule of the Gaussian mixture model is as follows: 1) For unmatched Gaussian distributions, their mean μ and covariance matrix remain unchanged; 2) The mean μ and covariance matrix of the matched Gaussian distribution Gi are updated as follows:

[0064] μ i,t =(1-ρ)·μ i,t-1 +ρ·X t

[0065] ∑ j,t =(1-ρ)·∑ j,t +ρ·diag[(X t -μ i,t ) T ](X t -μ i,t )

[0066]

[0067] Where α is the learning rate for parameter estimation.

[0068] There are two important factors that affect whether a distribution is a background distribution: (1) the proportion of data generated by the distribution; (2) the variance of the distribution. Based on these two factors, the following method is used for estimation:

[0069] Sort the k Gaussian distributions that make up each pixel's mixed Gaussian model by the ratio of ωk / |∑k| from large to small. ωk represents the proportion of data generated by the kth distribution, and ∑k represents the variance of the kth distribution. Select the first B Gaussian distributions in the above sequence as the background pixel model:

[0070]

[0071] Wherein, T is a predetermined threshold (0.5≤T≤1); B is the first B Gaussian distributions among the sorted K Gaussian distributions that best describe the background pixel.

[0072] Recheck each pixel value X at time t t The matching relationship with the first B Gaussian distributions obtained is that if the pixel value X t If it matches any of the first B Gaussian distributions, the pixel is a background point, otherwise the pixel is classified as a foreground, i.e., a moving object.

[0073] See also Figure 4 Taking video image a as an example, select the previous frame video image a' and the next frame video image a", and extract the background of images a, a', and a" by simulating each background in images a, a', and a", and check whether the pixels in the background of images a, a', and a" match the background Gaussian model. If the pixel matches the background Gaussian model, it is the background, otherwise it is the foreground.

[0074] Specifically, the parameters in the predefined Gaussian model are initialized to obtain the required parameters. Afterwards, the first pixel in the consecutive video frame images a, a', a" is processed to determine whether the first pixel matches a certain model, and the first pixel is a pixel at the same position arbitrarily selected in the video frame images a, a', a". If the first pixel matches a certain model, the first pixel is included in the model, and the model is updated according to the new pixel value. If the first pixel does not match any model, a Gaussian model is established with the first pixel. Initialize the parameters to replace the least likely model in the original model; select the first few most likely models to obtain an approximate background model, and then obtain the foreground image.

[0075] See also Figure 5 , Figure 5 is the foreground binary image processed by the mixed Gaussian model. The original example scene video image is processed by step 102 to obtain the following Figure 5 The foreground binary image shown.

[0076] In step 103, object contour detection is performed on the foreground binary image to obtain an object contour set including a plurality of object contours.

[0077] Exemplarily, the findContours() function may be used to detect the contours of objects in the foreground binary image, and the drawContours() function may be used to draw the contours of objects in the foreground binary image.

[0078] Specifically, when performing contour extraction, a one-dimensional array is used to record the information of the eight neighborhoods around the pixel point. If the grayscale value of the pixel point in the eight neighborhoods is the same as the grayscale value of the center point, the point is inside the object and can be deleted; otherwise, the point is at the edge of the image and needs to be retained. Each pixel in the image is processed in turn, and the remaining is the contour of the image.

[0079] Exemplarily, a boundary tracking algorithm may be used to find a boundary point a from the image, and starting from the boundary point, find the next boundary point b according to the search criteria until the subsequent points tracked return to the starting boundary point a.

[0080] See also Figure 6 , select boundary point a from the foreground binary image A, traverse and track all boundary points in the foreground binary image A until boundary point a' coincides with boundary point a, stop the operation, and obtain the object contour set of the foreground binary image.

[0081] Optionally, before step 103, the dust identification method may further include: performing corrosion and expansion processing on the foreground binary image to remove image noise points in the foreground binary image.

[0082] Among them, erosion and dilation are both morphological methods used to process the highlight part of the image. Dilation and erosion operations are to convolve the image with the kernel. The kernel can be of any shape and size and has an independently defined reference point. Dilation is an operation to find the local maximum value. By convolving the image with a preset kernel, the maximum value of the pixel points in the area covered by the preset kernel is calculated, and the maximum value is assigned to the pixel specified by the reference point, so that the highlight area in the image gradually increases. Erosion is an operation to find the local minimum value. By convolving the image with a preset kernel, the minimum value of the pixel points in the area covered by the preset kernel is calculated, and the minimum value is assigned to the pixel specified by the reference point, so that the highlight area in the image gradually decreases.

[0083] Specifically, since the noise in the foreground binary image is not just one pixel, it cannot be completely removed. By using the connected domain method, the noise is treated as a connected domain; by finding a threshold, the connected domain with a size below the threshold is removed to remove the noise.

[0084] For example, the bwareaopen function in Matlab can be used to remove connected domains. Since the connected domain operation requires the white part to be operated, if the part used as the connected domain in the image to be processed is black, a gray inversion step needs to be added. When the image is a binary image: img = 1-img. After multiple experiments, the best threshold is selected for operation, and the connected domain below the threshold is removed, thereby removing noise points.

[0085] In step 104, based on at least one of the color feature, shape feature and diffusion feature of the dust, each object contour in the object contour set is detected to determine whether each object contour contains dust.

[0086] In some embodiments, step 104 may specifically include: extracting the regional contour of each motion region in the object contour, determining the minimum circumscribed rectangle of the vertical boundary of the regional contour, and the pixel area and centroid coordinates of the regional contour; identifying the dust in the regional contour based on the aspect ratio and rectangularity of the minimum circumscribed rectangle to obtain a suspected dust region, where the rectangularity is the ratio of the contour area to the minimum circumscribed rectangle area; calculating the pixel grayscale value range corresponding to the dust in the dust color feature detection rule, and identifying the suspected dust region based on the pixel grayscale value range. Then, the real dust region is determined based on the diffusion characteristics of the dust.

[0087] For example, the diffusion characteristics of dust can be expressed as changes in area. Therefore, after the suspected dust area is identified based on the pixel grayscale value range to obtain the first dust area, it is possible to determine whether the first dust area is a real dust area based on the rate of change of the area of ​​the same area contour corresponding to adjacent video frames.

[0088] Exemplarily, the above-mentioned determination of whether the first dust area is a real dust area based on the rate of change of the area of ​​the contour of the same area corresponding to adjacent video frames may include: for any two objects, if the coordinates of the center of gravity of the motion contour satisfy a preset relationship, then the two objects are determined to be the same object and the two objects are counted; otherwise, two objects are added to the preset suspected dust list, and the two objects are counted and timed. If the number of second objects in the suspected dust list exceeds the number threshold, and the growth rate of the area of ​​the second object reaches the growth rate threshold, then the second object is determined to be dust. If the existence time of the first object in the suspected dust list exceeds the time threshold, then the first object is eliminated from the suspected dust list, and the first object is any object in the suspected dust list.

[0089] For example, by extracting the contours of each moving area in the image, the minimum circumscribed rectangle, pixel area, and centroid coordinates of the vertical boundary of each moving area contour can be obtained. Afterwards, based on the fact that the initial shape of the dust is nearly circular, the aspect ratio of the minimum circumscribed rectangle and the rectangularity (contour area / circumscribed rectangle area) are used to perform preliminary dust identification to obtain the suspected dust area. Then, through the representation method of dust color features obtained by experiment, that is, the dust color feature detection rule, the pixel gray value range is obtained by calculation and used as the color feature to further identify the suspected dust area. Finally, by detecting the growth rate of the moving area in continuous video frames, the dust diffusivity shown by the change in the moving target area is used to complete the identification and confirmation of the dust.

[0090] See also Figure 7 , determining whether the image contains dust based on the diffusion characteristics of dust may include:

[0091] A suspected dust list is established, which includes a counter, a timer, a center of gravity coordinates, a pixel area, etc. Afterwards, the center of gravity coordinates of the two moving contours of the suspected dust are determined to be within a certain range to determine whether the two moving contours of the suspected dust are the same object. If they are not the same object, the object is added to the suspected dust list, the count is 1 and the time is kept; if they are the same object, the dust information is updated and the counter is increased by 1. Based on the fact that dust will not float in the air for too long, if an object suspected of dust in the suspected dust list exists for more than a certain time, the object is eliminated from the suspected dust list. If the counter of a suspected dust object in the suspected dust list exceeds a certain number and the area growth rate reaches a certain threshold, the suspected dust object is determined to be dust.

[0092] See also Figure 8 , Figure 8 This is an image that is judged to be dust based on its diffusion characteristics, and also has an experimental visual calibration.

[0093] The above-mentioned dust identification method, after acquiring the video stream of the scene through the camera, can obtain a foreground binary image by identifying the moving objects in space; by identifying the outer contour of the foreground object, the dust distribution of the scene is identified according to the characteristics of the dust, and then the dust is effectively monitored and controlled; it saves the step of densely arranging a large number of dust detectors, avoids the impact of the installation process on production activities, reduces the complexity of the installation, and reduces the cost of commissioning and equipment maintenance.

[0094] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0095] Corresponding to the dust identification method described in the above embodiment, Figure 7 A structural block diagram of a dust identification device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0096] See also Fig. 9 The dust identification device in the embodiment of the present application may include an image acquisition module 201, an image processing module 202, an object contour recognition module 203 and a dust identification module 204.

[0097] Among them, the image acquisition module 201 is used to acquire a video image of the target area. The image processing module 202 is used to process the video image to obtain a foreground binary image. The object contour recognition module 203 is used to perform object contour detection on the foreground binary image to obtain an object contour set containing multiple object contours. The dust recognition module 204 is used to detect each object contour in the object contour set based on at least one of the color feature, shape feature and diffusion feature of the dust, and determine whether the object contour set contains a dust contour.

[0098] In some embodiments, see Fig.10 ,based on Fig. 9 In the illustrated embodiment, the dust recognition module 204 may include a motion region unit 2041 , a first recognition unit 2042 , and a second recognition unit 2043 .

[0099] The contour extraction unit 2041 is used to extract the region contour of each motion region in the object contour, determine the minimum circumscribed rectangle of the vertical boundary of the region contour, and the pixel area and center of gravity coordinates of the region contour.

[0100] A shape recognition unit 2042 is used to recognize dust in the area outline based on the aspect ratio and rectangularity of the minimum circumscribed rectangle to obtain a suspected dust area, wherein the rectangularity is the ratio of the outline area to the area of ​​the minimum circumscribed rectangle;

[0101] The color recognition unit 2043 is used to calculate the pixel gray value range corresponding to the dust in the dust color feature detection rule, and recognize the suspected dust area based on the pixel gray value range.

[0102] Optionally, the dust identification module 204 may further include: a diffusion identification unit 2044, configured to determine whether the first dust region is a real dust region based on a rate of change of an area of ​​a same region contour corresponding to adjacent video frames.

[0103] Exemplarily, the determining unit may be specifically configured to:

[0104] For any two objects, if the coordinates of the centroids of the motion profiles satisfy a preset relationship, the two objects are determined to be the same object and the two objects are counted; otherwise, the two objects are added to the preset suspected dust list, and the two objects are counted and timed;

[0105] If the number of second objects in the suspected dust list exceeds a number threshold, and a growth rate of the area of ​​the second objects reaches a growth rate threshold, the second object is determined to be dust.

[0106] If the existence time of the first object in the suspected dust list exceeds a time threshold, the first object is eliminated from the suspected dust list, and the first object is any object in the suspected dust list.

[0107] Optionally, the image processing module 202 is specifically used to: perform defogging on the video image to obtain a grayscale image; perform background modeling on the grayscale image, extract foreground moving objects by a background difference method, and obtain the foreground binary image.

[0108] Exemplarily, the above-mentioned defogging of the video image to obtain a grayscale image includes:

[0109] Calculate the grayscale histogram of the video image by channel to obtain a maximum grayscale value and a minimum grayscale value, wherein the maximum grayscale value and the minimum grayscale value constitute an extreme value range;

[0110] Dividing the video image into a plurality of image blocks;

[0111] For a first image block whose grayscale value is outside the extreme value range, the grayscale of the first image block is assigned a value of 0 or 255; for a second image block whose grayscale value is within the extreme value range, the grayscale of the second image block is linearly transformed to between 0 and 255 to obtain the grayscale image;

[0112] Among them, the coefficients of the linear transformation are k′ is a constant, is the internal grayscale variance of the second image block, is the image noise variance of the second image block.

[0113] Optionally, the dust identification device may further include: a denoising module, which is used to perform corrosion and expansion processing on the foreground binary image to remove image noise points in the foreground binary image. After the denoising module removes the image noise points in the foreground binary image, the object contour identification module performs object contour detection on the foreground binary image after the image noise points are removed.

[0114] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0115] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0116] The present application also provides a terminal device, see Fig.11 The terminal device 500 may include: at least one processor 510, a memory 520, and a computer program stored in the memory 520 and executable on the at least one processor 510. When the processor 510 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented, for example: Figure 2Steps 101 to 104 in the illustrated embodiment. Alternatively, when the processor 510 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Fig. 9 Functions of modules 201 to 204 are shown.

[0117] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 520, and executed by the processor 510 to complete the present application. The one or more modules / units may be a series of computer program segments capable of completing specific functions, and the program segments are used to describe the execution process of the computer program in the terminal device 500.

[0118] Those skilled in the art will understand that Fig.11 It is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, buses, etc.

[0119] The processor 510 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0120] The memory 520 may be an internal storage unit of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 520 is used to store the computer program and other programs and data required by the terminal device. The memory 520 may also be used to temporarily store data that has been output or is to be output.

[0121] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0122] The dust identification method provided in the embodiments of the present application can be applied to industrial robots, computers, wearable devices, vehicle-mounted devices, tablet computers, laptop computers, netbooks, personal digital assistants (PDA), augmented reality (AR) / virtual reality (VR) devices, mobile phones and other terminal devices. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0123] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each embodiment of the above-mentioned dust identification method can be implemented.

[0124] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in each embodiment of the above-mentioned dust identification method when executing the computer program product.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0126] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A dust identification method, characterized in that: include: Acquire a video image of a target area; Processing the video image to obtain a foreground binary image; Performing object contour detection on the foreground binary image to obtain an object contour set including a plurality of object contours; Based on at least one of the color feature, shape feature and diffusion feature of the dust, each object contour in the object contour set is detected to determine whether each object contour contains dust; The detecting each object contour in the object contour set based on at least one of the color feature, shape feature and diffusion feature of the dust to determine whether each object contour contains dust comprises: Extracting the area contour of each motion area in the object contour, determining the minimum circumscribed rectangle of the vertical boundary of the area contour, and the pixel area and center of gravity coordinates of the area contour; Identify the dust in the area contour based on the aspect ratio and rectangularity of the minimum circumscribed rectangle to obtain a suspected dust area, wherein the rectangularity is the ratio of the contour area to the area of ​​the minimum circumscribed rectangle; Calculating a pixel grayscale value range corresponding to dust in a dust color feature detection rule, identifying the suspected dust region based on the pixel grayscale value range, and obtaining a first dust region; Determining whether the first dust region is a real dust region based on a rate of change of areas of contours of the same region corresponding to adjacent video frames; The determining whether the first dust region is a real dust region based on the change rate of the area of ​​the same region contour corresponding to adjacent video frames includes: For any two objects, if the coordinates of the centroids of the motion profiles satisfy a preset relationship, the two objects are determined to be the same object and the two objects are counted; otherwise, the two objects are added to the preset suspected dust list, and the two objects are counted and timed; If the number of second objects in the suspected dust list exceeds a number threshold, and a growth rate of the area of ​​the second objects reaches a growth rate threshold, the second objects are determined to be dust; If the existence time of the first object in the suspected dust list exceeds a time threshold, the first object is eliminated from the suspected dust list, and the first object is any object in the suspected dust list.

2. The dust identification method according to claim 1, characterized in that: The step of processing the video image to obtain a foreground binary image includes: Performing defogging processing on the video image to obtain a grayscale image; Background modeling is performed on the grayscale image, and foreground moving objects are extracted by a background difference method to obtain the foreground binary image.

3. The dust identification method according to claim 2, characterized in that: The defogging process is performed on the video image to obtain a grayscale image, including: Calculate the grayscale histogram of the video image by channel to obtain a maximum grayscale value and a minimum grayscale value, wherein the maximum grayscale value and the minimum grayscale value constitute an extreme value range; Dividing the video image into a plurality of image blocks; For a first image block whose grayscale value is outside the extreme value range, the grayscale of the first image block is assigned a value of 0 or 255; for a second image block whose grayscale value is within the extreme value range, the grayscale of the second image block is linearly transformed to between 0 and 255 to obtain the grayscale image; Among them, the coefficients of the linear transformation are k′ is a constant, is the internal grayscale variance of the second image block, is the image noise variance of the second image block.

4. The dust identification method according to claim 1, characterized in that: Before performing object contour detection on the foreground binary image, the method includes: The foreground binary image is subjected to erosion and dilation processing to remove image noise points in the foreground binary image.

5. A dust identification device, characterized in that: include: An acquisition module, used for acquiring a video image of a target area; A processing module, used for processing the video image to obtain a foreground binary image; An object contour recognition module is used to perform object contour detection on the foreground binary image to obtain an object contour set including multiple object contours; a dust recognition module, configured to detect each object contour in the object contour set based on at least one of a color feature, a shape feature, and a diffusion feature of the dust, and determine whether the object contour set contains a dust contour; The detecting each object contour in the object contour set based on at least one of the color feature, shape feature and diffusion feature of the dust to determine whether each object contour contains dust comprises: Extracting the area contour of each motion area in the object contour, determining the minimum circumscribed rectangle of the vertical boundary of the area contour, and the pixel area and center of gravity coordinates of the area contour; Identify the dust in the area contour based on the aspect ratio and rectangularity of the minimum circumscribed rectangle to obtain a suspected dust area, wherein the rectangularity is the ratio of the contour area to the area of ​​the minimum circumscribed rectangle; Calculating a pixel grayscale value range corresponding to dust in a dust color feature detection rule, identifying the suspected dust region based on the pixel grayscale value range, and obtaining a first dust region; Determining whether the first dust region is a real dust region based on a rate of change of areas of contours of the same region corresponding to adjacent video frames; The determining whether the first dust region is a real dust region based on the change rate of the area of ​​the same region contour corresponding to adjacent video frames includes: For any two objects, if the coordinates of the centroids of the motion profiles satisfy a preset relationship, the two objects are determined to be the same object and the two objects are counted; otherwise, the two objects are added to the preset suspected dust list, and the two objects are counted and timed; If the number of second objects in the suspected dust list exceeds a number threshold, and a growth rate of the area of ​​the second objects reaches a growth rate threshold, the second objects are determined to be dust; If the existence time of the first object in the suspected dust list exceeds a time threshold, the first object is eliminated from the suspected dust list, and the first object is any object in the suspected dust list.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images