Image dust filtering method, device and equipment and medium
By preprocessing and deep neural network recognition of driverless loader images, combined with filtering technology to filter dust noise, the problem of insufficient dust filtering accuracy and information integrity in the existing technology has been solved, and the performance of driverless loaders has been improved. Perception and job safety.
Patent Information
- Application Number
- CN202411305329.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2025-01-21
AI Technical Summary
The existing image dust filtering methods have shortcomings in terms of accuracy and information integrity, making it difficult to effectively filter dust noise, affecting the perception and operation safety of unmanned loaders.
By acquiring the target image and preprocessing its pixels, the pre-trained deep neural network is used to identify and filter dust pixels with high confidence, and combined with reflection intensity filtering and radius filtering, a clean working object image is generated.
It improves the accuracy of dust filtration, ensures the integrity of image information, and enhances the perception capabilities and operating safety of unmanned loaders.
Smart Images
Figure CN119338710A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to an image dust filtering method, device, equipment and medium. Background Art
[0002] During the operation of the unmanned loader, shoveling and unloading operations are very likely to cause dust. Especially in outdoor environments, the frequent occurrence of rain and fog further increases the difficulty of the unmanned vehicle to perceive the surrounding environment, seriously affecting its safe and efficient execution of operating tasks.
[0003] To ensure the stability and reliability of perception information, the following methods are currently used: a combination of LiDAR and millimeter-wave radar is used to filter noise data, and rain, fog and dust point cloud clusters are filtered out by fusing multi-source information to enhance the ability to identify obstacles. However, although this multi-radar post-fusion solution has a certain effect in improving the filtering of rain, fog and dust noise, it still has obvious limitations. Due to the presence of dust noise, the real target may be obscured, and false alarms are prone to occur when using millimeter-wave radar, while LiDAR is difficult to accurately judge the target situation when it is interfered by dust.
[0004] In summary, the existing dust filtering method for images used to ensure the stability and reliability of perception information of unmanned loaders has the problem of low dust filtering accuracy and poor image information integrity of images after dust filtering. Summary of the invention
[0005] The present invention provides a method, device, equipment and medium for filtering dust from an image, which can solve the problems of low dust filtering accuracy and poor image information integrity of the image after dust filtering in the existing method for filtering dust from an image.
[0006] In a first aspect, an embodiment of the present invention provides a method for filtering dust from an image, the method comprising:
[0007] Acquire a target image, and pre-process all pixel points contained in the target image to obtain a target point information set; wherein the target point information includes pixel coordinates and pixel information of the pixel points;
[0008] The target point information of each pixel is processed by a pre-trained deep neural network to obtain the target category, information strength and confidence matching each pixel; wherein the target category includes: dust and working object;
[0009] Extract the pixels whose target category is dust and whose confidence meets the preset threshold as target pixels, obtain the information intensity of each target pixel, filter each target pixel using reflection intensity filtering and radius filtering, and obtain each quasi-operation object pixel that meets the filtering conditions;
[0010] A result image is generated based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
[0011] In a second aspect, an embodiment of the present invention provides a dust filtering device for an image, the device comprising:
[0012] A preprocessing module, used to obtain a target image and preprocess all pixel points contained in the target image to obtain a target point information set; wherein the target point information includes pixel coordinates and pixel information of the pixel points;
[0013] A deep neural network module is used to process the target point information of each pixel through a pre-trained deep neural network to obtain the target category, information strength and confidence matching each pixel; wherein the target category includes: dust and working object;
[0014] A filtering module is used to extract the pixels whose target category is dust and whose confidence meets a preset threshold as target pixels, obtain the information intensity of each target pixel, and filter each target pixel using reflection intensity filtering and radius filtering to obtain each quasi-operation object pixel that meets the filtering conditions;
[0015] The image generation module is used to generate a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
[0016] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:
[0017] at least one processor; and
[0018] a memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the dust filtering method for an image described in any embodiment of the present invention.
[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a dust filtering method for an image as described in any embodiment of the present invention when executed.
[0021] The technical solution of the embodiment of the present invention obtains a target image and pre-processes all the pixels contained in the target image to obtain a target point information set, then processes the target point information of each pixel through a pre-trained deep neural network to obtain a target category, information strength and confidence matching each pixel, then extracts pixels with a target category of dust and a confidence satisfying a preset threshold as target pixels, obtains the information strength of each target pixel, filters each target pixel using reflection intensity filtering and radius filtering to obtain each quasi-operation object pixel that meets the filtering conditions, and finally generates a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each pixel with a target category of an operation object, thereby solving the problems of low dust filtering accuracy and poor image information integrity of the image after dust filtering in the existing image dust filtering method, realizing image dust filtering, improving dust filtering accuracy and ensuring image information integrity.
[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0024] Figure 1 is a flow chart of a dust filtering method for an image provided according to a first embodiment of the present invention;
[0025] Figure 2 is a flow chart of a dust filtering method for an image provided according to a second embodiment of the present invention;
[0026] Figure 3 is a schematic structural diagram of a dust filtering device for an image provided according to a third embodiment of the present invention;
[0027] Figure 4 The present invention is a schematic diagram of the structure of an electronic device for implementing a method for filtering dust from an image according to an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variation of the terms "including" and "having" is intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0030] Embodiment 1
[0031] Figure 1 A flowchart of a method for filtering dust from an image is provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of filtering dust from an image. The method can be executed by a device for filtering dust from an image. The device for filtering dust from an image can be implemented in the form of hardware and / or software. The device for filtering dust from an image can be configured in a terminal or server having an image dust filtering function.
[0032] like Figure 1 As shown, the method includes:
[0033] S110, acquiring a target image, and preprocessing all pixel points contained in the target image to obtain a target point information set.
[0034] The target point information includes the pixel coordinates and pixel information of the pixel point.
[0035] Specifically, the target image refers to an image with specific features or specific uses, such as a picture of a certain area taken by a surveillance camera, an image of a specific geographical area obtained by satellite remote sensing, etc. Then all the pixels contained in the target image are preprocessed; further, the target point information includes the pixel coordinates of the pixel, which is the position identifier of the pixel in the image. For example, in a 1080P image, the coordinates of a certain pixel may be (500, 300); and pixel information, which includes specific data such as the color, brightness, and saturation of the pixel, for example, the color of a pixel is red, and the brightness value is 200.
[0036] Among them, all the pixels contained in the target image are preprocessed to obtain a target point information set, including: performing image preprocessing on the target image to obtain a standard image that meets preset rules; extracting all the pixels contained in the standard image as standard pixels, and calculating the average distance from each standard pixel to its nearest n points, where n is the preset number of pixels; constructing a Gaussian distribution according to the average distance of each standard pixel, and calculating the distance threshold according to the distance threshold calculation formula μ+σ based on the mean μ and variance σ of the constructed Gaussian distribution; filtering the standard pixels whose average distance is greater than the distance threshold to obtain each Gaussian pixel; filtering each Gaussian pixel using a voxel filtering method according to a preset voxel value to obtain each target point; acquiring the target point information of each target point according to the target image to obtain a target point information set.
[0037] Further, the Gaussian distribution: also known as the normal distribution, is a common continuous probability distribution, and its probability density function presents the shape of a bell curve and is symmetrical. Its characteristic is that most of the data is concentrated near the mean, and the farther away from the mean, the lower the probability of the data appearing; further, the voxel filtering method is a filtering method commonly used in image processing and three-dimensional data processing. Voxel refers to a small cubic unit in a three-dimensional space. The voxel filtering method operates on voxels to achieve filtering processing of images or three-dimensional data. Its purpose is usually to reduce noise, smooth data or simplify data structure, while trying to retain important features and shape information. It is widely used in processing three-dimensional point cloud data and other scenes. It should be noted that in this embodiment, the voxel value can be preset and adjusted according to the actual application scenario, and this embodiment does not limit this; illustratively, in this embodiment, when using the voxel filtering method, each Gaussian pixel point can be divided into a plurality of small voxels according to the voxel size of 0.005, and then the center point is selected in each voxel as the target point for output.
[0038] Furthermore, the target point information of each pixel is processed by a pre-trained deep neural network to obtain the confidence matching each pixel, including: processing the target point information and pixel features of each pixel by an object detection module to obtain the bounding box quality B, target confidence O and target consistency C matching each pixel; the confidence of each pixel is calculated according to the confidence calculation formula conf=αB+βO+γC; wherein α, β, and γ are pre-set weight parameters, and α+β+γ=1.
[0039] Among them, the operation object detection module can be a computing module that supports the Transformer model; further, the Transformer model is a deep learning model architecture widely used in fields such as natural language processing. In this embodiment, the Transformer model can be used to detect specific target objects in the image, determine their position and category, divide the image into different regions, each region has similar features or belongs to the same category, and extract representative and discriminative image features. It should be noted that relevant personnel in this field should understand that the bounding box quality B, target confidence O and target consistency C of the pixel point are all obtainable intermediate products of the Transformer model when performing image processing. Therefore, by setting the model, the bounding box quality B, target confidence O and target consistency C of the pixel point can be directly obtained in the process of the Transformer model processing the target information and pixel features of the pixel point, and then the confidence is calculated.
[0040] On the basis of the above steps, the target image is preprocessed to obtain a standard image that meets the preset rules, including: acquiring the target image and inputting the target image into a distortion correction model to obtain a first image after distortion correction; processing the first image through a normalization model to obtain a normalized second image; acquiring a preset image standard specification, and performing image scaling and image cropping processing on the second image based on the image standard specification to obtain a standard image that meets the image standard specification.
[0041] Among them, the distortion correction model is a mathematical model for correcting image distortion caused by factors such as optical systems or shooting conditions. Image distortion may include radial distortion (such as barrel distortion, pincushion distortion), tangential distortion, etc. The model analyzes the features and rules in the image and uses mathematical algorithms and parameters to deform and adjust the image to eliminate or reduce distortion and make the image closer to the real scene; further, the normalization model is a model for standardizing the pixel values or feature values of the image. Its purpose is to unify data of different ranges or distributions into a specific range or standard. It is common to map the values to the interval [0,1] or [-1,1]. This can eliminate the dimensional differences between the data, making different images or different parts of the image comparable in subsequent processing and analysis, and also helps to improve the stability and convergence of the algorithm.
[0042] S120, processing the target point information of each pixel through a pre-trained deep neural network to obtain a target category, information strength, and confidence level matching each pixel.
[0043] The target categories include: dust and work objects.
[0044] In this embodiment, the specific classification and content of the work object can be manually preset according to actual needs. For example, if the current application scenario is a construction site, the sub-classification of the work object can specifically include vehicles, people, walls, materials, ground, etc. Through the specific classification of sub-objects, the final result image can classify and display objects with the same sub-classification when displaying objects, thereby increasing the completeness of the image information.
[0045] Among them, the target point information of each pixel point is processed by a pre-trained deep neural network to obtain the target category and information strength matching each pixel point, including: inputting the target point information of each pixel point into a pre-trained category model to obtain the target category matching each pixel point; inputting the target point information of each pixel point into a pre-trained feature extraction model to obtain the pixel features matching each pixel point; processing the target point information and pixel features of each pixel point through an operation object detection module to obtain the information strength matching each pixel point.
[0046] The information intensity may be the reflection intensity of the pixel point.
[0047] Exemplarily, in a specific implementation scenario of this embodiment, first, the target point information of each pixel point is input one by one into a category model that has been trained in advance through a large amount of data and complex algorithms. The category model mentioned here is a mathematical model that can classify and judge the input data, such as a classification model used to distinguish between people and scenery in image recognition. Through the analysis and calculation of this category model, the target categories that match each pixel point are obtained respectively. The target category can be various, such as pedestrians, vehicles, buildings, etc. in unmanned driving scenes. Then, the target point information of each pixel point is input into a feature extraction model that has been repeatedly trained and optimized in advance. The so-called feature extraction model is a model specifically used to extract key features from input data, such as a model used to extract facial contours, facial features, etc. in face recognition. After processing and extraction by this model, pixel features that match each pixel point are obtained. Pixel features can be color features, texture features, shape features, etc. of pixel points. Subsequently, the target point information of each pixel point and the acquired pixel features are processed through a special operation object detection module to obtain the information intensity matching each pixel point.
[0048] S130, extracting the pixel points whose target category is dust and whose confidence meets the preset threshold as the target pixel points, obtaining the information intensity of each target pixel point, filtering each target pixel point using reflection intensity filtering and radius filtering, and obtaining each quasi-operation object pixel point that meets the filtering conditions.
[0049] Exemplarily, in a specific implementation scenario of this embodiment, firstly, the pixels whose target category is determined to be dust are extracted, and the confidence of these pixels must meet the preset threshold set in advance, and such pixels are determined as target pixels. The confidence here refers to the credibility of the judgment that the pixel belongs to the category of dust; the preset threshold is a pre-set standard value, such as 0.8, and only the pixels with a confidence greater than or equal to 0.8 will be identified as target pixels. Next, the information intensity of each target pixel is obtained. The information intensity can be understood as the reflection intensity contained in the pixel. Then, the two filtering methods of reflection intensity filtering and radius filtering are used to filter each target pixel. Reflection intensity filtering is a method of screening pixels according to the reflection intensity attribute of the pixel. For example, when detecting dust on the metal surface, the intensity of the light reflected by the metal is used to determine whether there are dust pixels. The radius filter is based on the distribution radius of the pixel to screen, such as setting a radius range and only retaining the pixels within this range.
[0050] S140, generating a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
[0051] The method described in this embodiment, based on the above steps, further filters the pixels classified as dust, thereby retaining the quasi-operation object pixels among the pixels classified as dust and removing the pixels that are actually dust, thereby avoiding the problem of the prior art that the pixels related to the operation target are misjudged due to the need to filter the dust, thereby affecting the integrity of the image information, and further avoiding the problem that the prior art obscures the real target due to the presence of dust noise, and easily generates false alarms when using millimeter wave radar, making it difficult to accurately judge the target situation.
[0052] The technical solution of the embodiment of the present invention obtains a target image and pre-processes all the pixels contained in the target image to obtain a target point information set, then processes the target point information of each pixel through a pre-trained deep neural network to obtain a target category, information strength and confidence matching each pixel, then extracts pixels with a target category of dust and a confidence level that meets a preset threshold as target pixels, obtains the information strength of each target pixel, filters each target pixel using reflection intensity filtering and radius filtering to obtain each quasi-operating object pixel that meets the filtering conditions, and finally generates a result image based on the pixel information and pixel coordinates of each quasi-operating object pixel and each pixel with a target category of operating objects, thereby realizing dust filtering of the image, improving the accuracy of dust filtering, and ensuring the integrity of image information.
[0053] Embodiment 2
[0054] Figure 2 A flow chart of a dust filtering method for an image provided in the second embodiment of the present invention. This embodiment is refined on the basis of the above-mentioned embodiment. In this embodiment, the information intensity of each target pixel point is obtained, and each target pixel point is filtered using reflection intensity filtering and radius filtering to obtain each quasi-operation object pixel point that meets the filtering conditions.
[0055] like Figure 2 As shown, the method includes:
[0056] S210: Acquire a target image, and pre-process all pixel points contained in the target image to obtain a target point information set.
[0057] The target point information includes the pixel coordinates and pixel information of the pixel point.
[0058] S220, processing the target point information of each pixel through a pre-trained deep neural network to obtain a target category, information strength, and confidence level that matches each pixel.
[0059] The target categories include: dust and work objects.
[0060] S230, extracting pixels whose target category is dust and whose confidence meets a preset threshold as target pixels, and calculating an intensity threshold according to the information intensity of each target pixel and the number of target pixels.
[0061] The intensity threshold may be an average value of the information intensity of each target pixel.
[0062] S240, setting the reflection intensity value of the reflection intensity filter as the intensity threshold, and using the set reflection intensity filter to filter each target pixel point to filter out the target pixel points whose information intensity is less than the intensity threshold, to obtain each first pixel point.
[0063] S250, filtering each first pixel point through radius filtering to obtain each quasi-operation object pixel point that meets the filtering condition.
[0064] Specifically, each first pixel point is filtered through radius filtering to obtain each quasi-operation object pixel point that meets the filtering conditions, including: respectively obtaining the pixel coordinates of each first pixel point and a pre-set radius threshold and quantity threshold; for each first pixel point, calculating the number of other first pixel points within the radius threshold around the first pixel point according to each pixel coordinate to obtain the number of pixels; filtering each first pixel point based on the number threshold and the number of pixels of each first pixel point to filter out first pixel points whose number of pixels is less than the number threshold to obtain each quasi-operation object pixel point.
[0065] Specifically, each first pixel point is filtered to obtain each quasi-operation object pixel point that meets the filtering conditions. The specific steps include: obtaining the pixel coordinates of each first pixel point respectively, and presetting the radius threshold and the number threshold. For each first pixel point, the number of other first pixel points within the radius threshold range around the first pixel point is calculated according to its respective pixel coordinates, so as to obtain the number of pixels. Then, based on the set number threshold and the number of pixels corresponding to each first pixel point, each first pixel point is filtered. In this process, those first pixel points whose number of pixels is less than the number threshold are filtered out, and finally the quasi-operation object pixel points are obtained; wherein, the radius threshold and the number threshold can be set and adjusted by relevant personnel according to the specific implementation scenario.
[0066] S260, generating a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
[0067] The technical solution of the embodiment of the present invention obtains a target image and pre-processes all pixels contained in the target image to obtain a target point information set, then processes the target point information of each pixel through a pre-trained deep neural network to obtain a target category, information strength and confidence matching each pixel, then extracts pixels whose target category is dust and whose confidence meets a preset threshold as target pixels, calculates an intensity threshold according to the information strength of each target pixel and the number of target pixels, and sets the reflection intensity value of the reflection intensity filter as the intensity threshold, filters each target pixel using the set reflection intensity filter to filter out target pixels whose information strength is less than the intensity threshold to obtain each first pixel, then filters each first pixel through a radius filter to obtain each quasi-operation object pixel that meets the filtering condition, and finally generates a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each pixel whose target category is an operation object, thereby realizing dust filtering of the image, improving the accuracy of dust filtering, and ensuring the integrity of image information.
[0068] Embodiment 3
[0069] Figure 3 This is a schematic structural diagram of a dust filtering device for an image provided in Embodiment 3 of the present invention.
[0070] like Figure 3 As shown, the device comprises:
[0071] The preprocessing module 310 is used to obtain a target image and preprocess all the pixels contained in the target image to obtain a target point information set; wherein the target point information includes the pixel coordinates and pixel information of the pixel points;
[0072] The deep neural network module 320 is used to process the target point information of each pixel point through a pre-trained deep neural network to obtain the target category, information strength and confidence matching each pixel point; wherein the target category includes: dust and working object;
[0073] The filtering module 330 is used to extract the pixels whose target category is dust and whose confidence meets the preset threshold as target pixels, obtain the information intensity of each target pixel, and filter each target pixel using reflection intensity filtering and radius filtering to obtain each quasi-operation object pixel that meets the filtering conditions;
[0074] The image generation module 340 is used to generate a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
[0075] The technical solution of the embodiment of the present invention obtains a target image and pre-processes all the pixels contained in the target image to obtain a target point information set, then processes the target point information of each pixel through a pre-trained deep neural network to obtain a target category, information strength and confidence matching each pixel, then extracts pixels with a target category of dust and a confidence level that meets a preset threshold as target pixels, obtains the information strength of each target pixel, filters each target pixel using reflection intensity filtering and radius filtering to obtain each quasi-operating object pixel that meets the filtering conditions, and finally generates a result image based on the pixel information and pixel coordinates of each quasi-operating object pixel and each pixel with a target category of operating objects, thereby realizing dust filtering of the image, improving the accuracy of dust filtering, and ensuring the integrity of image information.
[0076] Based on the above embodiment, the preprocessing module 310 includes:
[0077] An image preprocessing unit, used to perform image preprocessing on a target image to obtain a standard image that complies with preset rules;
[0078] The average distance calculation unit is used to extract all the pixels contained in the standard image as standard pixels, and calculate the average distance from each standard pixel to its nearest n points, where n is a preset number of pixels;
[0079] A distance threshold calculation unit is used to construct a Gaussian distribution according to the average distance of each standard pixel point, and calculate the distance threshold according to the distance threshold calculation formula μ+σ based on the mean μ and variance σ of the constructed Gaussian distribution;
[0080] A pixel point filtering unit is used to filter standard pixel points whose average distance is greater than a distance threshold to obtain Gaussian pixel points;
[0081] A voxel filtering unit, used to filter each Gaussian pixel point using a voxel filtering method according to a preset voxel value to obtain each target point;
[0082] The information set generating unit is used to obtain the target point information of each target point according to the target image to obtain the target point information set.
[0083] Based on the above embodiment, the deep neural network module 320 includes:
[0084] A target category acquisition unit is used to input the target point information of each pixel into a pre-trained category model to obtain a target category matching each pixel;
[0085] A pixel feature extraction unit is used to input the target point information of each pixel into a pre-trained feature extraction model to obtain pixel features that match each pixel;
[0086] The information intensity acquisition unit is used to process the target point information and pixel features of each pixel point through the operation object detection module to obtain the information intensity matching each pixel point.
[0087] Based on the above embodiment, the deep neural network module 320 further includes:
[0088] The operation object detection unit is used to process the target point information and pixel features of each pixel point through the operation object detection module to obtain the bounding box quality B, target confidence O and target consistency C matching each pixel point respectively;
[0089] The confidence calculation unit is used to calculate the confidence of each pixel point according to the confidence calculation formula conf=αB+βO+γC; wherein α, β, and γ are pre-set weight parameters, and α+β+γ=1.
[0090] Based on the above embodiment, the filtering module 330 includes:
[0091] An intensity threshold calculation unit, used to calculate an intensity threshold according to the information intensity of each target pixel and the number of target pixels;
[0092] A reflection intensity filtering unit, used to set the reflection intensity value of the reflection intensity filtering as the intensity threshold, and use the set reflection intensity filtering to filter each target pixel point to filter out the target pixel points whose information intensity is less than the intensity threshold, so as to obtain each first pixel point;
[0093] The radius filtering unit is used to filter each first pixel point through radius filtering to obtain each quasi-operation object pixel point that meets the filtering condition.
[0094] Based on the above embodiment, the radius filtering unit further includes:
[0095] A data acquisition unit, used to respectively acquire the pixel coordinates of each first pixel point and a preset radius threshold and a quantity threshold;
[0096] a pixel point number calculation unit, configured to calculate, for each first pixel point, the number of other first pixel points within a radius threshold around the first pixel point according to each pixel coordinate, to obtain the number of pixel points;
[0097] The first pixel point filtering unit is used to filter each first pixel point based on a quantity threshold and the number of pixels of each first pixel point, so as to filter out first pixel points whose number of pixels is less than the quantity threshold, and obtain each quasi-operation object pixel point.
[0098] Based on the above embodiment, the image preprocessing unit further includes:
[0099] A distortion correction unit, used for acquiring a target image and inputting the target image into a distortion correction model to obtain a first image after distortion correction;
[0100] A normalization unit, used for processing the first image through a normalization model to obtain a normalized second image;
[0101] The specification processing unit is used to obtain a preset image standard specification, and perform image scaling and image cropping processing on the second image based on the image standard specification to obtain a standard image that meets the image standard specification.
[0102] An image dust filtering device provided by an embodiment of the present invention can execute an image dust filtering method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method.
[0103] Embodiment 4
[0104] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0105] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0106] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0107] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a dust filtering method for an image.
[0108] Accordingly, the method comprises:
[0109] Acquire a target image, and pre-process all pixel points contained in the target image to obtain a target point information set; wherein the target point information includes pixel coordinates and pixel information of the pixel points;
[0110] The target point information of each pixel is processed by a pre-trained deep neural network to obtain the target category, information strength and confidence matching each pixel; wherein the target category includes: dust and working object;
[0111] Extract the pixels whose target category is dust and whose confidence meets the preset threshold as target pixels, obtain the information intensity of each target pixel, filter each target pixel using reflection intensity filtering and radius filtering, and obtain each quasi-operation object pixel that meets the filtering conditions;
[0112] A result image is generated based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
[0113] In some embodiments, a dust filtering method for an image may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the dust filtering method for an image described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a dust filtering method for an image in any other appropriate manner (e.g., by means of firmware).
[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0115] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0116] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0117] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0118] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0119] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
Claims
1. A dust filtering method for an image, characterized in that: include: Acquire a target image, and pre-process all pixel points contained in the target image to obtain a target point information set; wherein the target point information includes pixel coordinates and pixel information of the pixel points; The target point information of each pixel is processed by a pre-trained deep neural network to obtain the target category, information strength and confidence matching each pixel; wherein the target category includes: dust and working object; Extract the pixels whose target category is dust and whose confidence meets the preset threshold as target pixels, obtain the information intensity of each target pixel, filter each target pixel using reflection intensity filtering and radius filtering, and obtain each quasi-operation object pixel that meets the filtering conditions; A result image is generated based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
2. The method according to claim 1, characterized in that Preprocess all the pixels contained in the target image to obtain the target point information set, including: Perform image preprocessing on the target image to obtain a standard image that meets preset rules; All pixels contained in the standard image are extracted as standard pixels, and the average distance from each standard pixel to its nearest n points is calculated, where n is the preset number of pixels; A Gaussian distribution is constructed according to the average distance of each standard pixel point, and the distance threshold is calculated according to the distance threshold calculation formula μ+σ based on the mean μ and variance σ of the constructed Gaussian distribution; Filter the standard pixel points whose average distance is greater than the distance threshold to obtain Gaussian pixel points; Using the voxel filtering method to filter each Gaussian pixel point according to the preset voxel value, each target point is obtained; The target point information of each target point is obtained according to the target image to obtain a target point information set.
3. The method according to claim 1, characterized in that The target point information of each pixel is processed by a pre-trained deep neural network to obtain the target category and information strength matching each pixel, including: Input the target point information of each pixel into the pre-trained category model to obtain the target category matching each pixel; Input the target point information of each pixel into the pre-trained feature extraction model to obtain the pixel features that match each pixel; The target point information and pixel features of each pixel are processed by the operation object detection module to obtain the information intensity matching each pixel.
4. The method according to claim 1, characterized in that: The target point information of each pixel is processed by a pre-trained deep neural network to obtain the confidence level of each pixel, including: The target point information and pixel features of each pixel are processed by the object detection module to obtain the bounding box quality B, target confidence O and target consistency C matching each pixel respectively; The confidence of each pixel is calculated according to the confidence calculation formula conf=αB+βO+γC; wherein α, β, and γ are pre-set weight parameters, and α+β+γ=1.
5. The method according to claim 1, characterized in that The information intensity of each target pixel is obtained, and each target pixel is filtered using reflection intensity filtering and radius filtering to obtain each quasi-operation object pixel that meets the filtering conditions, including: The intensity threshold is calculated according to the information intensity of each target pixel and the number of target pixels; The reflection intensity value of the reflection intensity filter is set as the intensity threshold, and each target pixel point is filtered using the set reflection intensity filter to filter out the target pixel points whose information intensity is less than the intensity threshold, thereby obtaining each first pixel point; Each first pixel point is filtered through radius filtering to obtain each quasi-operation object pixel point that meets the filtering condition.
6. The method according to claim 5, characterized in that Filtering each first pixel point through radius filtering to obtain each quasi-operation object pixel point that meets the filtering condition, including: Respectively obtain the pixel coordinates of each first pixel point and a preset radius threshold and quantity threshold; For each first pixel point, the number of other first pixel points within a radius threshold around the first pixel point is calculated according to each pixel coordinate to obtain the number of pixel points; The first pixels are filtered based on the quantity threshold and the number of pixels of the first pixels to filter out the first pixels whose number of pixels is less than the quantity threshold, and the quasi-operation object pixels are obtained.
7. The method according to claim 2, characterized in that: Perform image preprocessing on the target image to obtain a standard image that meets the preset rules, including: Acquire a target image and input the target image into a distortion correction model to obtain a first image after distortion correction; Processing the first image through a normalization model to obtain a normalized second image; A preset image standard specification is obtained, and image scaling and image cropping processing is performed on the second image based on the image standard specification to obtain a standard image that meets the image standard specification.
8. A dust filtering device for an image, characterized in that: include: A preprocessing module, used to obtain a target image and preprocess all pixel points contained in the target image to obtain a target point information set; wherein the target point information includes pixel coordinates and pixel information of the pixel points; A deep neural network module is used to process the target point information of each pixel through a pre-trained deep neural network to obtain the target category, information strength and confidence matching each pixel; wherein the target category includes: dust and working object; A filtering module is used to extract the pixels whose target category is dust and whose confidence meets a preset threshold as target pixels, obtain the information intensity of each target pixel, and filter each target pixel using reflection intensity filtering and radius filtering to obtain each quasi-operation object pixel that meets the filtering conditions; The image generation module is used to generate a result image based on the pixel information and pixel coordinates of each quasi-operation object pixel and each target category being an operation object pixel.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a dust filtering method for an image according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a dust filtering method for an image as described in any one of claims 1 to 7 when executed.