Image processing method, camera device and robot

Abstract

The invention discloses an image processing method, a camera device and a robot, and the method comprises the steps: S1, the camera device firstly obtains a frame of environment image as a background frame, then controls a light supplement unit to work, and then obtains a frame of environment image as a foreground frame; s2, the camera device preprocesses the acquired background frame, and then binarizes the foreground frame and the preprocessed background frame; and S3, the camera device performs similarity comparison on the foreground frame and the background frame which are subjected to binarization processing, and then noise points are removed according to matching values of points in the foreground frame. And the camera device performs similarity comparison operation on the background frame and the foreground frame after binaryzation to determine the confidence coefficient of the ambient light in the foreground frame so as to remove the ambient light in the foreground frame.

Description

Technical Field

[0001] The present invention relates to the field of image technology, and in particular to an image processing method, a camera device and a robot. Background Art

[0002] In the application of the visual measurement sensor, there will be interference from ambient light in many scenarios, such as sunlight and stray light that happens to be within the receiving band of the sensor. Therefore, there is a need for a method to filter out ambient light to prevent the sensor from identifying such interference as an obstacle and prevent misjudgment. The commonly used method is the differential elimination method, which collects images before and after turning on the active light source, and then differentiates the two frames of images to eliminate the background light and leave the effective light source. However, in actual application scenarios, the detection subject may be a moving robot, or the environment to be measured may change, so that the position of the ambient light in the two frames of images is not completely consistent, resulting in unsatisfactory effect of filtering out ambient light. Summary of the invention

[0003] In order to solve the above problems, the present invention provides an image processing method, a camera device and a robot. The specific technical solution of the present invention is as follows: An image processing method comprises the following steps: S1: the camera device first acquires a frame of environment image as a background frame, and then controls a fill light unit in the camera device to work and then acquires another frame of environment image as a foreground frame; S2: the camera device pre-processes the acquired background frame, and then performs binarization processing on the foreground frame and the pre-processed background frame; S3: the camera device compares the similarity of the binarized foreground frame and the background frame, and then removes noise points according to the matching values ​​of the points in the foreground frame.

[0004] Furthermore, after acquiring the background frame and the foreground frame, the camera device constructs an N*N pixel area in the foreground frame as the region of interest based on the center point of the foreground frame, and then constructs the region of interest at the corresponding position of the background frame.

[0005] Further, in step S2, the camera device preprocesses the acquired background frame, including the following steps: the camera device calculates the weight of the area of ​​interest of the background frame according to a two-dimensional normal distribution function; the camera device performs a convolution operation on the weight of the area of ​​interest of the background frame with each pixel point of the area of ​​interest of the background frame to obtain a preprocessed background frame.

[0006] Further, in step S2, the camera device performs binarization processing on the foreground frame and the preprocessed background frame, including the following steps: the camera device calculates the segmentation thresholds of the regions of interest of the foreground frame and the background frame respectively according to the inter-class variance formula; the camera device sets the grayscale values ​​of the pixels in the region of interest of the foreground frame whose grayscale values ​​are greater than or equal to the segmentation threshold of the foreground frame to 1, and sets the grayscale values ​​of the pixels in the region of interest of the foreground frame whose grayscale values ​​are less than the segmentation threshold of the foreground frame to 0, so as to obtain a binary image of the region of interest of the foreground frame; the camera device sets the grayscale values ​​of the pixels in the region of interest of the background frame whose grayscale values ​​are greater than or equal to the segmentation threshold of the background frame to 1, and sets the grayscale values ​​of the pixels in the region of interest of the background frame whose grayscale values ​​are less than the segmentation threshold of the background frame to 0, so as to obtain a binary image of the region of interest of the background frame.

[0007] Further, in step S3, the camera device performs a similarity comparison on the foreground frame and the background frame after the binarization processing, including the following steps: the camera device performs contour search on the region of interest of the foreground frame and the background frame after the binarization processing; the camera device calculates the Hu moment of the contour in the region of interest of the foreground frame and the background frame according to the Hu moment calculation formula; the camera device calculates the similarity of the Hu moment of the contour in the region of interest of the foreground frame and the background frame according to the similarity calculation formula, and obtains the matching value of the pixel point in the region of interest of the foreground frame.

[0008] Further, in step S3, the camera device performs a similarity comparison on the foreground frame and the background frame after the binarization process, including the following steps: the camera device uses the region of interest of the background frame as a matching template, and then matches the region of interest of the foreground frame through the matching template; during the matching process of the camera device, the camera device uses the sum of the squares of the differences between the pixels of the matching template and the pixels of the region of interest of the foreground frame as the matching value of the pixel points of the region of interest of the foreground frame.

[0009] Further, in step S3, the camera device performs a similarity comparison on the foreground frame and the background frame after the binarization processing, including the following steps: the camera device performs contour search on the area of ​​interest of the foreground frame and the background frame after the binarization processing; the camera device matches the matching values ​​of the contour points of the contour in the area of ​​interest of the foreground frame and the background frame according to the feature matching network.

[0010] Furthermore, in step S3, the camera device removes noise points according to the similarity of points in the foreground frame, including the following steps: the camera device traverses the contour points of the contour in the area of ​​interest of the foreground frame, and then removes the pixel points in the area of ​​interest of the foreground frame whose matching values ​​are greater than the set matching values ​​to obtain a processed image.

[0011] A camera device processes images using the above-mentioned image processing method. The camera device comprises a controller, a fill light unit and a camera. The camera is used to acquire images, the fill light unit is used to fill light, and the controller is used to process images acquired by the camera.

[0012] A mobile robot comprises the above-mentioned camera device.

[0013] Compared with the existing technology, the beneficial effect of the present invention lies in that: after acquiring two frames of images, the camera device described in the present application binarizes the pre-processed background frame and the foreground frame with active light source respectively, and then performs a similarity comparison operation on the two binarized frames of images to determine the confidence of the ambient light in the foreground frame, thereby removing the ambient light in the foreground frame and improving the filtering effect of the ambient light. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The figure is a flowchart of an image processing method in one embodiment of the present invention. Implementation

[0015] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions.

[0016] The following is a further description of the specific embodiments of the present invention in conjunction with the drawings of the specification, so that the technical solution and its beneficial effects of the present invention are clearer and more explicit. The following description of the embodiments with reference to the drawings is exemplary and intended to explain the present invention, but cannot be understood as limiting the present invention.

[0017] like Figure 1As shown, an image processing method, the camera device includes a controller, a fill light unit and a camera, the camera is used to obtain an image, the fill light unit is used to fill light, the fill light unit is an LED lamp, and the controller is used to process the image obtained by the camera. The method includes the following steps: S1: the camera device first obtains a frame of environmental image as a background frame, and then controls the fill light unit to work and then obtains a frame of environmental image as a foreground frame. The camera device can be fixed or set on a mobile robot. S2: the camera device preprocesses the obtained background frame. Then the foreground frame and the preprocessed background frame are binarized. The image is composed of a matrix. The RGB value of each point in the matrix is ​​different, and the colors presented are different. Finally, the overall presentation is a color image. The binarization process is to select a global threshold. If the RGB value of a point in the matrix is ​​greater than the global threshold, it is assigned to 255, otherwise it is 0, so that the entire image is divided into a binary image of either black or white. S3: the camera device compares the similarity of the foreground frame and the background frame after the binarization process, and then removes the noise points according to the matching value of the points in the foreground frame. Similarity comparison can be achieved through the matchShapes function method, template matching method or neural network, etc. After acquiring two frames of images, the camera device described in the present application performs binarization processing on the pre-processed background frame and the foreground frame with active light source respectively, and then performs a similarity comparison operation on the two binarized frames to determine the confidence of the ambient light in the foreground frame, thereby removing the ambient light in the foreground frame and improving the filtering effect and filtering speed of the ambient light. The image processing method can be used in line laser projection through image recognition. The method is used to filter the image of the acquired line laser projection, and then the robot obtains the position of the line laser projection in the image from the image of the line laser projection after filtering through a contour recognition algorithm. The robot then determines the situation of obstacles based on the position of the line laser projection in the image, thereby improving the accuracy of the robot in identifying obstacles.

[0018] As one of the embodiments, after the camera device acquires the background frame and the foreground frame, the camera device constructs an area of ​​N*N pixels in size in the foreground frame based on the center point of the foreground frame as a region of interest (ROI (region of interest). In machine vision and image processing, the area to be processed is outlined from the processed image in the form of a box, circle, ellipse, irregular polygon, etc., which is called a region of interest ROI. Various operators and functions are often used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest ROI and perform the next step of image processing), and then constructs the region of interest at the corresponding position of the background frame. Among them, N can be 20. The camera device can also first identify the contour of the region of interest in the foreground frame, and then use one of the contour points as the center point to construct the region of interest. The camera device compares the similarity of images by constructing the region of interest, reduces the amount of data that the camera device needs to calculate, and improves the calculation speed.

[0019] As one embodiment, in step S2, the camera device preprocesses the acquired background frame, including the following steps: the camera device calculates the weight of the region of interest of the background frame according to the two-dimensional normal distribution function. The camera device performs a convolution operation on the weight of the region of interest of the background frame and each pixel point of the region of interest of the background frame to obtain a preprocessed background frame. This method is a Gaussian filtering method, which can prevent the noise in the background frame from affecting the similarity comparison between the foreground frame and the background frame.

[0020] As one of the embodiments, in step S2, the camera performs binarization processing on the foreground frame and the preprocessed background frame, including the following steps: the camera calculates the segmentation thresholds of the regions of interest of the foreground frame and the background frame respectively according to the inter-class variance formula. The camera sets the grayscale values ​​of the pixels in the region of interest of the foreground frame whose grayscale values ​​are greater than or equal to the segmentation threshold of the foreground frame to 1, and sets the grayscale values ​​of the pixels in the region of interest of the foreground frame whose grayscale values ​​are less than the segmentation threshold of the foreground frame to 0, thereby obtaining a binarized image of the region of interest of the foreground frame. The camera sets the grayscale values ​​of the pixels in the region of interest of the background frame whose grayscale values ​​are greater than or equal to the segmentation threshold of the background frame to 1, and sets the grayscale values ​​of the pixels in the region of interest of the background frame whose grayscale values ​​are less than the segmentation threshold of the background frame to 0, thereby obtaining a binarized image of the region of interest of the background frame. The camera calculates the binarized segmentation threshold by the inter-class variance formula, and the calculation is simple and fast, and is not affected by the brightness and contrast of the image.

[0021] As one embodiment, in step S3, the camera compares the similarity of the foreground frame and the background frame after the binarization process, including the following steps: the camera searches for the contours of the regions of interest of the foreground frame and the background frame after the binarization process. When searching for the contours, the camera traverses the regions of interest of the background frame and the foreground frame, deletes the pixels whose adjacent points in the regions of interest of the background frame and the foreground frame are all black pixels, and obtains the contours in the regions of interest of the foreground frame and the background frame. The camera calculates the Hu moment of the contours in the regions of interest of the foreground frame and the background frame according to the calculation formula of the Hu moment. The camera calculates the similarity of the Hu moment of the contours in the regions of interest of the foreground frame and the background frame according to the similarity calculation formula, and obtains the matching value of the pixel points in the region of interest of the foreground frame. The Hu moment is seven invariant moments composed of linear combinations of second-order and third-order central moments.

[0022] As one of the embodiments, in step S3, the camera device compares the similarity of the foreground frame and the background frame after the binarization process, including the following steps: the camera device uses the region of interest of the background frame as a matching template (template matching is an advanced computer vision technology that can identify the part of the image that matches the predefined template. It is the process of moving the template across the entire image and calculating the similarity between the template and the covered window on the image. Template matching is achieved through two-dimensional convolution. In the convolution, the value of the output pixel is obtained by multiplying the elements of two matrices and summing the results. One of the matrices represents the image itself, and the other is the template, which is the convolution kernel), and then matches the region of interest of the foreground frame through the matching template. During the matching process of the camera device, the camera device uses the sum of the squares of the difference between the pixels of the matching template and the pixels of the region of interest of the foreground frame as the matching value of the pixel points of the region of interest of the foreground frame.

[0023] As one embodiment, in step S3, the camera device performs a similarity comparison on the foreground frame and the background frame after the binarization process, including the following steps: the camera device performs contour search on the region of interest of the foreground frame and the background frame after the binarization process. The camera device matches the matching values ​​of the contour points of the contours in the region of interest of the foreground frame and the background frame according to the feature matching network. This method is a feature matching algorithm based on a graph convolutional neural network, which can improve the matching speed of contour points.

[0024] As one of the embodiments, in step S3, the camera device removes noise points according to the similarity of points in the foreground frame, including the following steps: the camera device traverses the contour points of the contour in the area of ​​interest of the foreground frame, and then removes the pixel points in the area of ​​interest of the foreground frame whose matching values ​​are greater than the set matching values ​​to obtain a processed image.

[0025] A camera device processes images using the above-mentioned image processing method. The camera device comprises a controller, a fill light unit and a camera. The camera is used to acquire images, the fill light unit is used to fill light, and the controller is used to process images acquired by the camera.

[0026] A mobile robot comprises the above-mentioned camera device.

[0027] In the description of the specification, the description with reference to the terms "one embodiment", "preferably", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention, and the schematic expressions of the above terms in this specification do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. The connection method connected in the description of the specification has obvious effects and practical effectiveness.

[0028] Through the description of the above structure and principle, technicians in the relevant technical field should understand that the present invention is not limited to the above-mentioned specific implementation methods. Improvements and substitutions based on the present invention using the well-known technology in the field all fall within the protection scope of the present invention and should be defined by the claims.

Claims

1. An image processing method, characterized in that: The method comprises the following steps: S1: The camera device first obtains a frame of the environment image as a background frame, and then controls the fill light unit in the camera device to work and then obtains another frame of the environment image as a foreground frame; S2: the camera device pre-processes the acquired background frame, and then performs binarization processing on the foreground frame and the pre-processed background frame; S3: The camera device compares the similarity between the foreground frame and the background frame after the binarization process, and then removes noise points according to the matching values ​​of the points in the foreground frame.

2. An image processing method according to claim 1, characterized in that: After acquiring the background frame and the foreground frame, the camera device constructs an N*N pixel area in the foreground frame as the region of interest based on the center point of the foreground frame, and then constructs the region of interest at the corresponding position of the background frame.

3. An image processing method according to claim 2, characterized in that: In step S2, the camera device pre-processes the acquired background frame, including the following steps: The camera device calculates the weight of the region of interest of the background frame according to the two-dimensional normal distribution function; The camera device performs a convolution operation on the weight of the region of interest of the background frame and each pixel point of the region of interest of the background frame to obtain a preprocessed background frame.

4. An image processing method according to claim 3, characterized in that: In step S2, the camera device performs binarization processing on the foreground frame and the pre-processed background frame, including the following steps: The camera device calculates the segmentation thresholds of the regions of interest of the foreground frame and the background frame respectively according to the inter-class variance formula; The camera device sets the grayscale values ​​of pixels in the region of interest of the foreground frame whose grayscale values ​​are greater than or equal to the segmentation threshold of the foreground frame to 1, and sets the grayscale values ​​of pixels in the region of interest of the foreground frame whose grayscale values ​​are less than the segmentation threshold of the foreground frame to 0, thereby obtaining a binary image of the region of interest of the foreground frame; The camera device sets the grayscale values ​​of the pixels in the region of interest of the background frame whose grayscale values ​​are greater than or equal to the segmentation threshold of the background frame to 1, and sets the grayscale values ​​of the pixels in the region of interest of the background frame whose grayscale values ​​are less than the segmentation threshold of the background frame to 0, thereby obtaining a binary image of the region of interest of the background frame.

5. An image processing method according to claim 4, characterized in that: In step S3, the camera device performs similarity comparison on the foreground frame and the background frame after the binarization process, including the following steps: The camera device performs contour search on the region of interest of the foreground frame and the background frame after the binarization process; The camera device calculates the Hu moment of the contour in the region of interest of the foreground frame and the background frame according to the calculation formula of the Hu moment; The camera device calculates the similarity of the Hu moments of the contours in the regions of interest of the foreground frame and the background frame according to a similarity calculation formula to obtain matching values ​​of the pixel points in the region of interest of the foreground frame.

6. An image processing method according to claim 4, characterized in that: In step S3, the camera device performs similarity comparison on the foreground frame and the background frame after the binarization process, including the following steps: The camera device uses the region of interest of the background frame as a matching template, and then matches the region of interest of the foreground frame through the matching template; During the matching process, the camera device uses the sum of squares of differences between pixels of the matching template and pixels of the region of interest of the foreground frame as the matching value of the pixel points of the region of interest of the foreground frame.

7. An image processing method according to claim 4, characterized in that: In step S3, the camera device performs similarity comparison on the foreground frame and the background frame after the binarization process, including the following steps: The camera device performs contour search on the region of interest of the foreground frame and the background frame after the binarization process; The camera device matches matching values ​​of contour points of contours in the regions of interest of the foreground frame and the background frame according to a feature matching network.

8. An image processing method according to claim 5, 6 or 7, characterized in that: In step S3, the camera device removes noise points according to the similarity of points in the foreground frame, including the following steps: The camera device traverses the contour points of the contour in the region of interest of the foreground frame, and then removes the pixel points in the region of interest of the foreground frame whose matching values ​​are greater than a set matching value to obtain a processed image.

9. A camera device, characterized in that: The camera device processes images by the image processing method described in any one of claims 1 to 8, and the camera device includes a controller, a fill light unit and a camera, the camera is used to acquire images, the fill light unit is used to fill light, and the controller is used to process images acquired by the camera.

10. A mobile robot, characterized in that: The mobile robot includes the camera device according to claim 9.

Images