Recognition method of flexible and deformable objects in complex background based on convolutional neural network

Abstract

The present invention discloses a method for identifying flexible and deformable objects in a complex background based on a convolutional neural network, which relates to the field of automated processing of visual images and machine learning, and solves the problem of low recognition rate due to complex background interference or easy deformation of the object to be identified. The key points of its technical solution are to enhance the data of the original image samples through image processing technologies such as size scaling, movement and rotation, color conversion, background change, noise simulation, artificial occlusion, and pre-deformation, and then input the training image data set into the convolutional neural network for training, so that the neural network "autonomously learns" to identify flexible objects in a complex background; compared with existing technologies such as artificial extraction of image features and then machine learning or establishment of a model library for comparison and identification, the method proposed in this application greatly reduces the requirements of the neural network model on the sample size of the training data set, and does not require the establishment of a physical model. It has a high degree of automation and a wide range of applications, and can achieve extremely high recognition accuracy even when processing complex, flexible and deformable objects.

Description

Technical Field

[0001] The present disclosure relates to the fields of automated processing of visual images and machine learning, and in particular to a method for recognizing flexible and deformable objects in complex backgrounds based on convolutional neural networks. Background Art

[0002] Machine vision, as a new sensing technology, has been widely applied in fields such as quality inspection, security monitoring, facial recognition, robotic perception, and home services. This has led to the emergence of a number of image processing algorithms for object recognition and classification. Traditional image processing techniques primarily target scenes with simple backgrounds, fixed surroundings, and ideal lighting conditions, and the objects to be identified are often rigid and non-deformable. In recent years, using machine vision to identify flexible and easily deformable objects in complex environments has become a hot and challenging issue in many manufacturing and scientific research fields.

[0003] In existing technologies, complex environments can easily interfere with the object being identified. Even the brightness of background light can directly affect the recognition quality of the main object in the image. Furthermore, flexible objects are prone to deformation. Even for objects of the same category, their images can change if the camera's shooting angle or distance changes, or if they are deformed by external forces (for example, when a robot folds clothes). This can severely reduce the recognition success rate. To address this problem, there are currently two main technical approaches: 1) Collecting a large dataset of samples of the object to be identified in different scenarios, extracting multiple attribute features of different object categories, and training these features into a neural network or machine learning model to achieve flexible object recognition; 2) Building a library of physical models of different object categories, containing the geometric shapes of objects under various conditions. By comparing the collected images with the existing models in the library, the category of the object to be identified is determined. The former method requires a large sample dataset, and the model quality is easily affected by the extracted features, resulting in high time costs and insufficient recognition accuracy. The latter method requires building a physical model of the object to be identified, which is more difficult for flexible objects under complex force conditions and has significant application limitations.

[0004] Therefore, there is an urgent need for a method that has low requirements for training sample data sets, is highly versatile, and has high recognition accuracy to solve the problem of identifying flexible and deformable objects in complex backgrounds. Summary of the Invention

[0005] This paper provides a method for recognizing flexible and easily deformable objects in complex backgrounds based on a convolutional neural network. The initial training set is images with simple backgrounds. For the same test set and the same network structure, the network recognition rate is higher when the training set is trained with data augmentation. The specific implementation method is as follows:

[0006] First, read each image in the dataset in turn, and convert the image into:

[0007] Scaling: (x', y') = (x*m, y*m), where m is the scaling factor, which is random. When m is greater than 1, the image is enlarged, and when m is less than 1, the image is reduced. After processing, the image is stored as it was (add "-1" to the name to avoid overwriting the original image). Scaling can reduce the impact of shooting distance during actual classification;

[0008] Move a certain distance: (x', y') = (x + m, y + n), where m and n are the horizontal and vertical distances. The distances are random but small to avoid excessive information loss. After processing, the image is stored according to the original path (with "-2" added to the name to avoid overwriting other images). The movement process can reduce the impact of the flexible object being identified not being in the center of the image during actual recognition.

[0009] Rotate a certain angle: Where θ represents the counterclockwise rotation angle, which is random. After processing, the image is stored according to the original path (with "-3" added to the name to avoid overwriting other images).

[0010] Color change: (R i ,G i ,B i )=(R i +r,G i +r,B i +r). Where r is the amount of change in each component, and this change is modulo 256. After processing, it is stored according to the original path (with "-4" added to the name to avoid overwriting other images). Color changes can increase the number of data sets and reduce the impact of different light color differences during actual recognition;

[0011] Artificial occlusion: randomly select a rectangular area I on the image d , calculate the area of the region, if the selected area is greater than or equal to 10%, reselect the region to avoid losing too much information. If the selected area is less than 10%, then To achieve the occlusion effect. After processing, it is stored according to the original path (the name is added with "-5" to avoid covering other images). Occlusion processing can reduce the impact of flexible object deformation on the recognition results during the actual recognition process;

[0012] Adding noise: Where RGB is the RGB component of each pixel, m is the Gaussian noise value, μ is the mean, and σ is the standard deviation. After processing, the image is stored in the original path (with "-6" added to the name to avoid overwriting other images). Noise simulation can increase the diversity of the dataset;

[0013] Background change: Perform edge detection to obtain the contour point set I0, and then obtain the target area point set I d , set the background image I b Change the size of the image to be processed I p The size of the pixel (x i ,y i ) are judged: If (x i ,y i )∈I d , then keep the pixel unchanged; if Then there is After processing, the images are stored in the original path (with "-7" added to the name to avoid overwriting other images). Background changes can reduce the impact of background changes during the actual recognition process;

[0014] Pre-deformation: Apply force or couple to the middle, upper, or lower part of a flexible object, collect images of the flexible object after deformation under specified circumstances, and add them to the dataset.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects:

[0016] In the present invention, after the above 8 steps are performed, {I1} stores more information than {I0}, the data volume becomes at least 8 times the original amount and covers more information, thus achieving data enhancement.

[0017] The enhanced data is used to train the neural network, and the resulting network has higher accuracy and better recognition effect for flexible objects than the original network.

[0018] The present invention can be applied not only to the intelligent recognition of flexible and deformable objects in complex backgrounds, but also to scenes with simple backgrounds and rigid objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the data enhancement method of the present invention;

[0020] Figure 2 It is the overall flow chart of the training and application of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0022] Figure 1 Flowchart of the feature extraction method of the present invention, as shown in FIG. Figure 1 As shown, all image data in the original dataset {I0} are rotated and translated, resized, color-converted, background-changed, noise-simulated, and artificially occluded in sequence, so that more information can be stored in the dataset and the ability to classify data images is stronger.

[0023] As a specific embodiment, all images in the small sample data {I0} used for the experiment are subjected to data augmentation operations according to the above data augmentation method. The specific operations are as follows:

[0024] Scaling: (x', y') = (x*m, y*m). m represents the scaling factor, which should be between 0.6 and 1.3 to avoid excessive distortion.

[0025] Move and rotate: (x', y') = (x+m, y+b), Where m and n are the horizontal and vertical translations, which should not exceed 15% of the original size to avoid excessive information loss. θ represents the counterclockwise rotation angle, which can be anywhere between 0 and 360°.

[0026] Color change: (R i ,G i ,B i )=(R i +r,G i +r,B i +r). Where r is the change in each component, and this change is modulo 256.

[0027] Background change: Perform edge detection to obtain the contour point set I0, and then obtain the target area point set I d , set the background image I b Change the size of the image to be processed I p The size of the pixel (x i ,y i ) are judged: If (x i ,y i )∈I d , then keep the pixel unchanged; if Then there is

[0028] Noise simulation: (R', G', B') = (R+m, G+m, B+m), Where RGB is the RGB component of each pixel, m is the Gaussian noise value, μ is the mean, σ is the standard deviation, the mean is 1, and the variance is 0.3.

[0029] Artificial occlusion: randomly select a rectangular area I on the image d , calculate the area of the region, if the area is too large (more than 10% of the total area) then reselect the region, if the area meets the requirements, then make

[0030] The processed results are obtained, and the pre-deformed data is collected by the camera. The two types of data are combined to obtain the enhanced dataset {I1}.

[0031] Obviously, {I1} stores more information than {I0}.

[0032] Use the {I1} dataset to train a convolutional neural network to obtain network N.

[0033] Use network N to classify specific images: Use network N to test and verify the test set data. It is found that the accuracy of the results has been greatly improved.

[0034] The above are exemplary embodiments of the present invention, and the protection scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for recognizing flexible and deformable objects in complex backgrounds based on convolutional neural networks, characterized by: The specific steps include: S1. Obtain a small sample dataset {I0} of visual images of the object to be identified; S2, extract the part of the object to be identified in each image in the dataset, apply size scaling, movement and rotation, color transformation, background change, noise simulation, artificial occlusion, and data enhancement to the image; S21, extracting the image portion of the subject to be identified using a Moore neighborhood tracking algorithm or a Gabor filter method; S22. Size scaling: (x', y') = (x*m, y*m), where m represents the scaling factor. S23, move and rotate: (x', y') = (x + m, y + n), Among them, m and n are the horizontal and vertical translations; θ represents the counterclockwise rotation angle; S24, color change: (R i ,G i ,B i )=(R i +r,G i +r,B i +r). The change amount for each channel is the same, r, and the change is modulo 256; S25, background change: perform edge detection to obtain the contour point set I0, and then obtain the target area point set I d , set the background image I b Change the size of the image to be processed I p The size of each pixel (x i ,y i ) are judged: If (x i ,y i )∈I d , then keep the pixel unchanged; if Then there is S26, noise simulation: Where RGB is the RGB component of each pixel, m is the Gaussian noise value, μ is the mean, and σ is the standard deviation; S27, artificial occlusion: randomly select pixels on the image, and generate an area S by the pixels. r Occlusion Area I d ; Calculate the area of the region. If the area is too large, exceeding 10% of the total area, reselect the region. If the area meets the requirements, S28. Obtain the processed result and use the camera to collect the pre-deformed data; combine the two data to obtain the enhanced data set {I1} S29, pre-deformation: preset several external force application scenarios, collect images of the flexible object after deformation under the specified scenarios, and add them to the data set; S3. Use {I1} to train the convolutional neural network to obtain a recognition model; predict the category of the recognized object.

2. The method for recognizing flexible and deformable objects in complex backgrounds based on convolutional neural networks according to claim 1, wherein: Visual image small sample dataset {I0}, directly acquired through a planar camera; Or it is generated by removing the depth information from the image captured by the depth camera.

3. The method for recognizing flexible and deformable objects in complex backgrounds based on convolutional neural networks according to claim 1, wherein: Each image in {I0} contains only one subject target to be identified, and the entire dataset must cover all categories of the subject target.

4. The method for recognizing flexible and deformable objects in complex backgrounds based on convolutional neural networks according to claim 1, wherein: The convolutional neural network training method in S3 uses enhanced training set data to train the network model.

Images