A deep image data enhancement method using quantitative skew

Abstract

The present invention discloses a method for enhancing depth image data by using quantitative deflection. The method recognizes the framing center axis of a binocular camera and the image center axis of an image to be framed, overlaps the framing center axis with the image center axis to construct a Z axis, and constructs a rotating framing coordinate system of the depth image based on the Z axis. The bottom end of the image center axis is fixed at the origin of the rotating framing coordinate system and framing at the origin is completed, thereby fixing the basic position of the image to be framed. Subsequently, the image to be framed is deflected frame by frame and rotated at a fixed axis at a uniform speed to enhance the presentation of features at various angles of the image to be framed, and the enhanced and presented features are compared with the image to be framed in a rotated state, thereby comprehensively eliminating interference features. The present invention has the advantage that the feature recognition effect of the image to be framed can be enhanced, thereby adapting to the deep learning requirements of a visual detection system.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection, and in particular to a method for enhancing deep image data using quantitative skew. Background Art

[0002] In recent years, deep learning has been used more and more widely in the field of computer vision. The excellent performance of deep learning in dealing with many problems in the field of computer vision has led more and more researchers to start this research direction. The reason why deep learning can perform so well is that the deep convolutional network itself has a strong expressive ability and can train the required model results according to the training objectives. However, because of this, the network model itself requires a large amount of data to drive model training, otherwise the model may fall into the dilemma of overfitting. However, in actual situations, not all data sets have a large number of training samples. For this reason, in the actual training process, data enhancement has become an important step in model training. Effective data expansion can not only expand the number of training samples, but also increase the diversity of sample training. On the one hand, it can avoid model overfitting, and on the other hand, it can improve the performance of the model. Common image data enhancement methods include: horizontal flipping, random rotation, random scaling, random cropping, random translation, etc. These common image data enhancement methods are all applied in the field of RGB images. For other types of images, these methods are not applicable.

[0003] With the development of binocular cameras in recent years, the cost of binocular cameras has become lower and lower, and depth images collected by binocular cameras are being used by more and more researchers in the field of computer vision. For example: human skeleton key point detection, human behavior recognition, gesture recognition and other fields. However, the common data enhancement methods used in the field of RGB images are not applicable to depth images. Since the value stored in each pixel of the depth image is the depth distance from the location to the camera, directly using the data enhancement method in the RGB image will cause the image to be distorted. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a depth image data enhancement method using quantitative deflection to address the deficiencies of the prior art.

[0005] Technical solution: The method for enhancing deep image data using quantitative deflection described in the present invention comprises the following steps:

[0006] S1. Identify the viewfinder central axis of the binocular camera, treat the image central axis of the viewfinder image at the same time, overlap the viewfinder central axis with the image central axis to construct the Z axis, and construct a rotation viewfinder coordinate system of the depth image based on the Z axis;

[0007] S2, fixing the bottom end of the central axis of the image at the origin of the rotating framing coordinate system and completing framing at the origin;

[0008] S3, fix the bottom end of the central axis of the image, and make the central axis of the image take the bottom end of the central axis of the image as the center of the circle, and perform a deflection action in a quantitative manner frame by frame along a fixed vertical plane, and the binocular camera performs a shooting and framing action for each frame of the deflection action;

[0009] S4, output the deflection action of the binocular camera in S3 frame by frame, arrange the frames in order according to the deflection angle, and perform visual recognition frame by frame, so that the features of each angle in the deflection direction can be enhanced and presented

[0010] S5, fix the bottom end of the central axis of the image, and make the central axis of the image take the bottom end of the central axis of the image as the center of the circle, and perform a deflection action in a quantitative manner frame by frame along a fixed vertical plane in the opposite direction to that in S3, and when each frame of the deflection action is performed, the binocular camera performs a shooting and framing action, and then repeat S4;

[0011] S6: The whole scene is rotated at a constant speed with the central axis of the image as the fixed axis. A binocular camera is used to continuously frame the scene. The enhanced features presented in S4 and S5 are imported and compared continuously to completely eliminate interference features.

[0012] Preferably, in S6, the central axis of the image is used as the fixed axis to make the entire scene perform a clockwise fixed-axis uniform rotation and a counterclockwise uniform rotation, respectively adapting to the actions of S4 and S5.

[0013] Preferably, the angle of inclination of each grid of the image center axis in S3 and S5 is 0.1°.

[0014] Preferably, the angular velocity of the fixed-axis uniform rotation in S6 is 12° / s.

[0015] Compared with the prior art, the present invention has the following beneficial effects: by comparing the enhanced presented features with the image to be framed in a rotated state, interfering features can be completely eliminated, which can enhance the feature recognition effect of the image to be framed, and further can adapt to the deep learning requirements of the visual inspection system. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] In the description of the present invention, it is necessary to understand that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships are for the purpose of facilitating the description of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0018] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integrated connection; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0019] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Embodiment: A method for enhancing deep image data using quantitative deflection comprises the following steps:

[0021] S1. Identify the viewfinder central axis of the binocular camera, treat the image central axis of the viewfinder image at the same time, overlap the viewfinder central axis with the image central axis to construct the Z axis, and construct a rotation viewfinder coordinate system of the depth image based on the Z axis;

[0022] S2, fixing the bottom end of the central axis of the image at the origin of the rotating framing coordinate system and completing framing at the origin;

[0023] S3, fix the bottom end of the image axis, and make the image axis take the bottom end of the image axis as the center, and perform a deflection action in a quantitative manner frame by frame along a fixed vertical plane. When each deflection action is performed, the binocular camera performs a shooting and framing action, and the deflection angle of each frame is 0.1°;

[0024] S4, output the deflection action of the binocular camera in S3 frame by frame, arrange the frames in order according to the deflection angle, and perform visual recognition frame by frame, so that the features of each angle in the deflection direction can be enhanced and presented

[0025] S5, fix the bottom end of the image axis, and make the image axis take the bottom end of the image axis as the center, and perform a deflection action in a quantitative manner frame by frame along a fixed vertical plane in the opposite direction to S3, with the deflection angle of each frame being 0.1°. When performing each deflection action, the binocular camera performs a shooting and framing action, and then repeat S4;

[0026] S6: With the central axis of the image as the fixed axis, the entire scene is rotated once clockwise and once counterclockwise. The angular velocity of the fixed-axis uniform rotation is 12° / s, which are adapted to the actions of S4 and S5 respectively. At the same time, a binocular camera is used for continuous framing. The enhanced features presented in S4 and S5 are imported and compared continuously to completely eliminate interference features.

[0027] The advantage of this technical solution is that the enhanced presented features are compared with the image to be framed in a rotated state, so as to completely eliminate interfering features, which can enhance the feature recognition effect of the image to be framed, and further adapt to the deep learning requirements of the visual inspection system.

[0028] In the present invention, unless otherwise clearly specified and limited, the first feature being "above" or "below" the second feature may mean that the first feature is in direct contact with the second feature, or the first feature and the second feature are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "above" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature. In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example.

[0029] Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradicting each other.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for depth image data enhancement using quantitative skew, characterized in that: The following steps are involved: S1. Identify the viewfinder central axis of the binocular camera, treat the image central axis of the viewfinder image at the same time, overlap the viewfinder central axis with the image central axis to construct the Z axis, and construct a rotation viewfinder coordinate system of the depth image based on the Z axis; S2, fixing the bottom end of the central axis of the image at the origin of the rotating framing coordinate system and completing framing at the origin; S3, fix the bottom end of the central axis of the image, and make the central axis of the image take the bottom end of the central axis of the image as the center of the circle, and perform a deflection action in a quantitative manner frame by frame along a fixed vertical plane, and the binocular camera performs a shooting and framing action for each frame of the deflection action; S4, outputting the deflection action of the binocular camera in S3 for each grid frame by frame, arranging the frames in sequence according to the deflection angle, and performing visual recognition frame by frame, so that the features of each angle in the deflection direction can be enhanced and presented; S5, fix the bottom end of the central axis of the image, and make the central axis of the image take the bottom end of the central axis of the image as the center of the circle, and perform a deflection action in a quantitative manner frame by frame along a fixed vertical plane in the opposite direction to that in S3, and when each frame of the deflection action is performed, the binocular camera performs a shooting and framing action, and then repeat S4; S6: The whole scene is rotated at a constant speed with the central axis of the image as the fixed axis. A binocular camera is used to continuously frame the scene. The enhanced features presented in S4 and S5 are imported and compared continuously to completely eliminate interference features.

2. The method for depth image data enhancement using quantitative deflection according to claim 1, characterized in that: In S6, the image center axis is used as the fixed axis to make the entire scene perform a clockwise fixed axis uniform rotation and a counterclockwise uniform rotation, respectively adapting to the actions of S4 and S5.

3. The method for depth image data enhancement using quantitative deflection according to claim 1, characterized in that: The angle of deflection of the image axis in S3 and S5 is 0.1° per grid.

4. The method for depth image data enhancement using quantitative deflection according to claim 1, characterized in that: The angular velocity of the fixed-axis uniform rotation in S6 is 12° / s.