Automatic recognition method and device for continuous frame RGB images

Abstract

The present invention discloses a method and device for automatically identifying continuous frame RGB images, including: generating a series of continuous frame RGB images by rendering according to a set rendering trajectory, and outputting the camera attributes of each frame RGB image; after detecting and eliminating invalid images, black images, and blurred images in the RGB images, judging whether the quality of the two frames of RGB images is qualified according to the angle amplitude and similarity of two adjacent frames of RGB images calculated based on the camera attributes and pixel values; post-processing the RGB images of unqualified quality, including manual verification, re-rendering of corresponding camera attributes, and re-rendering of corresponding rendering attributes, so as to further detect the quality of the RGB images. The method and device realize the automatic recognition of the differences of continuous frame RGB images, and have high recognition efficiency and high recognition accuracy.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a method and a device for automatically recognizing continuous frame RGB images. Background Art

[0002] When the data generation platform generates image data, a single project delivery task can produce more than 100,000 RGB images. The image quality of these RGB images needs to be manually checked. Invalid images with defects appear as follows: black images, blurred images, abnormal renderings with too much noise, abnormal renderings with missing targets, abnormal renderings with missing materials, etc.

[0003] When there are too many RGB images, the image quality is difficult to pass manual inspection. Therefore, a tool that can automatically identify images is needed. It can automatically identify the differences between adjacent frames and filter out low-quality images. Manual attention needs to be paid to the images, which saves human resources and can improve the test coverage and quality of the delivered data.

[0004] Patent document CN104794189A discloses an image screening method, including: obtaining and storing the characteristic values ​​of sample images in advance; sorting all images to be screened in a specific time sequence; obtaining the characteristic values ​​of all images to be screened after sorting, and storing them in a specific time sequence; comparing the characteristic values ​​of all images to be screened with the characteristic values ​​of pre-stored sample images, and screening out all images that meet predetermined conditions. This image screening method is based on standard sample images for screening, but when the data generation platform generates images, there are no standard sample images, so this method is not applicable.

[0005] Patent document CN105512617A discloses an image screening method, including: judging whether the image to be screened meets the preset standard; if the image to be screened meets the preset standard, decomposing the image to be screened by face recognition technology to obtain multiple images to be compared of different facial parts; scoring each of the multiple images to be compared according to the reference image database to obtain the score of each image to be compared; obtaining the score of the image to be screened according to the scores of the multiple images to be compared; when the score of the image to be screened is greater than the preset threshold, determining that the image to be screened is an image that meets the screening conditions. This image screening method is based on the preset standard and combined with the reference image, scoring the image to be compared, and screening according to the score, but when the image is generated on the data generation platform, there is no standard sample image, so this method is not applicable. Summary of the invention

[0006] In view of the above, an object of the present invention is to provide a method and device for automatically identifying continuous frame RGB images, so as to realize automatic identification of differences of continuous frame RGB images, and have high identification efficiency and high identification accuracy.

[0007] To achieve the above-mentioned object of the invention, a first embodiment provides a method for automatically identifying a continuous frame RGB image, comprising the following steps:

[0008] Generate a series of continuous frame RGB images according to the set rendering trajectory, and output the camera properties of each frame RGB image;

[0009] After removing the black and blurred images in the RGB image, the angle amplitude and similarity of two adjacent RGB images are calculated based on the camera attributes and pixel values ​​to determine whether the quality of the two RGB images is qualified;

[0010] Post-processing is performed on RGB images with unqualified quality, including manual verification, re-rendering with corresponding camera attributes, and re-rendering with corresponding rendering attributes to further detect the quality of RGB images.

[0011] The camera attributes include the camera position and the camera viewing angle.

[0012] In one embodiment, the angle amplitude of two adjacent RGB images is calculated according to the camera properties, including:

[0013] The camera position of each frame image and the camera viewing angle form a vector, and the angle between the vectors of the two frames of image is calculated as the angle amplitude.

[0014] In one embodiment, calculating the similarity between two adjacent RGB images based on pixel values ​​includes:

[0015] The two adjacent RGB frames are converted into normalized histogram data, and the similarity between the two adjacent RGB frames is determined by performing a similarity comparison on the histogram data.

[0016] In one embodiment, calculating the similarity between two adjacent RGB images based on pixel values ​​includes:

[0017] The current frame RGB image and the two RGB images before and after it are regarded as a unit, and the three RGB images in the unit are converted into normalized histogram data. The similarity comparison is performed on the histogram data of two pairs of adjacent frames to determine two similarities, and then the two similarities are averaged as the similarity of each pair of adjacent RGB frames.

[0018] In one embodiment, the angle amplitude and similarity of two adjacent RGB image frames calculated based on camera attributes and pixel values ​​to determine whether the quality of the two RGB image frames is qualified includes:

[0019] Compare the rotation angle amplitude of two adjacent RGB image frames with the rotation angle amplitude threshold. When the rotation angle amplitude is greater than the rotation angle amplitude threshold, it is preliminarily determined that the quality of the two RGB image frames is unqualified and the camera IDs of the two RGB image frames are recorded.

[0020] When the angle amplitude is less than or equal to the angle amplitude threshold, the similarity between the two adjacent RGB images is compared with the similarity threshold. When the similarity is greater than the similarity threshold, the quality of the two RGB images is considered to be qualified; when the similarity is less than or equal to the similarity threshold, it is preliminarily judged that the quality of the two RGB images is unqualified and the camera IDs of the two RGB images are recorded.

[0021] In one embodiment, the post-processing of the RGB image of unqualified quality includes:

[0022] First, we manually check the RGB images that do not meet the quality standards, and re-render the RGB images that are confirmed to have quality problems with the corresponding camera attributes.

[0023] Then, the re-rendered RGB images are manually checked for the second time. For the RGB images confirmed to have quality problems, the corresponding rendering attributes are re-rendered. If the quality of the re-rendered RGB images is qualified, the rendering engine is considered to be unstable. The rendering engine is re-rendered using the corresponding camera attributes and the corresponding rendering attributes to obtain a qualified RGB image.

[0024] If the quality of the RGB image re-rendered with the corresponding rendering attributes is unqualified, the cause of the unqualified quality is manually checked and corrected.

[0025] To achieve the above-mentioned object of the invention, a second embodiment provides an automatic recognition device for continuous frame RGB images, comprising:

[0026] The rendering generation module is used to generate a series of continuous frame RGB images according to the set rendering trajectory, and output the camera properties of each frame of RGB image;

[0027] The automatic recognition module is used to detect and remove the black and blurred images in the RGB image, and then determine whether the quality of the two adjacent RGB images is qualified based on the angle amplitude and similarity of the two adjacent RGB images calculated according to the camera attributes and pixel values;

[0028] The post-processing module is used to post-process the RGB images with unqualified quality, including manual verification, re-rendering according to the corresponding camera attributes, and re-rendering according to the corresponding rendering attributes, so as to further detect the quality of the RGB images.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] By comparing the angle amplitude and similarity, we screen out RGB images with unqualified quality as images of manual attention. For the RGB images with unqualified quality that we manually pay attention to, we use manual verification, re-rendering of corresponding camera attributes, and re-rendering of corresponding rendering attributes to further detect the RGB quality. This method can automatically identify and filter out images with unqualified quality, and automatically identify the differences between RGB images of adjacent frames. Since the RGB images of manual attention are a small number of images filtered by the algorithm, human resources are greatly saved. The use of machine automatic recognition and detection and manual verification methods greatly improves the test coverage and quality of the delivered RGB images, and also improves the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 is a flow chart of a method for automatically identifying continuous frame RGB images provided by an embodiment;

[0033] Figure 2 is a flow chart of a method for automatically identifying continuous frame RGB images provided by another embodiment;

[0034] Figure 3 It is a schematic diagram of the structure of the automatic recognition device for continuous frame RGB images provided in the embodiment. DETAILED DESCRIPTION

[0035] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0036] In order to automatically identify the quality and similarity of RGB images in order to solve the problem of defective data quality in image data generation tasks, an embodiment provides a method and device for automatically identifying continuous frame RGB images, which can automatically detect the quality of RGB images and filter out defective RGB images, thereby improving recognition accuracy and efficiency.

[0037] Figure 1 1 is a flow chart of the method for automatically identifying continuous frame RGB images provided by the embodiment. Figure 1 As shown, the automatic recognition method of continuous frame RGB images provided by the embodiment includes the following steps:

[0038] Step 1: Configure and generate an image data generation task, perform image rendering according to the image data generation task, generate a series of continuous frame RGB images, and output the camera properties of each frame RGB image.

[0039] In the embodiment, when configuring the image data generation task, it is necessary to set a three-dimensional scene model and a visualized camera motion trajectory. The camera movement speed and frame rate can be set for a bow-shaped trajectory roaming in the scene. The camera movement speed is required to be slower and the frame rate is higher. The specific camera movement speed and frame rate are set according to actual conditions. At the same time, the camera's up and down pitch angles (pitch) and height (height) are also set to remain unchanged.

[0040] The VRay rendering engine renders the 3D scene model according to the image data generation task, generates a series of continuous frame RGB images and outputs them, and also outputs the camera properties of each frame RGB image and stores them in the camera list. Among them, the camera properties include the camera position (location), the camera's viewing angle (lookat), the field of view (fov), etc. The camera properties are used to calculate the angle of rotation between two frames of images.

[0041] Step 2: Detect and remove black and blurred images in the RGB image.

[0042] In the embodiment, a removal script for removing basic defect images is also prepared. After the RGB image is obtained, the detection and removal of black images and blurred images in the RGB image are realized by running the removal script. Specifically, Python can be used to complete the recognition of all-black images and all-blurred images, and these images are marked and removed as invalid images, and the camera ID of the invalid image is also recorded.

[0043] Step 3: According to the rotation angle and similarity of two adjacent RGB image frames calculated based on the camera attributes and pixel values, determine whether the quality of the two RGB image frames is qualified.

[0044] In the embodiment, quality detection is performed on the remaining RGB image after the detection in step 2. Specifically, the difference between two adjacent frames of RGB images is detected by combining the angle amplitude and similarity to achieve quality detection.

[0045] In the embodiment, judging whether the quality of two adjacent RGB images is qualified based on the angle amplitude and similarity of the two adjacent RGB images calculated according to the camera attributes and pixel values ​​includes:

[0046] (a) Calculate the rotation angle of two adjacent RGB images based on the camera properties. That is, the camera position and the camera viewing angle of each frame form a vector, and calculate the angle between the vectors of the two frames as the rotation angle.

[0047] (b) comparing the rotation angle amplitude of two adjacent RGB image frames with the rotation angle amplitude threshold. When the rotation angle amplitude is greater than the rotation angle amplitude threshold, it is preliminarily determined that the quality of the two RGB image frames is unqualified and the camera IDs of the two RGB image frames are recorded and output;

[0048] (c) When the angle amplitude is less than or equal to the angle amplitude threshold, the similarity between two adjacent RGB image frames is calculated based on the pixel value. In one embodiment, the two adjacent RGB image frames can be converted into normalized histogram data, and the similarity between the two adjacent RGB image frames is determined by performing a similarity comparison on the histogram data.

[0049] In another embodiment, the current frame RGB image and the two RGB images before and after it are combined into a unit, and the three RGB images in the unit are converted into normalized histogram data. The histogram data of two pairs of adjacent frames are compared for similarity to determine two similarities, and then the two similarities are averaged as the similarity of each pair of adjacent RGB frames.

[0050] When performing similarity comparison, the principle of using the histogram comparison method is: convert the two frames of pictures to be compared into histogram data, and then normalize the histogram data for easy comparison, and finally obtain a similarity index. By setting the boundary of the similarity index, it can be determined whether the picture similarity meets expectations. This comparison method is simple and fast.

[0051] (d) Compare the similarity between two adjacent RGB images with the similarity threshold. When the similarity is greater than the similarity threshold, the quality of the two RGB images is considered to be qualified; when the similarity is less than or equal to the similarity threshold, it is preliminarily judged that the quality of the two RGB images is unqualified and the camera IDs of the two RGB images are recorded and output.

[0052] It should be noted that the similarity threshold, as a standard for judging the difference between two adjacent RGB image frames, can be set according to data quality requirements, and the value is 0-1. The larger the value, the greater the similarity.

[0053] It should also be noted that in order to avoid errors caused by excessive changes in the camera's lookat angle, a corner amplitude threshold is set as a standard for judging the corner amplitude. The setting is specified based on data quality requirements. The smaller the corner amplitude threshold, the more accurate the result of judging the similarity is.

[0054] In the embodiment, the RGB images that have been tested and confirmed to be of qualified quality are stored in the database and delivered. The RGB images that have been tested and confirmed to be of unqualified quality and their camera IDs are added to the manual verification list and wait for manual verification.

[0055] Step 4: Post-process the RGB images with unqualified quality, including manual verification, re-rendering according to the corresponding camera attributes, and re-rendering according to the corresponding rendering attributes, to further detect the quality of the RGB images.

[0056] In the embodiment, post-processing is performed on the RGB image with unqualified quality, including:

[0057] First, the first manual inspection is performed on the RGB images with unqualified quality. The RGB images that pass the first manual inspection and have no quality problems are saved in the database. For the RGB images confirmed to have quality problems, the corresponding camera attributes are re-rendered, that is, the location, lookat, fov, etc. of the RGB images with quality problems are used for re-rendering.

[0058] Then, the re-rendered RGB image is manually checked for the second time, and the RGB images that pass the second manual check and have no quality problems are saved in the database; the RGB images confirmed to have quality problems are re-rendered with corresponding rendering attributes. Specifically, no parameters in the rendering task should be changed, and they are manually re-rendered in the rendering engine; if the re-rendered RGB image is of qualified quality, the rendering engine is considered unstable, and the rendering engine is re-rendered using corresponding camera attributes and corresponding rendering attributes to obtain a qualified RGB image. The qualified RGB image replaces the unqualified RGB image and is stored in the database. If the RGB image re-rendered with the corresponding rendering attributes is of unqualified quality, the cause of the unqualified quality is manually investigated and corrected.

[0059] When manually troubleshooting the reasons for the unqualified RGB image quality, check the rendering engine, verify and correct the rendering program, and then re-execute the rendering task in step 1.

[0060] Figure 3 It is a schematic diagram of the structure of the automatic recognition device for continuous frame RGB images provided in the embodiment.

[0061] like Figure 3 As shown, the automatic recognition device for continuous frame RGB images provided by the embodiment includes:

[0062] The rendering generation module is used to generate a series of continuous frame RGB images according to the set rendering trajectory, and output the camera properties of each frame of RGB image;

[0063] The automatic recognition module is used to detect and remove the black and blurred images in the RGB image, and then determine whether the quality of the two adjacent RGB images is qualified based on the angle amplitude and similarity of the two adjacent RGB images calculated according to the camera attributes and pixel values;

[0064] The post-processing module is used to post-process the RGB images with unqualified quality, including manual verification, re-rendering according to the corresponding camera attributes, and re-rendering according to the corresponding rendering attributes, so as to further detect the quality of the RGB images.

[0065] It should be noted that the automatic recognition device for continuous frame RGB images provided in the above embodiment should be illustrated by the division of the above functional modules when performing automatic recognition of continuous frame RGB images. The above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the automatic recognition device for continuous frame RGB images provided in the above embodiment and the automatic recognition method embodiment of continuous frame RGB images belong to the same concept. The specific implementation process is detailed in the automatic recognition method embodiment of continuous frame RGB images, which will not be repeated here.

[0066] The automatic recognition method and device for continuous frame RGB images provided in the embodiment screen out RGB images of unqualified quality as images of manual attention through corner amplitude comparison and similarity comparison. For the RGB images of unqualified quality of manual attention, manual verification, re-rendering of corresponding camera attributes, and re-rendering of corresponding rendering attributes are used to further detect the RGB quality. In this way, it is possible to automatically identify and filter images of unqualified quality, and automatically identify the differences between adjacent frame RGB images. Since the RGB images of manual attention are a small number of images filtered by an algorithm, human resources are greatly saved. The use of machine automatic recognition and detection and manual verification methods greatly improves the test coverage and quality of the delivered RGB images, and also improves the test efficiency.

[0067] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically recognizing continuous frame RGB images, characterized in that: The following steps are involved: Generate a series of continuous frame RGB images according to the set rendering trajectory, and output the camera properties of each frame RGB image; After detecting and removing the black image and the blurred image in the RGB image, the angle amplitude of two adjacent RGB frames calculated according to the camera attributes and the similarity of two adjacent RGB frames calculated according to the pixel values ​​are used to judge whether the quality of the two RGB frames is qualified, including: comparing the angle amplitude of the two adjacent RGB frames with the angle amplitude threshold. When the angle amplitude is greater than the angle amplitude threshold, it is preliminarily judged that the quality of the two RGB frames is unqualified and the camera ID of the two RGB frames is recorded; when the angle amplitude is less than or equal to the angle amplitude threshold, the similarity of the two adjacent RGB frames is compared with the similarity threshold. When the similarity is greater than the similarity threshold, it is considered that the quality of the two RGB frames is qualified; when the similarity is less than or equal to the similarity threshold, it is preliminarily judged that the quality of the two RGB frames is unqualified and the camera ID of the two RGB frames is recorded; Post-processing is performed on RGB images with unqualified quality, including manual verification, re-rendering with corresponding camera attributes, and re-rendering with corresponding rendering attributes to further detect the quality of RGB images.

2. The method for automatically identifying continuous frame RGB images according to claim 1, characterized in that: The camera attributes include the camera position and the camera viewing angle.

3. The automatic recognition method of continuous frame RGB images according to claim 2 is characterized in that: The rotation angle between two adjacent RGB images is calculated based on the camera properties, including: The camera position of each frame image and the camera viewing angle form a vector, and the angle between the vectors of the two frames of image is calculated as the angle amplitude.

4. The method for automatically identifying continuous frame RGB images according to claim 1, characterized in that: The similarity between two adjacent RGB images is calculated based on pixel values, including: The two adjacent RGB frames are converted into normalized histogram data, and the similarity between the two adjacent RGB frames is determined by performing a similarity comparison on the histogram data.

5. The method for automatically identifying continuous frame RGB images according to claim 1, characterized in that: The similarity between two adjacent RGB images is calculated based on pixel values, including: The current frame RGB image and the two RGB images before and after it are regarded as a unit, and the three RGB images in the unit are converted into normalized histogram data. The similarity comparison is performed on the histogram data of two pairs of adjacent frames to determine two similarities, and then the two similarities are averaged as the similarity of each pair of adjacent RGB frames.

6. The method for automatically identifying continuous frame RGB images according to claim 1, characterized in that: The post-processing of the unqualified RGB image includes: First, we manually check the RGB images that do not meet the quality standards, and re-render the RGB images that are confirmed to have quality problems with the corresponding camera attributes. Then, the re-rendered RGB images are manually checked for the second time. For the RGB images confirmed to have quality problems, the corresponding rendering attributes are re-rendered. If the quality of the re-rendered RGB images is qualified, the rendering engine is considered to be unstable. The rendering engine is re-rendered using the corresponding camera attributes and the corresponding rendering attributes to obtain a qualified RGB image. If the quality of the RGB image re-rendered with the corresponding rendering attributes is unqualified, the cause of the unqualified quality is manually checked and corrected.

7. An automatic recognition device for continuous frame RGB images, characterized in that: include: The rendering generation module is used to generate a series of continuous frame RGB images according to the set rendering trajectory, and output the camera properties of each frame of RGB image; The automatic identification module is used to detect and eliminate black images and blurred images in the RGB image, and then determine whether the quality of the two adjacent RGB frames is qualified based on the angle amplitude of the two adjacent RGB frames calculated according to the camera attributes and the similarity of the two adjacent RGB frames calculated according to the pixel value, including: comparing the angle amplitude of the two adjacent RGB frames with the angle amplitude threshold, when the angle amplitude is greater than the angle amplitude threshold, preliminarily determining that the quality of the two RGB frames is unqualified and recording the camera ID of the two RGB frames; when the angle amplitude is less than or equal to the angle amplitude threshold, comparing the similarity of the two adjacent RGB frames with the similarity threshold, when the similarity is greater than the similarity threshold, it is considered that the quality of the two RGB frames is qualified; when the similarity is less than or equal to the similarity threshold, preliminarily determining that the quality of the two RGB frames is unqualified and recording the camera ID of the two RGB frames; The post-processing module is used to post-process the RGB images with unqualified quality, including manual verification, re-rendering according to the corresponding camera attributes, and re-rendering according to the corresponding rendering attributes, so as to further detect the quality of the RGB images.

Images