Method for correcting perspective distortion of fuzzy edge video based on single-frame reference

The target contour information is extracted through a single frame reference method and the perspective transformation matrix is ​​calculated. The problem of blurred contour of the target object caused by perspective distortion and high dynamic range in laser perforation experiments is solved, and efficient and accurate video correction is achieved, which is suitable for different experimental scenarios. , improving the quality of experimental data.

CN118967529AInactive Publication Date: 2024-11-15XIDIAN UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410985390.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In laser perforation experiment, the video has blurred outlines due to perspective distortion and high dynamic range, and the existing technology is difficult to completely solve, especially for non-vertical mounted cameras and high brightness environments. The existing methods have limitations and cannot be met. Experimental requirements.

Method used

The fuzzy edge video perspective distortion correction method based on single frame reference is adopted. By obtaining pictures with clear target contours, extracting contour information, calculating the perspective transformation matrix and applying the inverse matrix for correction, the perspective distortion correction of the entire video is achieved.

Benefits of technology

It improves the accuracy and efficiency of video processing, is suitable for different types of experimental scenarios, and can accurately correct perspective distortion under high dynamic range and low grayscale conditions, improving the quality and reliability of experimental data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967529A_ABST
    Figure CN118967529A_ABST
Patent Text Reader

Abstract

A fuzzy edge video perspective distortion correction method based on single-frame reference comprises the following steps: step 1, acquiring pictures converted from a video frame by frame, and selecting a picture with a clear target contour from the pictures; step 2, extracting contour information of the picture with the clear target contour obtained in the step 1; 3, acquiring a perspective transformation matrix by using the contour information extracted in the step 2; 4, inversing the perspective transformation matrix obtained in the step 3 to obtain an inverse matrix of the perspective transformation matrix, applying the inverse matrix of the perspective transformation matrix to the picture obtained by converting the video frame by frame in the step 1 for perspective correction to obtain an image after perspective distortion correction, extracting an image only containing a target area, and generating a picture; step 5, serializing all the pictures after perspective distortion correction generated in the step 4, and outputting a video; according to the invention, perspective distortion correction is carried out by using the target contour clear picture at the initial stage of shooting, and perspective distortion correction of the contour of the target object in the whole video can be efficiently and accurately realized in combination with a correction algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of laser processing and experimental video processing, and particularly relates to a method for correcting perspective distortion of fuzzy edge video based on single frame reference. Background Art

[0002] In the field of laser processing and experiments, especially in the process of laser perforation, shooting experimental videos to record and analyze the change process of materials is a crucial task. However, due to the limitations of experimental conditions, such as camera installation angle, light source intensity and other factors, the captured videos often have some problems, such as perspective distortion, blurred outline of the target object, etc. These problems not only affect the quality of the video, but also limit the accuracy and reliability of subsequent experimental data. Perspective distortion is the phenomenon of image deformation caused by factors such as shooting angle and focal length during the camera shooting process. In laser perforation experiments, the perspective distortion problem is particularly prominent because the camera cannot be installed completely perpendicular to the object being measured. This distortion not only changes the shape and size of the target object, but also may cause important experimental details to be ignored or misjudged. In addition, laser perforation experimental videos usually have the characteristics of high dynamic range, that is, the brightness of the laser light point is extremely high, while the brightness of the surrounding environment is relatively low. This high dynamic range causes the outline of the target object to gradually become blurred in the video, or even unrecognizable. This is a huge challenge for experiments that require accurate analysis of the change process of the target object. In order to solve these problems, researchers and engineers have been exploring various image processing techniques and algorithms. However, most of the existing solutions have certain limitations and cannot fully meet the needs of laser perforation experiment video processing.

[0003] The most recent implementations focus on two aspects: using camera calibration to correct perspective distortion and using checkerboard corner calibration to correct perspective distortion.

[0004] A common method for perspective distortion correction is to use geometric correction technology. This method estimates and eliminates perspective distortion by measuring the internal and external parameters of the camera, as well as the position and shape of the target object in the image. However, this method requires precise camera calibration and complex calculation process, and the correction effect may not be ideal for non-vertically mounted cameras. And it cannot be completed for experimental videos that have already been shot.

[0005] Another approach is to use machine learning or deep learning algorithms to predict and correct perspective distortion. This approach trains models to identify perspective distortion patterns in images and automatically applies corresponding correction strategies. However, this approach requires a large amount of training data and computing resources, and may require retraining the model for different types of experimental scenarios.

[0006] In terms of contour recognition, one approach is to use local contrast enhancement technology. This method analyzes the local contrast of each pixel in the image and applies a corresponding enhancement strategy to improve the clarity of the image. However, this method may not completely solve the problem of blurred contours of the target object, especially when the brightness of the laser light point is extremely high.

[0007] In summary, although the existing implementation schemes can solve some problems in laser perforation experiment video processing to a certain extent, they all have certain limitations and cannot meet all experimental requirements.

[0008] The patent application document with publication number CN104867113B discloses a method and system for correcting image perspective distortion. The method mainly uses two cameras to obtain two images of the object at one time, generates three-dimensional vector information of the two images respectively, analyzes the overlapping area, and performs a large number of calculations. Then, the perspective distortion correction algorithm is called to correct the perspective distortion of the three-dimensional image. However, when performing perspective distortion correction, the invention needs to obtain two images of the object at one time through two cameras, analyze the overlapping area and perform a large number of calculations.

[0009] The patent application document with publication number CN111145111A discloses a correction method based on infrared video. The invention mainly uses a prepared checkerboard prop, irradiates it with sunlight for a period of time, and then takes 15-20 infrared photos of it, sharpens it, grayscales it, performs corner point detection on the grayscale image to extract the corner points of the checkerboard, and obtains the correction parameters of the infrared camera to achieve the purpose of calibrating the camera, thereby realizing the correction of the video. However, this method must first make the checkerboard and take infrared photos of the checkerboard in the current scene before correction. For the video that has been shot, it cannot be guaranteed that the purpose of correction can be achieved while restoring its scene. Secondly, the calibration mentioned in this method is mainly aimed at the camera, so as to achieve video correction of the entire scene.

[0010] Existing perspective distortion correction methods usually require precise camera calibration and complex calculation processes, which may be difficult to achieve or time-consuming under experimental conditions. And the calibration process cannot be implemented for videos that have already been shot. Methods that rely on machine learning and deep learning require a large amount of training data and computing resources, and may need to retrain the model for different types of experimental scenarios, which limits their flexibility and applicability. Although local contrast enhancement technology can improve image clarity, it may not be able to completely solve the problem of blurred target object outlines when processing laser perforation experimental videos, especially when the brightness of the laser light output point is extremely high. Summary of the invention

[0011] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a method for correcting the perspective distortion of blurred edge videos based on single-frame reference. By utilizing a picture with a clear target contour obtained at the initial stage of shooting to correct the perspective distortion, combined with a correction algorithm, the perspective distortion correction of the target object contour in the entire video can be achieved efficiently and accurately, and the influence of perspective distortion on the video can be eliminated efficiently and at low cost. The present invention improves the accuracy and reliability of experimental data, and provides strong technical support for research and experiments in related fields.

[0012] In order to achieve the above object, the technical solution adopted by the present invention is:

[0013] A method for correcting perspective distortion of blurred edge video based on single frame reference comprises the following steps:

[0014] Step 1, obtain the images converted frame by frame from the video, and select the images with clear target outlines;

[0015] Step 2, extracting the contour information of the image with clear target contour obtained in step 1;

[0016] Step 3: Obtain the perspective transformation matrix using the contour information extracted in step 2;

[0017] Step 4: Invert the perspective transformation matrix obtained in step 3 to obtain the inverse matrix of the perspective transformation matrix, apply the inverse matrix of the perspective transformation matrix to the picture converted frame by frame from the video obtained in step 1 to perform perspective correction, obtain the image after perspective distortion correction, extract the image containing only the target area, and generate a picture;

[0018] Step 5: Serialize all the pictures after perspective distortion correction generated in step 4 and output them into a video.

[0019] The specific method of step 1 is:

[0020] Before the laser is emitted, use a high-dynamic camera to capture an initial clear outline video of the target object, decompose the video frame by frame into images using software or programming, obtain the images converted frame by frame from the video, and select an image with a clear target outline.

[0021] The specific method of step 2 is:

[0022] Use image processing technology to extract the target object's outline and hierarchical structure information from the image with clear target outline obtained in step 1, use the outline approximation function to obtain the approximate outline, focus on the four key points of the approximate rectangular shape of the approximate outline, and correspond to the four corners of the rectangle respectively to obtain the rectangular outline information.

[0023] The specific method of step 3 is:

[0024] Using the contour information obtained in step 2, four key points are used to define the corresponding four corner points in the image. The perspective transformation matrix is ​​obtained by solving the positions of the corner points in three-dimensional space and their coordinates in the image using the least squares method.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1) Accuracy and efficiency of perspective distortion correction:

[0027] The present invention uses a selected clear target outline image to extract outline information and realizes distortion correction of the target image in the entire video. This method is simpler and more direct than the traditional camera calibration method, and can more accurately reflect the perspective distortion under experimental conditions. It can efficiently correct the perspective distortion of each frame in the video without the need for complex calculations for each frame. This greatly improves the efficiency of video processing and ensures the accuracy and consistency of the correction.

[0028] 2) For low grayscale videos, the present invention transfers the contour information obtained in step 2 to the picture obtained in step 1 of the video frame-by-frame conversion. The picture of the video frame-by-frame conversion includes pictures with blurred or unclear contours and other edge contours that are difficult to detect, making the image contour extraction of low grayscale videos more convenient, accurate and reliable. Combined with step 3 and step 4, the inverse matrix of the perspective transformation matrix of the single image obtained in step 4 is applied to the picture of the video frame-by-frame conversion obtained in step 1 for perspective correction. This algorithm for correcting the perspective distortion of all images in the entire video through a single frame image is not only simple and easy, but also can show good performance under various experimental conditions.

[0029] 3) Flexibility and applicability: The technical solution of the present invention is applicable to different types of laser perforation experimental scenarios, without the need for complex adjustments and optimizations for each scenario. This makes the present invention more flexible and applicable, and can meet the needs of different users. At the same time, the present invention can also be combined with other image processing technologies to form a more complete video processing system, providing users with a more comprehensive and efficient solution.

[0030] 4) Innovation and foresight: The present invention proposes novel solutions in perspective distortion correction and high dynamic range video processing. It realizes correction of the entire video through a single-frame image, solving the problem that the target contour in low-grayscale images is difficult to identify and extract. It has high innovation. This innovation is not only reflected in the design of the algorithm, but also in the optimization of the entire video processing flow. With the continuous development of laser processing and experimental fields, the requirements for video processing technology are getting higher and higher. As a forward-looking technical solution, the present invention can meet the higher demands for video processing technology in future experimental scenarios.

[0031] In summary, the present invention realizes efficient processing of laser perforation experimental videos through an adaptive perspective distortion correction algorithm and low-grayscale video enhancement technology. This technical solution can not only accurately correct the perspective distortion and improve the clarity and contrast of the video, but also effectively correct the perspective distortion of the target object in the low-grayscale video, providing strong technical support for research and experiments in the field of laser processing and experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of the implementation method of the present invention.

[0033] Figure 2 This is an example diagram of an embodiment of the present invention without laser irradiation.

[0034] Figure 3 This is an example diagram of contour detection according to an embodiment of the present invention.

[0035] Figure 4 This is an example of a low grayscale video frame according to an embodiment of the present invention. Figure 1 .

[0036] Figure 5 This is an example of profile information migration in an embodiment of the present invention. Figure 1 .

[0037] Figure 6 This is an example of low grayscale video frame target correction extraction in an embodiment of the present invention. Figure 1 .

[0038] Figure 7 This is an example of a low grayscale video frame according to an embodiment of the present invention. Figure 2 .

[0039] Figure 8 This is an example of profile information migration in an embodiment of the present invention. Figure 2 .

[0040] Fig. 9 This is an example of low grayscale video frame target correction extraction in an embodiment of the present invention. Figure 2 . DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0042] A method for correcting perspective distortion of blurred edge video based on single frame reference comprises the following steps:

[0043] Step 1, obtain the images converted frame by frame from the video, and select the images with clear target outlines;

[0044] The specific method of step 1 is:

[0045] Since the laser has extremely high brightness, strong emission capability and highly concentrated energy, this high brightness characteristic makes it difficult for ordinary cameras to capture all the details of the laser, and it is easy to have overexposure or underexposure problems. High dynamic cameras can better handle high-brightness scenes and capture the complete details of the laser. Therefore, before the laser emits light, use a high dynamic camera to shoot an initial clear outline video of the target object, or during the laser irradiation process, for the recorded laser perforation process video, when there is a clear target outline picture in the entire video, it can also be corrected according to the following steps. Use software or programming to disassemble the video into pictures frame by frame, obtain the pictures converted frame by frame of the video, and select a picture with a clear target outline. Use it as the input picture for step 2 to extract the target outline information.

[0046] Step 2, extracting the contour information of the image with clear target contour obtained in step 1;

[0047] The specific method of step 2 is:

[0048] The image processing technology is used to extract the contour and hierarchical structure information of the target object from the image with clear target contour obtained in step 1, and the approximated contour is obtained using the contour approximation function. The present invention only performs perspective correction on rectangular experimental materials, so the obtained contour object presents an approximate rectangular shape. The approximate contour focuses on four key points of the approximate rectangular shape, which correspond to the four corners of the rectangle respectively, to obtain the rectangular contour information.

[0049] The image processing technology includes edge detection, threshold segmentation, morphological segmentation and other methods.

[0050] Step 3: Obtain the perspective transformation matrix using the contour information extracted in step 2;

[0051] The specific method of step 3 is:

[0052] Using the contour information obtained in step 2, four key points are used to define the corresponding four corner points in the image. The camera imaging matrix is ​​obtained by using the least squares method through the positions of the corner points in three-dimensional space and their coordinates in the image. This matrix describes how the camera maps points in three-dimensional space to two-dimensional images.

[0053] Step 4: Invert the perspective transformation matrix obtained in step 3 to obtain the inverse matrix of the perspective transformation matrix, apply the inverse matrix of the perspective transformation matrix to the picture converted frame by frame from the video obtained in step 1 to perform perspective correction, obtain the image after perspective distortion correction, extract the image containing only the target area, and generate a picture;

[0054] This step restores the image from a two-dimensional plane to its original appearance in three-dimensional space, thus eliminating perspective distortion.

[0055] Step 5: All the pictures after perspective distortion correction generated in step 4 are serialized and output as a video according to a certain encoding format and frame rate.

[0056] The video obtained at this time accurately retains the main features of the target image, effectively eliminates unnecessary background and miscellaneous scenes, and greatly improves the image quality and analysis efficiency.

[0057] Example

[0058] like Figure 1 As shown, a method for correcting perspective distortion of blurred edge video based on single frame reference comprises the following steps:

[0059] Step 1: Obtain the images converted frame by frame from the video and select the images with clear target outlines;

[0060] like Figure 2 As shown, before the laser emits light, a high-dynamic camera is used to capture an initial clear outline video of the target object, and the video is decomposed into pictures frame by frame using software or programming to obtain pictures converted frame by frame from the video, and a picture with a clear target outline is selected.

[0061] Step 2, extracting the contour information of the image with clear target contour obtained in step 1;

[0062] like Figure 3 As shown, the image processing technology is used to extract the contour and hierarchical structure information of the target object from the image with clear target contour obtained in step 1, and the approximated contour is obtained using the contour approximation function. The approximated contour is focused on four key points of the approximate rectangular shape, which correspond to the four corners of the rectangle respectively, to obtain the rectangular contour information.

[0063] Step 3: Migration and application of perspective distortion correction information;

[0064] The contour information extracted in step 2 is transferred to the picture converted frame by frame from the video obtained in step 1. Through the correction algorithm, each frame in the video is corrected for perspective distortion, so as to accurately correct the perspective distortion of the target object contour in the entire video, and extract the target according to the contour information, so as to obtain all pictures containing only the target area and completing the perspective distortion correction. Figure 4-Figure 9 As shown in FIG. 1 , they are schematic diagrams showing the extraction effects of target areas with low grayscale in the video.

[0065] Step 4: Serialize all the perspective distortion corrected images obtained in step 3 and output them as a video.

[0066] All the perspective distortion corrected images obtained in step 3 are re-synthesized into a video in the order of the original video sequence to obtain an image video containing only the target area.

[0067] The technical solution advantages of the present invention are:

[0068] 1. Efficient and accurate perspective distortion correction: By using the initial contour photo to extract the contour information of the target image, and combining it with the correction algorithms of steps 3 and 4, the contour of the target object in the entire video can be accurately corrected.

[0069] 2. Efficient processing of high dynamic range and low grayscale video: Through a single frame image, the perspective distortion of all images in the entire video is corrected, and based on the contour information, an image video containing only the target area is obtained. This fine video image can provide strong support for in-depth research on the specific impact and changes of the laser on the object. It not only provides key data references for laser ignition experiments, but also provides technical solutions and decision-making basis for video research after laser emission.

[0070] 3. Flexibility and applicability: This technical solution is suitable for different types of laser perforation experimental scenarios and has strong flexibility and applicability.

[0071] The key points and protection points of the present invention mainly include the following aspects:

[0072] The key points of the present invention are:

[0073] 1. Utilization of initial target outline clear pictures: The key of the present invention is to utilize the target outline clear pictures obtained in the initial shooting stage to perform perspective distortion correction. This step is the basis for subsequent video correction and ensures the accuracy and reliability of the correction information.

[0074] 2. Migration of perspective distortion correction information: Migrate the perspective distortion correction information obtained in the early stage to the entire video sequence to achieve correction of the contour of the target object in the entire video. This step is one of the key technologies for realizing video correction in the present invention.

[0075] 3. Adaptive correction system: The entire system can adaptively deal with problems such as perspective distortion and blurred target object outlines produced under different experimental conditions, ensuring the accuracy and efficiency of video correction.

[0076] The protection point of the present invention is:

[0077] 1. Perspective distortion correction method and steps: The perspective distortion correction method and steps protected by the present invention, in particular, the technology of using an initial picture with a clear target outline for correction, and migrating the correction information to the entire video sequence.

[0078] 2. Design and implementation of the adaptive correction system: The design and implementation of the adaptive correction system protected by the present invention includes the system architecture, algorithm flow and related software configuration.

[0079] 3. System application scope and effect: The application scope and application effect of the system protected by the present invention in the field of laser perforation experiment video processing, especially its contribution to improving video research and accuracy.

[0080] By clarifying these key points and protection points, the present invention aims to protect its unique technological innovation and application value, and provide effective technical support and solutions for the field of laser perforation experiment video processing.

[0081] The camera of the present invention should be in a fixed position during the shooting process. Through frame extraction, a picture with a clear target outline is obtained at the beginning of video shooting, i.e. when the laser is not emitted. Image operations such as grayscale, edge detection, and contour detection are performed to obtain vertex information of the target object outline (mainly rectangular outline in this experiment). It is transferred to other sequence pictures, and its perspective distortion is corrected using the contour information. The target area is extracted, and finally the corrected video is serialized and output.

[0082] For shooting laser irradiation of metal objects, especially in the experimental video of laser perforation, due to the limitation of experimental conditions, it is impossible to ensure that the camera is installed perpendicular to the object being measured, which leads to perspective distortion in the system. In the high dynamic range visible light video of laser perforation, the outline of the perforated object (mainly rectangular metal materials) is clear at the beginning of shooting, which is easy to identify and correct for perspective distortion. However, with the emission of high-intensity laser, the brightness of the laser light point is significantly increased, while the brightness of the surrounding environment is relatively reduced, which causes the outline of the target object to gradually become blurred or even unrecognizable. For low gray value videos where the target object is difficult to distinguish from the surrounding complex environment, when perspective distortion occurs, the identification of the target outline becomes particularly difficult, which further increases the difficulty of correction. Perspective distortion is mainly caused by focal length and shooting distance, but when the camera remains stationary, the perspective distortion it brings is constant.

[0083] In order to solve this problem, the present invention proposes an innovative method. The method first uses a picture with a clear target outline obtained at the initial stage of shooting to correct the perspective distortion. By analyzing the perspective distortion in the photo, accurate correction information can be obtained. Then, these correction information are migrated to the entire video sequence to achieve the correction of the target object outline in the entire video. This method can not only effectively deal with the problem of blurred target object outline caused by changes in laser brightness, but also solve challenges such as low grayscale value and severe perspective distortion, thereby improving the quality and accuracy of the video. By using clear outline photos at the initial stage of shooting, the present invention provides an effective and practical solution for perspective distortion correction of laser perforation experimental videos.

[0084] The technical problems to be solved by the present invention mainly include the following aspects:

[0085] 1) Perspective distortion correction: Due to experimental conditions, the camera cannot be installed perpendicular to the metal object being measured, resulting in perspective distortion when shooting the laser perforation process. This distortion will affect the shape and size of the target object in the video, and further affect the subsequent experimental data analysis and processing. The present invention solves the problem of how to accurately correct perspective distortion under non-vertical installation conditions.

[0086] 2) Correction of perspective distortion of target objects under high dynamic range: With the emission of high-intensity laser, the brightness of the laser light point increases significantly, while the brightness of the surrounding environment decreases relatively, causing the outline of the target object to gradually become blurred or even unrecognizable. This poses a challenge to the accuracy and reliability of experimental data. The present invention solves the problem of how to obtain outline information through the front image under such a high dynamic range and successfully correct the target image.

[0087] 3) Migration and application of correction information: After obtaining accurate perspective distortion correction information, the present invention solves how to effectively migrate this information to the entire video sequence and correct the contour of the target object in the entire video.

Claims

1. A method for correcting perspective distortion of blurred edge video based on single frame reference, characterized in that: The following steps are involved: Step 1, obtain the images converted frame by frame from the video, and select the images with clear target outlines; Step 2, extracting the contour information of the image with clear target contour obtained in step 1; Step 3: Obtain the perspective transformation matrix using the contour information extracted in step 2; Step 4: Invert the perspective transformation matrix obtained in step 3 to obtain the inverse matrix of the perspective transformation matrix, apply the inverse matrix of the perspective transformation matrix to the picture converted frame by frame from the video obtained in step 1 to perform perspective correction, obtain the image after perspective distortion correction, extract the image containing only the target area, and generate a picture; Step 5: Serialize all the pictures after perspective distortion correction generated in step 4 and output them into a video.

2. The method for correcting perspective distortion of blurred edge video based on single frame reference according to claim 1, characterized in that: The specific method of step 1 is: Before the laser is emitted, use a high-dynamic camera to capture an initial clear outline video of the target object, decompose the video frame by frame into images using software or programming, obtain the images converted frame by frame from the video, and select an image with a clear target outline.

3. The method for correcting perspective distortion of blurred edge video based on single frame reference according to claim 1 or 2, characterized in that: The specific method of step 2 is: Use image processing technology to extract the target object's outline and hierarchical structure information from the image with clear target outline obtained in step 1, use the outline approximation function to obtain the approximate outline, focus on the four key points of the approximate rectangular shape of the approximate outline, and correspond to the four corners of the rectangle respectively to obtain the rectangular outline information.

4. The method for correcting perspective distortion of blurred edge video based on single frame reference according to claim 1, characterized in that: The specific method of step 3 is: Using the contour information obtained in step 2, four key points are used to define the corresponding four corner points in the image. The perspective transformation matrix is ​​obtained by solving the positions of the corner points in three-dimensional space and their coordinates in the image using the least squares method.

Citation Information

Patent Citations

  • Tool setting method of mechanical arm feeding type laser etching system

    CN111604598A

  • Method and device for correcting video content deformation caused by shooting angle

    CN116260918A

  • Apparatus and method for image processing

    JP2003288588A