Video processing method and device, electronic equipment and computer readable storage medium

By obtaining and analyzing video equipment parameters, pixel point correction is performed on the video, solving the problem of video distortion shooting in mid- and low-end devices, realizing video distortion correction on the playback end, improving the video viewing experience and equipment production efficiency.

CN119067896APending Publication Date: 2024-12-03BEIJING QIHOOD TECHNOLOGY CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202310601757.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-25
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In order to compress manufacturing costs, mid- and low-end video shooting equipment uses lenses of average quality, resulting in a certain degree of distortion in the captured videos, affecting the viewing experience.

Method used

By obtaining the device parameters of the target video and the shooting device, determining the correction parameters, correcting the pixel points of each frame of the video, using Taylor expansion to establish the mapping relationship of pixel points, correcting the pixel values, and obtaining the corrected video .

Benefits of technology

Without increasing the cost of shooting equipment, correct video distortion, improve the video display effect, and save production costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067896A_ABST
    Figure CN119067896A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a video processing method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that a target video and equipment parameters of target shooting equipment corresponding to the target video are acquired; and determining correction parameters of the target video based on the equipment parameters, and performing correction processing on pixel points corresponding to each frame of image of the target video based on the correction parameters to obtain a target corrected video. The correction parameters corresponding to the equipment parameters are used for correcting the pixel points corresponding to the frames of images of the target video shot by the target shooting equipment, so that the lens cost of the target shooting equipment can be saved, and the video subjected to distortion correction can be obtained at the same time. Moreover, the distortion correction processing is not carried out on one side of the target shooting equipment, but is carried out in the electronic equipment playing the target video, and the configuration of the target shooting equipment is not required, so that the production cost of the target shooting equipment can be further saved, and a better video display effect can be obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image technology, and in particular to a video processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] In recent years, with the rapid popularization of filming equipment, a large number of users use filming equipment to shoot videos and share them. In the process of video shooting, the lens is often the core component of imaging. Generally, in order to reduce manufacturing costs, low-end and mid-range video shooting equipment will use lenses of average quality. The videos shot by such lenses generally have a certain degree of distortion, which affects the viewing experience of subsequent videos. Summary of the invention

[0003] The embodiments of the present application provide a video processing method, device, electronic device and computer-readable storage medium, which can solve the technical problem that in order to reduce manufacturing costs, mid- and low-end shooting equipment will use lenses of average quality. The videos shot by such lenses generally have a certain degree of distortion, which affects the viewing experience of subsequent videos.

[0004] In a first aspect, an embodiment of the present application provides a video processing method, the method comprising:

[0005] Acquire a target video and device parameters of a target shooting device corresponding to the target video;

[0006] The correction parameters of the target video are determined based on the device parameters, and the pixel points corresponding to each frame image of the target video are corrected based on the correction parameters to obtain a target corrected video.

[0007] Optionally, determining the correction parameter of the target video based on the device parameter includes:

[0008] Based on a pre-established correction mapping table of device parameters and correction parameters, the correction parameters corresponding to the device parameters are queried, the correction parameters at least including a first direction scaling coefficient and a first direction strength coefficient corresponding to the first direction of each frame image in the target video, and a second direction scaling coefficient and a second direction strength coefficient corresponding to the second direction of each frame image in the target video, the first direction and the second direction being set at an angle on each frame image.

[0009] Optionally, the first direction scaling factor and the second direction scaling factor are determined based on at least a lens parameter in the device parameters, the first direction scaling factor is used to correct an overall scaling degree of each frame image in the first direction, and the second direction scaling factor is used to correct an overall scaling degree of each frame image in the second direction;

[0010] The first directional intensity coefficient and the second directional intensity coefficient are determined based on a sample test image captured by the target shooting device and a standard test image corresponding to the sample test image. The first directional intensity coefficient is used to correct the radial distortion degree of each frame image in the first direction of its corresponding distortion center, and the second directional intensity coefficient is used to correct the radial distortion degree of each frame image in the second direction of its corresponding distortion center.

[0011] Optionally, the performing correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameter includes:

[0012] Determine the pixel value and first position information of each pixel point corresponding to each frame image of the target video;

[0013] Based on the second position information of each blank pixel point of the quasi-corrected image, the coordinate origin information corresponding to the second position information and the Taylor expansion determined by the correction parameter, a mapping relationship between the second position information of each blank pixel point of the quasi-corrected image and the first position information of each pixel point is established;

[0014] Based on the mapping relationship, the pixel values ​​of each pixel point corresponding to each frame image are assigned to the blank pixel points of the corresponding quasi-corrected image, so as to obtain each corrected image corresponding to each frame image of the target video.

[0015] Optionally, the first position information includes first direction position information corresponding to the first direction, and the second position information includes quasi-first direction position information corresponding to the first direction;

[0016] The Taylor expansion includes a first Taylor expansion, which is used to establish a first sub-mapping relationship between the quasi-first direction position information of each blank pixel point of the quasi-corrected image and the first direction position information of each pixel point; the expansion coefficient of the first Taylor expansion includes the first direction intensity coefficient, and the mapping relationship includes a first sub-mapping relationship.

[0017] Optionally, the first position information further includes second direction position information corresponding to the second direction, and the second position information further includes quasi-second direction position information corresponding to the second direction;

[0018] The Taylor expansion includes a second Taylor expansion, which is used to establish a second sub-mapping relationship between the quasi-second direction position information of each blank pixel point of the quasi-corrected image and the second direction position information of each pixel point; the expansion coefficient of the second Taylor expansion includes the second direction intensity coefficient, and the mapping relationship also includes a second sub-mapping relationship.

[0019] Optionally, the performing correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameter includes:

[0020] The graphics processor of the terminal is controlled to perform correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters.

[0021] In a second aspect, an embodiment of the present application provides a video processing device, the device comprising:

[0022] An acquisition module, adapted to acquire a target video and device parameters of a target shooting device corresponding to the target video;

[0023] The correction module is adapted to determine the correction parameters of the target video based on the device parameters, and to perform correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters to obtain a target corrected video.

[0024] Optionally, the correction module is also suitable for querying the correction parameters corresponding to the device parameters based on a pre-established correction mapping table of device parameters and correction parameters, the correction parameters at least including a first direction scaling coefficient and a first direction strength coefficient corresponding to the first direction of each frame image in the target video, and a second direction scaling coefficient and a second direction strength coefficient corresponding to the second direction of each frame image in the target video, the first direction and the second direction being set at an angle on each frame image.

[0025] Optionally, the first direction scaling factor and the second direction scaling factor are determined based on at least a lens parameter in the device parameters, the first direction scaling factor is used to correct an overall scaling degree of each frame image in the first direction, and the second direction scaling factor is used to correct an overall scaling degree of each frame image in the second direction;

[0026] The first directional intensity coefficient and the second directional intensity coefficient are determined based on a sample test image captured by the target shooting device and a standard test image corresponding to the sample test image. The first directional intensity coefficient is used to correct the radial distortion degree of each frame image in the first direction of its corresponding distortion center, and the second directional intensity coefficient is used to correct the radial distortion degree of each frame image in the second direction of its corresponding distortion center.

[0027] Optionally, the correction module is further adapted to determine the pixel value and first position information of each pixel point corresponding to each frame image of the target video;

[0028] Based on the second position information of each blank pixel point of the quasi-corrected image, the coordinate origin information corresponding to the second position information and the Taylor expansion determined by the correction parameter, a mapping relationship between the second position information of each blank pixel point of the quasi-corrected image and the first position information of each pixel point is established;

[0029] Based on the mapping relationship, the pixel values ​​of each pixel point corresponding to each frame image are assigned to the blank pixel points of the corresponding quasi-corrected image, so as to obtain each corrected image corresponding to each frame image of the target video.

[0030] Optionally, the first position information includes first direction position information corresponding to the first direction, and the second position information includes quasi-first direction position information corresponding to the first direction;

[0031] The Taylor expansion includes a first Taylor expansion, which is used to establish a first sub-mapping relationship between the quasi-first direction position information of each blank pixel point of the quasi-corrected image and the first direction position information of each pixel point; the expansion coefficient of the first Taylor expansion includes the first direction intensity coefficient, and the mapping relationship includes a first sub-mapping relationship.

[0032] Optionally, the first position information further includes second direction position information corresponding to the second direction, and the second position information further includes quasi-second direction position information corresponding to the second direction;

[0033] The Taylor expansion includes a second Taylor expansion, which is used to establish a second sub-mapping relationship between the quasi-second direction position information of each blank pixel point of the quasi-corrected image and the second direction position information of each pixel point; the expansion coefficient of the second Taylor expansion includes the second direction intensity coefficient, and the mapping relationship also includes a second sub-mapping relationship.

[0034] Optionally, the correction module is further adapted to control a graphics processor of the terminal to perform correction processing on pixel points corresponding to each frame image of the target video based on the correction parameters.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0036] Processor; and

[0037] A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the method of any one of the above embodiments.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by a processor, they implement the method described in any one of the above embodiments.

[0039] The beneficial effects brought by the technical solutions provided in some embodiments of the present application include at least: using the correction parameters corresponding to the device parameters to correct the pixel points corresponding to each frame image of the target video captured by the target shooting device, so that the lens cost of the target shooting device can be saved while obtaining a distortion-corrected video. In addition, the distortion correction process is not performed on the side of the target shooting device, but in the electronic device that plays the target video, and there is no requirement for the configuration of the target shooting device, so that the production cost of the target shooting device can be further saved and a better video display effect can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 An exemplary system architecture diagram of a video processing method provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a video processing method provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a process for determining each corrected image corresponding to each frame image of a target video provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of establishing a coordinate system in a frame image of a target video provided by an embodiment of the present application;

[0045] Figure 5 A schematic diagram of establishing a coordinate system in a quasi-rectified image provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of the structure of a video processing device provided in an embodiment of the present application;

[0047] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the embodiments of the present application.

[0049] With the rapid rise of various video sharing platforms, a large number of users shoot videos with camera devices and share them on video sharing platforms. When shooting videos with camera devices, the lens is often the core component of the imaging.

[0050] A lens is an optical lens. The thickness of the center and edge of the lens is different. A lens that is thick in the middle and thin at the edge is called a convex lens, and a lens that is thin in the middle and thick at the edge is called a concave lens. A lens is generally an optical device composed of multiple groups of lenses. Due to the inherent characteristics of the lens, a convex lens will converge light, while a concave lens will diverge light. Therefore, whether it is a convex lens or a concave lens, it will cause a certain degree of image distortion due to the perspective of the optical lens.

[0051] Using high-quality optical glass to manufacture lenses can reduce this perspective distortion to a very low level, but in order to reduce manufacturing costs, mid- and low-end shooting equipment will use cost-effective lenses. The videos shot by such shooting equipment generally have a certain degree of distortion, which will affect the subsequent user viewing experience to a certain extent. Here, distortion refers to the distortion of the object captured by the shooting equipment relative to the object itself, resulting in the deformation of the captured object.

[0052] Therefore, in order to solve the technical problems existing in the prior art, the present application provides a video processing method, which includes: obtaining a target video and device parameters of a target shooting device corresponding to the target video; determining correction parameters of the target video based on the device parameters, and performing correction processing on pixel points corresponding to each frame image of the target video based on the correction parameters to obtain a target corrected video.

[0053] The embodiment of the present application uses the electronic device at the playback end to obtain the device parameters of the target shooting device, and uses the correction parameters corresponding to the device parameters to correct the pixel points corresponding to each frame image of the target video shot by the target shooting device, thereby saving the lens cost of the target shooting device while obtaining a distortion-corrected video. In addition, the distortion correction process is not performed on the side of the target shooting device, but in the electronic device that plays the target video, and there is no requirement for the configuration of the target shooting device, thereby further saving the production cost of the target shooting device and obtaining a better video display effect.

[0054] See also Figure 1 , Figure 1 An exemplary system architecture diagram of a video processing method provided in an embodiment of the present application.

[0055] like Figure 1 As shown, the system architecture may include an electronic device 101, a network 102, and a target shooting device 103. The network 102 is used to provide a medium for a communication link between the electronic device 101 and the target shooting device 103. The network 102 may include various types of wired communication links or wireless communication links, for example: the wired communication link includes an optical fiber, a twisted pair, or a coaxial cable, and the wireless communication link includes a Bluetooth communication link, a Wireless-Fidelity (Wi-Fi) communication link, or a microwave communication link.

[0056] The electronic device 101 can interact with the target shooting device 103 through the network 102 to receive messages from the target shooting device 103 or send messages to the target shooting device 103, or the electronic device 101 can interact with the target shooting device 103 through the network 102. The electronic device 101 can be hardware or software. When the electronic device 101 is hardware, it can be various electronic devices, including but not limited to smart watches, smart phones, tablet computers, laptop portable computers, and desktop computers. When the electronic device 101 is software, it can be installed in the electronic devices listed above, and it can be implemented as multiple software or software modules (for example: used to provide distributed services), or it can be implemented as a single software or software module, which is not specifically limited here.

[0057] In an embodiment of the present application, the electronic device 101 can obtain a target video and device parameters of a target shooting device corresponding to the target video; determine correction parameters of the target video based on the device parameters, and perform correction processing on pixel points corresponding to each frame image of the target video based on the correction parameters to obtain a target corrected video.

[0058] The target shooting device 103 may be a camera, a camcorder, a smart watch with a lens, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.

[0059] Alternatively, the system architecture may also not include the target shooting device 103. In other words, the target shooting device 103 may be an optional device in the embodiments of this specification, that is, the method provided in the embodiments of this specification may be applied to a system structure that only includes the electronic device 101. In this case, the electronic device 101 may obtain the target video and the device parameters of the target shooting device corresponding to the target video through a storage device, the Internet, etc., and the embodiments of this application are not limited to this.

[0060] It should be understood that Figure 1 The number of electronic devices, networks, and target shooting devices in the figure is only illustrative, and any number of electronic devices, networks, and target shooting devices may be used according to implementation requirements.

[0061] See also Figure 2 , Figure 2 A flowchart of a video processing method provided in an embodiment of the present application. The execution subject of the embodiment of the present application can be an electronic device that executes the video processing method, or a processor in an electronic device that executes the video processing method, or a video processing service in an electronic device that executes the video processing method. For the convenience of description, the specific execution process of the device video processing method is introduced below by taking the execution subject as an example of a processor in an electronic device.

[0062] like Figure 2 As shown, the method includes:

[0063] S202: Acquire a target video and device parameters of a target shooting device corresponding to the target video.

[0064] The target video can be obtained from the cloud or from a storage device connected to the electronic device. The target video can be a video actually shot by the target shooting device, rather than a non-real video synthesized by video or graphic processing software, and it is not a video in which the pixel positions in the video actually shot by the target shooting device are offset by video or graphic processing software.

[0065] Generally, the device parameters of the target shooting device corresponding to the target video can be recorded in the basic parameters of the target video. Therefore, after the target video is obtained and processed, the device parameters of the target shooting device corresponding to the target video can be obtained.

[0066] When an electronic device obtains a target video, the target video is generally encapsulated in a certain encapsulation format, such as MP4 (Moving Picture Experts Group 4), flv (streaming media format, Flashvideo), mov (movie format), etc. Therefore, the electronic device can decapsulate the target video to obtain the corresponding video frame. Here, in order to facilitate transmission and storage, the video frame here can be a compressed and encoded video frame. The video frame type is not limited here, and it can be H264, H265 encoding, or VP8 (video compression format or video compression specification), VP9 (video compression format or video compression specification) or AV1 (Open Media Video Alliance, AOMedia Video 1).

[0067] Generally, in order to ensure smooth playback of the target video on the electronic device, several video frames can be decapsulated first to facilitate the decoding of the subsequent video frames, so a buffer is needed to store the video frames. The compressed video frames are converted into computer-renderable video frames through decoding. Generally, after decoding, the video frames are converted into image formats such as YUV. YUV can also be called YCbCr. Y represents the brightness component: if only Y is displayed, the image will look like a black and white photo. U (Cb) represents the chrominance component: the blue part of the image is removed from the brightness component (Y). V (Cr) represents the chrominance component: the red part of the image is removed from the brightness component (Y).

[0068] The device parameters of the target shooting device may include the model, hardware parameters and software parameters of the target device, wherein the hardware parameters may include lens parameters, and the lens parameters include but are not limited to the focal length, f-stop coefficient, aperture and depth of field of the lens.

[0069] S204: Determine correction parameters of the target video based on the device parameters, and perform correction processing on pixel points corresponding to each frame image of the target video based on the correction parameters to obtain a target corrected video.

[0070] The target shooting device can be tested during manufacturing to determine the correction parameters corresponding to different device parameters. After the electronic device obtains the device parameters of the target shooting device corresponding to the target video, it can query the correction parameters of the corresponding device parameters. Then, based on the correction parameters, the position of the pixel points corresponding to each frame image of the target video is corrected, so that the pixel points on the distorted image return from the distorted position to the normal position in the corrected image. Finally, the target corrected video is obtained based on each corrected frame image.

[0071] Optionally, determining the correction parameters of the target video based on the device parameters in step S204 includes:

[0072] The correction parameters corresponding to the device parameters are queried based on a pre-established correction mapping table of device parameters and correction parameters. The correction parameters include at least a first direction scaling coefficient and a first direction strength coefficient corresponding to the first direction of each frame image in the target video, and a second direction scaling coefficient and a second direction strength coefficient corresponding to the second direction of each frame image in the target video. The first direction and the second direction are set at an angle on each frame image.

[0073] The acquired device parameters of the target shooting device corresponding to the target video are used to query, and the corresponding correction parameters are queried in the correction mapping table. The first direction scaling coefficient is used to correct the overall displacement scaling distortion in the first direction of the pixel points corresponding to each frame image in the target video captured by the target shooting device. The second direction scaling coefficient is used to correct the overall displacement scaling distortion in the second direction of the pixel points corresponding to each frame image in the target video captured by the target shooting device.

[0074] The first direction intensity coefficient is used to correct the radial distortion degree of each frame image in the first direction of the corresponding distortion center, and the second direction intensity coefficient is used to correct the radial distortion degree of each frame image in the second direction of the corresponding distortion center. Radial distortion is the change of the vector endpoint along the length direction, that is, the change of the radial vector. Image radial distortion is the position deviation of the image pixel point along the radial direction with the distortion center as the center point, which causes the image formed in the image to deform.

[0075] The first direction and the second direction are set at an angle on each frame image, and the angle between the first direction and the second direction can be greater than or equal to 0° and less than 180°. Generally, in order to facilitate the establishment of the correction mapping table, the angle between the first direction and the second direction can be 90°.

[0076] Optionally, the first direction scaling factor and the second direction scaling factor are determined based at least on the lens parameters in the device parameters, the first direction scaling factor is used to correct the overall scaling degree of each frame image in the first direction, and the second direction scaling factor is used to correct the overall scaling degree of each frame image in the second direction.

[0077] Based on the lens parameters, the overall scaling size of the image in the first direction and the second direction after the lens imaging can be calculated. Generally, the image is scaled as a whole after the lens imaging, so the first direction scaling factor and the second direction scaling factor can be the same. Of course, in the face of some special lenses, that is, when the imaging in the first direction and the second direction are different, the first direction scaling factor and the second direction scaling factor can be different. The first direction scaling factor is used to correct the overall scaling degree of each frame image in the first direction, and actually makes an overall correction adjustment to the position of the pixel points of each frame image in the first direction. The second direction scaling factor is used to correct the overall scaling degree of each frame image in the second direction, and actually makes an overall correction adjustment to the position of the pixel points of each frame image in the second direction.

[0078] Optionally, the first directional intensity coefficient and the second directional intensity coefficient are determined based on a sample test image captured by a target shooting device and a standard test image corresponding to the sample test image. The first directional intensity coefficient is used to correct the degree of radial distortion of each frame image in a first direction of its corresponding distortion center, and the second directional intensity coefficient is used to correct the degree of radial distortion of each frame image in a second direction of its corresponding distortion center.

[0079] In order to determine the first direction strength coefficient and the second direction strength coefficient, a sample test image corresponding to the same physical object and a standard test image corresponding to the sample test image can be used for determination. The sample test image can be an image taken by a target shooting device at a preset position, and the standard test image can be a standard normal image taken by a standard shooting device at a preset position. The first direction strength coefficient and the second direction strength coefficient can be changed based on their respective corresponding preset values. At this time, the sample test image is corrected using the first direction strength coefficient, the second direction strength coefficient, the first direction scaling coefficient, and the second direction scaling coefficient. Different sample test images can be obtained based on different first direction strength coefficients and second direction strength coefficients. The similarity between the sample test image and the standard test image is detected. When the similarity between the sample test image and the standard test image is greater than or equal to the preset similarity, the corresponding first direction strength coefficient and the second direction strength coefficient of the target shooting device are obtained.

[0080] Here, the first direction intensity coefficient is used to correct the radial distortion degree of each frame image in the first direction of the corresponding distortion center. Each frame image corresponds to a distortion center, and the radial distortion degree in the first direction of the distortion center can be understood as the position offset of the pixel points in the first direction corresponding to each angle and radius with the distortion center as the center of the circle. The second direction intensity coefficient is used to correct the radial distortion degree of each frame image in the second direction of the corresponding distortion center. The radial distortion degree in the second direction of the distortion center can be understood as the position offset of the pixel points in the second direction corresponding to each angle and radius with the distortion center as the center of the circle.

[0081] See also Figure 3 , Figure 3 A schematic diagram of a process for determining each corrected image corresponding to each frame image of a target video provided in an embodiment of the present application. Optionally, in step S204, correcting the pixel points corresponding to each frame image of the target video based on the correction parameter includes:

[0082] S302: Determine the pixel value and first position information of each pixel point corresponding to each frame image of the target video.

[0083] The first position information of each pixel point can be determined based on a coordinate system established in each frame image, wherein an X axis can be established in a first direction in each frame image, and a Y axis can be established in a second direction in each frame image to obtain a coordinate system, and a unit of the coordinate system is set based on an actual scene. When each frame image is an RGB image, the pixel value of each pixel point can include a red pixel value, a green pixel value, and a blue pixel value.

[0084] S304: Based on the second position information of each blank pixel point of the quasi-corrected image, the coordinate origin information corresponding to the second position information and the Taylor expansion determined by the correction parameter, a mapping relationship between the second position information of each blank pixel point of the quasi-corrected image and the first position information of each pixel point is established.

[0085] like Figure 4 As shown, Figure 4 A schematic diagram of establishing a coordinate system in a frame image of a target video provided by an embodiment of the present application. Figure 5 As shown, Figure 5 A schematic diagram of establishing a coordinate system in a quasi-rectified image provided in an embodiment of the present application, wherein the direction of the X-axis may be a first direction, the direction of the Y-axis may be a second direction, and the origin may be a distortion center.

[0086] If the resolution of the target video frame image is 640x360, that is, the width and height of the frame image are 640x360, the core principle of image correction is to convert each pixel of 230400 (640x360) pixels into Figure 4 Find the corresponding pixel point in Figure 5 Point P1 should be Figure 4 When looking for a pixel point on the distortion map, it can be point P2(x2, y2).

[0087] Therefore, based on the second position information P1 (x1, y1) of each blank pixel of the quasi-corrected image and the coordinate origin information (0, 0) corresponding to the second position information and the Taylor expansion determined by the correction parameters, a mapping relationship between the second position information of each blank pixel of the quasi-corrected image and the first position information P1 (x2, y2) of each pixel can be established. Here, x1 in the second position information P1 (x1, y1) of each blank pixel of the quasi-corrected image does not include the first direction scaling factor, and y1 does not include the second direction scaling factor. A mapping relationship between P1 (x1, y1) mapped to the coordinates in the frame image of the corresponding target video based on the first direction scaling factor and the second direction scaling factor and P2 (x2, y2) can be established.

[0088] Optionally, the first position information includes first direction position information corresponding to the first direction, and the second position information includes quasi-first direction position information corresponding to the first direction;

[0089] The Taylor expansion includes a first Taylor expansion, which is used to establish a first sub-mapping relationship between the quasi-first direction position information of each blank pixel point of the quasi-corrected image and the first direction position information of each pixel point; the expansion coefficient of the first Taylor expansion includes a first direction intensity coefficient, and the mapping relationship includes a first sub-mapping relationship.

[0090] Specifically, the angle between P1 and the positive direction of the y-axis is θ, the first direction intensity coefficient is INTENSITY_X, and the first direction scaling coefficient is SCALE_X. The second direction intensity coefficient is INTENSITY_Y, and the second direction scaling coefficient is SCALE_Y. The first direction position information corresponding to the first direction can refer to the horizontal coordinate of the first position information, and the quasi-first direction position information corresponding to the first direction can refer to the horizontal coordinate of the second position information.

[0091] The first Taylor expansion can be 1+INTENSITY_X·R2, that is, the first few terms of the Taylor series expansion around the origin, the first sub-mapping relationship x2=R·sinθ / (1+INTENSITY_X·R), where R is the length of the origin distance P1, and the second-order expansion coefficient of the first Taylor expansion includes the first direction intensity coefficient. θ=arctan((x1·SCALE_X) / (y1·SCALE_Y)); R2=(x1·SCALE_X)2+(y1·SCALE_Y)2.

[0092] Optionally, the first position information further includes second direction position information corresponding to the second direction, and the second position information further includes quasi-second direction position information corresponding to the second direction;

[0093] The Taylor expansion includes a second Taylor expansion, which is used to establish a second sub-mapping relationship between the quasi-second direction position information of each blank pixel point of the quasi-corrected image and the second direction position information of each pixel point; the expansion coefficient of the second Taylor expansion includes a second direction intensity coefficient, and the mapping relationship also includes a second sub-mapping relationship.

[0094] The second direction position information corresponding to the second direction may refer to the ordinate of the second position information, and the quasi-second direction position information corresponding to the second direction may refer to the ordinate of the second position information.

[0095] The second Taylor expansion can be 1+INTENSITY_Y·R2, that is, the first few terms of the Taylor series expansion around the origin, the second sub-mapping relationship y2=R·cosθ / (1+INTENSITY_Y·R), and the second-order expansion coefficient of the second Taylor expansion includes the second directional intensity coefficient.

[0096] S306: Assigning pixel values ​​of each pixel point corresponding to each frame image to blank pixel points of the corresponding quasi-corrected image based on the mapping relationship, so as to obtain corrected images corresponding to each frame image of the target video.

[0097] Specifically, the values ​​of x2 and y2 are calculated through the coordinates (x1, y1) of P1 and INTENSITY_X, INTENSITY_Y, SCALE_X and SCALE_Y, and then the pixel value of point P2 is obtained using (x2, y2) as the coordinate point.

[0098] Will Figure 4 The pixel value corresponding to each pixel in the image is assigned to the mapping relationship between the second position information of each blank pixel in the corrected image and the first position information of each pixel. Figure 3 The corresponding pixel point in the image is obtained by analogy, and the pixel values ​​of each pixel point in the entire quasi-corrected image after mapping and assignment can be obtained to obtain the corrected frames. Finally, the target corrected video is obtained through the corrected frames.

[0099] Optionally, performing correction processing on pixel points corresponding to each frame image of the target video based on the correction parameter includes:

[0100] The graphics processor of the control terminal performs correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters.

[0101] The processor will perform correction processing on the pixels corresponding to each frame of the target video and place it in the GPU (graphics processing unit). The distortion correction process is completed in the GPU, which has little impact on the overall image rendering efficiency, almost no impact on the playback performance, and no impact on the operating efficiency of the processor.

[0102] Specifically, when using the GPU for distortion correction, a rendering texture is created based on the processor in advance and an FBO (Frame Buffer Object) is created using the generated texture. During correction, the GPU is used to render the image into the FBO, and the correction is performed using the Shader (a shader programming language) language using the correction method mentioned above. That is, the image rendered in the FBO is the corrected image, and finally the image is rendered to the electronic device for display.

[0103] An embodiment of the present application further provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of any method in the above embodiments.

[0104] Figure 6 FIG. 1 is a schematic diagram of a video processing device provided in an embodiment of the present application. Figure 6 shown.

[0105] The data retrieval device 600 comprises:

[0106] An acquisition module 610 is adapted to acquire a target video and device parameters of a target shooting device corresponding to the target video;

[0107] The correction module 620 is adapted to determine the correction parameters of the target video based on the device parameters, and to perform correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters to obtain the target corrected video.

[0108] Optionally, the correction module 620 is also suitable for querying the correction parameters corresponding to the device parameters based on a pre-established correction mapping table of device parameters and correction parameters, the correction parameters at least including a first direction scaling coefficient and a first direction strength coefficient corresponding to the first direction of each frame image in the target video, and a second direction scaling coefficient and a second direction strength coefficient corresponding to the second direction of each frame image in the target video, the first direction and the second direction being set at an angle on each frame image.

[0109] Optionally, the first direction scaling factor and the second direction scaling factor are determined based on at least the lens parameters in the device parameters, the first direction scaling factor is used to correct the overall scaling degree of each frame image in the first direction, and the second direction scaling factor is used to correct the overall scaling degree of each frame image in the second direction;

[0110] The first directional intensity coefficient and the second directional intensity coefficient are determined based on a sample test image captured by a target shooting device and a standard test image corresponding to the sample test image. The first directional intensity coefficient is used to correct the radial distortion degree of each frame image in a first direction of its corresponding distortion center, and the second directional intensity coefficient is used to correct the radial distortion degree of each frame image in a second direction of its corresponding distortion center.

[0111] Optionally, the correction module 620 is further adapted to determine the pixel value and first position information of each pixel point corresponding to each frame image of the target video;

[0112] Based on the second position information of each blank pixel point of the quasi-corrected image, the coordinate origin information corresponding to the second position information, and the Taylor expansion determined by the correction parameter, a mapping relationship between the second position information of each blank pixel point of the quasi-corrected image and the first position information of each pixel point is established;

[0113] Based on the mapping relationship, the pixel value of each pixel point corresponding to each frame image is assigned to the blank pixel point of the corresponding quasi-corrected image, so as to obtain each corrected image corresponding to each frame image of the target video.

[0114] Optionally, the first position information includes first direction position information corresponding to the first direction, and the second position information includes quasi-first direction position information corresponding to the first direction;

[0115] The Taylor expansion includes a first Taylor expansion, which is used to establish a first sub-mapping relationship between the quasi-first direction position information of each blank pixel point of the quasi-corrected image and the first direction position information of each pixel point; the expansion coefficient of the first Taylor expansion includes a first direction intensity coefficient, and the mapping relationship includes a first sub-mapping relationship.

[0116] Optionally, the first position information further includes second direction position information corresponding to the second direction, and the second position information further includes quasi-second direction position information corresponding to the second direction;

[0117] The Taylor expansion includes a second Taylor expansion, which is used to establish a second sub-mapping relationship between the quasi-second direction position information of each blank pixel point of the quasi-corrected image and the second direction position information of each pixel point; the expansion coefficient of the second Taylor expansion includes a second direction intensity coefficient, and the mapping relationship also includes a second sub-mapping relationship.

[0118] Optionally, the correction module 620 is further adapted to control a graphics processor of the terminal to perform correction processing on pixel points corresponding to each frame image of the target video based on the correction parameters.

[0119] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 may include: at least one processor 701 , at least one network interface 704 , a user interface 703 , a memory 705 , and at least one communication bus 702 .

[0120] The communication bus 702 is used to realize the connection and communication between these components.

[0121] The user interface 703 may include a display screen (Display) and a camera (Camera), and the optional user interface 703 may also include a standard wired interface and a wireless interface.

[0122] The network interface 704 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0123] Among them, the processor 701 may include one or more processing cores. The processor 701 uses various interfaces and lines to connect the various parts of the entire experimental operation electronic device 700, and executes various functions and processes data of the electronic device 700 by running or executing instructions, programs, code sets or instruction sets stored in the memory 705, and calling data stored in the memory 705. Optionally, the processor 701 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 701 can integrate one or more combinations of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 701, and it can be implemented by a single chip.

[0124] Among them, the memory 705 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory, ROM). Optionally, the memory 705 includes a non-transitory computer-readable storage medium. The memory 705 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 705 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 705 may also be optionally at least one storage device located away from the aforementioned processor 701. As Figure 7 As shown, the memory 705 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a video processing program.

[0125] exist Figure 7In the electronic device 700 shown, the user interface 703 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 701 can be used to call the video processing program stored in the memory 705 and specifically perform the following operations:

[0126] Obtain a target video and device parameters of a target shooting device corresponding to the target video;

[0127] The correction parameters of the target video are determined based on the device parameters, and the pixel points corresponding to each frame image of the target video are corrected based on the correction parameters to obtain the target corrected video.

[0128] Optionally, when the processor 701 determines the correction parameters of the target video based on the device parameters, the following steps are specifically performed:

[0129] The correction parameters corresponding to the device parameters are queried based on a pre-established correction mapping table of device parameters and correction parameters. The correction parameters include at least a first direction scaling coefficient and a first direction strength coefficient corresponding to the first direction of each frame image in the target video, and a second direction scaling coefficient and a second direction strength coefficient corresponding to the second direction of each frame image in the target video. The first direction and the second direction are set at an angle on each frame image.

[0130] Optionally, the first direction scaling factor and the second direction scaling factor are determined based on at least the lens parameters in the device parameters, the first direction scaling factor is used to correct the overall scaling degree of each frame image in the first direction, and the second direction scaling factor is used to correct the overall scaling degree of each frame image in the second direction;

[0131] The first directional intensity coefficient and the second directional intensity coefficient are determined based on a sample test image captured by a target shooting device and a standard test image corresponding to the sample test image. The first directional intensity coefficient is used to correct the radial distortion degree of each frame image in a first direction of its corresponding distortion center, and the second directional intensity coefficient is used to correct the radial distortion degree of each frame image in a second direction of its corresponding distortion center.

[0132] Optionally, when the processor 701 performs correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameter, the following steps are specifically performed:

[0133] Determine the pixel value and first position information of each pixel point corresponding to each frame image of the target video;

[0134] Based on the second position information of each blank pixel point of the quasi-corrected image, the coordinate origin information corresponding to the second position information, and the Taylor expansion determined by the correction parameter, a mapping relationship between the second position information of each blank pixel point of the quasi-corrected image and the first position information of each pixel point is established;

[0135] Based on the mapping relationship, the pixel value of each pixel point corresponding to each frame image is assigned to the blank pixel point of the corresponding quasi-corrected image, so as to obtain each corrected image corresponding to each frame image of the target video.

[0136] Optionally, the first position information includes first direction position information corresponding to the first direction, and the second position information includes quasi-first direction position information corresponding to the first direction;

[0137] The Taylor expansion includes a first Taylor expansion, which is used to establish a first sub-mapping relationship between the quasi-first direction position information of each blank pixel point of the quasi-corrected image and the first direction position information of each pixel point; the expansion coefficient of the first Taylor expansion includes a first direction intensity coefficient, and the mapping relationship includes a first sub-mapping relationship.

[0138] Optionally, the first position information further includes second direction position information corresponding to the second direction, and the second position information further includes quasi-second direction position information corresponding to the second direction;

[0139] The Taylor expansion includes a second Taylor expansion, which is used to establish a second sub-mapping relationship between the quasi-second direction position information of each blank pixel point of the quasi-corrected image and the second direction position information of each pixel point; the expansion coefficient of the second Taylor expansion includes a second direction intensity coefficient, and the mapping relationship also includes a second sub-mapping relationship.

[0140] Optionally, when the processor 701 performs correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameter, the following steps are specifically performed:

[0141] The graphics processor of the control terminal performs correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters.

[0142] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0143] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0145] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0146] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present application.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] The above is a description of a video processing method, device, electronic device and computer-readable storage medium provided in an embodiment of the present application. For technicians in this field, according to the ideas of the embodiments of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

Claims

1. A video processing method, in, The method comprises: Acquire a target video and device parameters of a target shooting device corresponding to the target video; The correction parameters of the target video are determined based on the device parameters, and the pixel points corresponding to each frame image of the target video are corrected based on the correction parameters to obtain a target corrected video.

2. The video processing method according to claim 1, in, The determining the correction parameter of the target video based on the device parameter includes: Based on a pre-established correction mapping table of device parameters and correction parameters, the correction parameters corresponding to the device parameters are queried, the correction parameters at least including a first direction scaling coefficient and a first direction strength coefficient corresponding to the first direction of each frame image in the target video, and a second direction scaling coefficient and a second direction strength coefficient corresponding to the second direction of each frame image in the target video, the first direction and the second direction being set at an angle on each frame image.

3. The video processing method according to claim 2, in, The first direction scaling factor and the second direction scaling factor are determined based on at least the lens parameters in the device parameters, the first direction scaling factor is used to correct the overall scaling degree of each frame image in the first direction, and the second direction scaling factor is used to correct the overall scaling degree of each frame image in the second direction; The first directional intensity coefficient and the second directional intensity coefficient are determined based on a sample test image captured by the target shooting device and a standard test image corresponding to the sample test image. The first directional intensity coefficient is used to correct the radial distortion degree of each frame image in the first direction of its corresponding distortion center, and the second directional intensity coefficient is used to correct the radial distortion degree of each frame image in the second direction of its corresponding distortion center.

4. The video processing method according to claim 2, in, The correcting the pixel points corresponding to each frame image of the target video based on the correction parameter includes: Determine the pixel value and first position information of each pixel point corresponding to each frame image of the target video; Based on the second position information of each blank pixel point of the quasi-corrected image, the coordinate origin information corresponding to the second position information and the Taylor expansion determined by the correction parameter, a mapping relationship between the second position information of each blank pixel point of the quasi-corrected image and the first position information of each pixel point is established; Based on the mapping relationship, the pixel values ​​of each pixel point corresponding to each frame image are assigned to the blank pixel points of the corresponding quasi-corrected image, so as to obtain each corrected image corresponding to each frame image of the target video.

5. The video processing method according to claim 4, in, The first position information includes first direction position information corresponding to the first direction, and the second position information includes quasi-first direction position information corresponding to the first direction; The Taylor expansion includes a first Taylor expansion, which is used to establish a first sub-mapping relationship between the quasi-first direction position information of each blank pixel point of the quasi-corrected image and the first direction position information of each pixel point; the expansion coefficient of the first Taylor expansion includes the first direction intensity coefficient, and the mapping relationship includes a first sub-mapping relationship.

6. The video processing method according to claim 5, in, The first position information further includes second direction position information corresponding to the second direction, and the second position information further includes quasi-second direction position information corresponding to the second direction; The Taylor expansion includes a second Taylor expansion, which is used to establish a second sub-mapping relationship between the quasi-second direction position information of each blank pixel point of the quasi-corrected image and the second direction position information of each pixel point; the expansion coefficient of the second Taylor expansion includes the second direction intensity coefficient, and the mapping relationship also includes a second sub-mapping relationship.

7. The video processing method according to claim 1, in, The correcting the pixel points corresponding to each frame image of the target video based on the correction parameter includes: The graphics processor of the terminal is controlled to perform correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters.

8. A video processing device, in, The device comprises: An acquisition module, adapted to acquire a target video and device parameters of a target shooting device corresponding to the target video; The correction module is adapted to determine the correction parameters of the target video based on the device parameters, and to perform correction processing on the pixel points corresponding to each frame image of the target video based on the correction parameters to obtain a target corrected video.

9. An electronic device, in, The electronic device includes: Processor; and A memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method according to any one of claims 1 to 7.

10. A computer-readable storage medium, in, The computer-readable storage medium stores one or more programs, which, when executed by a processor, implement the method of any one of claims 1 to 7.

Citation Information

Patent Citations

  • Image processing method and device

    CN102236790A

  • Information processing method and electronic equipment

    CN104427241A

  • Scene reconstruction method and device, equipment and storage medium

    CN114723887A

  • Method and device for image correction and storage medium

    EP3751508A1

  • Image processing method and apparatus

    US20020196472A1