Video processing method and device, equipment and computer storage medium

Abstract

The invention discloses a video processing method, device and equipment and a computer storage medium. The method comprises the following steps: acquiring a video acquired by a shooting device; dividing the video into a plurality of regions according to information associated with the global motion state between frames of the video, the plurality of regions including regions of interest and regions of non-interest; and performing different image processing on the region of interest and the non-region of interest, so that the definition of the region of interest and the definition of the non-region of interest are different. According to the invention, occupation of transmission resources can be effectively reduced on the basis of ensuring subjective image quality experience of a user.

Description

Video processing method, device, equipment and computer storage medium Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video processing method, apparatus, device and computer storage medium. Background Art

[0002] In image transmission applications, captured images and videos often need to be transmitted in real time or with low latency, which consumes significant transmission resources. Traditionally, all regions within a single image frame are encoded using a unified strategy. Given limited transmission resources, this unified encoding strategy cannot provide users with a clear view of their Region of Interest (ROI). Therefore, how to conserve transmission bandwidth while maintaining the user's subjective image quality experience is a pressing issue.

[0003] Summary of the Invention

[0004] In order to solve the above problems, the present application provides a video processing method, apparatus, device and computer storage medium, which can save transmission bandwidth.

[0005] In a first aspect, an embodiment of the present application provides a video processing method, the method comprising:

[0006] Obtaining a video captured by a camera;

[0007] dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video, wherein the plurality of regions include a region of interest and a region of no interest;

[0008] Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

[0009] In a second aspect, an embodiment of the present application provides a video processing method, the method comprising:

[0010] Obtaining a video captured by a camera;

[0011] determining, based on information associated with a global motion state between frames of the video, an area of ​​a region of interest and / or a region of no interest in the video;

[0012] Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

[0013] In a third aspect, an embodiment of the present application provides a video processing device, the device comprising:

[0014] memory for storing computer programs;

[0015] The processor is configured to call the computer program, and when the computer program is executed by the processor, causes the video processing device to perform the following operations:

[0016] Obtaining a video captured by a camera;

[0017] dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video, wherein the plurality of regions include a region of interest and a region of no interest;

[0018] Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

[0019] In a fourth aspect, an embodiment of the present application provides a video processing device, characterized by comprising:

[0020] memory for storing computer programs;

[0021] The processor is configured to call the computer program, and when the computer program is executed by the processor, causes the video processing device to perform the following operations:

[0022] Obtaining a video captured by a camera;

[0023] determining, based on information associated with a global motion state between frames of the video, an area of ​​a region of interest and / or a region of no interest in the video;

[0024] Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

[0025] In a fifth aspect, an embodiment of the present application provides a device, comprising:

[0026] a photographing device, the photographing device being mounted on the device;

[0027] And the video processing device according to any one of the third aspect or the fourth aspect.

[0028] In a sixth aspect, an embodiment of the present application provides a computer storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, they are used to execute the video processing method as described in any one of the first aspect or the second aspect.

[0029] By implementing the video processing method of the embodiment of the present application, the video is divided into multiple areas including ROI areas and non-ROI areas based on information associated with the global motion state between frames of the video, and different image processing methods are performed on the ROI areas and non-ROI areas in the video to make the clarity of the ROI areas and non-ROI areas different. While improving the user's subjective picture quality experience, it also reduces the overall occupancy of transmission resources and improves the timeliness of transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments.

[0031] FIG1 is a schematic scene diagram of an unmanned aerial vehicle provided in an embodiment of the present application;

[0032] FIG2 is a schematic flow chart of a video processing method provided in an embodiment of the present application.

[0033] FIG3 is a flowchart of dividing multiple areas in a video provided by an embodiment of the present application.

[0034] FIG4 is a schematic diagram of the position change of the ROI region in the video provided by an embodiment of the present application.

[0035] FIG5 is a schematic flowchart of another video processing method provided in an embodiment of the present application.

[0036] FIG6 is a schematic block diagram of a video processing device provided in an embodiment of the present application.

[0037] FIG7 is a schematic block diagram of another video processing device provided in an embodiment of the present application.

[0038] Description of main components of the drawings:

[0039] Unmanned aerial vehicle 10, remote controller 20, smart terminal 30, camera 11, gimbal 12;

[0040] Memory 301, processor 302;

[0041] Memory 401, processor 402. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0043] The purpose of the terms used herein is only to describe specific embodiments and is not intended to limit the present application. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "comprising" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items. "First" and "second" are merely for the needs of expression, and the data relationship between the two is not limited and can be the same or different. When used herein, A is positively correlated with B, that is, as A increases, B also increases, and A is negatively correlated with B, that is, A increases and B decreases.

[0044] The figures are for illustration only and do not necessarily include all contents and operations / steps, nor must they be performed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may change according to actual circumstances.

[0045] The embodiments of the present application are applicable to video transmission / communication scenarios of any device with a video acquisition function, including but not limited to a handheld gimbal, a sports camera, a removable device that directly or indirectly carries a shooting device through a carrier, and electronic devices such as mobile phones, tablet computers, smart wearable devices, and computers with a shooting function.

[0046] A movable device can be a vehicle capable of self-propelled movement. The vehicle can have one or more propulsion units that can allow the vehicle to move within an environment. The movable device can traverse on land or underground, on water or in water, in the air, in space, or any combination thereof. The movable device can be an aircraft (e.g., a rotorcraft, a fixed-wing aircraft), a land-based vehicle, a water-based vehicle, or an air-based vehicle. The movable device can be manned or unmanned.

[0047] The carrier may include one or more devices configured to accommodate the camera and / or allow the camera to be adjusted (e.g., rotated) relative to the movable device. For example, the carrier may be a gimbal. The carrier may be configured to allow the camera to rotate around one or more rotation axes, including a yaw axis, a pitch axis, or a roll axis. In some scenarios, the carrier may be configured to allow rotation of 360° or more around each axis to allow for better control of the viewing angle of the camera.

[0048] Please refer to Figure 1, which takes the unmanned aerial vehicle scene in a mobile device as an example for explanation. The unmanned aerial vehicle can be an aerial camera or a flying drone. In the unmanned aerial vehicle aerial photography or first-person view (FPV) flight experience scenario, the unmanned aerial vehicle 10 is flying in the air, and the shooting device 11 is mounted on the unmanned aerial vehicle 10 through the gimbal 12. The shooting device 11 captures video and then transmits the video to the remote control end 20 through the wireless image transmission system. The remote control end 20 then transmits the image to a smart terminal 30 with a display function for display. In some scenarios, the video captured by the shooting device 11 can also be directly transmitted to the smart terminal 30 for display through the wireless image transmission system. The smart terminal can be a wearable device such as smart glasses, goggles or head-mounted displays, or it can be a user device such as a mobile phone, computer, or tablet. The smart terminal can include any type of wearable computer or device that combines augmented reality (AR) or virtual reality (VR) technology. Users can watch videos captured by UAVs on smart devices, such as by wearing smart glasses and viewing videos transmitted by mobile devices, providing an immersive aerial photography and racing experience. For high-speed drone or FPV flight scenarios, UAVs need to transmit a large amount of video data back to the remote controller or smart device, and the image or video transmission must be real-time or have low latency. Therefore, saving transmission resources is an urgent issue.

[0049] Please refer to FIG2 , which is a flow chart of a video processing method provided in an embodiment of the present application.

[0050] As an example, the method 100 includes steps S110 to S130:

[0051] S110: Obtaining a video captured by a camera;

[0052] S120: Divide the video into a plurality of regions according to information associated with a global motion state between frames of the video, wherein the plurality of regions include a ROI region and a non-ROI region;

[0053] S130: Perform different image processing on the ROI region and the non-ROI region to make the clarity of the ROI region and the non-ROI region different.

[0054] The information associated with the global motion state between frames of a video refers to information that can reflect the global motion state between the current frame and the historical frames of the video, and reflects the global change information between the current frame and the historical frame images, where the historical frame can be any frame before the current frame. If the global motion change represented by the information associated with the global motion state between frames of the video meets the preset change conditions, the video is divided into multiple regions. The greater the global motion change between frames, the more drastic the global change between frames, and the more high-speed motion content in the video that is not sensitive to the human eye, the more likely it is to be divided.

[0055] Optionally, the information associated with the global motion state between frames of the video includes at least one of the following information or a combination thereof: global motion information between frames of the video; information associated with the motion state of the target object, wherein the target object includes at least any one of a shooting device and a device carrying the shooting device.

[0056] It should be noted that, in addition to the above information, the information associated with the global motion state between frames of the video may also include other information that can reflect the global motion state between the current frame and historical frames of the video, and this application does not impose any restrictions on this.

[0057] Optionally, the global motion information between video frames includes: a global motion vector (GMV) between video frames;

[0058] Optionally, the information associated with the motion state of the target object includes: the motion speed of the target object and the relative distance between the target object and the subject; or the motion speed of the target object and the relative height between the target object and the subject.

[0059] In some scenarios, the target object is a camera, and the information associated with the target object's motion state is the information associated with the camera's motion state. In other scenarios, given the ease of access to information associated with the target object's motion state, the information associated with the target object's motion state can be reflected by the motion state associated with the device carrying the camera. For example, in a handheld gimbal, the information associated with the target object's motion state can be represented by the gimbal carrying the camera. For example, in a mobile device, when the camera is directly mounted on the mobile device, the information associated with the target object's motion state can be represented by the motion state of the mobile device. When the camera is mounted on a carrier, the information associated with the target object's motion state can be represented by the motion state of the mobile device or the carrier. When the mobile device is specifically an aircraft, for example, the information associated with the target object's motion state can specifically include the aircraft's flight speed and the relative altitude between the aircraft and the subject. The aircraft's flight speed and altitude information can be easily obtained through the aircraft's own navigation system, mapped using the user's stick input, or measured using motion sensors carried by the carrier, camera, or mobile device itself; this is not a limitation.

[0060] In some embodiments, dividing a video into multiple regions based on information associated with the global motion state between frames of the video includes dividing the video into multiple regions if a global motion change represented by the information associated with the global motion state between frames of the video satisfies a preset change condition. Optionally, if the global motion change represented by the information associated with the global motion state between frames of the video satisfies the preset change condition, at least one of the following situations may occur:

[0061] Case 1: If the absolute value of the GMV between frames of the video is greater than the GMV threshold, the video is divided into multiple regions.

[0062] Scenario 2: When the relative height between the target object and the subject remains constant, if the absolute value of the target object's moving speed is greater than the moving speed threshold, the video is divided into multiple areas.

[0063] Case 3: When the target object's moving speed remains constant, if the relative height between the target object and the subject is less than the height threshold, the video is divided into multiple areas.

[0064] It should be noted that any one of the above-mentioned situations, or the satisfaction of multiple situations, will trigger the execution of the step of dividing the video into multiple regions. In addition to the above-mentioned situations, the judgment of division may also include using other information that can reflect the global motion state correlation between the current frame and historical frames of the video, and this application does not limit this.

[0065] It should be noted that the target's speed and the relative distance between the target and the subject are used as examples here. In other scenarios, such as aircraft scenarios, the target's speed and the relative height between the target and the subject are used as information associated with the target's motion state. This applies to the following text. The relative height between the target and the subject can also be expressed in other forms, such as the height between the target and the ground or the height of the target from the starting point (such as the take-off point).

[0066] In some embodiments, when transmission resources are sufficient, that is, when the transmission conditions of the transmission device corresponding to the shooting device meet the preset transmission conditions, the video may not be divided into ROI areas and non-ROI areas, that is, the image quality of the entire image is high, and there is no need to sacrifice the image quality of the non-ROI area to improve the image quality of the ROI area. In the case of insufficient transmission resources, that is, when the transmission adjustment of the transmission device corresponding to the shooting device does not meet the preset transmission conditions, the video is divided into multiple areas based on information associated with the global motion state between frames of the video. Optionally, the transmission condition can be represented by a transmission bit rate. A transmission bit rate higher than a bit rate threshold can be a specific case where the transmission condition meets the preset transmission condition, and a transmission bit rate lower than a bit rate threshold can be a specific case where the transmission condition does not meet the preset transmission condition. It should also be understood that in addition to the transmission bit rate, other methods of determining using information that can reflect the transmission conditions of the transmission device are also included, and this application does not limit this.

[0067] Optionally, please refer to Figure 3. An embodiment of the present application provides a specific method for determining the division of ROI areas and non-ROI areas. The specific steps are as follows: first determine whether the transmission bit rate of the transmission device corresponding to the shooting device is lower than the bit rate threshold. If not, do not divide it. If so, further determine whether the absolute value of the GMV between frames of the video is greater than the GMV threshold; if so, divide it; if not, further determine the movement speed of the target object and the relative height between the target object and the subject; if the movement speed of the target object is greater than the first movement speed threshold, and the relative height between the target object and the subject is less than the first height threshold; if so, divide it; if not, do not divide it.

[0068] It should be noted that at a constant motion speed, the smaller the relative height, the more significant the global changes between frames; at a constant relative height, the greater the motion speed, the more significant the global changes between frames. In some scenarios, the motion speed threshold and the relative height threshold can be set to several levels. At the same motion speed, but due to meeting different height threshold conditions, different classification judgment results may correspond to different results. At the same relative height, but due to meeting different motion speed threshold conditions, different classification judgment results may correspond to different results. The relationship between the target object's motion speed and the relative distance between the objects is similar and will not be elaborated on.

[0069] It should be noted that in the above embodiment, there is no restriction on the order in which the preset transmission conditions and the information associated with the global motion state between video frames are determined. When multiple pieces of information associated with the global motion state between video frames are combined, there is no restriction on the order in which the determinations are made between the pieces of information. In addition to the specific method for determining the division of ROI and non-ROI regions described above, other optional division and determination methods can be selected based on actual scenarios and are not further described here.

[0070] In some embodiments, after step S110 and before step S130, the method further includes determining the area of ​​the ROI region and / or the area of ​​the non-ROI region based on information associated with the global motion state between video frames. While ensuring user subjective image quality, the method adaptively adjusts the reasonable allocation of transmission resources between the ROI region and the non-ROI region based on information associated with the global motion state between video frames.

[0071] Optionally, the area of ​​the ROI region is negatively correlated with the global motion change represented by the information associated with the global motion state between frames of the video; and / or, the area of ​​the non-ROI region is positively correlated with the global motion change represented by the information associated with the global motion state between frames of the video. Among them, the greater the global motion change, the more drastic the global change between frames, the more high-speed motion content in the image that the human eye is not sensitive to, the smaller the area of ​​the ROI region that the human eye pays attention to can be set, and accordingly, the larger the area of ​​the non-ROI region can be set, and more transmission resources can be saved overall. Optionally, the area adjustment method of the ROI region and / or the non-ROI region includes at least one of the following situations:

[0072] In case 1, the area of ​​the ROI region is negatively correlated with the absolute value of the GMV between frames of the video (for example, the larger the absolute value, the smaller the area of ​​the ROI region), and / or the area of ​​the non-ROI region is positively correlated with the absolute value of the GMV between frames of the video (for example, the larger the absolute value, the larger the area of ​​the non-ROI region).

[0073] In the second scenario, when the relative height between the target object and the subject remains constant, the area of ​​the ROI region is negatively correlated with the absolute value of the target object's movement speed, and / or the area of ​​the non-ROI region is positively correlated with the absolute value of the target object's movement speed.

[0074] Case three: when the target object's moving speed remains constant, the area of ​​the ROI region is positively correlated with the relative height between the target object and the subject, and / or the area of ​​the non-ROI region is negatively correlated with the relative height between the target object and the subject.

[0075] It should be noted that if any one of the above-mentioned situations is met, or if multiple situations are met, the dynamic adjustment of the area of ​​the ROI region and / or non-ROI region in the video will be triggered. In addition to the above-mentioned situations, other situations in which the area of ​​the ROI region and / or non-ROI region is adjusted by using other information that can reflect the global motion state association between the current frame and historical frames of the video may also be included, and this application does not limit this.

[0076] It should be noted that, when the information associated with the global motion state between video frames remains constant, the size of the entire image is represented by the field of view (FOV) size of the shooting device, and the area of ​​the ROI region and / or non-ROI region is positively correlated with the FOV size.

[0077] It should be noted that in the above embodiment, when transmission resources are sufficient, that is, when the transmission conditions of the transmission device corresponding to the shooting device meet the preset transmission conditions, dynamic adjustment of the area of ​​the ROI region and the non-ROI region of the video may not be performed. When transmission resources are insufficient, that is, when the transmission adjustment of the transmission device corresponding to the shooting device does not meet the preset transmission conditions, dynamic adjustment of the area of ​​the ROI region and the non-ROI region is performed based on information associated with the global motion state between frames of the video. Optionally, the transmission condition can be represented by a transmission bit rate. A transmission bit rate higher than a bit rate threshold can be a specific case in which the transmission condition meets the preset transmission condition, and a transmission bit rate lower than a bit rate threshold can be a specific case in which the transmission condition does not meet the preset transmission condition. It should also be understood that in addition to the transmission bit rate, other methods of determining using information that can reflect the transmission conditions of the transmission device are also included, and this application does not limit this.

[0078] The present application provides a specific method for dynamically adjusting the area of ​​the ROI region, based on the different transmission conditions of the transmission device. The transmission bit rate is used as an example for illustration. It should also be understood that in addition to using the bit rate for determination, other information that can reflect the transmission conditions of the transmission device can also be used for adjustment, and this application does not limit this. It includes at least one of the following situations:

[0079] Case 1: The transmission rate is lower than the first rate threshold

[0080] 1) If the absolute value of the GMV between frames is greater than the first GMV threshold, or the target's movement speed is greater than the first movement speed threshold and the relative height is less than the first height threshold, then the area of ​​the ROI region is determined to be 1 / 4 FOV;

[0081] 2) If the absolute value of the GMV between frames is greater than the second GMV threshold and less than the first GMV threshold, or the target's movement speed is less than the first movement speed threshold and greater than the second movement speed threshold, and the relative height is less than the second height threshold, then the area of ​​the ROI region is determined to be 1 / 3 FOV;

[0082] 3) Otherwise, the area of ​​the ROI region is determined to be 1 / 2 FOV.

[0083] Case 2: The transmission rate is higher than the first rate threshold and lower than the second rate threshold

[0084] 1) If the absolute value of the GMV between frames is greater than the first GMV threshold, or the target's movement speed is greater than the first movement speed threshold and the relative height is less than the first height threshold, then the area of ​​the ROI region is determined to be 1 / 3 FOV;

[0085] 2) If the absolute value of the GMV between frames is greater than the second GMV threshold and less than the first GMV threshold, or the target's movement speed is less than the first movement speed threshold and greater than the second movement speed threshold, and the relative height is less than the second height threshold, then the area of ​​the ROI region is determined to be 1 / 2 FOV;

[0086] 3) Otherwise, the area of ​​the ROI region is determined to be 2 / 3 FOV.

[0087] The first bitrate threshold is lower than the second bitrate threshold, the first GMV threshold is higher than the second GMV threshold, the first motion speed threshold is higher than the second motion speed threshold, and the first height threshold is higher than the second height threshold. It should also be understood that there is no limit to the number of levels for the transmission bitrate threshold, GMV threshold, motion speed threshold, and relative height threshold.

[0088] Optionally, when information associated with the global motion state between video frames remains constant, the area of ​​the ROI region is positively correlated with the transmission resource. For example, the smaller the transmission bit rate, the smaller the area of ​​the ROI region.

[0089] Optionally, in order to avoid infinitely enlarging or reducing the proportion of the ROI area in the picture, a threshold for the proportion of the ROI area in the entire picture can be set, such as the minimum and maximum proportions relative to the FOV, to ensure a reasonable range of dynamic adjustment.

[0090] It should be noted that in addition to the above adjustment methods, other methods for dynamically adjusting the area of ​​the ROI region and / or the area of ​​the non-ROI region, such as not considering the limitations of transmission resources, can also be adaptively selected according to actual scenarios and will not be repeated here.

[0091] In some embodiments, the method 100 further includes: determining a position change of the ROI region according to information associated with the posture change of the target object.

[0092] When the target's attitude changes, such as when the aircraft turns or the gimbal's viewing angle changes, the ROI's position in the image is adaptively adjusted to enhance the user's visual experience. For example, the change in the gimbal's viewing angle can be a change in the attitude of at least one of the yaw, pitch, or roll axes. Information associated with the target's attitude change can be user-inputted. For example, in an aircraft scenario, it can be mapped to the amount of stick movement the user makes; the greater the amount of stick movement, the greater the offset in the image center of the ROI. Alternatively, the information can be measured using an attitude sensor on the gimbal, camera, or mobile device itself.

[0093] Optionally, to prevent ROI position shifts caused by unstable control or system errors, a posture change threshold is preset. If the information associated with the target's posture change meets a preset condition, for example, if the information associated with the target's posture change is greater than the posture change threshold, then the ROI position change is determined. If the information associated with the target's posture change is less than the posture change threshold, it is considered to be caused by unstable control or system errors, such as unexpected jitter. In this case, the position of the ROI and / or non-ROI areas does not change accordingly.

[0094] The horizontal displacement of the ROI region is positively correlated with the horizontal component corresponding to the information associated with the posture change of the target object, and / or the vertical displacement of the ROI region is positively correlated with the vertical component corresponding to the information associated with the posture change of the target object.

[0095] Information associated with the target's posture change includes at least one of the target's posture change velocity, posture change linear velocity, posture change angular velocity, posture change acceleration, and posture change angular acceleration. For example, when the posture change angular velocity is used for representation, the posture change angle is decomposed into a horizontal component and a vertical component, where a component is understood to be a vector including a direction and an amplitude. The horizontal displacement of the ROI region is positively correlated with the horizontal component of the target's posture change angular velocity, and / or the vertical displacement of the ROI region is positively correlated with the vertical component of the target's posture change angular velocity.

[0096] Alternatively, as shown in Figure 5, the position change of the ROI region can be represented by the coordinate change of a characteristic identification point in the ROI region. For example, in an unmanned aerial vehicle scenario, the user's line of sight is typically focused on the center of the wearable device. Therefore, the coordinate change of the ROI region's center point can also be used to reflect the position change of the ROI region. In Figure 5, the ROI region's center point changes from center point 1 (x1, y1) to center point 2 (x2, y2). x1 plus the horizontal offset component yields x2, and y1 plus the vertical offset component yields y2. The horizontal and vertical offset components are positively correlated with the horizontal and vertical components of the information associated with the target's posture change.

[0097] In some embodiments, different image processing is performed on the ROI region and the non-ROI region to make the clarity of the ROI region and the non-ROI region different.

[0098] The clarity of the non-ROI area is lower than that of the ROI area.

[0099] While maintaining constant information about the global motion state between video frames, the clarity of the ROI and / or non-ROI regions is related to the transmission conditions of the transmission device corresponding to the camera. For example, the clarity of the ROI and / or non-ROI regions decreases as the transmission bitrate decreases.

[0100] The division method of the ROI area and the non-ROI area includes, but is not limited to, rectangle, square, circle, ellipse, triangle or any other suitable shape, which is not limited in this application.

[0101] Optionally, the ROI area may include multiple non-ROI areas, and the clarity of the non-ROI areas changes gradually, for example, including a first non-ROI area close to the ROI area and a second non-ROI area far from the ROI area, wherein the clarity of the first non-ROI area is higher than that of the second non-ROI area.

[0102] Optionally, the ROI region also includes multiple ROI regions, and the multiple ROI regions gradually change from the outside to the inside, and the clarity of the outer ROI region is lower than that of the inner ROI region.

[0103] The above-mentioned gradual change in clarity and smooth transition can better meet the viewing comfort of the human eye and avoid the viewing discomfort caused by abrupt changes in clarity.

[0104] Optionally, different image processing is performed on the ROI region and the non-ROI region, including at least one of the following situations:

[0105] Blurring of non-ROI areas;

[0106] Sharpening is performed on the ROI area and blurring is performed on the non-ROI area, and the two are optimized jointly;

[0107] Different image processing is performed on the ROI region and the non-ROI region, so that the quantizer parameter (QP) of the ROI region is smaller than the QP of the non-ROI region.

[0108] For example, the QP parameter used when encoding pixels in the ROI region is smaller than the QP parameter used when encoding pixels in the non-ROI region.

[0109] It should be noted that any other manner that can make the clarity of the ROI region and the non-ROI region different should be understood to be included in the different image processing referred to in this application.

[0110] It should be noted that the execution subject of the above-mentioned method 100 may be a device such as a camera, a handheld gimbal, an action camera, a mobile device, a mobile phone, a tablet computer, a smart wearable device, or a computer. In other possible implementations, the camera, the gimbal equipped with the camera, or the mobile device may upload the video and information associated with the global motion state between video frames to a cloud server or a third-party device for data processing, receive the results of the data processing performed by the cloud server or the third-party device, and then perform different image processing on the ROI area and the non-ROI area based on the feedback results. By utilizing the high-speed processing performance of the cloud server or the third-party device, local processing efficiency can be improved.

[0111] The embodiments of the present application fully take into account the characteristic that the human eye is insensitive to high-speed motion content, and determine whether to divide the video into multiple regions based on information associated with the global motion state between frames of the video, and the multiple regions include ROI regions and non-ROI regions. In some embodiments, the area of ​​the ROI region and the non-ROI region in the picture is dynamically and adaptively adjusted based on information associated with the global motion state between frames of the video, so as to effectively save the occupancy of transmission resources while satisfying the user's subjective image quality. In some embodiments, the change in the position of the ROI region and the non-ROI region is determined based on information associated with the posture change of the target object, so as to achieve self-correction of the position of the ROI region and the non-ROI region when the posture of the target object changes.

[0112] Optionally, in some embodiments, in order to avoid the user's subjective visual flickering experience caused by frequent switching between the divided and non-divided scenes of the ROI area and the non-ROI area, when it is determined that the video needs to be switched from non-divided to divided, the transition can be completed gradually within a period of time T1 (multi-frame images). When it is determined that the division needs to be changed to non-divided, the transition is completed gradually within a period of time T2. Compared with instantaneous flickering, the gradual transition can better meet the visual adaptation of the human eye and avoid eye fatigue. T1 and T2 can be the same or different and can be set according to actual needs. For example, the original need to switch from non-divided to divided ROI area and non-ROI area within two adjacent frames can be switched to gradually switching from non-divided to divided ROI area and non-ROI area in continuous multiple frames through time domain filtering, which can avoid the problem of poor user experience caused by the jump of the picture.

[0113] Please refer to FIG. 5 , another video processing method 200 is provided in this embodiment of the present application.

[0114] As an example, the method 200 includes steps S210 to S230:

[0115] S210: Obtaining a video captured by a camera;

[0116] S220: Determine the area of ​​the ROI region and / or the non-ROI region in the video based on information associated with the global motion state between frames of the video;

[0117] S230: Perform different image processing on the ROI region and the non-ROI region to make the clarity of the ROI region and the non-ROI region different.

[0118] Method 200 dynamically adjusts the area of ​​the ROI and / or non-ROI regions in a video. The specific steps for performing this dynamic adjustment, including determining the positional change of the ROI region based on information associated with the target's posture changes, are similar to those of method 100 and are not further described here for the sake of brevity. Method 200 does not impose any particular restrictions on the manner or basis for dividing the ROI and non-ROI regions in the video.

[0119] Optionally, in some embodiments, in order to avoid the user's subjective visual flickering experience caused by frequent switching between the divided and non-divided scenes of the ROI area and the non-ROI area, when it is determined that the video needs to be switched from non-divided to divided, the transition can be completed gradually within a period of time T1 (multi-frame images). When it is determined that the division needs to be changed to non-divided, the transition is completed gradually within a period of time T2. Compared with instantaneous flickering, the gradual transition can better meet the visual adaptation of the human eye and avoid eye fatigue. T1 and T2 can be the same or different and can be set according to actual needs. For example, the original need to switch from non-divided to divided ROI area and non-ROI area within two adjacent frames can be switched to gradually switching from non-divided to divided ROI area and non-ROI area in continuous multiple frames through time domain filtering, which can avoid the problem of poor user experience caused by the jump of the picture.

[0120] The embodiments of this application fully consider the human eye's insensitivity to high-speed motion content. Based on information associated with the global motion state between video frames, the area of ​​the ROI and non-ROI regions in the image are dynamically and adaptively adjusted to effectively conserve transmission resources while still meeting the user's subjective image quality. In some embodiments, the positional changes of the ROI and non-ROI regions are also determined based on information associated with the target's posture changes, enabling automatic positional correction of the ROI and non-ROI regions in the event of a change in the target's posture.

[0121] Referring to FIG. 6 , an embodiment of the present application further provides a video processing device 300 , including:

[0122] Memory 301, used for storing computer programs;

[0123] The processor 302 is configured to call a computer program. When the computer program is executed by the processor, the video processing apparatus performs the following operations:

[0124] Obtaining a video captured by a camera;

[0125] Dividing the video into a plurality of regions according to information associated with a global motion state between frames of the video, wherein the plurality of regions include a ROI region and a non-ROI region;

[0126] Different image processing is performed on the ROI region and the non-ROI region so that the clarity of the ROI region and the non-ROI region is different.

[0127] It should be noted that the specific implementation of the above operations performed by the processor 302 can refer to the relevant description in the aforementioned method 100, which will not be repeated here.

[0128] Referring to FIG. 7 , an embodiment of the present application further provides another video processing device 400 , including:

[0129] Memory 401, used for storing computer programs;

[0130] The processor 402 is configured to call a computer program. When the computer program is executed by the processor, the video processing apparatus performs the following operations:

[0131] Obtaining a video captured by a camera;

[0132] Determining the area of ​​a ROI region and / or a non-ROI region in the video based on information associated with a global motion state between frames of the video;

[0133] Different image processing is performed on the ROI region and the non-ROI region so that the clarity of the ROI region and the non-ROI region is different.

[0134] It should be noted that the specific implementation of the above operations performed by the processor 402 can refer to the relevant description in the aforementioned method 200, which will not be repeated here.

[0135] Specifically, the processor 302 or the processor 402 may be a micro-controller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0136] Specifically, the memory 301 or 401 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0137] The processor is used to run the computer program stored in the memory and implement the operations of the aforementioned video processing method when executing the computer program.

[0138] The specific principles and implementation methods of the video processing device provided in the embodiments of the present application are similar to the video processing methods of the aforementioned corresponding embodiments and will not be repeated here.

[0139] The present application also provides a device, comprising:

[0140] A camera device, which is mounted on the device;

[0141] And the video processing device as in the above embodiment.

[0142] Optionally, the device includes but is not limited to: a handheld gimbal, a sports camera, a movable device, a mobile phone, a tablet computer, a smart wearable device, a computer, and the like.

[0143] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the video processing method provided in the above embodiment.

[0144] One or more computer program instructions can be stored on a computer-readable storage medium, and the processor can run the program instructions stored in the storage device to implement the functions (implemented by the processor) in the embodiments of the present application herein and / or other desired functions, such as to execute the corresponding steps of the video processing method according to the embodiments of the present application. Various applications and various data can also be stored in the computer-readable storage medium, such as various data used and / or generated by the application.

[0145] It should be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application.

[0146] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A video processing method, characterized in that: include: Obtaining a video captured by a camera; dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video, wherein the plurality of regions include a region of interest and a region of no interest; Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

2. The method according to claim 1, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: In a case where a transmission condition of a transmission device corresponding to the shooting apparatus does not satisfy a preset transmission condition, the video is divided into a plurality of regions according to information associated with a global motion state between frames of the video.

3. The method according to claim 1 or 2, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: If the global motion change represented by the information associated with the global motion state between frames of the video meets a preset change condition, the video is divided into a plurality of regions.

4. The method according to claim 1, wherein The information associated with the global motion state between frames of the video includes at least one of the following: Global motion information between frames of the video; Information associated with a motion state of a target object, wherein the target object includes at least any one of the photographing device and a device carrying the photographing device.

5. The method according to claim 4, characterized in that The global motion information between frames of the video includes: a global motion vector between frames of the video; The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: If the absolute value of the global motion vector between frames of the video is greater than a global motion vector threshold, the video is divided into a plurality of regions.

6. The method according to claim 4, characterized in that The information associated with the motion state of the target object includes: the motion speed of the target object; and the relative distance or relative height between the target object and the subject.

7. The method according to claim 6, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: In a case where the relative distance or relative height between the target object and the subject remains constant, if the absolute value of the movement speed of the target object is greater than a movement speed threshold, the video is divided into a plurality of areas.

8. The method according to claim 6, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: In a case where the moving speed of the target object remains constant, if the relative distance between the target object and the subject is less than a distance threshold or the relative height is less than a height threshold, the video is divided into a plurality of areas.

9. The method according to any one of claims 1 to 6, further comprising: The area of ​​the region of interest and / or the area of ​​the region of no interest is determined according to information associated with a global motion state between frames of the video.

10. The method according to claim 9, characterized in that The area of ​​the region of interest is negatively correlated with the global motion change represented by the information associated with the global motion state between frames of the video; and / or, The area of ​​the non-interest region is positively correlated with the global motion change represented by the information associated with the global motion state between frames of the video.

11. The method according to claim 9, characterized in that In a case where the global motion information between frames of the video includes a global motion vector between frames of the video: The area of ​​the region of interest is negatively correlated with the absolute value of the global motion vector between frames of the video, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the global motion vector between frames of the video.

12. The method according to claim 9, characterized in that In the case where the information associated with the motion state of the target includes the motion speed of the target and the relative distance or relative height between the target and the subject, and the relative distance / relative height between the target and the subject remains constant: The area of ​​the region of interest is negatively correlated with the absolute value of the movement speed of the target object, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the movement speed of the target object.

13. The method according to claim 9, characterized in that In the case where the information associated with the motion state of the target object includes the motion speed of the target object and the relative distance or relative height between the target object and the subject, and the motion speed of the target object remains constant: The area of ​​the region of interest is positively correlated with the relative distance or relative height between the target object and the subject, and / or the area of ​​the non-region of interest is negatively correlated with the relative distance or relative height between the target object and the subject.

14. The method according to claim 9, characterized in that In a case where the information associated with the global motion state between frames of the video remains constant, the area of ​​the region of interest and / or the region of no interest is positively correlated with the field of view of the camera.

15. The method according to claim 1 or 9, characterized in that The method further comprises: determining a position change of the region of interest based on information associated with a posture change of the target object; The target object at least includes any one of the photographing device and the equipment carrying the photographing device.

16. The method according to claim 15, characterized in that The determining the position change of the region of interest based on information associated with the posture change of the target object includes: If the information associated with the posture change of the target object meets a preset condition, the position change of the region of interest is determined.

17. The method according to claim 15, characterized in that The position change of the region of interest includes a horizontal displacement and / or a vertical displacement of the region of interest.

18. The method according to claim 17, characterized in that The horizontal displacement of the region of interest is positively correlated with the horizontal component corresponding to the information associated with the posture change of the target object, and / or the vertical displacement of the region of interest is positively correlated with the vertical component corresponding to the information associated with the posture change of the target object.

19. The method according to claim 18, characterized in that The information associated with the posture change of the target object includes at least one of a posture change speed, a posture change linear speed, a posture change angular speed, a posture change acceleration, and a posture change angular acceleration of the target object.

20. The method according to claim 1, wherein The definition of the non-interest region is lower than that of the interest region.

21. The method according to claim 20, characterized in that In a case where the information associated with the global motion state between frames of the video remains constant, the clarity of the region of interest and / or the region of no interest is related to the transmission condition corresponding to the transmission device corresponding to the shooting apparatus.

22. The method according to claim 21, characterized in that The transmission condition includes a transmission bit rate, and the clarity of the region of interest and / or the region of no interest decreases as the transmission bit rate decreases.

23. The method according to claim 20, characterized in that The region of interest includes a first non-interest region close to the region of interest and a second non-interest region far from the region of interest, wherein the clarity of the first non-interest region is higher than that of the second non-interest region.

24. The method according to claim 20, characterized in that The performing different image processing on the region of interest and the region of no interest so as to make the clarity of the region of interest and the region of no interest different comprises: Blur areas of no interest; or Different image processing is performed on the region of interest and the region of no interest, so that a quantization parameter of the region of interest is smaller than a quantization parameter of the region of no interest.

25. The method according to claim 24, characterized in that The performing blur processing on the non-interest area includes: A sharpness process is performed on the region of interest, and a blur process is performed on the non-region of interest.

26. A video processing method, characterized in that: include: Obtaining a video captured by a camera; determining, based on information associated with a global motion state between frames of the video, an area of ​​a region of interest and / or a region of no interest in the video; Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

27. The method according to claim 26, characterized in that The determining, based on information associated with a global motion state between frames of the video, the area of ​​the region of interest and / or the region of no interest in the video includes: When the transmission conditions of the transmission equipment corresponding to the shooting device do not meet the preset transmission conditions, the area of ​​the area of ​​interest and / or the area of ​​the area of ​​no interest in the video are determined based on information associated with the global motion state between frames of the video.

28. The method according to claim 26 or 27, characterized in that The information associated with the global motion state between frames of the video includes at least one of the following: Global motion information between frames of the video; Information related to the target's motion state, The target object at least includes any one of the photographing device and the equipment carrying the photographing device.

29. The method according to claim 28, characterized in that The global motion information between frames of the video includes a global motion vector between frames of the video; The area of ​​the region of interest is negatively correlated with the absolute value of the global motion vector between frames of the video, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the global motion vector between frames of the video.

30. The method according to claim 28, wherein The information associated with the motion state of the target object includes: The moving speed of the target object; and The relative distance or relative height between the target object and the subject.

31. The method according to claim 30, wherein When the relative distance or relative height between the target object and the subject remains constant, the area of ​​the region of interest is negatively correlated with the absolute value of the movement speed of the target object, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the movement speed of the target object.

32. The method according to claim 30, wherein When the movement speed of the target object remains constant, the area of ​​the region of interest is positively correlated with the relative distance or relative height between the target object and the subject, and / or the area of ​​the non-region of interest is negatively correlated with the relative distance or relative height between the target object and the subject.

33. The method according to claim 26 or 27, characterized in that In a case where information associated with a global motion state between frames of the video remains constant, the area of ​​the region of interest and / or the region of no interest is positively correlated with the field of view of the camera.

34. The method according to claim 26, wherein The method further includes: determining a position change of the region of interest based on information associated with a posture change of a target object; wherein the target object includes at least one of the photographing device and a device carrying the photographing device.

35. The method according to claim 34, wherein The determining the position change of the region of interest based on information associated with the posture change of the target object includes: If the information associated with the posture change of the target object meets a threshold condition, a position change of the region of interest is determined.

36. The method according to claim 34 or 35, characterized in that The position change of the region of interest includes a horizontal displacement and / or a vertical displacement of the region of interest.

37. The method according to claim 36, wherein The horizontal displacement of the region of interest is positively correlated with the horizontal component corresponding to the information associated with the posture change of the target object, and / or the vertical displacement of the region of interest is positively correlated with the vertical component corresponding to the information associated with the posture change of the target object.

38. The method according to claim 37, wherein The information associated with the posture change of the target object includes at least one of a posture change speed, a posture change linear speed, a posture change angular speed, a posture change acceleration, and a posture change angular acceleration of the target object.

39. A video processing device, characterized in that: include: memory for storing computer programs; The processor is configured to call the computer program, and when the computer program is executed by the processor, causes the video processing device to perform the following operations: Obtaining a video captured by a camera; dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video, wherein the plurality of regions include a region of interest and a region of no interest; Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

40. The apparatus of claim 39, wherein dividing the video into regions of interest and regions of non-interest based on information associated with a global motion state between frames of the video comprises: In a case where a transmission condition of a transmission device corresponding to the shooting apparatus does not satisfy a preset transmission condition, the video is divided into a plurality of regions according to information associated with a global motion state between frames of the video.

41. The device according to claim 39 or 40, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: If the global motion change represented by the information associated with the global motion state between frames of the video meets a preset change condition, the video is divided into a plurality of regions.

42. The device according to claim 39, characterized in that The information associated with the global motion state between frames of the video includes at least one of the following: Global motion information between frames of the video; Information related to the target's motion state, The target object at least includes any one of the photographing device and the equipment carrying the photographing device.

43. The device according to claim 42, characterized in that The global motion information between frames of the video includes: a global motion vector between frames of the video; The dividing the video into multiple regions based on information associated with the global motion state between frames of the video includes: if the absolute value of the global motion vector between frames of the video is greater than a global motion vector threshold, dividing the video into multiple regions.

44. The device according to claim 42, characterized in that The information associated with the motion state of the target object includes: The moving speed of the target object; and The relative distance or relative height between the target object and the subject.

45. The device according to claim 44, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: In a case where the relative distance or relative height between the target object and the subject remains constant, if the absolute value of the movement speed of the target object is greater than a movement speed threshold, the video is divided into a plurality of areas.

46. ​​The device according to claim 44, characterized in that The step of dividing the video into a plurality of regions based on information associated with a global motion state between frames of the video comprises: In a case where the moving speed of the target object remains constant, if the relative distance between the target object and the subject is less than a distance threshold or the relative height is less than a height threshold, the video is divided into a plurality of areas.

47. The apparatus according to any one of claims 39 to 46, wherein the processor further comprises: The area of ​​the region of interest and / or the area of ​​the region of no interest is determined according to information associated with a global motion state between frames of the video.

48. The device according to claim 47, characterized in that The area of ​​the region of interest is negatively correlated with the global motion change represented by the information associated with the global motion state between frames of the video, and / or, The area of ​​the non-interest region is positively correlated with the global motion change represented by the information associated with the global motion state between frames of the video.

49. The device according to claim 47, characterized in that In a case where the global motion information between frames of the video includes a global motion vector between frames of the video; The area of ​​the region of interest is negatively correlated with the absolute value of the global motion vector between frames of the video, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the global motion vector between frames of the video.

50. The device according to claim 47, characterized in that In the case where the information associated with the motion state of the target object includes the motion speed of the target object and the relative distance or relative height between the target object and the subject, and the relative distance / relative height between the target object and the subject remains constant: The area of ​​the region of interest is negatively correlated with the absolute value of the movement speed of the target object, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the movement speed of the target object.

51. The device according to claim 47, characterized in that In the case where the information associated with the motion state of the target object includes the motion speed of the target object and the relative distance or relative height between the target object and the subject, and the motion speed of the target object remains constant: The area of ​​the region of interest is positively correlated with the relative distance or relative height between the target object and the subject, and / or the area of ​​the non-region of interest is negatively correlated with the relative distance or relative height between the target object and the subject.

52. The device according to claim 47, characterized in that In a case where information associated with a global motion state between frames of the video remains constant, the area of ​​the region of interest and / or the region of no interest is positively correlated with the field of view of the camera.

53. The device according to claim 39 or 47, characterized in that The processor further includes: determining a position change of the region of interest based on information associated with a posture change of a target object; wherein the target object includes at least one of the photographing device and a device carrying the photographing device.

54. The device according to claim 53, characterized in that The determining the position change of the region of interest based on information associated with the posture change of the target object includes: If the information associated with the posture change of the target object meets a preset condition, the position change of the region of interest is determined.

55. The device according to claim 54, characterized in that The position change of the region of interest includes a horizontal displacement and / or a vertical displacement of the region of interest.

56. The device according to claim 55, characterized in that The horizontal displacement of the region of interest is positively correlated with the horizontal component corresponding to the information associated with the posture change of the target object, and / or the vertical displacement corresponding to the region of interest is positively correlated with the vertical component of the information associated with the posture change of the target object.

57. The device according to claim 56, characterized in that The information associated with the posture change of the target object includes at least one of the posture change speed, posture change linear velocity, posture change angular velocity, posture change acceleration, and posture change angular acceleration of the target object.

58. The device according to claim 39, characterized in that The definition of the non-interest region is lower than that of the interest region.

59. The device according to claim 58, characterized in that In a case where the information associated with the global motion state between frames of the video remains constant, the clarity of the region of interest and / or the region of no interest is related to the transmission condition corresponding to the transmission device corresponding to the shooting apparatus.

60. The method according to claim 59, wherein The transmission condition includes a transmission bit rate, and the clarity of the region of interest and / or the region of no interest decreases as the transmission bit rate decreases.

61. The device according to claim 58, characterized in that When the information associated with the global motion state between frames of the video remains constant, the clarity of the region of interest and / or the region of no interest decreases as the transmission bit rate of the transmission device corresponding to the shooting apparatus decreases.

62. The device according to claim 58, characterized in that The region of interest includes a first non-interest region close to the region of interest and a second non-interest region far from the region of interest, wherein the first non-interest region has a higher definition than the second non-interest region.

63. The device according to claim 58, characterized in that The performing different image processing on the region of interest and the region of no interest so as to make the clarity of the region of interest and the region of no interest different comprises: Performing blurring on the non-interested region; or, Different image processing is performed on the region of interest and the region of no interest, so that a quantization parameter of the region of interest is smaller than a quantization parameter of the region of no interest.

64. The device according to claim 63, characterized in that The performing blur processing on the non-interest area includes: A sharpness process is performed on the region of interest, and a blur process is performed on the non-region of interest.

65. A video processing device, characterized in that include: memory for storing computer programs; The processor is configured to call the computer program, and when the computer program is executed by the processor, causes the video processing device to perform the following operations: Obtaining a video captured by a camera; determining, based on information associated with a global motion state between frames of the video, an area of ​​a region of interest and / or a region of no interest in the video; Different image processing is performed on the region of interest and the region of no interest, so that the clarity of the region of interest and the region of no interest is different.

66. The device according to claim 65, characterized in that The determining, based on information associated with a global motion state between frames of the video, the area of ​​the region of interest and / or the region of no interest in the video includes: When the transmission conditions of the transmission equipment corresponding to the shooting device do not meet the preset transmission conditions, the area of ​​the area of ​​interest and / or the area of ​​the area of ​​no interest in the video are determined based on information associated with the global motion state between frames of the video.

67. The device according to claim 65 or 66, characterized in that The information associated with the global motion state between frames of the video includes at least one of the following: Global motion information between frames of the video; Information related to the target's motion state, The target object at least includes any one of the photographing device and the equipment carrying the photographing device.

68. The device according to claim 67, characterized in that The global motion information between frames of the video includes a global motion vector between frames of the video; The area of ​​the region of interest is negatively correlated with the absolute value of the global motion vector between frames of the video, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the global motion vector between frames of the video.

69. The device according to claim 67, characterized in that The information associated with the motion state of the target object includes: The moving speed of the target object; and The relative distance or relative height between the target object and the subject.

70. The device according to claim 69, characterized in that When the relative distance or relative height between the target object and the subject remains constant, the area of ​​the region of interest is negatively correlated with the absolute value of the movement speed of the target object, and / or the area of ​​the non-region of interest is positively correlated with the absolute value of the movement speed of the target object.

71. The device according to claim 69, characterized in that When the movement speed of the target object remains constant, the area of ​​the region of interest is positively correlated with the relative distance or relative height between the target object and the subject, and / or the area of ​​the non-region of interest is negatively correlated with the relative distance or relative height between the target object and the subject.

72. The device according to claim 65 or 66, characterized in that In a case where information associated with a global motion state between frames of the video remains constant, the area of ​​the region of interest and / or the region of no interest is positively correlated with the field of view of the camera.

73. The device according to claim 65, characterized in that The device further includes: determining a position change of the region of interest based on information associated with a posture change of a target object; wherein the target object includes at least one of the photographing device and a device carrying the photographing device.

74. The device according to claim 73, characterized in that The determining the position change of the region of interest based on information associated with the posture change of the target object includes: If the information associated with the posture change of the target object meets a threshold condition, a position change of the region of interest is determined.

75. The device according to claim 74, characterized in that The position change of the region of interest includes a horizontal displacement and / or a vertical displacement of the region of interest.

76. The device according to claim 75, characterized in that The horizontal displacement of the region of interest is positively correlated with the horizontal component corresponding to the information associated with the posture change of the target object, and / or the vertical displacement of the region of interest is positively correlated with the vertical component corresponding to the information associated with the posture change of the target object.

77. The device according to claim 76, characterized in that The information associated with the posture change of the target object includes at least any one of the posture change speed, posture change linear velocity, posture change angular velocity, posture change acceleration, and posture change angular acceleration of the target object.

78. A device, characterized in that The device comprises: a photographing device, the photographing device being mounted on the device; And the video processing device according to any one of claims 39-77.

79. The apparatus according to claim 78, wherein The device includes: any one of a mobile phone, a tablet computer, a smart wearable device, a handheld gimbal, and a movable device.

80. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by a processor, are used to execute the video processing method according to any one of claims 1 to 38.

Images