Video image correction method and apparatus, computer readable medium, and electronic device

Abstract

The present disclosure provides a video image correction method and device, a computer readable medium and an electronic device, and relates to the technical field of video image stabilization. The method comprises: acquiring a current video image captured in a current camera pose, and acquiring a current gravity vector corresponding to the current camera pose; determining a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector; determining a rotation axis angle relationship for converting from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector; and performing horizontal correction on the current video image through the rotation axis angle relationship to obtain a horizontally corrected current video image. The present disclosure can realize horizontal line correction of image content, improve the accuracy of the horizontal line correction result, has a wide range of applications, and has a small amount of calculation, and can be applied to a real-time processing system.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video image stabilization, in particular to a video image correction method, a video image correction device, a computer readable medium and an electronic device. BACKGROUND

[0002] With the rapid development of science and technology, people are increasingly fond of recording life through the camera on the smart phone, and video image stabilization technology is also increasingly valued. For example, electric image stabilization (EIS) and optical image stabilization (OIS) are relatively mature video image stabilization technologies. However, although video image stabilization can reduce the shaking phenomenon of the video image to a certain extent, it cannot solve the problem of the tilt of the video image.

[0003] At present, in the related technical solutions, either the horizontal line correction of the video image cannot be realized, or only the horizontal correction of the image with obvious key feature points such as a license plate image can be performed, the application scene is relatively single, the applicable range is small, and the estimation of the horizontal direction through the image content is relatively dependent on the quality of the image content. Different light intensity, light color and image definition will affect the final estimation result, the accuracy of the horizontal correction result is low, and the robustness is poor. SUMMARY

[0004] The purpose of the present disclosure is to provide a video image correction method, a video image correction device, a computer readable medium and an electronic device, so as to at least realize the horizontal line correction of the video image in any scene, have a wide application range, and improve the accuracy and robustness of the horizontal correction result.

[0005] According to a first aspect of the present disclosure, a video image correction method is provided, comprising:

[0006] obtaining a current video image captured in a current camera pose, and obtaining a current gravity vector corresponding to the current camera pose;

[0007] determining a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector;

[0008] determining a rotation axis angle relationship for converting from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector;

[0009] performing horizontal correction on the current video image through the rotation axis angle relationship to obtain a horizontally corrected current video image.

[0010] According to a second aspect of the present disclosure, a video image correction apparatus is provided, comprising:

[0011] a video image obtaining module configured to obtain a current video image captured at a current camera pose, and obtain a current gravity vector corresponding to the current camera pose;

[0012] a gravity vector determining module configured to determine a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector;

[0013] a rotation axis angle relationship determining module configured to determine a rotation axis angle relationship from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector;

[0014] a horizontal correction module configured to perform horizontal correction on the current video image by the rotation axis angle relationship to obtain a horizontally corrected current video image.

[0015] According to a third aspect of the present disclosure, a computer readable medium is provided, having a computer program stored thereon, the computer program, when executed by a processor, implements the method described above.

[0016] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0017] a processor; and

[0018] a memory configured to store one or more programs, the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method described above.

[0019] The video image correction method provided by one embodiment of the present disclosure can acquire a current video image captured in a current camera pose, acquire a current gravity vector corresponding to the current camera pose, then determine a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector, and further determine a rotation axis angle relationship for converting from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector, and correct a horizontal line of the current video image through the rotation axis angle relationship to obtain a current video image corrected horizontally. On one hand, the rotation axis angle relationship is determined according to the current gravity vector corresponding to the current camera pose and the horizontal gravity vector, and the current video image is corrected horizontally through the rotation axis angle relationship, compared with the scheme of determining a horizontal direction through key points in image content and correcting horizontally, the current gravity vector detected by the gravity sensor has higher accuracy, which effectively improves the accuracy of the horizontal line correction result, and at the same time, the horizontal line correction result is only associated with the current gravity vector and is not interfered by other error data, thereby ensuring the robustness of the horizontal line correction result. On the other hand, the current video image is corrected horizontally through the detected current gravity vector, which can be applied to various types of scenes, has a wide range of applications, and has a low time complexity compared with the way of calculating a horizontal direction according to image content, and can be well adapted to real-time processing systems.

[0020] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0021] The drawings incorporated in the specification and constituting a part of the specification illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings. In the drawings:

[0022] Figure 1 A schematic diagram of an exemplary system architecture to which embodiments of the present disclosure can be applied is shown;

[0023] Figure 2 A schematic diagram of an electronic device to which embodiments of the present disclosure can be applied is shown;

[0024] Figure 3 A schematic diagram of a camera motion coordinate axis in an exemplary embodiment of the present disclosure is shown schematically;

[0025] Figure 4 A flowchart of a video image correction method in an exemplary embodiment of the present disclosure is shown schematically;

[0026] Figure 5A flowchart schematically showing a process of determining a maximum correctable angle in an exemplary embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram schematically showing a principle of determining a maximum correctable angle in an exemplary embodiment of the present disclosure;

[0028] Figure 7 A flowchart schematically showing a process of determining a rotational axis angle relationship in an exemplary embodiment of the present disclosure;

[0029] Figure 8 A flowchart schematically showing another process of determining a rotational axis angle relationship in an exemplary embodiment of the present disclosure;

[0030] Figure 9 A flowchart schematically showing a process of performing a horizontal line correction on a current video image in an exemplary embodiment of the present disclosure;

[0031] Figure 10 A schematic diagram schematically showing a video image requiring a horizontal line correction in an exemplary embodiment of the present disclosure;

[0032] Figure 11 A schematic diagram schematically showing a principle of performing a horizontal line correction on a current video image in an exemplary embodiment of the present disclosure;

[0033] Figure 12 A flowchart schematically showing a process of performing a horizontal correction output on a current video image in an exemplary embodiment of the present disclosure;

[0034] Figure 13 A schematic diagram schematically showing a composition of a video image correction device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0036] Further, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification. The drawings are not necessarily to scale, the dimensions of certain features being exaggerated from others for illustrative purposes. Same reference numerals in different drawings represent same or similar elements unless otherwise specified.

[0037] Figure 1 A schematic diagram of a system architecture showing an exemplary application environment of a video image rectification method and apparatus according to an embodiment of the present disclosure is shown.

[0038] As shown in Figure 1 , the system architecture 100 can include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The terminal devices 101, 102, 103 can be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smart phones, and tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0039] The video image rectification method provided by the embodiments of the present disclosure is generally executed by the terminal devices 101, 102, 103, and accordingly, the video image rectification apparatus is generally provided in the terminal devices 101, 102, 103. However, it is easily understood by those skilled in the art that the video image rectification method provided by the embodiments of the present disclosure can also be executed by the server 105, and accordingly, the video image rectification apparatus can also be provided in the server 105, which is not specially limited in the present exemplary embodiment.

[0040] For example, in an exemplary embodiment, the user can upload the current video image photographed by the terminal device 101, 102, 103 and the current gravity vector corresponding to the current camera pose collected to the server 105, and the server generates the horizontal rectified current video image by the video image rectification method provided by the embodiments of the present disclosure, and then transmits the horizontal rectified current video image to the terminal device 101, 102, 103, etc. for display.

[0041] An exemplary embodiment of the present disclosure provides an electronic device for implementing a video image rectification method, which can be a terminal device 101, 102, 103 or a server 105 in Figure 1 The electronic device at least includes a processor and a memory for storing executable instructions of the processor, and the processor is configured to execute the video image rectification method via executing the executable instructions.

[0042] The following takes the mobile terminal 200 in Figure 2 as an example to exemplarily illustrate the configuration of the electronic device. Those skilled in the art should understand that, in addition to the components specially used for mobile purposes, Figure 2 The configuration in Figure 2 can also be applied to devices of a fixed type. In other embodiments, the mobile terminal 200 can include more or fewer components than those shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between components is only schematically shown, and does not constitute a structural limitation on the mobile terminal 200. In other embodiments, the mobile terminal 200 can also use different interface connection manners, or a combination of multiple interface connection manners.

[0043] As shown in Figure 2 , the mobile terminal 200 can specifically include a processor 210, an internal memory 221, an external memory interface 222, a universal serial bus (USB) interface 230, a charge management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a loudspeaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a key 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 can include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0044] The processor 210 can include one or more processing units, for example: the processor 210 can include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), a controller, a video codec, a digital signal processor (Digital Signal Processor, DSP), a baseband processor and / or a neural network processing unit (Neural-Network Processing Unit, NPU) and the like. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0045] The NPU is a neural network (Neural-Network, NN) computing processor, which can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, intelligent cognitive applications of the mobile terminal 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0046] The processor 210 is provided with a memory. The memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions and notification instructions, and the execution is controlled by the processor 210.

[0047] The charging management module 240 is used to receive charging input from the charger. The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, and supplies power to the processor 210, the internal memory 221, the display screen 290, the camera module 291 and the wireless communication module 260, etc.

[0048] The wireless communication function of the mobile terminal 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor, and the baseband processor, etc. Among them, the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals; the mobile communication module 250 can provide a solution including 2G / 3G / 4G / 5G wireless communication applied to the mobile terminal 200; the modem processor can include a modulator and a demodulator; the wireless communication module 260 can provide a solution including wireless local area network (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), etc. wireless communication applied to the mobile terminal 200. In some embodiments, the antenna 1 of the mobile terminal 200 and the mobile communication module 250 are coupled, and the antenna 2 and the wireless communication module 260 are coupled, so that the mobile terminal 200 can communicate with the network and other devices through wireless communication technology.

[0049] The mobile terminal 200 can realize the display function through the GPU, the display screen 290, and the application processor, etc. The GPU is a microprocessor for image processing, connected with the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 can include one or more GPUs that execute program instructions to generate or change display information.

[0050] The mobile terminal 200 can realize the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290, and the application processor, etc. Among them, the ISP is used to process the data fed back by the camera module 291; the camera module 291 is used to capture still images or videos; the digital signal processor is used to process digital signals, which can process not only digital image signals but also other digital signals; the video codec is used to compress or decompress digital videos, and the mobile terminal 200 can also support one or more video codecs.

[0051] The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to realize the expansion of the storage capacity of the mobile terminal 200. The external memory card communicates with the processor 210 through the external memory interface 222 to realize the data storage function. For example, the music, video, etc. files are saved in the external memory card.

[0052] The internal memory 221 can be used to store computer executable program codes, the executable program codes including instructions. The internal memory 221 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like. The data storage area can store data (such as audio data, a phone book, etc.) created during use of the mobile terminal 200, and the like. In addition, the internal memory 221 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one of a magnetic disk storage device, a flash memory device, a Universal Flash Storage (UFS), and the like. The processor 210 performs various function applications and data processing of the mobile terminal 200 by running instructions stored in the internal memory 221 and / or instructions stored in a memory disposed in the processor.

[0053] The mobile terminal 200 can implement an audio function through the audio module 270, the speaker 271, the receiver 272, the microphone 273, the earphone interface 274, the application processor, and the like. For example, music playing, recording, and the like.

[0054] The depth sensor 2801 is used to obtain depth information of a scene. In some embodiments, the depth sensor can be disposed in the camera module 291.

[0055] The pressure sensor 2802 is used to sense a pressure signal, and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 2802 can be disposed in the display screen 290. There are many types of pressure sensors, such as a resistance type pressure sensor, an inductance type pressure sensor, a capacitance type pressure sensor, and the like.

[0056] The gyroscope sensor 2803 can be used to determine a motion posture of the mobile terminal 200. In some embodiments, the angular velocity of the mobile terminal 200 around three axes (i.e., x, y, and z axes) can be determined through the gyroscope sensor 2803. The gyroscope sensor 2803 can be used for shooting anti-shake, navigation, motion sensing game scenarios, and the like.

[0057] In addition, other functional sensors can also be disposed in the sensor module 280 according to actual needs, such as a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0058] The mobile terminal 200 may also include other devices that provide auxiliary functions. For example, buttons 294 may include a power button, volume buttons, etc., allowing users to input key signals related to user settings and function control of the mobile terminal 200. Other examples include indicators 292, motors 293, and SIM card interfaces 295.

[0059] Most terminal devices with image acquisition capabilities, such as smartphones, are equipped with gravity sensors. Gravity sensors can be used to measure acceleration caused by gravity, calculate the tilt angle of the terminal device relative to the horizontal plane, and analyze the movement mode of the terminal device by analyzing dynamic gravitational acceleration.

[0060] For ease of explanation and understanding, the camera coordinate system in the embodiments of this disclosure can be as follows: Figure 3 As shown, when the terminal device is in the default screen orientation, the X-axis extends horizontally to the right, the Y-axis extends vertically upwards, and the Z-axis extends perpendicularly outwards from the screen. It's easy to understand that in... Figure 3 In the camera coordinate system shown, the coordinates behind the screen will have negative Z values. Generally, the gravity sensor can directly output the components of gravitational acceleration along the three axes of the camera.

[0061] In the embodiments of this disclosure, a camera orientation where the X-axis is parallel to the horizon (horizontal line) or the Y-axis is parallel to the horizon (horizontal line) can be defined as a horizontal camera orientation. In an image captured by a camera in a horizontal orientation, a straight line parallel to the horizon is also parallel to the edge of the image. Because the direction of gravitational acceleration is always perpendicular to the horizon and points towards the Earth's center, the reading of the gravity sensor should be (g...) when the camera is in a horizontal orientation. x ,0,g z ) or (0,g y ,g z ).

[0062] In the related art, one technical solution is to correct the license plate image horizontally according to the image content, which estimates the horizontal inclination angle of the current image by identifying the video image content, locating the key points in the license plate image, and calculating the horizontal inclination angle of the key point line. Another technical solution is to correct the license plate image horizontally, which finds the largest face in the video frame or image by face recognition, and takes the direction of the largest face as the target horizontal direction, and then uses the gyroscope information to correct the video frame or image to the direction of the largest face. However, these two technical solutions are only applicable to images with specific key features, such as license plate images and face images, and are not universal. Moreover, both of these two technical solutions estimate the horizontal direction based on image content, which is susceptible to the quality of the image. Different light intensity, light color and image clarity will affect the final estimation result, resulting in low accuracy and poor robustness of the image correction result. In addition, the calculation of the horizontal direction based on image content has high time complexity, which is not suitable for some real-time processing systems.

[0063] Based on one or more problems in the related art, the present disclosure first provides a video image correction method. The video image correction method of the exemplary embodiment of the present disclosure is described in detail below with the terminal device executing the method as an example.

[0064] Figure 4 The flow of a video image correction method in the present exemplary embodiment is shown, which includes the following steps S410 to S440:

[0065] In step S410, a current video image captured in a current camera pose is obtained, and a current gravity vector corresponding to the current camera pose is obtained.

[0066] In an exemplary embodiment, the current camera pose refers to the camera pose of the camera when capturing the current video image at the current time. For example, the current camera pose can be a camera pose with an inclination angle of 45° with respect to the horizontal line, or a horizontal camera pose. The present exemplary embodiment does not make special limitations on this.

[0067] The current gravity vector refers to the gravity acceleration measured by the gravity sensor under the current camera pose. For example, the current gravity vector under the current camera pose can be represented as (g x ,g y ,g z ), where g x represents the component of the gravity acceleration on the X-axis, g y represents the component of the gravity acceleration on the Y-axis, and g zmay represent the component of the gravity acceleration on the Z axis; if the current camera pose is a horizontal pose, the corresponding current gravity vector can be represented as (g x ,0,g z ) or (0,g y ,g z ).

[0068] In step S420, a horizontal gravity vector corresponding to a horizontal camera pose is determined based on the current gravity vector.

[0069] In an exemplary embodiment, the horizontal gravity vector refers to the gravity acceleration output by the gravity sensor when the camera is in the horizontal camera pose. The horizontal gravity vector corresponding to the conversion of the camera from the current camera pose to the horizontal camera pose can be obtained by converting the current gravity vector. For example, assuming that the current gravity vector under the current camera pose is (g x ,g y ,g z ) and the horizontal camera pose is a camera pose parallel to the Y axis, the horizontal gravity vector corresponding to the horizontal camera pose can be represented as Of course, this is only an illustrative example and should not impose any special limitation on the present exemplary embodiment.

[0070] In step S430, a rotation axis angle relationship for converting the camera from the current camera pose to the horizontal camera pose is determined based on the current gravity vector and the horizontal gravity vector.

[0071] In an exemplary embodiment, the rotation axis angle relationship refers to the rotation relationship for converting the current camera pose to the horizontal camera pose. When transforming one coordinate system A to another coordinate system B, the coordinate system A can be rotated by an angle σ around a straight line axis, and this rotation expression is called the axis-angle representation. The rotation axis angle relationship can include a rotation angle and a rotation axis. Assuming that the rotation axis is a straight line axis and the rotation angle is an angle σ, the rotation axis angle relationship can be represented as (axis, σ), and the current camera pose can be converted to the horizontal camera pose through the rotation axis angle relationship.

[0072] In step S440, the current video image is horizontally corrected through the rotation axis angle relationship to obtain a horizontally corrected current video image.

[0073] In an exemplary embodiment, the horizontally corrected current video image refers to the video image corresponding to the conversion of the current camera pose corresponding to the current video image to the horizontal camera pose. When the current camera pose corresponding to the current video image is converted to the horizontal camera pose, the image content in the current video image will also change, and the corresponding video image under the horizontal camera pose, i.e., the horizontally corrected current video image, is obtained.

[0074] The steps S410 to S440 are described in detail as follows.

[0075] In an example embodiment, since the display content in the current image frame will also change when the current camera pose is converted to the horizontal camera pose, part of the content in the current video image after horizontal correction may exceed the image display boundary, and therefore, in order to ensure better image quality, the current video image after horizontal correction needs to be cropped.

[0076] Optionally, the output image size can be determined according to the image size of the current video image and the preset cropping ratio, and the current video image after horizontal correction is cropped at the center and output based on the output image size.

[0077] The image size of the current video image refers to data used to represent the size of the current video image, for example, the image size of the current video image can be the length and width corresponding to the current video image, or the diagonal length corresponding to the current video image, and of course, can also be other data capable of representing the size of the current video image, which is not specially limited in the example embodiment.

[0078] The output image size refers to data used to represent the size of the cropped video image, and accordingly, when the image size of the current video image is the length and width corresponding to the current video image, the output image size can be the length and width corresponding to the cropped video image, and when the image size of the current video image is the diagonal length corresponding to the current video image, the output image size can be the diagonal length corresponding to the cropped video image, which is not specially limited in the example embodiment.

[0079] For example, if the preset cropping ratio is 50%, the image size of the current video image is 10 in length and 6 in width, and then the output image size is 5 in length and 3 in width, and of course, the preset cropping ratio can also be 30% and the like, and the specific cropping ratio can be set according to actual application conditions, such as setting different cropping ratios for different inclination angles of terminal devices, which is not specially limited in the example embodiment.

[0080] In the example embodiment, since after the horizontal line correction of the current video image, only the content of the center part of the current video image after horizontal line correction can completely restore the display content in the current video image, and other parts may exceed the image display boundary, therefore, after the output image size is determined, the center of the cropping area consistent with the output image size is aligned with the center of the current video image after horizontal line correction, and the video image content in the cropping area is retained, that is, the center cropping of the current video image after horizontal line correction is implemented, and the integrity of the output video image is ensured.

[0081] Optionally, the video image after the center cutting can be scaled by a scaling ratio consistent with the cutting ratio, so that the size of the final output video image is consistent with the size of the current video image, ensuring the consistency of the size of the video image after the horizontal line correction, and improving the user experience.

[0082] In an example embodiment, the maximum correctable angle corresponding to the current video image can be calculated according to the image size of the current video image and the output image size. The output image size calculated by the image size of the current video image and the cutting ratio can automatically calculate the maximum correctable angle of the current video image, avoiding the problem of manually inputting the maximum correctable angle in the related art, improving the processing efficiency, and reducing the error caused by manual input.

[0083] Further, the maximum correctable angle corresponding to the current video image can be calculated according to the image size of the current video image and the output image size by the steps in Figure 5 , and reference is made to Figure 5 , which can specifically include:

[0084] Step S510, calculating a first included angle formed by the lower boundary of the current video image and the diagonal line of the cutting region in the current video image in a state without horizontal correction according to the image size of the current video image and the output image size.

[0085] Step S520, calculating a second included angle formed by the lower boundary of the current video image and the diagonal line of the cutting region in the current video image in a state with the horizontal correction at the maximum correction angle according to the image size of the current video image and the output image size.

[0086] Step S530, taking the difference between the first included angle and the second included angle as the maximum correctable angle corresponding to the current video image.

[0087] The state without horizontal correction refers to the ideal state that the current video image is in the horizontal direction and the cutting region corresponding to the output image size is in the horizontal direction. The cutting region in the current video image refers to the region that needs to be cut in the current video image based on the output image size.

[0088] The first included angle refers to the included angle formed by the lower boundary of the current video image and the diagonal line of the cutting region in the state without horizontal line correction, and the second included angle refers to the included angle formed by the lower boundary of the current video image and the diagonal line of the cutting region when the current video image and the cutting region are at the maximum correction angle in the state of horizontal line correction.

[0089] It should be noted that the "first" and "second" in the "first included angle" and "second included angle" here are only used to distinguish the included angle formed by the lower boundary of the current video image and the diagonal line of the region to be cut in different states, and do not have any other meanings, and should not cause any special limitations on the present example embodiment.

[0090] Figure 6 A schematic diagram illustrating the principle of determining the maximum correctable angle in an example embodiment of the present disclosure is schematically shown.

[0091] Reference Figure 6 As shown, it is assumed that the image size of the input current video image 601 is (W, H), and the output image size of the region to be cut (output image) 602 corresponding to the current video image 601 is (w, h). Then, when the current video image 601 is not corrected, i.e., the current video image 601 and the region to be cut 602 are in a state 603 without horizontal correction, the first included angle formed by the lower boundary of the current video image 601 (since the current video image 601 is in the state 603 without horizontal correction, the lower boundary of the current video image 601 is parallel to the lower boundary of the region to be cut 602, in order to facilitate description and understanding, the first included angle is calculated by the included angle between the lower boundary of the current video image 601 and the diagonal line in the present embodiment) and the diagonal line of the region to be cut 602 can be represented as relationship (1):

[0092]

[0093] Wherein, α can represent the angle value of the first included angle, BC can represent the opposite side of the first included angle in the right triangle ABC, i.e., the short side of the region to be cut 602, AC can represent the lower boundary of the region to be cut 602, i.e., the long side of the region to be cut 602, h can represent the length of BC, i.e., the width of the region to be cut 602, and w can represent the length of AC, i.e., the length of the region to be cut 602.

[0094] When the current video image 601 and the region to be cut 602 are horizontally corrected to the state 604 of the maximum correction angle, the second included angle formed by the lower boundary of the current video image 601 and the diagonal line of the region to be cut 602 can be represented as relationship (2):

[0095]

[0096] Wherein, β can represent the angle value of the second included angle, BD can represent the opposite side of the second included angle in the right triangle ABD, AB can represent the diagonal line of the region to be cut 602 (i.e., the lower boundary of the current video image 601), h can represent the width of the region to be cut 602, w can represent the width of the region to be cut 602, and H can represent the width of the current video image 601.

[0097] After the first angle and the second angle are calculated by the relationship (1) and the relationship (2), the difference between the first angle and the second angle can be taken as the maximum correctable angle corresponding to the current video image, assuming that the maximum correctable angle is θ max , then the maximum correctable angle can be expressed as θ max = β - α. Of course, the above is only an illustrative example and should not cause any special limitation to the present example embodiment.

[0098] In an example embodiment, the rotation axis angle relationship can include a rotation angle and a rotation axis, and when the tilt angle is less than or equal to the maximum correctable angle, the rotation axis angle relationship can be calculated by the steps in Figure 7 , and reference is made to Figure 7 , which can specifically include:

[0099] Step S710, calculating a tilt angle according to the current gravity vector and the horizontal gravity vector;

[0100] Step S720, if the tilt angle is less than or equal to the maximum correctable angle, taking the tilt angle as a rotation angle for converting from the current camera pose to the horizontal camera pose;

[0101] Step S730, performing cross multiplication calculation on the current gravity vector and the horizontal gravity vector to obtain a rotation axis for converting from the current camera pose to the horizontal camera pose.

[0102] The tilt angle refers to an angle needed for rotating the current camera pose corresponding to the current gravity vector to the horizontal camera pose corresponding to the horizontal gravity vector.

[0103] For example, assuming that the current time t i is g

[0104] i = (g x , g y , g z )

[0105] If it is determined that g x > g y , it can be considered that the terminal device is possibly horizontally lying, i.e., the Y axis is parallel to the horizontal line, and therefore, the horizontal gravity vector g′ i of the horizontal camera pose can be calculated based on the current gravity vector.

[0106]

[0107] It is easy to understand that if it is determined that g x > g y , it can be considered that the terminal device is possibly horizontally lying, i.e., the Y axis is parallel to the horizontal line, and therefore, the horizontal gravity vector g′ i of the horizontal camera pose can be calculated based on the current gravity vector.y >g x , it can be considered that the terminal device is likely to be held vertically, i.e., the X axis is parallel to the horizontal line, and thus a horizontal gravity vector g' of the horizontal camera pose is calculated based on the current gravity vector i The following can be obtained:

[0108]

[0109] In determining the horizontal gravity vector g' of the horizontal camera pose, the current gravity vector g and the horizontal gravity vector g' can be used. i Afterwards, the tilt angle can be calculated according to the current gravity vector and the horizontal gravity vector, and the calculation of the tilt angle can be represented as relation (3):

[0110]

[0111] wherein θ can represent the angle required for conversion from the current camera pose to the horizontal camera pose, i.e., the tilt angle, g i may represent the current gravity vector, g' i may represent the horizontal gravity vector, |g i may represent the length of the current gravity vector g i , |g' i may represent the length of the horizontal gravity vector g' i .

[0112] In the example embodiment, if the tilt angle is detected to be less than or equal to the maximum correctable angle, i.e., θ≤θ max , it can be considered that the current tilt angle is within the adjustable range, and the tilt angle is directly taken as the rotation angle for conversion from the current camera pose to the horizontal camera pose, i.e., letting the rotation angle σ i = θ.

[0113] Further, since the rotation conversion from the current camera pose to the horizontal camera pose is essentially a conversion of the component of the current gravity acceleration in the camera coordinate system from the current gravity vector g i to the horizontal gravity vector g' i , the rotation axis for conversion from the current camera pose to the horizontal camera pose can be obtained by calculating the cross product of the current gravity vector and the horizontal gravity vector.

[0114] For example, the rotation axis can be determined by relation (4):

[0115] axis i = g i × g' i (4)

[0116] wherein axis i may represent the rotation axis for conversion from the current camera pose to the horizontal camera pose, gi may represent the current gravity vector, g' i may represent the horizontal gravity vector.

[0117] Therefore, when the tilt angle is less than or equal to the maximum correctable angle, the rotation axis angle relationship can be represented as

[0118] In an exemplary embodiment, when the tilt angle is greater than the maximum correctable angle, the rotation axis angle relationship can be calculated by the steps in Figure 8 , as shown in Figure 8 , which can specifically include:

[0119] Step S810, if the tilt angle is greater than the maximum correctable angle, the maximum correctable angle is taken as the rotation angle from the current camera pose to the horizontal camera pose; and

[0120] Step S820, the target horizontal gravity vector is determined according to the current gravity vector and the maximum correctable angle, and cross product calculation is performed on the current gravity vector and the target horizontal gravity vector to obtain the rotation axis from the current camera pose to the horizontal camera pose.

[0121] The target horizontal gravity vector refers to the horizontal gravity vector re-determined when the tilt angle is greater than the maximum correctable angle, which can be determined by the current gravity vector and the maximum correctable angle.

[0122] For example, assuming that the current gravity vector g i =(g x ,g y ,g z ) and g x >g y , when the tilt angle is greater than the maximum correctable angle, i.e. θ>θ max , only the current video image is corrected by θ max , so it is necessary to re-calculate the horizontal gravity vector under the horizontal camera pose, which can be specifically calculated by the target horizontal gravity vector through the relationship (5):

[0123]

[0124] wherein g″ i may represent the target horizontal gravity vector, θ can represent the tilt angle, θ can be calculated by the relationship (3), and θ max may represent the maximum correctable angle, and θ max may be calculated by the relationship (1) and the relationship (2).

[0125] Similarly, the corresponding target horizontal gravity vector when g y >g x is calculated by formula (5), which will not be described here.

[0126] After the target horizontal gravity vector is calculated, the rotation axis from the current camera pose to the horizontal camera pose can be calculated by cross multiplication of the current gravity vector and the target horizontal gravity vector, for example, the determined rotation axis can be represented as:

[0127] axis i =g i ×g″ i

[0128] wherein g i may represent the current gravity vector, and g″ i may represent the target horizontal gravity vector.

[0129] Therefore, when the tilt angle is greater than the maximum correctable angle, the rotation axis angle relationship can be represented as (axis i =g i ×g″ i , σ i =θ max =β-α).

[0130] In an exemplary embodiment, after the rotation axis angle relationship is calculated, the current video image can be horizontally corrected by the steps in Figure 9 , which will be described with reference to Figure 9 , and specifically can include:

[0131] Step S910, converting and projecting the current video image in the two-dimensional coordinate system into the three-dimensional coordinate system based on a preset projection mode to obtain a set of projection coordinates;

[0132] Step S920, rotating the points in the set of projection coordinates according to the rotation axis angle relationship to obtain a set of horizontally corrected projection coordinates;

[0133] Step S930, de-projecting the set of horizontally corrected projection coordinates according to the projection mode to obtain a horizontally corrected current video image.

[0134] The preset projection mode refers to a processing mode for converting and projecting the pixel coordinates of the current video image in the two-dimensional coordinate system into the three-dimensional coordinate system, for example, the preset projection mode can be a camera pinhole imaging model or a coordinate conversion matrix, which is not specially limited in the present example embodiment.

[0135] For example, a 2D plane image can be projected onto a 3D sphere by a preset projection mode, such as by a camera pinhole imaging model. Assuming that the pixel point coordinates on the 2D plane image captured by the camera are (x, y), the corresponding pixel point coordinates on the 3D sphere obtained by reverse projection by the camera pinhole imaging model are (X, Y, Z), that is, a projection coordinate set. Then, the pixel point coordinates (X, Y, Z) on the 3D sphere are rotated to obtain new pixel point coordinates (X', Y', Z') by using the rotation axis angle relationship. Finally, the pixel point coordinates (X', Y', Z') on the 3D sphere obtained by rotation are projected back to a 2D plane by the camera pinhole imaging model to obtain (x', y'). Each pixel point on the current video image is rotated to obtain a new video image, that is, a horizontally corrected current video image, and then the horizontally corrected current video image is center cropped according to a cropping ratio and output.

[0136] Figure 10 A schematic diagram of a video image that needs to be horizontally corrected in an example embodiment of the present disclosure is schematically shown.

[0137] Reference is made to Figure 10 As shown, for a real scene 1001, a video image 1003 is captured by a camera 1002 in a horizontal camera pose at a current time, and the image content in the video image 1003 is parallel to the horizontal line. For the real scene 1001, a video image 1005 is captured by a camera 1004 in a non-horizontal camera pose (a tilted camera pose) at a current time, and the image content in the video image 1005 is not parallel to the horizontal line. The video image can be horizontally corrected by the video image correction method in the example embodiment to obtain a video image similar to the image content in the video image 1003.

[0138] Figure 11 A schematic diagram of a principle of implementing horizontal line correction of a current video image in an example embodiment of the present disclosure is schematically shown.

[0139] Reference is made to Figure 10 As shown, in step S1010, pixel point coordinates (x, y) on a current video image 1101 are projected onto a 3D sphere 1102 to obtain pixel point coordinates (X, Y, Z) by a preset projection mode, such as by a camera pinhole imaging model.

[0140] In step S1020, the pixel point coordinates (X, Y, Z) on the 3D sphere 1102 are rotated to obtain new pixel point coordinates (X', Y', Z') on a 3D sphere 1103 by using the calculated rotation axis angle relationship.

[0141] Step S1030, the new pixel point coordinate (X', Y', Z') on the 3D sphere 1103 is inversely projected to obtain a pixel point coordinate (x', y') according to a preset projection mode such as a camera pinhole imaging model, and a current video image after horizontal line correction is obtained through the new pixel point coordinate (x', y'), and the current video image after horizontal line correction is center cropped according to a cropping ratio to obtain a final output video image 1104.

[0142] Figure 12 An exemplary flowchart of the current video image horizontal correction output in the exemplary embodiment of the present disclosure is schematically shown.

[0143] Reference Figure 12 As shown, step S1210, reading the gravity sensor reading at the current time, i.e., the current gravity vector;

[0144] Step S1220, calculating the horizontal gravity vector of the horizontal camera pose based on the current gravity vector;

[0145] Step S1230, calculating the rotation axis angle relationship from the current gravity vector to the horizontal gravity vector;

[0146] Step S1240, rotating the current video image at the current time through the rotation axis angle relationship to obtain a current video image after horizontal line correction, and outputting after center cropping, ending the horizontal line correction of the current video image, and continuing the horizontal line correction of the next time video image.

[0147] In summary, in the exemplary embodiment, the current video image captured in the current camera pose can be obtained, and the current gravity vector corresponding to the current camera pose can be obtained, then the horizontal gravity vector corresponding to the horizontal camera pose is determined based on the current gravity vector, and then the rotation axis angle relationship from the current camera pose to the horizontal camera pose can be determined according to the current gravity vector and the horizontal gravity vector, and the current video image is horizontally corrected through the rotation axis angle relationship to obtain the current video image after horizontal correction. On the one hand, the rotation axis angle relationship is determined according to the current gravity vector and the horizontal gravity vector corresponding to the current camera pose, and the current video image is horizontally corrected through the rotation axis angle relationship, compared with the scheme of determining the horizontal direction through the key points in the image content and horizontally correcting, the accuracy of the current gravity vector detected by the gravity sensor is higher, which effectively improves the accuracy of the horizontal correction result, at the same time, the horizontal correction result is only associated with the current gravity vector, without the interference of other error data, which guarantees the robustness of the horizontal correction result; on the other hand, the current video image is horizontally corrected through the detected current gravity vector, which can be applied to various types of scenes, has a wide range of applications, and has a low time complexity compared with the way of calculating the horizontal direction according to the image content, which can better adapt to real-time processing systems.

[0148] It should be noted that the above-described figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above-described figures do not indicate or limit the time sequence of the processes. In addition, it is also easy to understand that the processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0149] Further, referring to Figure 13 In the embodiment of the present example, a video image correction apparatus 1300 is also provided, which can include a video image acquisition module 1310, a gravity vector determination module 1320, a rotation axis angle relationship determination module 1330, and a horizontal correction module 1340. Wherein:

[0150] The video image acquisition module 1310 is configured to acquire a current video image taken at a current camera pose, and acquire a current gravity vector corresponding to the current camera pose;

[0151] The gravity vector determination module 1320 is configured to determine a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector;

[0152] The rotation axis angle relationship determination module 1330 is configured to determine a rotation axis angle relationship for converting from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector;

[0153] The horizontal correction module 1340 is configured to perform horizontal correction on the current video image by the rotation axis angle relationship to obtain a horizontally corrected current video image.

[0154] In an exemplary embodiment, the video image correction apparatus 1300 can include a cropping unit, which can be configured to:

[0155] determine an output image size according to an image size of the current video image and a preset cropping ratio;

[0156] perform center cropping on the horizontally corrected current video image based on the output image size and output.

[0157] In an exemplary embodiment, the video image correction apparatus 1300 can include a maximum correctable angle determination unit, which can be configured to:

[0158] calculate a maximum correctable angle corresponding to the current video image according to an image size of the current video image and the output image size.

[0159] In an exemplary embodiment, the maximum correctable angle determination unit can be configured to:

[0160] According to the image size of the current video image and the output image size, a first included angle between a lower boundary of the current video image and a diagonal line of the region to be cropped in the current video image is calculated in a state where no horizontal correction is performed;

[0161] According to the image size of the current video image and the output image size, a second included angle between the lower boundary of the current video image and the diagonal line of the region to be cropped in the current video image is calculated in a state where the horizontal correction is performed at a maximum correction angle;

[0162] A difference between the first included angle and the second included angle is taken as a maximum correctable angle corresponding to the current video image.

[0163] In an example embodiment, the rotation axis-angle relationship can include a rotation angle and a rotation axis, and the rotation axis-angle relationship determining module 1330 can be configured to:

[0164] According to the current gravity vector and the horizontal gravity vector, a tilt angle is calculated;

[0165] If the tilt angle is less than or equal to the maximum correctable angle, the tilt angle is taken as a rotation angle for converting from the current camera pose to the horizontal camera pose.

[0166] The current gravity vector and the horizontal gravity vector are cross-multiplied to obtain a rotation axis for converting from the current camera pose to the horizontal camera pose.

[0167] In an example embodiment, the rotation axis-angle relationship determining module 1330 can be configured to:

[0168] If the tilt angle is greater than the maximum correctable angle, the maximum correctable angle is taken as a rotation angle for converting from the current camera pose to the horizontal camera pose; and

[0169] According to the current gravity vector and the maximum correctable angle, a target horizontal gravity vector is determined, and the current gravity vector and the target horizontal gravity vector are cross-multiplied to obtain a rotation axis for converting from the current camera pose to the horizontal camera pose.

[0170] In an example embodiment, the horizontal correction module 1340 can be configured to:

[0171] According to a preset projection mode, the current video image in a two-dimensional coordinate system is projected into a three-dimensional coordinate system to obtain a set of projection coordinates,

[0172] According to the rotation axis-angle relationship, points in the set of projection coordinates are rotated to obtain a set of horizontally corrected projection coordinates.

[0173] The horizontal corrected projection coordinate set is inverse projected according to the projection mode, to obtain a horizontal corrected current video image.

[0174] The specific details of each module in the above device have been described in detail in the method part of the embodiments, and the undisclosed details can refer to the embodiment content of the method part, and thus will not be described again.

[0175] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0176] The exemplary embodiments of the present disclosure also provide a computer readable storage medium having stored thereon a program product capable of implementing the above-mentioned method of the present disclosure. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing the terminal device to perform the steps described in the above "exemplary method" section according to various exemplary embodiments of the present disclosure when the program product is run on the terminal device.

[0177] It should be noted that the computer readable medium shown in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0178] In this disclosure, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for use by or in connection with an instruction execution system, apparatus, or device. The computer readable program code can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of these. A computer readable medium can include any medium that can be read by a computer. Examples of computer readable mediums include but are not limited to memory, optical disc, etc.

[0179] Further, the program code for implementing the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0180] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure including those variations that can be incorporated into the above detailed description and making use of the general principles of the present disclosure. It is intended that the present disclosure include all such as fall within the scope of the appended claims and their equivalents.

[0181] It is to be understood that the present disclosure is not limited to the precise construction described and as shown in the drawings, and that various modifications and changes can be effected therein by those skilled in the art without departing from the scope of the disclosure. The scope of the disclosure is to be limited only by the appended claims.

Claims

1. A method of video image rectification, characterized by, The method comprises: obtaining a current video image captured at a current camera pose, and obtaining a current gravity vector corresponding to the current camera pose; determining a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector; determining a rotation axis angle relationship from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector; performing horizontal correction on the current video image through the rotation axis angle relationship to obtain a horizontally corrected current video image, comprising: projecting the current video image in a two-dimensional coordinate system into a three-dimensional coordinate system based on a preset projection mode to obtain a projection coordinate set, rotating points in the projection coordinate set according to the rotation axis angle relationship to obtain a horizontally corrected projection coordinate set; and de-projecting the horizontally corrected projection coordinate set according to the projection mode to obtain the horizontally corrected current video image.

2. The method of claim 1, wherein, The method further comprises: determining an output image size according to an image size of the current video image and a preset cropping ratio; and performing center cropping on the horizontally corrected current video image based on the output image size and outputting the horizontally corrected current video image.

3. The method of claim 2, wherein, The method further comprises: calculating a maximum correctable angle corresponding to the current video image according to the image size of the current video image and the output image size.

4. The method of claim 3, wherein, The method further comprises: calculating a first included angle between a lower boundary of the current video image and a diagonal line of a region to be cropped in the current video image in a state without horizontal correction according to the image size of the current video image and the output image size; calculating a second included angle between the lower boundary of the current video image and the diagonal line of the region to be cropped in the current video image in a state with horizontal correction at a maximum correction angle according to the image size of the current video image and the output image size; and taking a difference between the first included angle and the second included angle as the maximum correctable angle corresponding to the current video image.

5. The method of claim 1, wherein, The rotation axis angle relationship comprises a rotation angle and a rotation axis, and the method of determining the rotation axis angle relationship from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector comprises: calculating an inclination angle according to the current gravity vector and the horizontal gravity vector; if the inclination angle is less than or equal to a maximum correctable angle, taking the inclination angle as the rotation angle from the current camera pose to the horizontal camera pose; performing cross multiplication calculation on the current gravity vector and the horizontal gravity vector to obtain the rotation axis from the current camera pose to the horizontal camera pose.

6. The method of claim 5, wherein, The method of determining the rotation axis angle relationship from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector further comprises: if the inclination angle is greater than the maximum correctable angle, taking the maximum correctable angle as the rotation angle from the current camera pose to the horizontal camera pose; and A target horizontal gravity vector is determined according to the current gravity vector and the maximum correctable angle, and a cross product calculation is performed on the current gravity vector and the target horizontal gravity vector to obtain a rotation axis for converting from the current camera pose to the horizontal camera pose.

7. A video image correction apparatus characterized by comprising: The method comprises the steps of: acquiring a current video image captured in a current camera pose and a current gravity vector corresponding to the current camera pose; determining a horizontal gravity vector corresponding to a horizontal camera pose based on the current gravity vector; determining a rotation axis angle relationship for converting from the current camera pose to the horizontal camera pose according to the current gravity vector and the horizontal gravity vector; performing horizontal correction on the current video image through the rotation axis angle relationship to obtain a horizontally corrected current video image, comprising: converting and projecting the current video image in a two-dimensional coordinate system into a three-dimensional coordinate system to obtain a projection coordinate set based on a preset projection mode, rotating points in the projection coordinate set according to the rotation axis angle relationship to obtain a horizontally corrected projection coordinate set, and inversely projecting the horizontally corrected projection coordinate set according to the projection mode to obtain the horizontally corrected current video image.

8. A computer readable medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the method of any one of claims 1 to 6.

9. An electronic device, comprising: The method comprises the steps of: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method of any one of claims 1 to 6 by executing the executable instructions. The computer program, when executed by a processor, implements the method of any one of claims 1 to 6.

Images