Image direction correction method and device, electronic equipment and storage medium

Abstract

The invention relates to an image direction correction method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first native direction of a first display screen in calling end equipment and a second native direction of a second display screen in called end equipment; determining a first sensing direction of a first camera in the calling end equipment and a second sensing direction of a second camera in the called end equipment; obtaining a correction angle based on the first native direction, the second native direction, the first sensing direction and the second sensing direction; and rotating the direction of the first image obtained by the second camera in the current direction by the correction angle, so that the second image obtained after rotation is consistent with the direction of a third image obtained by the first camera in the current direction. Therefore, after the second image is transmitted to the calling end equipment, the image pictures displayed on the first display screen are all forward pictures, so that the user experience is greatly improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer communications, and in particular to an image direction correction method, device, electronic equipment and storage medium. Background Art

[0002] In a multi-device networking environment, the need for users to use hardware such as cameras on other devices on one device is becoming more and more common; for example, the camera on the called device (such as a mobile phone) is directly called on the calling device (such as a tablet or laptop) to achieve remote image transmission. However, since the installation direction (Camera Sensor Orientation) of the camera image sensors on the calling device and the called device is not fixed or uniform, the direction of the image obtained by the called device is different from that of the calling device, so that after the image obtained by the called device is transmitted to the calling device, the image displayed on the display screen of the calling device may be non-positive, requiring the user to manually adjust the image transmitted by the called device, thereby affecting the user experience. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides an image direction correction method, device, electronic device and storage medium. The correction process can automatically adjust the image direction on the calling end device or the called end device without the user manually adjusting the called end device, which greatly improves the usability and user experience of the function.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for image direction correction is provided, the method comprising:

[0005] Determine a first native orientation of a first display screen in a calling device and a second native orientation of a second display screen in a called device;

[0006] Determine a first sensing direction of a first camera in the calling end device and a second sensing direction of a second camera in the called end device;

[0007] Obtaining a correction angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction;

[0008] The direction of the first image acquired by the second camera in the current direction is rotated by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction.

[0009] In some embodiments, when the current direction of the first camera is the first sensing direction and the current direction of the second camera is the second sensing direction, obtaining the corrected angle based on the first native direction, the second native direction, the first sensing direction and the second sensing direction includes:

[0010] Obtaining a first sensing angle based on an angle between the first sensing direction and the first native direction;

[0011] Obtaining a second sensing angle based on an angle between the second sensing direction and the second native direction;

[0012] The correction angle=(the second sensing angle-the first sensing angle+360°) / 360 modulo.

[0013] In some embodiments, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, the correction angle=(the second sensing angle-the first sensing angle-the first angle+360°) / 360 modulo.

[0014] In some embodiments, when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle=(the second sensing angle-the first sensing angle+the second angle+360°) / 360 modulo.

[0015] In some embodiments, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, and when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle = (the second sensing angle - the first sensing angle - the first angle + the second angle + 360°) / 360 modulo.

[0016] In some embodiments, the method further includes: cropping the second image so that a fourth image obtained after cropping has the same image ratio as the third image.

[0017] In some embodiments, the method further includes: scaling the fourth image so that a size of a fifth image obtained after the scaling is the same as a size of the third image.

[0018] In some embodiments, the method further includes: establishing a network connection between the calling end device and the called end device to implement camera virtualization processing between the first camera and the second camera; and

[0019] The fifth image is sent to the calling end device so as to display the fifth image in a positive direction on the first display screen.

[0020] According to a second aspect of an embodiment of the present disclosure, there is provided an image direction correction device, the device comprising:

[0021] A first determining module is configured to determine a first native orientation of a first display screen in a calling end device and a second native orientation of a second display screen in a called end device;

[0022] A second determining module is configured to determine a first sensing direction of a first camera in the calling end device and a second sensing direction of a second camera in the called end device;

[0023] an acquisition module, configured to obtain a correction angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction;

[0024] The rotation module is configured to rotate the direction of the first image acquired by the second camera in the current direction by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction.

[0025] In some embodiments, when the current direction of the first camera is the first sensing direction, and the current direction of the second camera is the second sensing direction, the acquisition module is specifically configured as follows:

[0026] Obtaining a first sensing angle based on an angle between the first sensing direction and the first native direction;

[0027] Obtaining a second sensing angle based on an angle between the second sensing direction and the second native direction;

[0028] The correction angle=(the second sensing angle-the first sensing angle+360°) / 360 modulo.

[0029] In some embodiments, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, the correction angle=(the second sensing angle-the first sensing angle-the first angle+360°) / 360 modulo.

[0030] In some embodiments, when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle=(the second sensing angle-the first sensing angle+the second angle+360°) / 360 modulo.

[0031] In some embodiments, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, and when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle = (the second sensing angle - the first sensing angle - the first angle + the second angle + 360°) / 360 modulo.

[0032] In some embodiments, the device further includes: a cropping module configured to crop the second image so that a fourth image obtained after cropping has the same image ratio as the third image.

[0033] In some embodiments, the device further includes: a scaling processing module configured to perform scaling processing on the fourth image so that the size of the fifth image obtained after the scaling processing is the same as the size of the third image.

[0034] In some embodiments, the apparatus further includes: a network connection module configured to establish a network connection between the calling end device and the called end device to implement camera virtualization processing between the first camera and the second camera; and

[0035] The fifth image is sent to the calling end device so as to display the fifth image in a positive direction on the first display screen.

[0036] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the method described in the above embodiment of the present disclosure.

[0037] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method described in the above embodiment of the present disclosure.

[0038] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0039] In the embodiment of the present disclosure, by determining the first native direction of the first display screen in the calling end device and determining the second native direction of the second display screen in the called end device; determining the first sensing direction of the first camera in the calling end device and determining the second sensing direction of the second camera in the called end device; and based on the first native direction, the second native direction, the first sensing direction and the second sensing direction, obtaining a correction angle; rotating the direction of the first image acquired by the second camera in the current direction by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction. Through the embodiment of the present disclosure, at least when the direction of the first image acquired by the second camera in the current direction is inconsistent with the direction of the third image acquired by the first camera in the current direction, the direction of the first image acquired by the second camera in the current direction can be rotated so that the direction of the second image obtained after the rotation is consistent with the direction of the third image.

[0040] Furthermore, after the calling device calls the camera of the called device, when the calling device or the called device is rotated at any angle, it can be ensured that the image from the camera of the called device displayed on the display screen of the calling device is always positive; and there is no need for the user to manually adjust the placement direction of the display screen, thereby improving the user experience.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] Figure 1 is a schematic diagram of a native orientation of a display screen according to an exemplary embodiment;

[0044] Figure 2 is a schematic diagram of a sensing direction of a camera according to an exemplary embodiment;

[0045] Figure 3 is a schematic diagram of an image rotation process when a mobile phone is a rear camera according to an exemplary embodiment;

[0046] Figure 4 is a schematic diagram of an image rotation process when a mobile phone is a front camera according to an exemplary embodiment;

[0047] Figure 5 is a schematic diagram showing images acquired by displaying screens of a calling end device and a called end device respectively before starting a three-party application according to an exemplary embodiment;

[0048] Figure 6 is a schematic diagram showing images acquired by respective display screens of a calling end device and a called end device after a three-party application is started according to an exemplary embodiment;

[0049] Figure 7 is a flow chart of an image direction correction method according to an exemplary embodiment;

[0050] Figure 8 is a schematic diagram showing a second image result displayed on a first display screen and a second display screen according to an exemplary embodiment;

[0051] Fig. 9 is a schematic diagram showing image ratios of various second images and third images according to an exemplary embodiment;

[0052] Fig.10 is a structural block diagram of an image direction correction device according to an exemplary embodiment;

[0053] Fig.11 The present invention is a block diagram of an electronic device including an image direction correction device according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0055] It should be noted that the device natural orientation (DNO) described in the embodiments of the present disclosure is the natural orientation or typical orientation of the display screen in the device (keeping the display screen facing the user); for example, refer to Figure 1 , Figure 1 is a schematic diagram of the native orientation of a display screen according to an exemplary embodiment; Figure 1 As shown, the native orientation of the display screen of the mobile phone 101 is portrait, that is, the height H1 of the display screen is greater than the width L1 of the display screen; the native orientation of the display screen of the laptop computer 102 is landscape, that is, the width L2 of the display screen is greater than the height H2 of the display screen. Figure 1The display screens of the mobile phone 101 and the laptop computer 102 shown in the figure are placed in their native orientations. Generally, the native orientation of the device display screen is one of landscape or portrait, and the native orientation of the device display screen does not change due to changes in the device placement or holding angle.

[0056] It should be noted that the device current orientation (Device Orientation, DO) mentioned in the embodiments of the present disclosure is the direction of the display screen of the current device (keeping the display screen facing the user) after rotation relative to its native orientation, and the rotation amplitude is usually indicated by the numerical value of the rotation angle; among which, the device current orientation will change with the change of the device placement or holding angle.

[0057] Exemplarily, when the mobile phone display screen (keeping the display screen facing the user) is placed in its native orientation, the current orientation of the mobile phone device is vertical, and the current orientation of the mobile phone device is rotated 0 degrees or 180 degrees relative to its native orientation; when the mobile phone display screen (keeping the display screen facing the user) is rotated to landscape mode, the current orientation of the mobile phone device is horizontal, that is, the current orientation of the mobile phone device is rotated 90 degrees or 270 degrees relative to its native orientation. Among them, when the mobile phone display screen (keeping the display screen facing the user) is in landscape mode after being rotated clockwise, the rotation angle of the mobile phone is 90 degrees; when the mobile phone display screen is in landscape mode after being rotated counterclockwise, the rotation angle value of the mobile phone is 270 (taking the supplementary angle of 360 degrees). In actual use, the rotation angle of the device display screen is usually one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0058] It should be noted that the sensing direction of the camera described in the embodiments of the present disclosure is the installation direction of the camera image sensor (Camera Sensor Orientation, CSO). It should be noted that the camera image sensor uses the photoelectric conversion function of the photoelectric device to convert the light image on the photosensitive surface into an electrical signal that is proportional to the light image. For example, refer to Figure 2 , Figure 2 is a schematic diagram of the sensing direction of a camera according to an exemplary embodiment; wherein, Figure 2 The placement direction 203 of the camera image sensor shown is the native direction of the camera image sensor, that is, the native direction of the camera; usually, the native direction 203 of the camera is consistent with the native direction of the device display screen. The sensing direction of the camera is the setting direction when the long side direction of the camera image sensor is aligned with the long side direction of the device display screen; in other words, no matter whether the native direction of the camera in the device is horizontal or vertical, the sensing direction of the camera is determined by aligning the length direction of the camera with the length direction of the screen; and the sensing direction of the camera does not change due to changes in the current placement or holding angle of the device. Exemplary, refer to Figure 2 , the sensing direction of the calling end device 202 is horizontal; the sensing direction of the called end device 201 is vertical.

[0059] It should be noted that the angle between the camera's sensing direction and the device's display screen's native direction is the sensing angle. When the sensing angle is 0 degrees, it indicates that the camera's image sensor is installed in the same direction as the display screen's native direction; when the sensing angle is 90 degrees, it indicates that the camera's image sensor is installed 90 degrees away from the display screen's native direction. For example, refer to Figure 2 , the native orientation of the display screen of the calling end device 202 (such as a laptop) is horizontal, the camera is a rear camera, and the sensing angle is 0 degrees, that is, the sensing direction of the camera is 0 degrees counterclockwise compared to the native orientation of the display screen. Figure 2 The native orientation of the display screen of the called device 201 (such as a mobile phone) is vertical, the camera is a rear camera, and the sensing angle is 90 degrees, that is, the sensing direction of the camera is rotated 90 degrees counterclockwise compared to the native direction of the display screen.

[0060] It should be understood that since the camera image sensor usually outputs data in a horizontal aspect ratio (for example, a horizontal aspect ratio of 4:3 or 16:9), for the called end device 201 whose native orientation of the device display screen is vertical, the angle of rotation of the sensing direction of its camera compared to the native orientation of the display screen is 90 degrees or 270 degrees; for the calling end device 202 whose native orientation of the device display screen is horizontal, the angle of rotation of the sensing direction of its camera compared to the native orientation of the display screen is 0 degrees or 180 degrees. In actual use, the sensing direction of the device camera is usually rotated by one of 0 degrees, 90 degrees, 180 degrees, and 270 degrees compared to the native orientation of the display screen.

[0061] Based on the above content, it can be known that after the device camera acquires an image, the acquired image can be rotated by a certain angle and then displayed in the positive direction on the device display screen. The rotation angle can be determined based on the camera's sensing direction, the display screen's native direction, and the device's current direction. Figure 3 and Figure 4 A detailed description is given of how to rotate the image acquired by the camera so that the image can be displayed forward on the display screen, wherein: Figure 3 is a schematic diagram of an image rotation process when a mobile phone is a rear camera according to an exemplary embodiment; Figure 4It is a schematic diagram of the image rotation process when the mobile phone is a front camera according to an exemplary embodiment. In addition, the setting position of the camera of the device (such as the front camera, the rear camera) is different, and the rotation angle is different. Among them, when the camera is a rear camera, the rotation angle = sensor angle + the angle of rotation of the current direction of the device compared to the native direction of the display screen; when the camera is a front camera, the rotation angle = (sensor angle - the angle of rotation of the current direction of the device compared to the native direction of the display screen + 360 degrees) / 360 modulo.

[0062] Exemplary, reference Figure 3 , Figure 3 The device shown (such as mobile phone 301) is a rear camera, the native orientation of the display screen of mobile phone 301 is vertical, the sensing direction of mobile phone 301 is rotated 90 degrees counterclockwise compared to the native orientation of the display screen of mobile phone 301 (i.e., sensing angle = 90 degrees), and the current orientation of mobile phone 301 is rotated 90 degrees clockwise compared to the native orientation of the display screen of mobile phone 301 (i.e., the rotation angle of the current orientation of the device = 90 degrees). At this time, after rotating the image 302 acquired by the camera of mobile phone 301 clockwise (sensing angle + rotation angle of the current orientation of the device), an image 303 with a positive image can be obtained on the display screen of mobile phone 301. Here, sensing angle + rotation angle of the current orientation of the device = 90 degrees + 90 degrees = 180 degrees.

[0063] Exemplary, reference Figure 4 , Figure 4 The device shown (such as mobile phone 401) is a front camera, the native orientation of the display screen of mobile phone 401 is vertical, the sensing direction of mobile phone 401 is rotated 270 degrees counterclockwise compared to the native orientation of the display screen of mobile phone 401 (i.e., sensing angle = 270 degrees), and the current orientation of the device of mobile phone 401 is rotated 90 degrees clockwise compared to the native orientation of the display screen of mobile phone 401 (i.e., the rotation angle of the current orientation of the device = 90 degrees). At this time, after rotating the image 402 acquired by the camera of mobile phone 401 clockwise (sensing angle - rotation angle of the current orientation of the device), an image 403 with a positive image can be obtained on the display screen of mobile phone 401. Here, sensing angle - rotation angle of the current orientation of the device = 270 degrees - 90 degrees = 180 degrees. In addition, it should be noted that in some embodiments, since the current device is a front camera, in order to achieve the "mirror" mode effect, the current image 403 is usually horizontally mirrored 404 after the rotation operation to obtain a mirror image 405.

[0064] Return to reference Figure 2, the current orientation of the display screen of the calling end device (such as a laptop) 202 has not been rotated, that is, the rotation angle of the current orientation of the device = 0 degrees, and the corresponding sensing angle of the laptop is 0 degrees (that is, the sensing direction of the camera in the laptop is 0 degrees counterclockwise relative to the native orientation of the laptop display screen). Based on this, the image obtained by the laptop through its camera can be displayed on the laptop display screen as a positive image without rotation. Continue to refer to Figure 2 , the current direction of the device of the called end device (mobile phone) 201 display screen has not been rotated, that is, the rotation angle of the current direction of the mobile phone's device = 0 degrees. The corresponding sensing angle of the mobile phone is 90 degrees (that is, the sensing direction of the mobile phone camera is rotated 90 degrees counterclockwise relative to the native direction of the mobile phone display screen). Based on this, the image obtained by the mobile phone through its camera can be displayed on the mobile phone display screen as a positive image after it is rotated 90 degrees clockwise. In other words, the image obtained by the mobile phone's camera determines the rotation angle according to the sensing direction of the mobile phone and the current direction of the device. After the image is rotated, a positive image can be displayed on the mobile phone display screen.

[0065] As mentioned above, in a multi-device networking environment, the need for users to use hardware such as cameras on other devices on one device is becoming more and more common. When the calling device directly calls the camera on the called device to achieve remote image transmission, how the image is rotated is explained below with an example.

[0066] In some embodiments, a network connection is established between the calling device and the called device to implement camera virtualization processing between the first camera and the second camera.

[0067] refer to Figure 5 , Figure 5 FIG. 1 is a schematic diagram showing images captured by the display screens of the calling end device and the called end device before the third-party application is started according to an exemplary embodiment. Figure 5 As shown, the first camera of the calling end device 502 is a rear camera, the first native orientation of the first display screen is horizontal, the current orientation of the device of the first display screen is horizontal, and the image of the first scene acquired by the first camera in the current orientation is displayed as a positive orientation on the first display screen. Figure 5As shown, the second camera of the called end device 501 is a rear camera, the second native direction of the second display screen is vertical, the current direction of the device of the second display screen is vertical, and the image of the second scene acquired by the second camera in the current direction is displayed as a positive direction on the second display screen. Here, the current directions of the first camera and the second camera are both the sensing directions of their cameras; wherein the network connection can be based on a network connection such as a local area network and the Internet. The calling end device and the called end device can be applicable to the Android operating system, the Windows operating system, the Linux operating system, etc., without limitation.

[0068] refer to Figure 6 , Figure 6 The figure is a schematic diagram showing that the calling end device and the called end device each display screen and obtain images after starting a three-party application according to an exemplary embodiment. At this time, the camera used in the three-party application is the first camera on the calling end device, that is, the local camera; here, the three-party application is, for example, a video conferencing application. Figure 6 As shown, after the third-party application is enabled, the transmission of instructions and data between the calling device 602 and the called device 601 can be realized through the control instruction network channel and the image data network channel, so as to realize the camera virtualization processing between the first camera in the calling device and the second camera in the called device; here, the camera virtualization processing can be understood as the calling device calling the second camera of the called device.

[0069] It should be noted that when the native orientation, sensor orientation, and current orientation of the display screens of the calling device and the called device are all consistent, the direction of the image acquired by the camera of the called device is consistent with the direction of the image acquired by the camera of the calling device. In this way, after the calling device calls the camera of the called device, the camera of the called device can directly display the image in the positive direction on the display screen of the calling device after acquiring the image.

[0070] In some embodiments, when the native orientation, sensing orientation, and current orientation of the display screens of the calling device and the called device are inconsistent, the user needs to manually rotate the called device, and while rotating the called device, the user needs to observe the orientation of the image displayed on the display screen of the calling device, and then select a suitable orientation from the four orientations corresponding to portrait, landscape, reverse portrait, and reverse landscape to place the called device. Furthermore, if the user subsequently wants to change the placement orientation of the calling device, the user needs to adjust the called device again, resulting in a poor user experience.

[0071] In other embodiments, when the native orientation, sensing orientation, and current orientation of the display screens of the calling device and the called device are inconsistent, a third-party application is used to solve the image rotation during the image transmission process; illustratively, during the image transmission process, after the camera of the called device acquires the image, it is transmitted to the third-party application, and the third-party application rotates the image and sends it to the calling device to display it in the forward direction on the display screen of the calling device. However, during the image transmission process of the third-party application, the software development kit (SDK) provided by the system is required to realize this function; the process is complicated and the user experience is poor.

[0072] Based on this, the embodiment of the present disclosure provides an image direction correction method, which can directly display the image in the positive direction on the display screen of the calling end device after the calling end device directly calls the camera on the called end device; the image can be adaptively rotated without the need for manual adjustment by the user and without access to the SDK, thereby ensuring that the user always sees the image from the camera of the called end device in the positive direction, thereby improving the user experience. Figure 7 , Figure 8 , the image direction correction method is described in detail.

[0073] Figure 7 is a flow chart of an image direction correction method according to an exemplary embodiment. Figure 7 As shown, the method mainly includes the following steps:

[0074] In step 701, a first native orientation of a first display screen in a calling device and a second native orientation of a second display screen in a called device are determined;

[0075] In step 702, a first sensing direction of a first camera in the calling device and a second sensing direction of a second camera in the called device are determined;

[0076] In step 703, a correction angle is obtained based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction;

[0077] In step 704, the direction of the first image acquired by the second camera in the current direction is rotated by the correction angle so that the second image obtained after the rotation has the same direction as the third image acquired by the first camera in the current direction.

[0078] It should be noted that the image direction correction method proposed in the present disclosure can be applied to a recommendation system, which may include an electronic device. Here, the electronic device may include a calling end device and a called end device; the calling end device and the called end device may both include: terminal devices, for example, mobile terminals or fixed terminals. Among them, mobile terminals may include: mobile phones, tablet computers, laptop computers, wearable electronic devices and other devices. Fixed terminals may include: desktop computers, smart TVs, vehicle-mounted devices, etc. In some embodiments, the image direction correction method can also be applied to applications installed on electronic devices. It should be noted that here and below, the calling end device is a laptop computer and the called end device is a mobile phone as an example for explanation, and the cameras of the calling end device and the called end device are both rear cameras.

[0079] In other embodiments, the image direction correction method in the embodiment of the present disclosure may be configured in an image direction correction device, and the image direction correction device may be set in a calling end device or a called end device, and the embodiment of the present disclosure does not limit this. It should be noted that the execution subject of the embodiment of the present disclosure may be a central processing unit (CPU) in an electronic device in terms of hardware, and may be a related background service in an electronic device in terms of software, and there is no limitation on this.

[0080] In some embodiments, the image direction correction method disclosed in the present invention can be used to implement the image direction correction function in different image transmission scenarios. For example, the image correction function during image transmission between the calling end device and the called end device can be implemented based on the virtual camera function of the local area network or the Internet. Here, the calling end device and the called end device can be different electronic devices located at different locations; illustratively, the calling end device is located at the local end, and the called device is located at the remote end, and the present disclosure does not limit this.

[0081] In step 701, the calling end device is a laptop computer, the first display screen is the display screen of the laptop computer, the native orientation of the first display screen is landscape, and the first native orientation is landscape. The called end device is a mobile phone, the second display screen is the display screen of the mobile phone, and the native orientation of the second display screen is portrait, and the second native orientation is portrait.

[0082] In step 702, the calling end device is a laptop computer, and the first sensing direction of the first camera is horizontal; the called end device is a mobile phone, and the second sensing direction of the second camera is vertical. Here, the angle between the first sensing direction and the first native direction is the first sensing angle, and the angle between the second sensing direction and the second native direction is the second sensing angle. For example, refer to Figure 2 , Figure 2The laptop computer 202 shown in the figure is a rear camera. In the calling end device (such as the laptop computer 202), the first sensing angle is 0 degrees, that is, the first sensing direction of the first camera in the calling end device (such as the laptop computer 202) is rotated 0 degrees counterclockwise relative to the first native direction of its display screen; Figure 2 The mobile phone 201 shown in the figure is a rear camera. In the called device (such as mobile phone 201), the second sensing angle is 90 degrees, that is, the second sensing direction of the second camera in the called device (such as mobile phone 201) is rotated 90 degrees counterclockwise relative to the second native direction of its display screen.

[0083] In step 703, both the calling device and the called device are placed or held in their corresponding native directions; that is, when the first camera of the calling device is placed or held in the first sensing direction, and the second camera of the called device is placed or held in the second sensing direction, the correction angle = (second sensing angle - first sensing angle + 360°) / 360 modulo. Among them, 360 degrees are added after the difference is taken to ensure that the value of the correction angle is always a positive number. Here, the correction angle is the angle between the image direction obtained by the second camera in the second sensing direction and the image direction obtained by the first camera in the first sensing direction. Exemplarily, the first sensing angle = 0 degrees, the second sensing angle = 90 degrees, the correction angle = (90 degrees - 0 degrees + 360°) / 360 modulo 90 degrees, then the correction angle = 90 degrees.

[0084] In step 704, when the calling end device is placed or held in its corresponding native direction, the current direction of the first camera is the first sensing direction, that is, the third image is the image acquired by the first camera in the calling end device in the first sensing direction; similarly, when the called end device is placed or held in its corresponding native direction, the current direction of the second camera is the second sensing direction, that is, the first image is the image acquired by the second camera of the called end device in the second sensing direction. At this time, the second image is obtained by rotating the first image by the correction angle; the direction of the second image is consistent with the direction of the third image.

[0085] Exemplary, reference Figure 8 , Figure 8 A schematic diagram of a first display screen and a second image result displayed on a second display screen provided by an embodiment is shown; wherein, when a calling device (e.g., a laptop computer 802) calls a second camera of a called device (e.g., a mobile phone 801), a correction angle = 90 degrees; wherein, the direction of the first image acquired by the second camera is rotated 90 degrees to obtain a second graphic; after the second image is sent to the calling device, the second image can be displayed in a positive direction on the first display screen of the calling device.

[0086] In this way, when the calling device calls the second camera on the called device to realize the transmission of the remote image, the above-mentioned correction method can be used to rotate the direction of the first image obtained by the second camera of the called device, so that after the rotated first image is sent to the calling device, the image displayed on the first display screen of the calling device is in the positive direction.

[0087] In some embodiments, when the calling device is rotated and the called device is placed or held in its corresponding native orientation, the correction angle changes with the rotation of the calling device. In other words, when the calling device is rotated, the current position of the first camera changes, and the corresponding correction angle also changes.

[0088] Exemplarily, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, the correction angle=(second sensing angle-first sensing angle-first angle+360°) / 360 modulo.

[0089] Among them, when the calling end device rotates counterclockwise by the first angle in its native direction, it means that the current direction of the device of the first display screen (keeping the display screen facing the user) is rotated counterclockwise by the first angle relative to the first native direction, and the first camera is rotated counterclockwise by the first angle based on the first sensing direction. Based on this, the correction angle = (second sensing angle - first sensing angle - first angle + 360°) / 360 modulo. In this way, when the first image obtained by the second camera in the second sensing direction is rotated by the above correction angle, it can be guaranteed to be consistent with the direction of the third image obtained by the first camera in the current direction.

[0090] In some embodiments, when the calling device is placed or held in its corresponding native orientation and the called device is rotated, the correction angle changes with the rotation of the called device. In other words, when the called device is rotated, the current position of the second camera changes, and the corresponding correction angle also changes.

[0091] Exemplarily, when the second camera is rotated counterclockwise by the second angle based on the second sensing direction, the correction angle=(second sensing angle-first sensing angle+second angle+360°) / 360 modulo.

[0092] Among them, when the called end device rotates counterclockwise by the second angle in its native direction, it means that the current direction of the device of the second display screen (keeping the display screen facing the user) is rotated clockwise by the second angle relative to the second native direction, and the second camera is rotated clockwise by the second angle based on the second sensing direction. Based on this, the correction angle = (second sensing angle - first sensing angle + second angle + 360°) / 360 modulo. In this way, when the first image obtained by the second camera in the rotated direction is rotated by the above correction angle, it can be guaranteed to be consistent with the direction of the third image obtained by the first camera in the first sensing direction.

[0093] In some embodiments, when both the calling device and the called device rotate, the correction angle changes with the rotation of the calling device and the called device. In other words, when the calling device rotates, the current position of the first camera changes; when the called device rotates, the current position of the second camera changes, and the corresponding correction angle also changes.

[0094] Exemplarily, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, and when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle = (second sensing angle - first sensing angle - first angle + second angle + 360°) / 360 modulo.

[0095] Among them, when the calling end device rotates counterclockwise by a first angle in its native direction, it means that the current direction of the device of the first display screen (keeping the display screen facing the user) is rotated counterclockwise by a first angle relative to the first native direction, and the first camera is rotated counterclockwise by the first angle based on the first sensing direction; when the called end device rotates counterclockwise by a second angle in its native direction, it means that the current direction of the device of the second display screen (keeping the display screen facing the user) is rotated clockwise by a second angle relative to the second native direction, and the second camera is rotated clockwise by a second angle based on the second sensing direction. Based on this, the correction angle = (second sensing angle - first sensing angle - first angle + second angle + 360°) / 360 modulo. In this way, when the first image obtained by the second camera in the rotated direction is rotated by the above correction angle, it can be guaranteed to be consistent with the direction of the third image obtained by the first camera in the rotated direction.

[0096] In some embodiments, the method further includes: cropping the second image so that the fourth image obtained after cropping has the same image ratio as the third image.

[0097] Exemplary, reference Fig. 9 , Fig. 9The figure is a schematic diagram of image ratios of various second images and third images according to an exemplary embodiment; wherein, when the direction of the second image is consistent with that of the third image, but the image ratio of the second image is different from that of the third image, the image ratio of the second image is adjusted, such as cropping, so that the image ratio of the adjusted second image (i.e., the fourth image) is the same as that of the third image. It should be noted that Fig. 9 The dotted box shown in FIG. 1 is a schematic diagram of a third image; Fig. 9 The solid line frame shown in FIG. 1 is a schematic diagram of the second image. It should be understood that the size of the fourth image may be the same as the size of the third image, or may be different from the size of the third image.

[0098] It should be noted that, on the one hand, when the second image and the third image are both horizontal, the cropping method can be selected and set according to the specific situation; for example, when the aspect ratio of the second image is greater than that of the third image, the cropping is performed in a left-right center manner ( Fig. 9 When the aspect ratio of the second image is smaller than that of the third image, the image is cropped in the middle of the top and bottom. Fig. 9 On the other hand, when the second image is horizontal and the third image is vertical, the cropping can be performed in the middle of the top and bottom ( Fig. 9 (c) in the figure), the present disclosure is not limited thereto.

[0099] In some embodiments, when the size of the fourth image is different from that of the third image, the method further includes: scaling the fourth image so that the size of the fifth image obtained after the scaling is the same as that of the third image.

[0100] In some embodiments, the method further includes: sending the fifth image to the calling end device to display the fifth image in a positive direction on the first display screen.

[0101] It should be noted that the above embodiment uses the example that both the calling device and the called device are rear cameras to introduce in detail the principles of rotation, cropping, and scaling. This principle is also applicable to image direction correction when both the calling device and the called device are front cameras.

[0102] In addition, it should be understood that the image direction correction method provided in the embodiment of the present disclosure can be implemented by an algorithm; for example, (1) correction angle A = (the second sensing angle b - the first sensing angle a + 360°) / 360 modulo, the corresponding algorithm formula for the correction angle is:

[0103] int A=(b_Y–a_X+360)%360;

[0104] (X=BACK or FRONT)

[0105] (Y=BACK or FRONY).

[0106] (2) Correction angle B = (second sensing angle b - first sensing angle a - first angle c + 360°) / 360° modulo, the corresponding correction angle algorithm formula is:

[0107] int B;

[0108] if(X == BACK){ / / Calling end device

[0109] B = (360–a)%360;

[0110] }else{

[0111] B = a;

[0112] }.

[0113] (3) Correction angle C = (second sensing angle b - first sensing angle a + second angle d + 360°) / 360° modulo, the corresponding correction angle algorithm formula is:

[0114] int C;

[0115] if(Y == BACK){ / / Called device

[0116] C = b;

[0117] }else{

[0118] C = (360–b)%360;

[0119] }.

[0120] (4) Correction angle D = (second sensing angle - first sensing angle - first angle + second angle + 360°) / 360° modulo, the corresponding correction angle algorithm formula is:

[0121] int D = (A + B + C) % 360.

[0122] It should be understood that the calculation process of the correction angle in the image direction correction method can be performed before the calling device and the called device establish a network connection, or can be performed during image transmission after the network connection is established.

[0123] Based on this, in the embodiments of the present disclosure, a general, cross-platform, and adaptive solution is proposed for the problem of the direction of the virtual camera image in the current camera virtualization function based on the local area network or the Internet. According to the embodiments of the present disclosure, at least when the direction of the first image acquired by the second camera is inconsistent with the direction of the third image acquired by the first camera in the current direction, the direction of the first image acquired by the second camera in the current direction can be rotated so that the direction of the second image obtained after rotation is consistent with the direction of the third image. Furthermore, no matter what angle the calling device or the called device is rotated, it can always be ensured that the picture from the camera of the called device displayed on the display screen of the calling device is always positive; and there is no need for the user to manually adjust the placement direction of the display screen, thereby improving the user experience.

[0124] According to a second aspect of an embodiment of the present disclosure, there is provided an image direction correction device; Fig.10 FIG. 1 is a structural block diagram of an image direction correction device according to an exemplary embodiment. Fig.10 As shown, the image direction correction device 1000 includes:

[0125] A first determining module 1001 is configured to determine a first native orientation of a first display screen in a calling device and a second native orientation of a second display screen in a called device;

[0126] The second determination module 1002 is configured to determine a first sensing direction of a first camera in the calling end device and a second sensing direction of a second camera in the called end device;

[0127] An acquisition module 1003 is configured to obtain a correction angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction;

[0128] The rotation module 1004 is configured to rotate the direction of the first image acquired by the second camera in the current direction by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction.

[0129] In some embodiments, when the current direction of the first camera is the first sensing direction, and the current direction of the second camera is the second sensing direction, the acquisition module is specifically configured as follows:

[0130] Obtaining a first sensing angle based on an angle between the first sensing direction and the first native direction;

[0131] Obtaining a second sensing angle based on an angle between the second sensing direction and the second native direction;

[0132] The correction angle=(the second sensing angle-the first sensing angle+360°) / 360 modulo.

[0133] In some embodiments, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, the correction angle=(the second sensing angle-the first sensing angle-the first angle+360°) / 360 modulo.

[0134] In some embodiments, when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle=(the second sensing angle-the first sensing angle+the second angle+360°) / 360 modulo.

[0135] In some embodiments, when the first camera is rotated counterclockwise by a first angle based on the first sensing direction, and when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle = (the second sensing angle - the first sensing angle - the first angle + the second angle + 360°) / 360 modulo.

[0136] In some embodiments, the device further includes: a cropping module configured to crop the second image so that a fourth image obtained after cropping has the same image ratio as the third image.

[0137] In some embodiments, the device further includes: a scaling processing module configured to perform scaling processing on the fourth image so that the size of the fifth image obtained after the scaling processing is the same as the size of the third image.

[0138] In some embodiments, the device also includes: a network connection module, configured to establish a network connection between the calling device and the called device to implement camera virtualization processing between the first camera and the second camera; and send the fifth image to the first display screen to display the fifth image forward on the first display screen.

[0139] It should be noted that when implementing image transmission between the calling device and the called device, the image method correction device can be integrated on the calling device or on the called device. It should be noted that when the image method correction device is integrated on the calling device, before the image (or image data) starts to be transmitted, the called device needs to send the camera image sensor direction and the device current direction information in advance through the control command network channel to the calling device; during the image (or image data) transmission process, the called device only needs to send the device current direction information to the calling device through the control command network channel when its own direction changes.

[0140] Similarly, exemplarily, when the image method correction device is integrated on the called device, before the image (or image data) starts to be transmitted, the calling device needs to send the camera image sensor direction and the device current direction information to the called device through the control command network channel in advance; during the image (or image data) transmission process, the calling device only needs to send the device current direction information to the called device through the control command network channel when its own direction changes.

[0141] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the method described in the above embodiment of the present disclosure. Fig.11 1 is a block diagram of an electronic device including an image direction correction device 1000 according to an exemplary embodiment. For example, the device 1100 may be a mobile phone, a mobile computer, etc.

[0142] Reference Fig.11 , the device 1100 may include one or more of the following components: a processing component 1102 , a memory 1104 , a power component 1106 , a multimedia component 1108 , an audio component 1110 , an input / output (I / O) interface 1112 , a sensor component 1114 , and a communication component 1116 .

[0143] The processing component 1102 generally controls the overall operation of the device 1100, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1102 may include one or more modules to facilitate interaction between the processing component 1102 and other components. For example, the processing component 1102 may include a multimedia module to facilitate interaction between the multimedia component 1108 and the processing component 1102.

[0144] The memory 1104 is configured to store various types of data to support the operation of the device 1100. Examples of such data include instructions for any application or method operating on the device 1100, contact data, phone book data, messages, pictures, videos, etc. The memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0145] The power supply component 1106 provides power to the various components of the device 1100. The power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1100.

[0146] The multimedia component 1108 includes a screen that provides an output interface between the device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1108 includes a front camera and / or a rear camera. When the device 1100 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0147] The audio component 1110 is configured to output and / or input audio signals. For example, the audio component 1110 includes a microphone (MIC), and when the device 1100 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1104 or sent via the communication component 1116. In some embodiments, the audio component 1110 also includes a speaker for outputting audio signals.

[0148] I / O interface 1112 provides an interface between processing component 1102 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0149] The sensor assembly 1114 includes one or more sensors for providing various aspects of the status assessment of the device 1100. For example, the sensor assembly 1114 can detect the open / closed state of the device 1100, the relative positioning of components, such as the display and keypad of the device 1100, the sensor assembly 1114 can also detect the position change of the device 1100 or a component of the device 1100, the presence or absence of user contact with the device 1100, the orientation or acceleration / deceleration of the device 1100, and the temperature change of the device 1100. The sensor assembly 1114 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1114 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1114 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0150] The communication component 1116 is configured to facilitate wired or wireless communication between the device 1100 and other devices. The device 1100 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 1116 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1116 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0151] In an exemplary embodiment, the device 1100 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned methods.

[0152] In an exemplary embodiment, a readable storage medium including instructions is also provided, such as a memory 1104 including instructions, and the instructions can be executed by a processor 1120 of the device 1100 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The method includes:

[0153] Determine a first native orientation of a first display screen in a calling device and a second native orientation of a second display screen in a called device;

[0154] Determine a first sensing direction of a first camera in the calling end device and a second sensing direction of a second camera in the called end device;

[0155] Obtaining a correction angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction;

[0156] The direction of the first image acquired by the second camera in the current direction is rotated by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction.

[0157] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present disclosure. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present disclosure, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure. The serial numbers of the embodiments of the present disclosure are for description only and do not represent the advantages and disadvantages of the embodiments.

[0158] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0159] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0160] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for image direction correction, characterized in that: The method comprises: Determine a first native orientation of a first display screen in a calling device and a second native orientation of a second display screen in a called device; Determine a first sensing direction of a first camera in the calling end device and a second sensing direction of a second camera in the called end device; Obtaining a correction angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction; The direction of the first image acquired by the second camera in the current direction is rotated by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction.

2. The method according to claim 1, characterized in that When the current direction of the first camera is the first sensing direction, and the current direction of the second camera is the second sensing direction, obtaining a corrected angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction includes: Obtaining a first sensing angle based on an angle between the first sensing direction and the first native direction; Obtaining a second sensing angle based on an angle between the second sensing direction and the second native direction; The correction angle=(the second sensing angle-the first sensing angle+360°) / 360 modulo.

3. The method according to claim 2, characterized in that When the first camera is rotated counterclockwise by a first angle based on the first sensing direction, the correction angle=(the second sensing angle-the first sensing angle-the first angle+360°) / 360 modulo.

4. The method according to claim 2, characterized in that: When the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle=(the second sensing angle-the first sensing angle+the second angle+360°) / 360° modulo.

5. The method according to claim 2, characterized in that: When the first camera is rotated counterclockwise by a first angle based on the first sensing direction, and when the second camera is rotated counterclockwise by a second angle based on the second sensing direction, the correction angle = (the second sensing angle - the first sensing angle - the first angle + the second angle + 360°) / 360 modulo.

6. The method according to claim 1, characterized in that The method further comprises: The second image is cropped so that a fourth image obtained after the cropping has the same image ratio as the third image.

7. The method according to claim 6, characterized in that The method further comprises: The fourth image is scaled so that a size of a fifth image obtained after the scaling is the same as a size of the third image.

8. The method according to claim 7, characterized in that The method further comprises: Establishing a network connection between the calling end device and the called end device to implement camera virtualization processing between the first camera and the second camera; and The fifth image is sent to the calling end device so as to display the fifth image in a positive direction on the first display screen.

9. An image direction correction device, characterized in that: The device comprises: A first determining module is configured to determine a first native orientation of a first display screen in a calling end device and a second native orientation of a second display screen in a called end device; A second determining module is configured to determine a first sensing direction of a first camera in the calling end device and a second sensing direction of a second camera in the called end device; an acquisition module, configured to obtain a correction angle based on the first native direction, the second native direction, the first sensing direction, and the second sensing direction; The rotation module is configured to rotate the direction of the first image acquired by the second camera in the current direction by the correction angle so that the second image obtained after the rotation is consistent with the direction of the third image acquired by the first camera in the current direction.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method as claimed in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 8.

Images