Communication method and apparatus

Abstract

The embodiment of the application discloses communication methods and devices, relates to the technical field of communication, and is used for reducing resource consumption in a video call process. The method comprises the following steps: acquiring first information; and in the case that a counterpart device does not support a portrait centering technology, performing cutting on a first video stream according to a local resolution to obtain a first cut video stream. The first information is used for indicating whether the counterpart device supports the portrait centering technology, and the first video stream is a video stream of the counterpart device. It can be understood that if the counterpart device does not support the portrait centering technology, it means that the video stream (i.e. the first video stream) of the device is an original video stream which has not been cut by the portrait centering technology. According to the scheme, the video stream of the counterpart device is cut only on the premise that it is confirmed that the counterpart device does not have the portrait centering technology. The purpose of the measure is to avoid repeated cutting of the video stream which has been centered on a portrait, effectively reduce resource waste, and thus reduce resource consumption in the video call process.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to communication methods and apparatus. Background Technology

[0002] With the continuous development of technology, the widespread use of electronic devices (such as terminals, computing power units, and smart screens) has become commonplace. These electronic devices generally have video calling capabilities; however, due to cost control and space constraints, they are usually not equipped with cameras that support rotating axes. Therefore, when users move continuously or are far away, their images are often difficult to center on the display interface, which directly leads to a decline in video call quality and a worse user experience.

[0003] To address this issue, electronic devices can utilize face-centering technology during video calls to crop the video frame and center the cropped face image, thereby optimizing the call experience. However, using face-centering technology during video calls incurs additional resource consumption. Summary of the Invention

[0004] This application provides a communication method and apparatus to reduce resource overhead during video calls. To achieve the above objective, this application adopts the following technical solution:

[0005] In a first aspect, embodiments of this application provide a communication method, the method comprising: acquiring first information; and, in the absence of the peer device supporting portrait centering technology, cropping a first video stream according to the local resolution to obtain a first cropped video stream. The first information is used to indicate whether the peer device supports portrait centering technology, and the first video stream is the video stream of the peer device.

[0006] It is understandable that if the other device does not support portrait centering technology, it means that the video stream of that device (i.e., the first video stream) is the original video stream that has not been cropped using portrait centering technology. The solution provided in this application embodiment will only crop the video stream if it is confirmed that the other device does not have portrait centering technology. This measure aims to avoid repeatedly cropping a video stream that has already been centered, effectively reducing resource waste and thus reducing resource consumption during video calls.

[0007] In one possible implementation, the method further includes: if the peer device does not support portrait centering technology, cropping the first video stream according to the peer resolution to obtain a second cropped video stream; if the peer device does not support portrait centering technology, sending the second cropped video stream or cropping information of the second cropped video stream. The cropping information includes at least one of cropping position or cropping frame size.

[0008] Understandably, by cropping the video stream according to the resolution of the peer device, a second cropped video stream is generated, and then this second cropped video stream or its cropping information is sent to the peer device. In this way, even peer devices that do not support portrait centering technology can present a video effect with the portrait centered, thus significantly improving the user experience of video calls.

[0009] In one possible implementation, the method further includes: cropping the second video stream according to the peer's resolution to obtain a third cropped video stream; cropping the second video stream according to the local resolution to obtain a fourth cropped video stream; and sending the third cropped video stream or cropping information of the third cropped video stream. The second video stream is the video stream of the local device, and the cropping information includes at least one of a cropping position or a cropping frame size.

[0010] Understandably, in some video call scenarios, there is a significant difference in resolution between the local and remote devices. The local device may crop its video stream based on the remote device's resolution, resulting in black bars on the local screen and severely impacting user experience. To address this, this solution employs a portrait-centered cropping technique to both crop a video stream to match the remote device's aspect ratio and save a separate video stream to match the local device's resolution. This effectively resolves the black bar issue when there is a significant difference in resolution between the local and remote devices, significantly improving the user experience of video calls.

[0011] In one possible implementation, the above-mentioned cropping of the first video stream according to the local resolution to obtain the first cropped video stream specifically includes: cropping the first video stream according to second information and the local resolution to obtain the first cropped video stream. The second information includes at least one of the cropping frame size of the first cropped video stream or the smoothing coefficient of the first cropped video stream.

[0012] As can be seen, in this solution, both the cropping frame size and the smoothing coefficient of the video stream are variable, significantly improving the flexibility of cropping. Specifically, the size of the cropping frame can be flexibly adjusted according to actual needs to adapt to various display devices and resolutions, ensuring that the cropped video stream makes full use of the display screen and effectively reduces or avoids black borders. Furthermore, adjusting the smoothing coefficient makes the video smoother, further optimizing the user experience. This flexible cropping strategy enables this solution to achieve excellent display results in diverse application scenarios.

[0013] In one possible implementation, the method further includes: receiving a user operation; and determining the second information based on the user operation.

[0014] As can be seen, this solution allows users to adjust the size of the cropping frame and the smoothing coefficient based on user actions (such as gestures, voice, and button presses). For example, users can increase or decrease the size of the cropping frame by swiping, or adjust the smoothing coefficient using specific voice commands to achieve a more satisfactory video display effect. This user-participatory cropping adjustment method not only further enhances the flexibility of cropping but also greatly improves the personalization of the user experience.

[0015] In one possible implementation, the first cropped video stream includes a first video frame and a second video frame, the first video frame including a first cropping frame, and the second video frame including a second cropping frame. The second cropping frame is obtained based on the first cropping frame and the smoothing coefficient.

[0016] As can be seen, this solution employs frame-by-frame detection and smooth motion techniques, eliminating the need for face detection and cropping for every single frame. Some video frames can reuse previously acquired cropping boxes, effectively improving the smoothness of video calls when Neural Processing Unit (NPU) resources are limited or the number of face detection and cropping frames is insufficient.

[0017] Secondly, embodiments of this application provide a communication device, which may be a terminal, a module applied to the terminal (such as a processor, chip or chip system, etc.), or a logical node, logical module or software that can realize all or part of the terminal functions. The device includes: a transceiver unit and a processing unit.

[0018] The aforementioned transceiver unit is used to acquire first information, which is used to indicate whether the peer device supports human face centering technology.

[0019] The aforementioned processing unit is configured to crop the first video stream according to the local resolution to obtain a first cropped video stream when the aforementioned peer device does not support the portrait centering technology. The first video stream is the video stream of the aforementioned peer device.

[0020] In one possible implementation, the processing unit is further configured to: if the peer device does not support portrait centering technology, crop the first video stream according to the peer resolution to obtain a second cropped video stream; if the peer device does not support portrait centering technology, send the second cropped video stream or cropping information of the second cropped video stream, wherein the cropping information includes at least one of cropping position or cropping frame size.

[0021] In one possible implementation, the processing unit is further configured to: crop the second video stream according to the peer resolution to obtain a third cropped video stream, wherein the second video stream is the video stream of the local device; crop the second video stream according to the local resolution to obtain a fourth cropped video stream; and send the third cropped video stream or cropping information of the third cropped video stream, wherein the cropping information includes at least one of cropping position or cropping frame size.

[0022] In one possible implementation, the processing unit is specifically used to: crop the first video stream according to the second information and the local resolution to obtain a first cropped video stream, wherein the second information includes at least one of the cropping frame size of the first cropped video stream or the smoothing coefficient of the first cropped video stream.

[0023] In one possible implementation, the processing unit is further configured to: receive a user operation; and determine the second information based on the user operation.

[0024] In one possible implementation, the first cropped video stream includes a first video frame and a second video frame, the first video frame includes a first cropping frame, the second video frame includes a second cropping frame, and the second cropping frame is obtained based on the first cropping frame and the smoothing coefficient.

[0025] Thirdly, embodiments of this application also provide a communication device, which includes: at least one processor, which, when the at least one processor executes program code or instructions, implements the method described in the first aspect or any possible implementation thereof.

[0026] Alternatively, the communication device may be a chip or a chip system.

[0027] Optionally, the device may further include at least one memory for storing the program code or instructions.

[0028] Fourthly, embodiments of this application also provide a chip, including: an input interface, an output interface, and at least one processor. Optionally, the chip further includes a memory. The at least one processor is used to execute code in the memory, and when the at least one processor executes the code, the chip implements the method described in the first aspect or any possible implementation thereof.

[0029] Alternatively, the chip described above can also be an integrated circuit.

[0030] Fifthly, embodiments of this application also provide a computer-readable storage medium for storing a computer program, the computer program including methods for implementing the first aspect or any possible implementation thereof.

[0031] Sixthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to implement the method described in the first aspect or any possible implementation thereof.

[0032] The communication device, computer storage medium, computer program product, and chip provided in this embodiment are all used to execute the communication method provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the communication method provided above, and will not be repeated here. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This application provides a schematic diagram of the structure of a communication system according to an embodiment of the present application.

[0035] Figure 2 A flowchart illustrating a communication method provided in an embodiment of this application;

[0036] Figure 3 This is a schematic flowchart of a video call solution provided in an embodiment of this application;

[0037] Figure 4 Another flowchart illustrating the video call solution provided in this application embodiment;

[0038] Figure 5 Another flowchart illustrating the video call solution provided in this application embodiment.

[0039] Figure 6 This is another flowchart illustrating a video call solution provided in an embodiment of this application.

[0040] Figure 7 This is another flowchart illustrating a video call solution provided in an embodiment of this application.

[0041] Figure 8 This is another flowchart illustrating a video call solution provided in an embodiment of this application.

[0042] Figure 9 This is a schematic diagram of the structure of a communication device provided in an embodiment of this application;

[0043] Figure 10 This is a schematic diagram of the structure of a chip provided in an embodiment of this application;

[0044] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the embodiments of this application.

[0046] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0047] The terms "first" and "second," etc., in the specification and drawings of the embodiments of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0048] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0049] It should be noted that in the description of the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0050] Currently, electronic devices can use image centering technology during video calls to crop the video frame and center the cropped image, thus optimizing the call experience. However, using image centering technology during video calls incurs additional resource consumption.

[0051] To this end, embodiments of this application provide a communication method and apparatus for reducing resource overhead during video calls.

[0052] The technical solutions provided in this application can be applied to communication systems.

[0053] Figure 1 A schematic diagram of one possible, non-limiting communication system described above is shown. Figure 1 As shown, the communication system 100 includes at least one local device 200 and at least one remote device 300.

[0054] Local device 200 is used to generate local video stream, i.e., the second video stream.

[0055] For example, the local device 200 may be a terminal, a computing host, a smart screen, a mobile phone, a cellular phone, a smartphone, a tablet computer, a set-top box, a camera, a screen, a smart screen, a personal digital assistant (PDA), a handset, a laptop computer, a smartwatch, or a smart bracelet.

[0056] The peer device 300 is used to generate the peer video stream, i.e., the first video stream.

[0057] For example, the peer device 300 may be a terminal, a computing host, a smart screen, a mobile phone, a cellular phone, a smartphone, a tablet computer, a set-top box, a camera, a screen, a smart screen, a personal digital assistant (PDA), a handset, a laptop computer, a smartwatch, or a smart bracelet.

[0058] The local device 200 and the remote device 300 can make video calls through a communication network.

[0059] For example, the communication network can be a long-term evolution (LTE) communication network, an LTE frequency division duplex (FDD) communication network, an LTE time division duplex (TDD) communication network, a public land mobile network (PLMN), a 5th generation (5G) communication network, and future communication networks, etc. Among them, the 5G communication network in this application embodiment includes a non-standalone (NSA) 5G mobile communication network or a standalone (SA) 5G mobile communication network.

[0060] As another example, the communication network can also be a satellite communication network, such as a nonterrestrial network (NTN).

[0061] For example, the communication network can also be a device-to-device (D2D) communication network, a side link (SL) communication network, a machine-to-machine (M2M) communication network, a machine-type communication (MTC) network, an Internet of Things (IoT) communication network, a vehicle-to-everything (V2X) communication network, an unrewed aerial vehicle (UAV) communication network, or other communication networks.

[0062] It needs to be explained that, Figure 1 The communication system 100 shown is merely one implementation of the embodiments of this application. In actual applications, the communication system 100 may include more or fewer components, which is not limited here.

[0063] Figure 2 This application illustrates a communication method provided in an embodiment, which can be executed by the local device 200 in the aforementioned communication system 100, such as... Figure 2 As shown, the method includes:

[0064] S201, Obtain first information.

[0065] The first piece of information is used to indicate whether the other device supports human face centering technology.

[0066] In one possible implementation, obtaining the first information specifically includes: the peer device can send the first information to the local device.

[0067] Accordingly, the local device can receive the first information sent by the remote device.

[0068] For example, the peer device can send a first message carrying first information to the local device.

[0069] Accordingly, the local device can receive messages carrying first information sent by the remote device.

[0070] Optionally, the first message mentioned above may be a session description protocol (SDP) message or other messages.

[0071] The aforementioned first information can be carried using existing parameters in the SDP message, or it can be carried using newly defined parameters in the SDP message. When using existing parameters in the SDP message, the corresponding parameter can be found in the SDP message, and its value can be used to determine whether the peer device supports portrait centering technology. When using newly defined parameters in the SDP message, a new parameter can be added to the SDP message specifically to indicate whether the peer device supports portrait centering technology. In this way, after receiving the SDP message, the local device can quickly and accurately determine whether the peer device supports portrait centering technology based on the value of the newly defined parameter, and then perform appropriate processing.

[0072] For example, a new portrait centering attribute can be added to the 'a' field (media attribute) at the media level of the SDP message: a = centerportrait:true 640*480 / 1280*768 / 1920*1080. This means that the device supports portrait centering capability and supports video streams of three different resolutions.

[0073] In one possible implementation, obtaining the first information further includes: the local device can send the first information to the remote device. Correspondingly, the remote device can receive the first information sent by the local device.

[0074] For example, the local device can send a first message carrying first information to the remote device.

[0075] Accordingly, the peer device can receive a message carrying the first information sent by the local device.

[0076] S202. If the other end device does not support the portrait centering technology, the first video stream is cropped according to the resolution of the local end to obtain the first cropped video stream.

[0077] The first video stream mentioned above is the video stream of the aforementioned peer device.

[0078] The local resolution can be any resolution supported by the local device. For example, if the local device supports three resolutions: 640×480, 1280×768, and 1920×1080, then the local resolution can be 640×480, 1280×768, or 1920×1080.

[0079] In one possible implementation, the above-mentioned cropping of the first video stream according to the local resolution to obtain the first cropped video stream specifically includes: cropping the first video stream according to the second information and the local resolution to obtain the first cropped video stream.

[0080] The second information mentioned above includes at least one of the cropping frame size of the first cropped video stream or the smoothing coefficient of the first cropped video stream.

[0081] For example, the call interface of the local device can crop the first video stream according to the second information and the local resolution to obtain a first cropped video stream.

[0082] like Figure 3 As shown, after acquiring the first video stream, the video stream processing unit of the local device can perform face detection (such as frame-by-frame face detection) on the first video stream and generate cropping boxes. Then, it sends multiple smoothing coefficients and multiple cropping box sizes to the call interface of the local device. The call interface of the local device can generate a motion mode selection interface based on the multiple smoothing coefficients sent by the video stream processing unit of the local device, and generate a cropping box selection interface based on the multiple cropping box sizes. Then, the first video stream is cropped according to the smoothing coefficients selected by the user through the motion mode selection interface and the cropping box sizes selected by the user through the cropping box selection interface to obtain the first cropped video stream.

[0083] As can be seen, in this solution, both the cropping frame size and the smoothing coefficient of the video stream are variable, significantly improving the flexibility of cropping. Specifically, the size of the cropping frame can be flexibly adjusted according to actual needs to adapt to various display devices and resolutions, ensuring that the cropped video stream makes full use of the display screen and effectively reduces or avoids black borders. Furthermore, adjusting the smoothing coefficient makes the video smoother, further optimizing the user experience. This flexible cropping strategy enables this solution to achieve excellent display results in diverse application scenarios.

[0084] In one possible implementation, the method further includes: determining the second information based on user information.

[0085] The aforementioned user information includes at least one of the following: portrait size or user movement frequency.

[0086] For example, the cropping frame size of the first cropped video stream can be determined based on the size of the human face.

[0087] For example, if the size of the human face increases, the cropping frame size of the first cropped video stream can be increased accordingly. Conversely, if the size of the human face increases, the cropping frame size of the first cropped video stream needs to be decreased.

[0088] As another example, the smoothing coefficient of the first cropped video stream can be determined based on the user's movement frequency.

[0089] For example, if the frequency of user movement increases, the smoothing coefficient of the first cropped video stream can be increased accordingly. Conversely, if the frequency of user movement decreases, the smoothing coefficient of the first cropped video stream should be decreased.

[0090] Understandably, increasing the smoothing coefficient when user movement frequency increases ensures smooth video streaming. Conversely, decreasing the smoothing coefficient when user movement frequency decreases effectively reduces device power consumption and extends battery life. This design not only significantly improves the user experience but also cleverly addresses the device's energy efficiency management.

[0091] In another possible implementation, the method further includes: receiving a user operation; and determining the second information based on the user operation.

[0092] For example, such as Figure 4 As shown, this solution offers several preset cropping frame modes, each corresponding to a specific cropping frame size. Users can choose the most suitable cropping frame mode from these options based on their needs. With this design, users no longer need to manually adjust the cropping frame size; they can quickly complete the video stream cropping operation simply by selecting the appropriate mode. This not only simplifies the operation and improves cropping efficiency but also further optimizes the user experience.

[0093] Optionally, the various cropping modes mentioned above may include parent-child call mode, normal mode, and multi-person conference mode.

[0094] The cropping frame size for the parent-child call mode is 320×240, for the normal mode it is 640×480, and for the multi-person conference mode it is 1080×810.

[0095] It should be noted that, generally speaking, if the portrait cropping frame remains unchanged, the head will appear too small at a distance and too large at close range. To solve this problem, this application can employ dynamic cropping technology based on the size of the face detection frame (portrait size), and allow users to configure the size of the cropping frame and select the cropping frame mode.

[0096] like Figure 4 As shown, this solution offers multiple trimming modes, such as parent-child call mode, normal mode, and multi-person conference mode, for users to choose from flexibly.

[0097] Among them, the parent-child call mode corresponds to the parent-child call mode scenario. When children talk to their parents who are away at home on a large screen, the cropping frame is made smaller to make the child's head clearer.

[0098] The multi-person meeting mode is for multi-person meetings / video interview scenarios. In order to frame as many people as possible, the cropping box is made larger accordingly.

[0099] like Figure 5 As shown, the size of the cropping frame in this solution can be adjusted according to user gestures. Users can shrink the cropping frame from 640×480 to 320×240 using a shrink gesture, and enlarge it from 640×480 to 1080×810 using a zoom gesture.

[0100] For example, such as Figure 6 As shown, this solution offers multiple preset motion modes, each corresponding to a specific smoothing coefficient. Users can choose the most suitable motion mode from these options based on their needs. With this design, users no longer need to manually adjust the smoothing coefficient; they can quickly complete the video stream trimming operation simply by selecting the appropriate mode. This not only simplifies the operation and improves trimming efficiency but also further optimizes the user experience.

[0101] Optionally, the various cropping modes mentioned above may include slow mode, motion mode, and normal mode.

[0102] The slow mode is suitable for scenarios where users sit on the sofa for extended periods to make calls, while the motion mode is suitable for scenarios where users are frequently on the move to make calls.

[0103] As can be seen, this solution allows users to adjust the size of the cropping frame and the smoothing coefficient based on user actions (such as gestures, voice, and button presses). For example, users can increase or decrease the size of the cropping frame by swiping, or adjust the smoothing coefficient using specific voice commands to achieve a more satisfactory video display effect. This user-participatory cropping adjustment method not only further enhances the flexibility of cropping but also greatly improves the personalization of the user experience.

[0104] In one possible implementation, the first cropped video stream includes a first video frame and a second video frame, wherein the first video frame includes a first cropping frame and the second video frame includes a second cropping frame.

[0105] The second cropping frame is obtained based on the first cropping frame and the smoothing coefficient.

[0106] like Figure 7 As shown, the first cropped video stream includes video frames 0 to 5. Face detection is performed on video frame 0, outputting a first cropping box; video frame 1 can reuse the first cropping box and output a second cropping box based on a smoothing coefficient. Similarly, face detection is performed on video frame 2, outputting a third cropping box; video frame 3 reuses the third cropping box and outputs a fourth cropping box based on a smoothing coefficient. Face detection is performed on video frame 4, outputting a fifth cropping box; and video frame 5 reuses the fifth cropping box and outputs a sixth cropping box based on a smoothing coefficient.

[0107] The second cropping frame can satisfy the following condition: second cropping frame = first cropping frame * smoothing coefficient.

[0108] The fourth cropping frame can satisfy the following condition: Fourth cropping frame = Third cropping frame * Smoothing coefficient.

[0109] The sixth cropping frame can satisfy the following condition: sixth cropping frame = fifth cropping frame * smoothness coefficient.

[0110] As can be seen, this solution employs frame-by-frame detection and smooth motion techniques, eliminating the need for face detection and cropping for every single frame. Some video frames can reuse previously acquired cropping boxes, effectively improving the smoothness of video calls when Neural Processing Unit (NPU) resources are limited or the number of face detection and cropping frames is insufficient.

[0111] In one possible implementation, the method further includes: if the peer device does not support portrait centering technology, cropping the first video stream according to the peer resolution to obtain a second cropped video stream; if the peer device does not support portrait centering technology, sending the second cropped video stream or cropping information of the second cropped video stream.

[0112] The aforementioned cutting information includes at least one of the following: cutting position or cutting frame size.

[0113] The resolution of the peer device can be any resolution supported by the peer device. For example, if the peer device supports three resolutions: 640×480, 1280×768, and 1920×1080, then the resolution of the peer device can be 640×480, 1280×768, or 1920×1080.

[0114] The resolution of the other end can be obtained through resolution negotiation between the local device and the other end device.

[0115] Understandably, by cropping the video stream according to the resolution of the peer device, a second cropped video stream is generated, and then this second cropped video stream or its cropping information is sent to the peer device. In this way, even peer devices that do not support portrait centering technology can present a video effect with the portrait centered, thus significantly improving the user experience of video calls.

[0116] In one possible implementation, the method further includes: cropping the second video stream according to the peer's resolution to obtain a third cropped video stream; cropping the second video stream according to the local resolution to obtain a fourth cropped video stream; and sending the third cropped video stream or cropping information of the third cropped video stream. The second video stream is the video stream of the local device, and the cropping information includes at least one of a cropping position or a cropping frame size.

[0117] like Figure 8 As shown, taking the local device as a large-screen device and the remote device as a mobile phone as an example, in a video call scenario, the large-screen device generates a video stream based on its own resolution by centering and cropping the portrait, and saves it for display on its own screen. Subsequently, based on the resolution agreed upon by the remote device (i.e., the remote device's resolution), it again crops a video stream that matches the remote device's resolution ratio and sends it to the remote device for display. This solution effectively solves the problem of black borders appearing on the local screen, while ensuring that the image quality is not affected when switching between large and small windows.

[0118] Understandably, in some video call scenarios, there is a significant difference in resolution between the local and remote devices. The local device may crop its video stream based on the remote device's resolution, resulting in black bars on the local screen and severely impacting user experience. To address this, this solution employs a portrait-centered cropping technique to both crop a video stream to match the remote device's aspect ratio and save a separate video stream to match the local device's resolution. This effectively resolves the black bar issue when there is a significant difference in resolution between the local and remote devices, significantly improving the user experience of video calls.

[0119] It is understandable that if the other device does not support the portrait centering technology, it means that the video stream of that device (i.e. the first video stream) is the original video stream that has not been cropped by the portrait centering technology.

[0120] As can be seen from S201 to S202, the solution provided in this application embodiment only performs video stream cropping when it is confirmed that the other end device does not have the image centering technology. This measure aims to avoid repeatedly cropping video streams that have already been image centered, effectively reducing resource waste and thus reducing resource consumption during video calls.

[0121] The following describes a communication device used to perform the above communication method.

[0122] It is understood that, in order to achieve the above-mentioned functions, the communication device includes hardware and / or software modules that perform the respective functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0123] This application embodiment can divide the communication device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0124] When dividing each function into modules according to its corresponding function. Figure 9 The diagram illustrates a possible composition of the communication device involved in the above embodiments. The device may be a terminal, a module applied to the terminal (such as a processor, chip, or chip system), or a logic node, logic module, or software that can implement all or part of the terminal functions.

[0125] like Figure 9 As shown, the communication device 900 may include a transceiver unit 901 and a processing unit 902.

[0126] The aforementioned transceiver unit 901 is used to acquire first information, which is used to indicate whether the peer device supports human face centering technology.

[0127] For example, transceiver unit 901 can perform Figure 2 S201 in the method shown.

[0128] The aforementioned processing unit 902 is used to crop the first video stream according to the local resolution to obtain a first cropped video stream when the aforementioned peer device does not support the portrait centering technology. The aforementioned first video stream is the video stream of the aforementioned peer device.

[0129] For example, processing unit 902 can perform Figure 2 S202 in the method shown.

[0130] In one possible implementation, the processing unit 902 is further configured to: if the peer device does not support portrait centering technology, crop the first video stream according to the peer resolution to obtain a second cropped video stream; if the peer device does not support portrait centering technology, send the second cropped video stream or cropping information of the second cropped video stream, wherein the cropping information includes at least one of cropping position or cropping frame size.

[0131] In one possible implementation, the processing unit 902 is further configured to: crop the second video stream according to the peer resolution to obtain a third cropped video stream, wherein the second video stream is the video stream of the local device; crop the second video stream according to the local resolution to obtain a fourth cropped video stream; and send the third cropped video stream or cropping information of the third cropped video stream, wherein the cropping information includes at least one of cropping position or cropping frame size.

[0132] In one possible implementation, the processing unit 902 is specifically used to: crop the first video stream according to the second information and the local resolution to obtain a first cropped video stream, wherein the second information includes at least one of the cropping frame size of the first cropped video stream or the smoothing coefficient of the first cropped video stream.

[0133] In one possible implementation, the processing unit 901 is further configured to: receive a user operation; and determine the second information based on the user operation.

[0134] In one possible implementation, the first cropped video stream includes a first video frame and a second video frame, the first video frame includes a first cropping frame, the second video frame includes a second cropping frame, and the second cropping frame is obtained based on the first cropping frame and the smoothing coefficient.

[0135] This application also provides a chip, which can be the chip of the above-mentioned communication device. Figure 10 A schematic diagram of a chip 1000 is shown. The chip 1000 includes one or more processors 1001 and interface circuits 1002. Optionally, the chip 1000 may also include a bus 1003.

[0136] The processor 1001 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above communication method can be completed through the integrated logic circuitry in the processor 1001 or through software instructions.

[0137] Optionally, the processor 1001 described above may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods and steps disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0138] The interface circuit 1002 can be used to send or receive data, instructions or information. The processor 1001 can use the data, instructions or other information received by the interface circuit 1002 to process the data, instructions or other information, and can send the processed information out through the interface circuit 1002.

[0139] Optionally, the chip may also include memory, which may include read-only memory and random access memory, providing operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0140] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).

[0141] Optionally, the chip can be used in the communication device or communication device involved in the embodiments of this application. Optionally, the interface circuit 1002 can be used to output the execution result of the processor 1001. For the communication methods provided by one or more embodiments of this application, please refer to the foregoing embodiments, which will not be repeated here.

[0142] It should be noted that the functions of the processor 1001 and the interface circuit 1002 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0143] Figure 11This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be a communication device, a chip within the communication device, or a functional module. For example... Figure 11 As shown, the electronic device 1100 includes a processor 1101, a transceiver 1102, and a communication line 1103.

[0144] The processor 1101 is used to execute any step of the communication method provided in the embodiments of this application, and in the process of executing any step of the communication method provided in the embodiments of this application, it may choose to call the transceiver 1102 and the communication line 1103 to complete the corresponding operation.

[0145] Furthermore, the electronic device 1100 may also include a memory 1104. The processor 1101, the memory 1104, and the transceiver 1102 can be connected via a communication line 1103.

[0146] The processor 1101 can be a processor, a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 1101 can also be other devices with processing capabilities, such as circuits, devices, or software modules, without limitation.

[0147] Transceiver 1102 is used to communicate with other devices or other communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Transceiver 1102 can be a module, circuit, transceiver, or any device capable of enabling communication.

[0148] The transceiver 1102 is mainly used for sending and receiving commands and information, and may include a transmitter and a receiver to send and receive commands and information, respectively; operations other than sending and receiving commands and information are implemented by the processor.

[0149] Communication line 1103 is used to transmit information between the various components included in electronic device 1100.

[0150] In one design, the processor can be viewed as a logic circuit, and the transceiver as an interface circuit.

[0151] Memory 1104 is used to store instructions. These instructions can be computer programs.

[0152] The memory 1104 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM). Memory 1104 can also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices. It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0153] It should be noted that the memory 1104 can exist independently of the processor 1101 or can be integrated with the processor 1101. The memory 1104 can be used to store instructions, program code, or some data, etc. The memory 1104 can be located inside or outside the electronic device 1100, without limitation. The processor 1101 is used to execute the instructions stored in the memory 1104 to implement the methods provided in the embodiments of this application.

[0154] In one example, processor 1101 may include one or more processor cores, for example Figure 11 The processor cores are 0 and 1.

[0155] As an optional implementation, the electronic device 1100 includes multiple processors, for example, besides Figure 11 In addition to processor 1101, it may also include processor 1107.

[0156] As an optional implementation, the electronic device 1100 also includes an output device 1105 and an input device 1106. For example, the input device 1106 is a device such as a keyboard, mouse, microphone, or joystick, and the output device 1105 is a device such as a display screen or speaker.

[0157] It should be noted that electronic device 1100 can be a chip system or... Figure 11 Devices with similar structures. The chip system can be composed of chips or include chips and other discrete components. Actions, terminology, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages used for interaction between devices in the embodiments of this application are merely examples; other names can be used in specific implementations without limitation. Furthermore, Figure 11 The structural composition shown does not constitute a limitation on the electronic device 1100, except... Figure 11 In addition to the components shown, the electronic device 1100 may include components that are larger than those shown. Figure 11 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.

[0158] The processor and transceiver described in the embodiments of this application can be implemented on integrated circuits (ICs), analog ICs, radio frequency integrated circuits, mixed-signal ICs, application-specific integrated circuits (ASICs), printed circuit boards (PCBs), electronic devices, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductors (CMOS), n-type metal-oxide-semiconductor (NMOS), p-type metal oxide semiconductors (PMOS), bipolar junction transistors (BJTs), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0159] This application also provides a communication device, which includes at least one processor. When the at least one processor executes program code or instructions, it implements the aforementioned method steps to achieve the communication method in the above embodiments.

[0160] Optionally, the device may further include at least one memory for storing the program code or instructions.

[0161] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a communication device, the communication device performs the aforementioned related method steps to implement the communication method in the above embodiments.

[0162] In one possible implementation, the computer-readable storage medium is a non-transitory computer-readable medium.

[0163] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the communication method described in the above embodiments.

[0164] This application also provides a communication device, which may specifically be a chip, integrated circuit, component, or module. Specifically, the device may include a connected processor and a memory for storing instructions, or the device may include at least one processor for fetching instructions from external memory. When the device is running, the processor can execute instructions to cause the chip to perform the communication methods described in the above-described method embodiments.

[0165] It should be understood that in various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0167] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0169] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0171] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this application, essentially, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0172] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be included within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A communication method characterized by comprising: include: Obtain first information, which is used to indicate whether the peer device supports human image centering technology; If the peer device does not support human image centering technology, the first video stream is cropped according to the local resolution to obtain a first cropped video stream, which is the video stream of the peer device.

2. The method of claim 1, wherein, The method further includes: If the peer device does not support human image centering technology, the first video stream is cropped according to the peer resolution to obtain a second cropped video stream; If the peer device does not support human image centering technology, the second cropped video stream or cropping information of the second cropped video stream is sent, and the cropping information includes at least one of cropping position or cropping frame size.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The second video stream is cropped according to the resolution of the peer device to obtain the third cropped video stream, where the second video stream is the video stream of the local device. The second video stream is cropped according to the local resolution to obtain a fourth cropped video stream; Send the third cropped video stream or cropping information of the third cropped video stream, wherein the cropping information includes at least one of cropping position or cropping frame size.

4. The method according to any one of claims 1 to 3, characterized in that, The step of cropping the first video stream according to the local resolution to obtain the first cropped video stream includes: The first video stream is cropped based on the second information and the local resolution to obtain a first cropped video stream. The second information includes at least one of the cropping frame size of the first cropped video stream or the smoothing coefficient of the first cropped video stream.

5. The method of claim 4, wherein, The method further includes: Receive user actions; The second information is determined based on the user's actions.

6. The method according to claim 4 or 5, characterized in that, The first cropped video stream includes a first video frame and a second video frame. The first video frame includes a first cropping frame, and the second video frame includes a second cropping frame. The second cropping frame is obtained based on the first cropping frame and the smoothing coefficient.

7. A communication device, characterized by include: Transceiver unit and processing unit; The transceiver unit is used to acquire first information, which is used to indicate whether the peer device supports human face centering technology. The processing unit is configured to crop the first video stream according to the resolution of the local device to obtain a first cropped video stream when the remote device does not support the portrait centering technology. The first video stream is the video stream of the remote device.

8. The apparatus of claim 7, wherein, The processing unit is also used for: If the peer device does not support human image centering technology, the first video stream is cropped according to the peer resolution to obtain a second cropped video stream; If the peer device does not support human image centering technology, the second cropped video stream or cropping information of the second cropped video stream is sent, and the cropping information includes at least one of cropping position or cropping frame size.

9. The apparatus of claim 7 or 8, wherein, The processing unit is also used for: The second video stream is cropped according to the resolution of the peer device to obtain the third cropped video stream, where the second video stream is the video stream of the local device. The second video stream is cropped according to the local resolution to obtain a fourth cropped video stream; Send the third cropped video stream or cropping information of the third cropped video stream, wherein the cropping information includes at least one of cropping position or cropping frame size.

10. The apparatus of any one of claims 7 to 9, wherein, The processing unit is specifically used for: The first video stream is cropped based on the second information and the local resolution to obtain a first cropped video stream. The second information includes at least one of the cropping frame size of the first cropped video stream or the smoothing coefficient of the first cropped video stream.

11. The apparatus according to claim 10, characterized in that, The processing unit is also used for: Receive user actions; The second information is determined based on the user's actions.

12. The apparatus according to claim 10 or 11, characterized in that, The first cropped video stream includes a first video frame and a second video frame. The first video frame includes a first cropping frame, and the second video frame includes a second cropping frame. The second cropping frame is obtained based on the first cropping frame and the smoothing coefficient.

13. A communication device comprising at least one processor and a memory, characterized in that, The at least one processor executes a program or instructions stored in a memory to cause the communication device to implement the method of any one of claims 1 to 6.

14. A computer readable storage medium for storing a computer program, characterized in that, When the computer program is run on a computer or processor, it causes the computer or processor to perform the method of any one of claims 1 to 6.

15. A computer program product, comprising instructions therein, wherein the computer program product is characterised in that, When the instructions are executed on a computer or processor, the computer or processor causes the computer or processor to perform the method of any one of claims 1 to 6.

Images