Image frame processing method and electronic device

Abstract

The present application provides an image frame processing method and an electronic device. A second electronic device cooperates with a first electronic device, and the call video on the first electronic device is switched to the second electronic device. The method includes: receiving camera parameters sent by the first electronic device, determining a first target capacity based on the camera parameters and applying for a first memory space and a second memory space, storing a first image and an image after color conversion of the first image in the first memory space and a second memory space in turn; obtaining an image after color conversion from the first memory space or the second memory space for encoding, and sending the encoded image to the first electronic device. In this way, the second electronic device reduces the time spent on frequent memory applications during the color conversion process of the image frame, and reduces the processing delay of each frame of the image in the video call.

Description

[0001] This application is a divisional application of the invention patent application with application number 202210859314.1 submitted to the China Patent Office, application date July 21, 2022, and invention name “Image frame processing method and electronic device”. Technical Field

[0002] The present application relates to the field of terminal devices, and in particular to an image frame processing method and electronic device. Background Art

[0003] When a user is using a mobile phone to make a video call, if he needs to use a computer to complete some work, it will be inconvenient for the user to use the mobile phone and the computer at the same time. At this time, the user can coordinate the mobile phone with the computer, switch the call video on the mobile phone to the computer, and then control the mobile phone to make a video call through the computer.

[0004] In this scenario, if the frame rate of the video image captured by the computer is greater than the frame rate of the image frames sent by the mobile phone to the application to which the video call belongs, the mobile phone needs to perform frame drop processing.

[0005] At present, in the related art, the frame loss processing method in the above scenario will cause continuous frame loss operations, which will cause frame skipping in the video screen of the mobile phone, resulting in a poor user experience. Summary of the invention

[0006] In order to solve the above technical problems, the present application provides an image frame processing method and an electronic device, which accumulates the proportion of redundant frames in the video image frames actually received by the electronic device, and determines whether to discard the current image frame according to the accumulated value of the proportion of redundant frames, thereby reducing the operation of continuous frame loss, avoiding video frame skipping, and improving user experience.

[0007] In the first aspect, the present application provides an image frame processing method. The method is applied to a first electronic device. The method includes: receiving a first image frame sent by a second electronic device, the first image frame belongs to a target application on the first electronic device, and the timestamp corresponding to the first image frame is a first timestamp; determining a first frame rate at which the first electronic device receives image data according to the timestamps of a first number of image frames belonging to the target application that are recently received at the time indicated by the first timestamp; obtaining a first frame loss rate corresponding to the first image frame according to the first frame rate and the target frame rate, the target frame rate being the frame rate at which the first electronic device sends image data to the target application; updating the cumulative frame loss rate according to the first frame loss rate; if the updated cumulative frame loss rate is greater than or equal to the first value, discarding the first image frame. In this way, by accumulating the proportion of redundant frames in the video image frames actually received by the electronic device, and determining whether to discard the current image frame according to the cumulative value of the proportion of redundant frames, the operation of continuous frame loss can be reduced, video frame skipping can be avoided, and the user experience can be improved.

[0008] According to the first aspect, determining the first frame rate of the first electronic device receiving image data according to the timestamps of the first number of image frames belonging to the target application that are recently received at the time indicated by the first timestamp includes: obtaining the time difference between the first timestamp and the first target timestamp, the first target timestamp being the smallest timestamp among the timestamps of the first number of image frames; determining the first frame rate of the electronic device currently receiving image data according to the time difference and the first number. In this way, the actual frame rate corresponding to each image frame can be obtained.

[0009] According to the first aspect, a first frame loss rate corresponding to a first image frame is obtained according to a first frame rate and a target frame rate, wherein the target frame rate is a frame rate at which the first electronic device sends image data to a target application, including: obtaining a frame rate difference between the first frame rate and the target frame rate; determining the first frame loss rate according to the frame rate difference and the first frame rate, wherein the first frame loss rate is equal to a quotient obtained by dividing the frame rate difference by the first frame rate. In this way, the frame loss rate corresponding to each image frame can be obtained.

[0010] According to the first aspect, updating the cumulative frame loss rate according to the first frame loss rate includes: obtaining the first cumulative frame loss rate corresponding to the image frame before the first image frame; determining the updated cumulative frame loss rate according to the first cumulative frame loss rate and the first frame loss rate, wherein the updated cumulative frame loss rate is equal to the sum of the first cumulative frame loss rate and the first frame loss rate. In this way, the cumulative frame loss rate up to the current image frame in each video call can be obtained, providing a basis for determining whether to discard the current image frame.

[0011] According to the first aspect, if the updated cumulative frame loss rate is greater than or equal to the first value, after discarding the first image frame, the method further includes: updating the value of the cumulative frame loss rate to the difference between the cumulative frame loss rate and the first value. In this way, the cumulative frame loss rate can be protected from overflow.

[0012] According to the first aspect, after the value of the cumulative frame loss rate is updated to the difference between the cumulative frame loss rate and the first value, it also includes: receiving a second image frame sent by a second electronic device, the second image frame belongs to a target application, and the second timestamp corresponding to the second image frame is greater than the first timestamp; determining the second frame rate at which the first electronic device receives image data according to the timestamps of the first number of image frames belonging to the target application that are recently received at the time indicated by the second timestamp; obtaining the second frame loss rate corresponding to the second image frame according to the second frame rate and the target frame rate; updating the cumulative frame loss rate according to the second frame loss rate; if the updated cumulative frame loss rate is less than or equal to the second value, sending the second image frame to the target application, the second value is less than the first value. In this way, when the cumulative frame loss rate is less than or equal to the second value, the current image frame is not discarded.

[0013] According to the first aspect, after sending the first target image frame to the target application so that the target application displays the first target image frame, the method further includes: updating the value of the cumulative frame loss rate to the difference between the cumulative frame loss rate and the second value. In this way, the cumulative frame loss rate can be protected from overflow.

[0014] According to the first aspect, after the value of the cumulative frame loss rate is updated to the difference between the cumulative frame loss rate and the first value, the method further includes: receiving a third image frame sent by the second electronic device, the third image frame belonging to the target application, and a third timestamp corresponding to the third image frame being greater than the first timestamp; determining a third frame rate at which the first electronic device receives image data according to the timestamps of a first number of image frames belonging to the target application that are most recently received at the time indicated by the third timestamp; obtaining a third frame loss rate corresponding to the third image frame according to the third frame rate and the target frame rate; and updating the cumulative frame loss rate according to the third frame loss rate;

[0015] If the updated accumulated frame loss rate is less than the first value and greater than the second value, the third image frame is sent to the target application.

[0016] According to the first aspect, the first number is smaller than a preset number value.

[0017] According to the first aspect, the first electronic device is a smart phone, and the second electronic device is a personal computer, a tablet or a smart screen.

[0018] In a second aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, and when the program instructions are executed by the processor, the electronic device executes any one of the image frame processing methods of the first aspect.

[0019] In a third aspect, the present application provides a computer-readable storage medium, comprising a computer program, which, when executed on an electronic device, enables the electronic device to execute any one of the image frame processing methods of the first aspect.

[0020] In a fourth aspect, the present application provides a memory management method. The method is applied to a second electronic device. The method includes: receiving camera parameters sent by a first electronic device, the camera parameters including width information and height information of the captured image; determining a first target capacity according to the width information and the height information; applying for a first memory space and a second memory space, the capacity of the first memory space and the capacity of the second memory space are both equal to the first target capacity; after capturing a first image frame, storing the first image data corresponding to the first image frame and at least one group of first related image data obtained after processing the first image frame in the first memory space and the second memory space in turn. In this way, after applying for memory once, it can be used multiple times until the memory is released when the camera is turned off after the video call ends, which reduces the time consumed by frequent memory applications during image frame processing and reduces the processing delay of each frame of the image in the video call.

[0021] According to the fourth aspect, after capturing the first image frame, the first image data corresponding to the first image frame and at least one group of second image data obtained after processing the first image frame are stored alternately in the capacity of the first memory space and the second memory space, and it also includes: capturing the second image frame; determining whether the data corresponding to the second image frame is equal to the first target capacity; if so, storing the second image data corresponding to the second image frame and at least one group of second related image data obtained after processing the second image frame alternately in the first memory space and the second memory space.

[0022] According to the fourth aspect, after capturing the first image frame, the first image data corresponding to the first image frame and at least one group of second image data obtained after processing the first image frame are stored alternately in the capacity of the first memory space and the second memory space, and it also includes: capturing the third image frame; determining whether the length of the third image data corresponding to the third image frame is equal to the first target capacity; if not, determining the second target capacity according to the length of the third image data; applying for the third memory space and the fourth memory space, the capacity of the third memory space and the capacity of the fourth memory space are both equal to the second target capacity; storing the third image data corresponding to the third image frame and at least one group of third related image data obtained after processing the third image frame alternately in the third memory space and the fourth memory space.

[0023] According to the fourth aspect, after capturing the first image frame, the first image data corresponding to the first image frame and at least one set of second image data obtained after processing the first image frame are stored alternately in the capacity of the first memory space and the second memory space, and it also includes: receiving an instruction to turn off the camera; releasing the capacity of the first memory space and the second memory space.

[0024] In a fifth aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, and when the program instructions are executed by the processor, the electronic device executes any one of the memory management methods of the third aspect.

[0025] In a sixth aspect, the present application provides a computer-readable storage medium, comprising a computer program, which, when executed on an electronic device, enables the electronic device to execute the memory management method of any one of the aforementioned third aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic structural diagram of an illustrative electronic device 100;

[0027] Figure 2 1 is a software structure block diagram of an electronic device 100 according to an embodiment of the present application;

[0028] Figure 3 is a schematic diagram showing an example of two users communicating instantly via mobile phones;

[0029] Figure 4 for Figure 3 Schematic diagram of user B using mobile phone B and computer C to collaborate;

[0030] Figure 5 is a flowchart of an exemplary method for processing an image frame;

[0031] Figure 6 An example flowchart of a memory management method is shown as an example;

[0032] Figure 7 Another example flowchart of a memory management method is shown in FIG. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0035] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0036] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0037] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0038] When a user is making a video call through a first electronic device, if he needs to use a second electronic device to complete some work, using two electronic devices at the same time will cause inconvenience to the user. In this case, the first electronic device can be coordinated with the second electronic device, and the audio and video can be switched to the second electronic device, so that the second electronic device can control the first electronic device to make a video call.

[0039] The image frame processing method in the embodiment of the present application can be applied to a first electronic device, which can be, for example, a smart phone, a tablet or other electronic device. The structure of the first electronic device can be as follows: Figure 1 shown.

[0040] Figure 1 FIG. 1 is a schematic diagram of the structure of an electronic device 100 exemplarily shown. It should be understood that Figure 1 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0041] See also Figure 1 The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, etc.

[0042] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0043] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0044] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0045] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0046] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0047] The mobile communication module 150 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0048] The wireless communication module 160 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0049] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0050] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0051] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0052] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0053] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0054] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0055] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0056] Figure 2 1 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0057] The layered architecture of the electronic device 100 divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system may include an application layer, an application framework layer, a system layer, and a kernel layer.

[0058] The application layer can include a series of application packages.

[0059] like Figure 2 As shown, the application package may include application programs such as camera, gallery, call, map, WLAN, Bluetooth, video, and instant messaging modules. The instant messaging module may be, for example, any application module that can make an instant video call. A user may make a video call with other users through the instant messaging module of the first electronic device.

[0060] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0061] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a resource manager, a view system, an image frame processing module, etc.

[0062] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0063] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0064] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0065] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0066] Among them, the image frame processing module is used to execute the image frame processing method of the embodiment of the present application. When a user makes a video call with other users through the instant communication module of the first electronic device, the video call can be coordinated to the second electronic device. The second electronic device uses its own camera to capture the image frame of the video call and sends it to the first electronic device. The image frame processing module of the first electronic device receives the image frame sent by the second electronic device, and processes the image frame using the image frame processing method of the embodiment of the present application. After the image frame is processed by the image frame processing module, if the processing result is to discard the image frame, the image frame processing module discards the image frame and no longer sends the image frame to the instant communication module; if the processing result is to display the image frame, the image frame processing module sends the image frame to the instant communication module, and the instant communication module displays the image frame.

[0067] The application framework layer may also include a phone manager ( Figure 2 The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connection, hang-up, etc.).

[0068] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0069] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0070] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0071] The system library can include multiple functional modules. Figure 2 As shown, in the embodiment of the present application, the system library may include

[0072] Surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (also known as 2D graphics engine, for example: SGL), etc.

[0073] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0074] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0075] The 2D graphics engine is a drawing engine for 2D drawing. The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0076] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0077] The kernel layer is the layer between hardware and software.

[0078] like Figure 2 As shown, the kernel layer may include modules such as display driver, Bluetooth driver, Wi-Fi driver, Bluetooth driver, audio driver, sensor driver, etc.

[0079] Understandably, Figure 2The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0080] The present application is described in detail below through examples.

[0081] Here, taking the first electronic device being a mobile phone and the second electronic device being a computer as an example, the application scenario of the image frame processing method of the embodiment of the present application is explained.

[0082] Figure 3 Schematic diagram showing an example of two users communicating instantly via mobile phones. Figure 3 , user A and user B communicate with each other through mobile phone A and mobile phone B. Both mobile phone A and mobile phone B have an instant messaging application installed, for example, the instant messaging application can be any application that can make an instant video call.

[0083] Figure 4 for Figure 3 Schematic diagram of user B using mobile phone B and computer C to collaborate. Figure 4 After mobile phone B and computer C collaborate, the call video on mobile phone B is switched to computer C. The camera in computer C collects video image frames of user B and sends them to mobile phone B. The camera in mobile phone B no longer needs to collect video image frames of user B. Mobile phone B processes the video image frames sent by computer C using the image frame processing and sending method of the embodiment of the present application.

[0084] exist Figure 4 In the collaborative scenario shown, mobile phone B is called the center-side device, and computer C is called the end-side device or device-side device. Figure 4 The terminal device in the example is a computer, but it should be understood that this is only an illustrative example of the terminal device.

[0085] The center-side device can set one or more video streams with different frame rates for each instant messaging application, and the terminal-side device uses the maximum frame rate of all video streams of each instant messaging application to capture images during the video call.

[0086] For example, assume that for an instant messaging application on mobile phone B, mobile phone B sets two video streams with different frame rates, one of which is a video stream that displays user B's video on the local display screen of mobile phone B, and the other video stream is a video stream that sends user B's video to mobile phone A. Computer C collects the video image frames of user B based on the video stream with the largest frame rate among the two video streams.

[0087] For example, suppose that mobile phone B sets two video streams with different frame rates, wherein the frame rate of the video stream that displays user B's video on the local display screen of mobile phone B is frame rate 1, and the frame rate of the video stream that sends user B's video to mobile phone A is frame rate 2, and frame rate 1 is greater than frame rate 2. Then, the frame rate of the video image frames collected by computer C is frame rate 1. Then, computer C sends the collected video image frames to mobile phone B at a frame rate of frame rate 1, that is, the frame rate of the video image frames received by mobile phone B from computer C is frame rate 1. At this time, for the video stream that displays user B's video on the local display screen of mobile phone B, since the frame rate of the video stream is equal to frame rate 1, mobile phone B can directly send the received video image frames to this video stream. However, for the video stream that sends user B's video to mobile phone A, since the frame rate of the video stream is less than frame rate 1, mobile phone B needs to drop frames for the video images received from computer C, and then send the remaining video image frames to this video stream.

[0088] For example, suppose computer C sends 30 frames of images to mobile phone B per second, and mobile phone B sends 15 frames of images to mobile phone A per second. At this time, mobile phone B needs to discard 15 frames of images per second and send the remaining 15 frames of images to mobile phone A.

[0089] Below Figure 4 Taking mobile phone B in the figure as an example, the image frame processing method of an embodiment of the present application is explained.

[0090] Figure 5 FIG. 1 is a flowchart of an exemplary method for processing an image frame. In this embodiment, the method for processing an image frame is applied to mobile phone B. The method may include the following steps:

[0091] S501. Mobile phone B receives an image frame sent by computer C. The image frame belongs to application 1 on mobile phone B, and the timestamp corresponding to the image frame is timestamp 1.

[0092] S502 : Determine the actual frame rate at which mobile phone B receives image data according to the timestamps of the x image frames belonging to application 1 that are most recently received at the time indicated by timestamp 1 .

[0093] Wherein, x is a natural number.

[0094] Assuming that the current image frame is the i-th image frame, the corresponding timestamp is timeStamp, and the timestamp corresponding to the (ix)-th image frame is timeStamp(ix), then the actual frame rate curFrameRate(i) of the current image frame can be calculated by the following formula (1).

[0095]

[0096] For example, i=11, x=10, the actual frame rate when mobile phone B receives the 11th image frame in the video can be calculated. The actual frame rate of the first image frame in the video does not need to be calculated, and the frame drop rate dropRate(1) of the first image frame defaults to 0. When calculating the actual frame rate of the second image frame in the video, x=1. When calculating the actual frame rates of the third to tenth image frames in the video, x can be a natural number from 1 to 9 that is less than i.

[0097] It should be noted that if the value of x is too large, the accuracy of the actual frame rate may be affected. Therefore, the value of x needs to be controlled within a certain range.

[0098] In one example, x is less than or equal to 60.

[0099] S503 . Obtain a frame loss rate 1 corresponding to the image frame according to the actual frame rate and the target frame rate. The target frame rate is the frame rate at which the mobile phone B sends image data to the application 1 .

[0100] Assuming that the target frame rate is targetFrameRate, the frame drop rate dropRate(i) corresponding to the i-th image frame can be calculated by the following formula (2).

[0101]

[0102] S504: Update the cumulative frame loss rate according to the frame loss rate 1.

[0103] The cumulative frame loss rate is obtained by adding up the frame loss rates of all image frames sent by computer C and received by mobile phone B.

[0104] The cumulative frame loss rate dropRate can be calculated using the following formula (3).

[0105] dropRate=∑dropRate(i) Formula (3)

[0106] S505: Determine whether the updated cumulative frame loss rate is greater than or equal to the first value, if so, execute step S507, otherwise execute step S506.

[0107] S506: Determine whether the updated cumulative frame loss rate is less than or equal to the second value. If so, execute step S509; otherwise, execute step S511. Updated cumulative frame loss rate ≤ second value

[0108] S507: Discard the image frame.

[0109] In this way, application 1 does not need to process the image frames.

[0110] S508. Update the value of the cumulative frame loss rate to the difference between the cumulative frame loss rate and the first value, and return to step S501.

[0111] For example, the first value is 1, then dropRate=dropRate-1.

[0112] S509: Send the image frame to application 1.

[0113] In this way, application 1 needs to process the image frames.

[0114] When the updated cumulative frame loss rate is less than or equal to the second value, if the time interval for application 1 to process image frames is full, but mobile phone B has not received a new image frame, then mobile phone B can send the most recent received image frame at this moment to application 1 so that application 1 can process the image frame at the set frame rate.

[0115] S510, update the value of the cumulative frame loss rate to the difference between the cumulative frame loss rate and the second value, and return to step S501.

[0116] For example, the second value is -1, then dropRate=dropRate-(-1)=dropRate+1.

[0117] S511, sending the image frame to application 1, and returning to step S501.

[0118] In this step, the value of the accumulated frame loss rate remains unchanged from the accumulated frame loss rate value calculated in step S504.

[0119] Assume that the video image frames received by mobile phone B from computer C include image frames a1 to a100, x=10, and the processing process of mobile phone B on image frames a1 to a100 is as follows:

[0120] The frame loss rate of the image frames a1 to a10 is 0, and the cumulative frame loss rate is also 0.

[0121] For image frame a11, the actual frame rate and cumulative frame loss rate are calculated according to formula (1) and formula (2). Assuming that the cumulative frame loss rate corresponding to image frame a11 is greater than -1 and less than 1, then image frame a11 is directly sent to the instant messaging application, assuming it is application 1.

[0122] The processing process of the image frames a12 to a20 is assumed to be the same as the processing process of the image frame a11.

[0123] For image frame a21, the actual frame rate and cumulative frame loss rate are calculated according to formula (1) and formula (2). Assuming that the cumulative frame loss rate corresponding to image frame a21 is greater than 1, mobile phone B discards image frame a21 and does not send image frame a21 to application 1. Then, the cumulative frame loss rate is updated according to dropRate=dropRate-1.

[0124] The processing process of the image frames a22 to a30 is assumed to be the same as the processing process of the image frame a11.

[0125] For image frame a31, the actual frame rate and cumulative frame loss rate are calculated according to formula (1) and formula (2). Assuming that the cumulative frame loss rate corresponding to image frame a21 is less than -1, mobile phone B sends image frame a30 to application 1. Then, the cumulative frame loss rate is updated according to dropRate = dropRate + 1.

[0126] …

[0127] The image frame processing method of the embodiment of the present application accumulates the proportion of redundant frames in the video image frames actually received by the electronic device, and determines whether to discard the current image frame based on the accumulated value of the proportion of redundant frames, thereby reducing the operation of continuous frame dropping, avoiding video frame skipping, and improving user experience.

[0128] Furthermore, the present invention directly adopts the floating point operation method, and the processing process is simple.

[0129] In addition, the error between the cumulative frame loss rate calculated by the image frame processing method of the embodiment of the present application and the theoretical frame loss rate is mainly due to the calculation error of the floating point number, while the frame loss standard dropRate is accumulated by the frame rate when each frame arrives, and each received frame affects the subsequent frame loss rate with the current frame rate, which is closer to the theoretical frame loss rate. Therefore, the frame loss processing of the image frame processing method of the embodiment of the present application is more accurate and reasonable.

[0130] Below, we will Figure 4 The computer C in the example is replaced by a tablet D to illustrate the memory management method of the embodiment of the present application. That is, in the following embodiment, the center side device is a mobile phone B, and the terminal side device is a tablet D. Among them, the hardware structure of the tablet D can be adopted Figure 1 In the software structure of tablet D, the application framework layer includes a memory management module, and the memory management module can be used to execute the memory management method of the embodiment of the present application.

[0131] exist Figure 4In the collaborative scenario shown, mobile phone B sends the camera parameters to computer C, tablet D captures the image frames in the video call, and then sends the image frames to mobile phone B. After tablet D captures the image frames, it stores the image frames in the memory, and then tablet D performs at least one color conversion process on the image frames. The image data obtained after each color conversion process is also stored in the memory. After all the color conversion processing steps are completed, tablet D encodes the corresponding image data and then sends the encoded image data to mobile phone B.

[0132] In the related art, memory is applied for storage of image data of image frames collected by the tablet D and image data obtained after each color conversion process of the image frames, and the memory is released after use.

[0133] For example, after tablet D captures image frame a0, it applies for memory 1 and stores image frame a0 in memory 1;

[0134] Next, tablet D takes out image frame a0 from memory 1, releases memory 1, applies for memory 2, performs the first color conversion process on image frame a0 to obtain image data a1, and stores image data a1 in memory 2;

[0135] Next, tablet D takes out image data a1 from memory 2, releases memory 2, applies for memory 3, performs a second color conversion process on image data a1 to obtain image data a2, and stores image data a2 in memory 3;

[0136] Then, tablet D takes out image data a2 from memory 2, releases memory 3, encodes image data a2 to obtain encoded image data a3, applies for memory 4, and stores encoded image data a3 in memory 4;

[0137] Then, tablet D takes out the encoded image data a3 from memory 4 and sends the encoded image data a3 to mobile phone B.

[0138] The above timings for releasing each memory are merely exemplary, and the embodiments of the present application do not limit the timing for releasing the memory.

[0139] In the above-mentioned memory application method, tablet D needs to apply for multiple memory for each frame of image captured, which is time-consuming and causes a large processing delay for each frame of image.

[0140] An embodiment of the present application provides a memory management method that can reduce the processing delay of the terminal device on the collected image frames in the above-mentioned collaborative scenario.

[0141] Figure 6 FIG. 1 is a flowchart of an exemplary memory management method. In this embodiment, the memory management method can be applied to Figure 4 Plate D shown. See Figure 6 In the process of tablet D and mobile phone B making a video call, the method may include the following steps:

[0142] S601. Tablet D receives camera parameters sent by mobile phone B. The camera parameters include width information and height information of a captured image.

[0143] In this embodiment, width is used to represent the width of the image, and hight is used to represent the height of the image. The camera parameters sent by mobile phone B to tablet D include width and hight.

[0144] The camera parameters may also include information such as image resolution, frame rate of captured images, and color space.

[0145] S602: Determine a first target capacity according to the width information and the height information.

[0146] After analysis, the inventors found that the memory size required for storing the collected original image data and for storing the image data after color conversion processing of the original image data is relatively fixed, and the memory size is only related to the width information and height information of the collected image.

[0147] The size of the first target capacity can be calculated using the following formula (4).

[0148]

[0149] Among them, memorySize indicates the memory capacity.

[0150] S603: Apply for a first memory space and a second memory space, wherein the capacity of the first memory space and the capacity of the second memory space are both equal to the first target capacity.

[0151] In this step, two memory spaces are applied for in one application process.

[0152] S604: After acquiring the first image frame, first image data corresponding to the first image frame and at least one group of first related image data obtained after processing the first image frame are stored alternately in the first memory space and the second memory space.

[0153] For example, after tablet D captures image frame a0, it stores image frame a0 in the first memory space;

[0154] Next, the tablet D takes out the image frame a0 from the first memory space, performs a first color conversion process on the image frame a0 to obtain image data a1, and stores the image data a1 in the second memory space;

[0155] Next, the tablet D takes out the image data a1 from the second memory space, performs a second color conversion process on the image data a1 to obtain the image data a2, and stores the image data a2 in the first memory space;

[0156] Then, the tablet D takes out the image data a2 from the first memory space, encodes the image data a2 to obtain the encoded image data a3, applies for the memory 4, and stores the encoded image data a3 in the memory 4;

[0157] Then, tablet D takes out the encoded image data a3 from memory 4 and sends the encoded image data a3 to mobile phone B.

[0158] After processing the image frame a0 and encoding the image data a3 to send to the mobile phone B, the tablet D does not release the first memory space and the second memory space, but directly uses the first memory space and the second memory space for processing subsequent acquired image frames.

[0159] For images of the same size such as image frame b0, image frame c0, image frame d0, etc. acquired after image frame a0, the tablet D stores the original image frame data and the processed image data in the same manner as that of processing image frame a0.

[0160] If during a collaborative video call, the width and height of the image frame captured by tablet D remain unchanged, the first memory space and the second memory space are kept until the video call ends and the camera is turned off and then released.

[0161] If the width and height of the image frame captured by tablet D change during a collaborative video call, tablet D will release the first memory space and the second memory space and reapply for two new memory spaces.

[0162] Therefore, for the second frame and subsequent image frames collected during the collaborative video call, the Figure 7 Process as shown.

[0163] Figure 7 Another example flow chart of a memory management method is shown in FIG. Figure 7 In the process of tablet D cooperating with mobile phone B to make a video call, the method may further include the following steps:

[0164] S701, collecting image frames.

[0165] S702, determine whether the data volume corresponding to the image frame is equal to the first target capacity, if so, execute step S703, otherwise, execute step S704.

[0166] S703: Store the second image data corresponding to the image frame and at least one group of second related image data obtained after processing the second image frame in the first memory space and the second memory space in turn, and the processing of the current image frame is completed.

[0167] S704: Determine a second target capacity according to the length of the image data corresponding to the image frame.

[0168] S705: Apply for a third memory space and a fourth memory space, wherein the capacity of the third memory space and the capacity of the fourth memory space are both equal to the second target capacity.

[0169] Before step S705, after step S704, or after step S705, the first memory space and the second memory space may be released.

[0170] S706. Store the second image data corresponding to the image frame and at least one group of second related image data obtained after processing the second image frame in the third memory space and the fourth memory space in turn, and the processing of the current image frame is completed.

[0171] Give an example.

[0172] After processing the above image frame a0, the tablet D continues to collect image frame b0.

[0173] The tablet D determines whether the data volume corresponding to the image frame b0 is equal to the first target capacity. If so, the tablet D stores the image frame b0 in the second memory space.

[0174] Next, the tablet D takes out the image frame b0 from the second memory space, performs a first color conversion process on the image frame b0 to obtain image data b1, and stores the image data b1 in the first memory space;

[0175] Next, the tablet D takes out the image data b1 from the first memory space, performs a second color conversion process on the image data b1 to obtain image data b2, and stores the image data b2 in the second memory space;

[0176] Then, the tablet D takes out the image data b2 from the second memory space, encodes the image data b2 to obtain the encoded image data b3, applies for the memory 4, and stores the encoded image data b3 in the memory 4;

[0177] Then, tablet D takes out the encoded image data b3 from memory 4 and sends the encoded image data b3 to mobile phone B.

[0178] Then, after the tablet D processes the above-mentioned image frame b0, the width and / or height of the acquired image changes, resulting in a change in the capacity of the memory to be applied for.

[0179] After processing the above-mentioned image frame b0, the tablet D continues to collect image frame c0.

[0180] The tablet D determines whether the amount of data corresponding to the image frame c0 is equal to the first target capacity. If not, the tablet D determines the second target capacity according to the length of the image data corresponding to the image frame c0.

[0181] Tablet D applies for a third memory space and a fourth memory space, and the capacity of the third memory space and the capacity of the fourth memory space are both equal to the second target capacity;

[0182] Tablet D stores image frame c0 in the third memory space;

[0183] Next, the tablet D takes out the image frame c0 from the third memory space, performs a first color conversion process on the image frame c0 to obtain image data c1, and stores the image data c1 in the fourth memory space;

[0184] Next, the tablet D takes out the image data c1 from the fourth memory space, performs a second color conversion process on the image data c1 to obtain image data c2, and stores the image data c2 in the third memory space;

[0185] Then, the tablet D takes out the image data c2 from the third memory space, encodes the image data c2 to obtain the encoded image data c3, and stores the encoded image data c3 in the memory 4;

[0186] Then, tablet D takes out application memory 4 from memory 4, encodes image data c3, and sends the encoded image data c3 to mobile phone B.

[0187] …

[0188] And so on.

[0189] In one example, when the collaborative video call ends, the memory management method may further include the following steps:

[0190] Receive a command to turn off the camera;

[0191] The first memory space and the second memory space are released.

[0192] The memory management method provided in the embodiment of the present application, after applying for memory, if the memory can be used by image frames all the time (that is, the size of the captured image frames has not changed), then the memory will not be released until the camera is turned off after the video call ends. In this way, there is no need to apply for and release memory for each captured image frame, thereby reducing the time consumption of image processing.

[0193] The memory management method provided in the embodiment of the present application can use the memory multiple times after applying for it once, and the memory will be released when the camera is turned off after the video call ends, thereby reducing the time consumed by frequent memory applications during image frame processing and reducing the processing delay of each frame of the image.

[0194] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is coupled to the processor, and the memory stores program instructions. When the program instructions are executed by the processor, the electronic device performs the image frame processing method performed by the aforementioned electronic device.

[0195] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is coupled to the processor, and the memory stores program instructions. When the program instructions are executed by the processor, the electronic device implements the memory management method executed by the aforementioned electronic device.

[0196] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0197] This embodiment also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image frame processing method in the above-mentioned embodiment.

[0198] This embodiment also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the memory management method in the above-mentioned embodiment.

[0199] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the image frame processing method in the above-mentioned embodiment.

[0200] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the memory management method in the above-mentioned embodiment.

[0201] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the image frame processing method in the above-mentioned method embodiments.

[0202] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0203] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0208] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0209] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0210] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0211] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0212] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image frame processing method, characterized in that: Applied to a second electronic device, the second electronic device cooperates with a first electronic device, and a call video on the first electronic device is switched to the second electronic device, the method comprising: receiving camera parameters sent by the first electronic device, where the camera parameters include width information and height information of a captured image; Determining a first target capacity according to the width information and the height information; Applying for a first memory space and a second memory space, wherein the capacity of the first memory space and the capacity of the second memory space are both equal to the first target capacity; After acquiring a first image frame, first image data corresponding to the first image frame and at least one group of first related image data obtained after color conversion processing is performed on the first image frame are stored alternately in the first memory space and the second memory space; After completing the color conversion process on the first image frame, acquiring the image frame after the color conversion from the first memory space or the second memory space as the image frame to be encoded; Encoding the image frame to be encoded, and applying for a fifth memory space; storing the encoded image frame into the fifth memory space; Acquire the encoded image frame from the fifth memory space, and send the encoded image frame to the first electronic device; After acquiring a first image frame, first image data corresponding to the first image frame and at least one set of second image data obtained after color conversion processing of the first image frame are stored alternately in the capacity of the first memory space and the second memory space, further comprising: When the video call ends, receiving an instruction to turn off the camera; The capacity of the first memory space and the second memory space are released.

2. The method according to claim 1, characterized in that After acquiring the first image frame, the method further includes: storing first image data corresponding to the first image frame and at least one group of first related image data obtained after color conversion processing is performed on the first image frame in the first memory space and the second memory space in turn: Acquiring a second image frame; determining whether data corresponding to the second image frame is equal to the first target capacity; If yes, the second image data corresponding to the second image frame and at least one group of second related image data obtained after color conversion processing is performed on the second image frame are stored in the first memory space and the second memory space in turn.

3. The method according to claim 1, characterized in that After acquiring a first image frame, first image data corresponding to the first image frame and at least one set of second image data obtained after color conversion processing of the first image frame are stored alternately in the capacity of the first memory space and the second memory space, further comprising: Acquiring a third image frame; determining whether the length of the third image data corresponding to the third image frame is equal to the first target capacity; If not, determining a second target capacity according to the length of the third image data; Applying for a third memory space and a fourth memory space, wherein the capacity of the third memory space and the capacity of the fourth memory space are both equal to the second target capacity; The third image data corresponding to the third image frame and at least one group of third related image data obtained after processing the third image frame are stored alternately in the third memory space and the fourth memory space.

4. The method according to claim 3, characterized in that After determining the second target capacity according to the length of the third image data, the method further includes: The first memory space and the second memory space are released.

5. The method according to any one of claims 1 to 4, characterized in that The camera parameters also include: image resolution, frame rate of captured images, and color space information.

6. The method according to claim 5, characterized in that The frame rate of the captured image includes: the frame rate of each video stream set by the call video application in the first electronic device; when the camera parameters include the frame rate of the captured image, the method further includes: The second electronic device obtains, by the first electronic device, a frame rate of each video stream set by a call video application; The second electronic device obtains the maximum frame rate of the call video application as a preset frame rate; The second electronic device captures the first image frame according to the preset frame rate.

7. The method according to claim 6, characterized in that Sending the encoded image to the first electronic device includes: The encoded image frames are sent to the first electronic device according to the preset frame rate, so that after receiving the encoded image frames, the first electronic device determines the first frame rate at which the first electronic device receives the image according to the first timestamp of the encoded image frames, obtains the first frame loss rate corresponding to the encoded image frames according to the first frame rate and the target frame rate, updates the cumulative frame loss rate according to the first frame loss rate, and discards the encoded image frames if the updated cumulative frame loss rate is greater than or equal to a first value, wherein the target frame rate is the frame rate at which the first electronic device sends image data to a target application.

8. An image frame processing method, characterized in that: Applied to a system including a first electronic device and a second electronic device, the second electronic device cooperates with the first electronic device, and a call video on the first electronic device is switched to the second electronic device, the method comprising: The first electronic device sends camera parameters to the second electronic device, where the camera parameters include width information and height information of the captured image; In response to receiving the camera parameter, the second electronic device determines a first target capacity according to the width information and the height information; The second electronic device applies for a first memory space and a second memory space, and the capacity of the first memory space and the capacity of the second memory space are both equal to the first target capacity; After acquiring the first image frame, the second electronic device stores the first image data corresponding to the first image frame and at least one group of first related image data obtained after performing color conversion processing on the first image frame in the first memory space and the second memory space in turn; After completing the color conversion process on the first image frame, the second electronic device obtains the image frame after the color conversion from the first memory space or the second memory space as the image frame to be encoded; The second electronic device encodes the image frame to be encoded and applies for a fifth memory space; The second electronic device stores the encoded image frame in the fifth memory space; The second electronic device obtains the encoded image frame from the fifth memory space, and sends the encoded image frame to the first electronic device; After acquiring the first image frame, the second electronic device stores first image data corresponding to the first image frame and at least one set of second image data obtained after color conversion processing of the first image frame in the capacity of the first memory space and the second memory space in turn, further comprising: When the video call ends, the second electronic device receives an instruction to turn off the camera; The second electronic device releases the capacity of the first memory space and the second memory space.

9. The method according to claim 8, characterized in that The method further comprises: In response to receiving the encoded image frame sent by the second electronic device, the first electronic device determines a first frame rate at which the first electronic device receives the image according to a first timestamp of the encoded image frame; The first electronic device obtains a first frame loss rate corresponding to the first image frame according to the first frame rate and a target frame rate, where the target frame rate is a frame rate at which the first electronic device sends image data to a target application; The first electronic device updates the accumulated frame loss rate according to the first frame loss rate; If the updated accumulated frame loss rate is greater than or equal to the first value, the first electronic device discards the received image frame.

10. The method according to claim 9, characterized in that The first electronic device determines, according to the first timestamp of the encoded image frame, a first frame rate at which the first electronic device receives the image, including: The first electronic device acquires a time difference between the first timestamp and a first target timestamp, where the first target timestamp is a minimum timestamp among the timestamps of a first number of image frames; The first electronic device determines a first frame rate at which the electronic device currently receives image data according to the time difference and the first number.

11. The method according to claim 9, characterized in that The first electronic device updates the accumulated frame loss rate according to the first frame loss rate, including: The first electronic device acquires a frame rate difference between the first frame rate and the target frame rate; The first electronic device determines a first frame loss rate according to the frame rate difference and the first frame rate, where the first frame loss rate is equal to a quotient obtained by dividing the frame rate difference by the first frame rate.

12. The method according to any one of claims 8 to 11, characterized in that After acquiring the first image frame, the second electronic device stores the first image data corresponding to the first image frame and at least one group of first related image data obtained after performing color conversion processing on the first image frame in the first memory space and the second memory space in turn, the method further includes: The second electronic device collects a second image frame; The second electronic device determines whether the data corresponding to the second image frame is equal to the first target capacity; If so, the second electronic device stores the second image data corresponding to the second image frame and at least one group of second related image data obtained after color conversion processing of the second image frame in the first memory space and the second memory space in turn.

13. The method according to any one of claims 8 to 11, characterized in that: After acquiring the first image frame, the second electronic device stores first image data corresponding to the first image frame and at least one set of second image data obtained after color conversion processing of the first image frame in the capacity of the first memory space and the second memory space in turn, further comprising: The second electronic device collects a third image frame; The second electronic device determines whether the length of the third image data corresponding to the third image frame is equal to the first target capacity; If not, the second electronic device determines a second target capacity according to the length of the third image data; The second electronic device applies for a third memory space and a fourth memory space, and the capacity of the third memory space and the capacity of the fourth memory space are both equal to the second target capacity; The second electronic device stores the third image data corresponding to the third image frame and at least one group of third related image data obtained after processing the third image frame in the third memory space and the fourth memory space in turn.

14. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the image frame processing method described in any one of claims 1-7.

15. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program runs on an electronic device, the electronic device executes the image frame processing method as described in any one of claims 1 to 7.

Images