Data processing method and device, mobile terminal and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device, a mobile terminal and a storage medium. The method comprises: obtaining a magnification corresponding to current frame image data in a process of performing a smooth zooming operation, the current frame image data being image data displayed by the mobile terminal in real time in the process of performing the smooth zooming operation; obtaining a data processing mode corresponding to the magnification; and processing a plurality of image data streams based on the data processing mode, the plurality of image data streams being image data streams output by the plurality of cameras. In this way, different magnifications correspond to different data processing modes, and the plurality of image data can be processed differently according to the current magnification, thereby improving the flexibility of processing the plurality of image data.

Description

Technical Field

[0001] This application belongs to the field of terminal control technology, specifically relating to a data processing method, device, mobile terminal, and storage medium. Background Technology

[0002] When taking a picture, the camera first zooms in (i.e., adjusts the focal length) before capturing the image. The focal length is the distance from the center of the lens to the point where the image appears sharpest on the sensor surface.

[0003] Since the zoom distance of a single camera is always limited, a second camera is added to mobile devices to meet the focal length needs of various usage scenarios, forming a dual-camera system. In mobile devices with dual cameras, one camera is designated as the main camera, suitable for everyday scenarios; the other camera serves as the secondary camera, with a different focal length than the main camera. When zooming is required to capture an object, the control chip in the mobile device controls the zoom process, directly switching from the main camera's focal length to the secondary camera's focal length. The zoom process is smooth, and the preview image does not flicker.

[0004] However, in some special applications, the magnification range covered by dual cameras includes multiple ranges, and the imaging effects corresponding to these ranges vary. In some magnification ranges, the imaging effect of dual cameras still does not meet the desired results. Therefore, additional cameras are added to the dual-camera setup to meet these needs. However, the flexibility of processing multi-channel data from multiple cameras still needs improvement. Summary of the Invention

[0005] In view of the above problems, this application proposes a data processing method, apparatus, mobile terminal, and storage medium to improve the above problems.

[0006] In a first aspect, embodiments of this application provide a data processing method applied to a mobile terminal, the mobile terminal including multiple cameras, the method including: during a smooth zoom operation, obtaining a magnification corresponding to the current frame image data, the current frame image data being the image data displayed in real time by the mobile terminal during the smooth zoom operation; obtaining a data processing method corresponding to the magnification; and processing multiple image data streams based on the data processing method, the multiple image data streams being the image data streams output by the multiple cameras.

[0007] Secondly, embodiments of this application provide a data processing apparatus operating on a mobile terminal, the mobile terminal including multiple cameras, the apparatus including: a magnification acquisition unit, used to acquire the magnification corresponding to the current frame image data during a smooth zoom operation, the current frame image data being the image data displayed in real time by the mobile terminal during the smooth zoom operation; a mode acquisition unit, used to acquire the data processing mode corresponding to the magnification; and a processing unit, used to process multiple image data streams based on the data processing mode, the multiple image data streams being the image data streams output by the multiple cameras.

[0008] Thirdly, embodiments of this application provide a mobile terminal, including one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run.

[0010] This application provides a data processing method, apparatus, mobile terminal, and storage medium. During a smooth zoom operation, the magnification corresponding to the current frame image data is obtained. The current frame image data is the image data displayed in real-time by the mobile terminal during the smooth zoom operation. Then, the data processing method corresponding to this magnification is obtained. Based on the data processing method, multiple image data streams, which are image data streams output by multiple cameras, are processed. Through this method, different magnifications correspond to different data processing methods, thus allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of processing multiple image data streams. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of a data processing method according to an embodiment of this application is shown;

[0013] Figure 2 A flowchart of a data processing method according to another embodiment of this application is shown;

[0014] Figure 3 A flowchart of a data processing method according to another embodiment of this application is shown;

[0015] Figure 4 A schematic diagram of the configuration of the preset outflow strategy in another embodiment of this application is shown;

[0016] Figure 5 A flowchart of a data processing method according to another embodiment of this application is shown;

[0017] Figure 6 This paper shows a structural block diagram of a data processing apparatus according to an embodiment of the present application;

[0018] Figure 7 This paper shows a structural block diagram of a data processing apparatus according to an embodiment of the present application;

[0019] Figure 8 This invention illustrates a structural block diagram of a mobile terminal used to execute a data processing method according to an embodiment of the present invention.

[0020] Figure 9 This application shows a storage unit for storing or carrying program code that implements the data processing method according to the embodiments of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] In current mobile terminal camera lens support solutions, smooth zoom is a crucial function, frequently required for scenarios such as video recording and photo taking. When taking a photo using a mobile terminal's camera, zooming (i.e., adjusting the focal length) is performed before shooting. The focal length is the distance from the center point of the lens to the point where the image appears sharpest on the sensor plane.

[0023] For mobile devices, there is usually support for 3 to 5 different types of lenses. Generally, the camera operation page of a multi-camera mobile device will have focal length switching options of 0.6X, 1X, 2X or 5X, 10X. The smaller the number representing the focal length, the wider the shooting range; the larger the number representing the focal length, the farther the shooting range. This roughly corresponds to the focal lengths of ultra-wide-angle lens, main lens, and telephoto lens.

[0024] Main camera lens: As we know, the main camera of most mobile terminals is generally a wide-angle lens with an equivalent focal length of about 28mm. This focal length is close to the field of view of "what the human eye sees". What you see is what you shoot. Therefore, the main camera is also the most frequently used camera.

[0025] When you open the camera's operation page on your mobile device, you'll see a 1X value displayed. 1X is the main camera's focal length, and the image quality at this focal length is the best among all focal lengths, making it suitable for shooting portraits, architecture, landscapes, documentaries, and more.

[0026] Ultra-wide-angle lens: The ultra-wide-angle lens provides a wider field of view than the main camera. It's important to note that the focal length of the ultra-wide-angle lens is displayed differently on some mobile devices. For example, some mobile devices display it as 0.5X, some as 0.6X, and some simply display it as "wide-angle" in text.

[0027] Compared to the main camera lens, the ultra-wide-angle lens can capture a wider field of view, making it suitable for shooting landscapes and architecture, and producing a more impactful visual effect.

[0028] Furthermore, ultra-wide-angle lenses offer a significantly wider field of view, allowing for a broader perspective from the same location, greatly simplifying post-processing cropping. When shooting landscapes, using an ultra-wide-angle lens not only allows more scenic elements to be captured, but with proper composition, it can also enhance the sense of depth and space, making the entire landscape photograph appear grand and imposing.

[0029] Furthermore, ultra-wide-angle lenses inherently possess lens distortion, which elongates and magnifies objects at the edges of a photograph. This distortion effect can be fully utilized for shooting from a low angle, creating a visually striking effect of "near objects appearing larger while distant objects appear smaller." When photographing architecture, the lens distortion of an ultra-wide-angle lens can be used to make buildings appear more magnificent.

[0030] Telephoto lenses: Focal lengths of 1X and above are generally referred to as telephoto. The higher the X before the number, the farther you can shoot. For example, some mobile devices have 5x optical zoom; when switching to 5X, you can switch from the main camera to the telephoto lens for shooting. Telephoto lenses can capture higher-quality photos from a distance, photographing distant objects or magnifying objects in the scene without the image quality degradation that digital zoom can cause. In situations where movement is inconvenient, or when framing a shot in cluttered buildings, the main camera of a mobile device may not be able to capture the subject clearly. In such cases, using the mobile device's telephoto lens can add depth and dimension to otherwise ordinary photos.

[0031] Telephoto lenses can "bring closer" the distance between the background and the foreground, thus creating a sense of spatial compression and making the overall image more complete. This "compression" is one of the characteristics of telephoto lenses.

[0032] Telephoto lenses have less distortion and weaker perspective, which can bring the foreground and background closer together, enhance their relationship, and thus create some unique visual effects.

[0033] This feature can also be used to guide the viewer's attention to the main subject in the depth of the image by using straight, extending objects such as roads and railings.

[0034] In conclusion, while multi-camera mobile devices cannot guarantee better photos, each lens has its own unique characteristics. Therefore, current camera shooting solutions can achieve real-time switching of camera lenses based on user needs in different shooting scenarios through smooth zoom.

[0035] For example, a mobile terminal uses the following fixed-focus lenses: 40MP, ultra-sensitive, 27mm equivalent focal length, F1.6; 20MP, ultra-wide-angle, 16mm equivalent focal length, F2.2; 8MP, periscope telephoto, 125mm equivalent focal length, F3.4.

[0036] The focal length changes continuously between 16mm-27mm and 27mm-125mm, and through the fusion of digital zoom, a continuous zoom of 16mm-125mm is ultimately achieved.

[0037] Ensuring smooth lens switching and minimizing image distortion during zooming are issues that current cameras need to address.

[0038] In current smooth zoom solutions, for example, some mobile terminals only start outputting the second image data stream when smooth zoom switching is triggered or when smooth zoom may be triggered, and spatial alignment processing is performed based on the second image data stream; for other mobile terminals, the three image data streams are in a normal outgoing state for a long time (the frame rates of the three image data streams will be different). These two solutions either increase the switching time and affect the switching effect; or the simultaneous output of the three image data streams for a long time will increase the power consumption of the system and may cause the mobile terminal to overheat.

[0039] The inventors discovered in their research on a solution for outputting three image data streams simultaneously over a long period that the flexibility in processing multiple data streams needs to be improved.

[0040] Therefore, the inventors have proposed the data processing method, apparatus, mobile terminal, and storage medium of this application. During a smooth zoom operation, the magnification corresponding to the current frame image data is obtained. The current frame image data is the image data displayed in real-time by the mobile terminal during the smooth zoom operation. Then, the data processing method corresponding to this magnification is obtained. Based on the data processing method, multiple image data streams, which are image data streams output by the multiple cameras, are processed. Through this method, different magnifications correspond to different data processing methods, thus allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of processing multiple image data streams.

[0041] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0042] Please see Figure 1 This application provides a data processing method applied to a mobile terminal, the mobile terminal including multiple cameras, the method comprising:

[0043] Step S110: During the smooth zoom operation, obtain the magnification corresponding to the current frame image data, wherein the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation.

[0044] As we know, zoom refers to the technology of changing the focal length of a camera to capture a clearer image. Zoom is generally implemented in two ways: optical zoom and digital zoom. Optical zoom relies on the physical movement of the lens assembly. It allows the camera to magnify distant objects while simultaneously increasing the number of pixels, ensuring image sharpness even when magnified. Digital zoom, on the other hand, relies on interpolation algorithms. It allows the camera to magnify distant objects and then use interpolation algorithms to process the magnified image for improved sharpness. Digital zoom has limitations in processing image sharpness, resulting in a final image quality that is generally inferior to optical zoom.

[0045] In addition, zoom lenses and prime lenses are often mentioned in video recordings. A zoom lens is a lens that can perform optical zoom, meaning the camera can adjust its focal length as needed to change the zoom ratio. A prime lens is a lens that cannot perform optical zoom, meaning the camera's focal length is fixed. If you want to change the image of an object, you can only do so by changing the shooting position or using digital zoom.

[0046] While optical zoom offers superior image quality, optical zoom cameras are generally bulky, which doesn't align with current mobile device design trends. To achieve a slim and lightweight design, mobile devices primarily employ a combination of fixed-focus lenses and digital zoom. To ensure image clarity even when objects are magnified, related technologies utilize cameras with varying focal lengths, such as integrating two, three, four, or even more different types of cameras into a single phone. When zooming, the mobile device switches between these cameras with different focal lengths to change the zoom ratio.

[0047] In this embodiment, the mobile terminal has more than two cameras, meaning the mobile terminal has a wider zoom range and produces better image quality. Magnification refers to the physical magnification of each of the multiple cameras.

[0048] Mobile terminals often feature multiple cameras of different types. These cameras have varying physical performance characteristics, resulting in corresponding physical magnification ranges. However, these ranges may overlap, necessitating the use of software to define optimal software magnification ranges for each camera. For instance, three cameras with different physical performance characteristics might each have a superior physical magnification range, with some overlap. The terminal's software can then set optimal software magnification ranges for each camera. These three ranges are consecutive, for example, [1x~2x), [2x~5x), and [5x~6x].

[0049] In this embodiment, the mobile terminal includes multiple cameras, each with a different physical magnification. When smooth zooming is detected, during the zoom process from the initial magnification to the desired magnification, the current frame image currently displayed on the mobile terminal is acquired in real time, and the magnification corresponding to the current frame image data is obtained. The initial magnification is the magnification of the mobile terminal's camera before the zoom operation, and the desired magnification is the final magnification used by the mobile terminal for recording video or taking photos after the zoom operation.

[0050] As one approach, the current frame image data displayed on the mobile terminal differs at different physical magnifications. Specifically, the characteristics of the image data obtained at each physical magnification can be analyzed in advance. Then, during the zoom operation, after acquiring the image data currently displayed on the mobile terminal, this image data can be analyzed to determine its characteristics, and the corresponding magnification can be determined based on these characteristics. A higher magnification generally means fewer objects are included in the image data. Therefore, the area or number of objects can be used as a characteristic of the image data currently displayed on the mobile terminal at different magnifications. Of course, other characteristics are also possible, but are not specifically limited here.

[0051] Step S120: Obtain the data processing method corresponding to the multiplier.

[0052] In this embodiment, the data processing method is used to process the image data stream output by each camera in the mobile terminal. The data processing methods for image data streams output by different cameras can be the same or different. Alternatively, some camera image data streams can be processed using the same method, while others can be processed differently.

[0053] Furthermore, in this embodiment, different magnifications correspond to different data processing methods. Specifically, different data processing methods can be preset for different magnifications. Then, when the magnification corresponding to the current frame image data is obtained, the data processing method corresponding to the magnification of the current frame image data can be determined according to the preset correspondence between magnification and data processing methods.

[0054] Step S130: Based on the data processing method, process the multiple image data streams, where the multiple image data streams are the image data streams output by the multiple cameras.

[0055] In this embodiment, after obtaining the corresponding data processing method, the multiple image data streams output by the mobile terminal are processed using the corresponding data processing method. Each camera outputs one image data stream, and the number of image data streams output corresponds to the number of cameras in the mobile terminal. For example, if the mobile terminal includes three cameras, then the corresponding multiple image data streams will include three image data streams.

[0056] As one approach, when processing multi-channel image data using corresponding data processing methods, it may include spatial alignment of the multi-channel image data streams, discarding some of the multi-channel image data streams, or processing some of the data in the multi-channel image data streams, i.e., partial processing of the multi-channel image data, without specific limitations here.

[0057] This application provides a data processing method that, during a smooth zoom operation, obtains the magnification corresponding to the current frame image data, where the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation. Then, it obtains the data processing method corresponding to this magnification, and processes multiple image data streams based on this method. These multiple image data streams are image data streams output by multiple cameras. Through this method, different magnifications correspond to different data processing methods, thus allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of multi-channel image data processing.

[0058] Please see Figure 2 This application provides a data processing method applied to a mobile terminal, the mobile terminal including multiple cameras, the method comprising:

[0059] Step S210: During the smooth zoom operation, obtain the magnification corresponding to the current frame image data, wherein the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation.

[0060] In this embodiment, during the zoom process from the initial magnification to the desired magnification, since the magnification changes gradually in real time, the magnification corresponding to the current frame image data can be obtained in real time during the smooth zoom process. The magnification can change at equal intervals according to a preset magnification value, or it can change according to different preset magnification values; no specific limitation is made here. To obtain better alignment results during spatial alignment processing, the preset magnification value can be set to a smaller value.

[0061] Optionally, the magnification interval from the initial magnification to the desired magnification is known in advance. Therefore, the magnification of the current frame image data can be acquired every preset time interval. The current frame image data at this time is the image data displayed on the mobile terminal at the moment the magnification is acquired. The preset time interval is a pre-defined time during which a specific magnification range changes.

[0062] Step S220: Obtain the range of the multiplier.

[0063] In this embodiment, the entire zoom range can be pre-divided into n magnification intervals, with each camera in the mobile terminal corresponding to its own suitable magnification interval. The i-th camera corresponds to the i-th magnification interval, and adjacent magnification intervals can be continuous but do not overlap. During zooming in the shooting process, the mobile terminal obtains the desired magnification set by the user (or determined by the autofocus program), which is the specified zoom magnification. The mobile terminal selects the camera to be used first by determining the magnification interval that matches the desired magnification.

[0064] After obtaining the magnification corresponding to the current frame image data, based on the pre-divided n magnification intervals, the magnification corresponding to the current frame image data is compared with the maximum and minimum magnification of each interval to determine the magnification interval in which the current frame image data falls. Specifically, if the magnification corresponding to the current frame image data is greater than or equal to the minimum magnification of the i-th magnification interval, and the magnification corresponding to the current frame image data is less than or equal to the maximum magnification of the i-th magnification interval, then the magnification interval in which the current frame image data falls is determined to be the i-th magnification interval.

[0065] Step S230: Determine the corresponding data processing method based on the said multiplier range.

[0066] In this embodiment, to avoid frequently switching processing methods for multiple image data streams and increasing the power consumption of the mobile terminal, a corresponding data processing method can be set for each magnification range. When the magnification of the current frame image data falls within a certain range, the multiple image data streams are continuously processed using the data processing method corresponding to that range. The pre-divided magnification ranges can include a first magnification range, a second magnification range, and a third magnification range; these ranges are continuous but do not overlap; the magnifications corresponding to the first, second, and third magnification ranges can gradually increase or decrease, without specific limitations.

[0067] In this embodiment, taking the example of gradually increasing magnification across the first, second, and third magnification ranges, the data processing method for the first magnification range is the first data processing method, the data processing method for the second magnification range is the second data processing method, and the data processing method for the third magnification range is the third data processing method. The first, second, and third data processing methods are different.

[0068] As one approach, if the magnification is within the first magnification range, the corresponding data processing method is determined to be the first data processing method. The first data processing method is to perform spatial alignment processing on at least two of the multi-channel image data streams and discard the image data streams other than the at least two image data streams.

[0069] Spatial alignment of at least two image data streams from a multi-channel image data stream can include aligning only two image data streams from the multi-channel image data stream; or it can include aligning three or more image data streams from the multi-channel image data stream. When aligning at least two image data streams from a multi-channel image data stream, a spatial alignment algorithm can be used.

[0070] As another approach, if the magnification is within the second magnification range, the corresponding data processing method is determined to be the second data processing method. The second data processing method is to perform spatial alignment processing on at least two of the image data streams in the multi-channel image data stream, and to perform alignment processing on the shooting parameters in the image data streams other than the at least two image data streams.

[0071] In this embodiment, the shooting parameters may include three parameters: automatic exposure, automatic focus, and automatic white balance. Aligning the shooting parameters in image data streams other than at least two image data streams can be understood as extracting the shooting parameters from these streams and aligning them using a preset alignment algorithm. This preset alignment algorithm may include automatic exposure algorithms, automatic focus algorithms, and automatic white balance algorithms.

[0072] Optionally, if the magnification is in the third magnification range, the corresponding data processing method is determined to be the third data processing method, which is to perform spatial alignment processing on the multi-channel image data stream.

[0073] In this embodiment of the application, when performing spatial alignment processing on multiple image data streams, two of the image data streams are first spatially aligned to obtain aligned image data streams. Then, the aligned image data streams are spatially aligned with one image data stream that has not yet been aligned. This method is used to perform spatial alignment processing on multiple image data streams.

[0074] In this embodiment, as one approach, since the mobile terminal includes multiple cameras, each corresponding to a different magnification range, after determining the magnification range of the current frame image data, the camera corresponding to that magnification range can be used as the camera for acquiring the current frame image data. For example, if the mobile terminal includes three cameras: an ultra-wide-angle camera, a wide-angle camera, and a telephoto camera, these three cameras correspond to the first, second, and third magnification ranges, respectively, and the image data streams corresponding to these three cameras are the first image data stream, the second image data stream, and the third image data stream, respectively. When it is determined that the magnification of the current frame image data falls within the first magnification range, it is determined that the current frame image data is acquired through the ultra-wide-angle camera. In this case, the corresponding data processing method is the first data processing method, which involves spatial alignment of the first and second image data streams and discarding the third image data stream.

[0075] When it is determined that the magnification of the current frame image data is in the second magnification range, it is determined that the current frame image data is collected by a wide-angle camera. In this case, the corresponding data processing method is the second data processing method, which will then perform spatial alignment processing on the first image data stream and the second image data stream, and perform alignment processing on the shooting parameters in the third image data stream.

[0076] When it is determined that the magnification of the current frame image data is in the third magnification range, it is determined that the current frame image data is collected by a telephoto camera. In this case, the corresponding data processing method is the third data processing method, and then the first image data stream, the second image data stream, and the third image data stream will be spatially aligned.

[0077] Optionally, if the alignment algorithm requires some alignment processing in advance, then during the non-triggered smooth zoom period, the three image data streams can be processed by the VNPU together (the frame rate is 20+7.5+3fps at this time), and the third image data stream can be processed separately (discarded or only the shooting parameters are extracted) only after the smooth zoom is triggered and spatial alignment processing is performed.

[0078] Step S240: Based on the data processing method, process the multiple image data streams, where the multiple image data streams are the image data streams output by the multiple cameras.

[0079] This application provides a data processing method that, during a smooth zoom operation, obtains the magnification corresponding to the current frame image data, where the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation. Then, it obtains the magnification range within which the magnification falls, determines the corresponding data processing method based on the magnification range, and finally processes multiple image data streams, which are image data streams output by multiple cameras, based on the data processing method. Through this method, different magnifications correspond to different data processing methods, thus allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of multi-channel image data processing.

[0080] Please see Figure 3 This application provides a data processing method applied to a mobile terminal, the mobile terminal including multiple cameras, the method comprising:

[0081] Step S310: When the camera is detected to be turned on, configure the multiple image data streams according to the preset outgoing stream strategy.

[0082] In this embodiment, a preset outflow strategy is used to control multiple cameras to output image data streams at a preset frame rate. Specifically, the preset frame rate varies for each camera at different magnifications. The preset outflow strategy can be as follows: Figure 4 As shown, Figure 4 The values ​​0.5X, 0.8X, 1X, ..., 20X, ... represent magnification. "UW" indicates an ultra-wide-angle camera, "W" indicates a wide-angle lens, and "T" indicates a telephoto camera. Figure 4 It can be seen that when the magnification is in the range of 0.5X to 1X, the ultra-wide-angle camera outputs image data stream at a frame rate of 30fps. When the magnification is in the range of 1X to 2.8X, the ultra-wide-angle camera outputs image data stream at a frame rate of 7.5fps. When the magnification is in the range of 2.8X to the maximum magnification, the ultra-wide-angle camera outputs image data stream at a frame rate of 0(3)fps.

[0083] When the magnification is in the range of 0.5X to 1X, the wide-angle camera outputs image data stream at a frame rate of 7.5fps. When the magnification is in the range of 1X to 5X, the wide-angle camera outputs image data stream at a frame rate of 30fps. When the magnification is in the range of 5X to the maximum magnification, the wide-angle camera outputs image data stream at a frame rate of 7.5fps.

[0084] When the magnification is in the range of 0.5X to 2.8X, the telephoto camera outputs image data stream at a frame rate of 0(3)fps. When the magnification is in the range of 2.8X to 5X, the telephoto camera outputs image data stream at a frame rate of 7.5fps. When the magnification is in the range of 5X to the maximum magnification, the telephoto camera outputs image data stream at a frame rate of 30fps.

[0085] Figure 4 This document describes the frame rate configuration strategy for the main, secondary, and third cameras when previewing or recording at a stable magnification value without triggering smooth zoom switching. The 3fps setting for the third camera bypasses the VNPU, meaning the third image data stream does not pass through the Image Embedded Neural Processing Unit (VNPU). 0fps means the image data from the third camera stream is discarded before entering the ISP (but the camera continues streaming, so the data stream can be quickly accessed when needed). When smooth zoom switching is triggered, the configuration of the main, secondary, and third cameras is determined based on the magnification range of the current frame image data.

[0086] Optionally, in this embodiment, multiple image data streams are activated simultaneously. Activating multiple image data streams simultaneously ensures that two image data streams can be quickly and simultaneously sent to the spatial alignment algorithm module during spatial alignment processing, without waiting for the camera to re-output the data. Since camera switching typically begins between the current main and secondary cameras, a short transition period is required before the third image data stream is fully utilized for spatial alignment. Therefore, processing of the third stream can begin only when a smooth zoom switch is triggered, preparing to resume the normal processing flow (the same as the current main and secondary cameras). The switching timing can be determined based on the magnification change of the current frame to ensure accuracy.

[0087] As one approach, when a camera is detected to be open, an image preview is initiated, and the frame rates of multiple cameras are configured according to the outflow strategy described above.

[0088] Step S320: In response to the magnification adjustment operation, a smooth zoom operation is triggered.

[0089] In this embodiment, the magnification adjustment operation is an operation to adjust the magnification. When the user's desired magnification is detected or the control for adjusting the magnification is pulled up, a smooth zoom switching process is triggered.

[0090] As one approach, step S320 further includes: adjusting the frame rate of the secondary camera among the plurality of cameras, and controlling the secondary camera to output image data stream at a target frame rate.

[0091] In this embodiment of the application, in order to not exceed the bandwidth limit of the VNPU, the frame rate of each camera can be adjusted in real time by changing the magnification of the current frame image data in the scenario that triggers smooth zoom, so as to reduce the bandwidth of the VNPU.

[0092] After the smooth zoom switching process is triggered, the frame rate adjustment of the secondary camera is triggered, and the frame rate of the secondary camera is adjusted to the target frame rate set above, controlling the secondary camera to output image data stream at the target frame rate.

[0093] Step S330: During the smooth zoom operation, obtain the magnification corresponding to the current frame image data, wherein the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation.

[0094] Step S340: Obtain the data processing method corresponding to the multiplier.

[0095] Step S350: Based on the data processing method, process the multiple image data streams, where the multiple image data streams are the image data streams output by the multiple cameras.

[0096] This application provides a data processing method. First, when a camera is detected to be on, multiple image data streams are configured according to a preset outgoing flow strategy. Then, in response to a magnification adjustment operation, a smooth zoom operation is triggered. During the smooth zoom operation, the magnification corresponding to the current frame image data is obtained, and then the data processing method corresponding to the magnification is obtained. Finally, based on the data processing method, the multiple image data streams, which are image data streams output by multiple cameras, are processed. Through this method, different magnifications correspond to different data processing methods, thus allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of processing multiple image data streams.

[0097] Please see Figure 5 This application provides a data processing method applied to a mobile terminal, the mobile terminal including multiple cameras, the method comprising:

[0098] Step S410: During the smooth zoom operation, obtain the magnification corresponding to the current frame image data, wherein the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation.

[0099] Step S420: Obtain the data processing method corresponding to the multiplier.

[0100] Step S430: Based on the data processing method, process the multi-channel image data stream, wherein the multi-channel image data stream is the image data stream output by the multiple cameras.

[0101] Step S440: If the magnification is equal to the target magnification, the smooth zoom operation ends.

[0102] In this embodiment, the target magnification is the aforementioned desired magnification. During the smooth zoom operation, the magnification of the current frame image data is acquired in real time. When the magnification corresponding to the current frame image data is detected to be equal to the desired magnification, the smooth zoom operation is determined to be complete.

[0103] Step S450: Based on the target magnification, re-determine the main camera from the plurality of cameras.

[0104] Step S460: Use the camera corresponding to the target magnification as the main camera.

[0105] In this embodiment of the application, the magnification range of the target magnification will be obtained, and the camera corresponding to the magnification range among multiple cameras will be used as the main camera.

[0106] Step S470: Preview the image using the main camera.

[0107] In this embodiment of the application, an image preview or video recording is performed using a designated main camera.

[0108] This application provides a data processing method that, during a smooth zoom operation, obtains the magnification corresponding to the current frame image data. The current frame image data is the image data displayed in real-time by the mobile terminal during the smooth zoom operation. Then, it obtains the data processing method corresponding to the magnification, and based on this method, processes multiple image data streams, which are image data streams output from multiple cameras. Through this method, different magnifications correspond to different data processing methods, thus allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of multi-channel image data processing.

[0109] Please see Figure 6 This application provides a data processing device 500 that operates on a mobile terminal, the mobile terminal including multiple cameras, and the device 500 includes:

[0110] The magnification acquisition unit 510 is used to acquire the magnification corresponding to the current frame image data during the smooth zoom operation, wherein the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation.

[0111] The method acquisition unit 520 is used to acquire the data processing method corresponding to the multiplier.

[0112] In one approach, the method acquisition unit 520 is specifically used to acquire the multiplier range in which the multiplier is located; and based on the multiplier range, to determine the corresponding data processing method.

[0113] Alternatively, the method acquisition unit 520 is specifically configured to: if the magnification is in a first magnification range, determine the corresponding data processing method as a first data processing method, wherein the first data processing method is to perform spatial alignment processing on at least two of the multi-channel image data streams and discard the image data streams other than the at least two image data streams; if the magnification is in a second magnification range, determine the corresponding data processing method as a second data processing method, wherein the second data processing method is to perform spatial alignment processing on at least two of the multi-channel image data streams and align the shooting parameters in the image data streams other than the at least two image data streams; if the magnification is in a third magnification range, determine the corresponding data processing method as a third data processing method, wherein the third data processing method is to perform spatial alignment processing on the multi-channel image data streams.

[0114] The processing unit 530 is used to process multiple image data streams based on the data processing method, wherein the multiple image data streams are image data streams output by the multiple cameras.

[0115] Please see Figure 7 The device 500 further includes:

[0116] Configuration unit 540 is used to configure multiple image data streams according to a preset outgoing stream strategy when the camera is detected to be turned on; and to trigger a smooth zoom operation in response to a magnification adjustment operation.

[0117] In one approach, the configuration unit 540 is further configured to adjust the frame rate of the secondary camera among the plurality of cameras, and control the secondary camera to output image data stream at a target frame rate.

[0118] The determining unit 550 is used to terminate the smooth zoom operation if the magnification is equal to the target magnification; and to re-determine the main camera from the plurality of cameras based on the target magnification.

[0119] In one manner, the determining unit 550 is also used to designate the camera corresponding to the target magnification as the main camera; and to perform image preview through the main camera.

[0120] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0121] The following will combine Figure 8This application describes a mobile terminal.

[0122] Please see Figure 8 Based on the aforementioned data processing method and apparatus, this application embodiment also provides another mobile terminal 800 capable of executing the aforementioned data processing method. The mobile terminal 800 includes one or more (only one shown in the figure) processors 802, a memory 804, and a network module 806 coupled together. The memory 804 stores programs capable of executing the contents of the aforementioned embodiments, and the processor 802 can execute the programs stored in the memory 804.

[0123] The processor 802 may include one or more processing cores. The processor 802 connects to various parts within the mobile terminal 800 using various interfaces and lines, and performs various functions and processes data of the mobile terminal 800 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 804, and by calling data stored in the memory 804. Optionally, the processor 802 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 802 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 802 and may be implemented separately using a communication chip.

[0124] The memory 804 may include random access memory (RAM) or read-only memory (ROM). The memory 804 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 804 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the mobile terminal 800 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0125] The network module 806 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby communicating with communication networks or other devices, such as audio playback devices. The network module 806 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM cards), memory, etc. The network module 806 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices through wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). For example, the network module 806 can interact with base stations.

[0126] Please refer to Figure 9 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 900 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0127] The computer-readable storage medium 900 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 910 may be compressed, for example, in a suitable form.

[0128] This application provides a data processing method, apparatus, mobile terminal, and storage medium. During a smooth zoom operation, the method acquires the magnification corresponding to the current frame image data, where the current frame image data is the image data displayed in real-time by the mobile terminal during the smooth zoom operation. Then, it acquires the data processing method corresponding to this magnification and processes multiple image data streams based on this method. These multiple image data streams are image data streams output by multiple cameras. Through this method, different magnifications correspond to different data processing methods, allowing for differentiated processing of multiple image data streams based on the current magnification, thereby improving the flexibility of processing multiple image data streams.

[0129] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A data processing method, characterized in that, Applied to a mobile terminal, the mobile terminal including multiple cameras, the multiple cameras being used to output multiple image data streams, the method includes: During the smooth zoom operation, the magnification corresponding to the current frame image data is obtained. The current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation. Obtain the range of the specified multiplier; If the magnification is within the first magnification range, the corresponding data processing method is determined to be the first data processing method. The first data processing method is to perform spatial alignment processing on at least two of the image data streams in the multi-channel image data streams and to discard the image data streams other than the at least two image data streams. If the magnification is in the second magnification range, the corresponding data processing method is determined to be the second data processing method. The second data processing method is to perform spatial alignment processing on at least two of the image data streams in the multi-channel image data stream, and to perform alignment processing on the shooting parameters in the image data streams other than the at least two image data streams. The alignment processing on the shooting parameters in the image data streams other than the at least two image data streams is to extract the shooting parameters in the image data streams other than the at least two image data streams and perform alignment processing through a preset alignment algorithm. If the magnification is in the third magnification range, the corresponding data processing method is determined to be the third data processing method. The third data processing method is to perform spatial alignment processing on the multi-channel image data stream, wherein the entire zoom focal length is divided into the first magnification range, the second magnification range and the third magnification range, and the magnification corresponding to the first magnification range, the second magnification range and the third magnification range gradually increases. Based on the aforementioned data processing method, multiple image data streams are processed.

2. The method according to claim 1, characterized in that, Before obtaining the magnification corresponding to the current frame image data during the smooth zoom process, the following steps are also included: When the camera is detected to be on, the multi-channel image data streams are configured according to the preset outgoing flow strategy; In response to a magnification adjustment, a smooth zoom operation is triggered.

3. The method according to claim 2, characterized in that, The response to the magnification adjustment operation, after triggering the smooth zoom operation, further includes: The frame rate of the secondary camera among the plurality of cameras is adjusted, and the secondary camera is controlled to output image data stream at a target frame rate.

4. The method according to claim 1, characterized in that, The method further includes: If the magnification is equal to the target magnification, the smooth zoom operation ends. Based on the target magnification, the main camera is re-determined from the plurality of cameras.

5. The method according to claim 4, characterized in that, The step of re-determining the main camera from the plurality of cameras based on the target magnification includes: The camera corresponding to the target magnification is used as the main camera; Image preview is performed using the main camera.

6. A data processing apparatus, characterized in that, Operating on a mobile terminal, the mobile terminal including multiple cameras, the multiple cameras being used to output multiple image data streams, the device comprising: The magnification acquisition unit is used to acquire the magnification corresponding to the current frame image data during the smooth zoom operation, wherein the current frame image data is the image data displayed in real time by the mobile terminal during the smooth zoom operation. The method acquisition unit is used to acquire the magnification range in which the magnification is located; if the magnification is in a first magnification range, the corresponding data processing method is determined to be a first data processing method, which involves spatially aligning at least two image data streams from the multi-channel image data streams and discarding the image data streams other than the at least two image data streams; if the magnification is in a second magnification range, the corresponding data processing method is determined to be a second data processing method, which involves spatially aligning at least two image data streams from the multi-channel image data streams and discarding the image data streams other than the at least two image data streams. The parameters are aligned. The alignment of shooting parameters in image data streams other than the at least two image data streams involves extracting shooting parameters from these streams and aligning them using a preset alignment algorithm. If the magnification is in the third magnification range, the corresponding data processing method is determined to be the third data processing method. This third data processing method involves spatial alignment of the multiple image data streams, wherein the entire zoom range is divided into the first magnification range, the second magnification range, and the third magnification range, with the magnifications of the first, second, and third magnification ranges gradually increasing. The processing unit is used to process multiple image data streams based on the data processing method.

7. A mobile terminal, characterized in that, It includes a memory, one or more processors; one or more programs are stored in the memory and configured to be executed by the one or more processors according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the program code, when executed by a processor, performs the method according to any one of claims 1-5.

Images