A shooting method, system, intelligent terminal and storage medium
By identifying the shooting scene in the smart terminal and determining the multi-camera strategy, differentiating processing and transmitting data, the problem of high data pressure on the back-end processing module under the multi-camera shooting function is solved, and the shooting performance and user experience of the terminal are improved .
Patent Information
- Application Number
- CN202210786042.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-04
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-07-04
AI Technical Summary
In smart terminals, under the multi-camera shooting function, the back-end processing module bears huge data processing pressure, resulting in poor experiences such as stuttering video image display, zoom picture jumps and terminal heating. The hardware size and system processing capabilities are limited, which is difficult to meet Higher and more professional shooting function requirements.
The front-end processing module recognizes the shooting scene, determines the corresponding multi-camera strategy, and differentiates the processing and transmits data to the back-end processing module, including the scene depth data switch, the camera advance start switch, the local area image acquisition switch and the camera image shooting partial frame transmission Switch, reduce data transmission volume and calculate pressure.
It effectively shares the computing pressure of the back-end processing module, reduces the data transmission bandwidth usage between the front and back-end processing modules, and improves the shooting and processing performance and user experience of the smart terminal.
Smart Images

Figure CN115190241B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to, but is not limited to, the technical field of smart terminals, and specifically to a shooting method, system, smart terminal, and storage medium. Background Art
[0002] With the continuous development of smart terminals, the shooting functions they have become increasingly powerful. In order to better meet the needs of multiple functions such as portraits, landscapes, close-ups, and distant views, more and more smart terminal shooting systems have introduced multiple cameras. The use of auxiliary cameras in conjunction with the main camera to achieve a dual-camera solution can provide shooting functions such as blurring and smooth zooming.
[0003] Under the constraints of limited hardware size and system processing power, how to meet users' higher and more professional shooting function requirements while providing a better operating experience has always been a direction that technical personnel in the field of smart terminals have been constantly exploring. Summary of the invention
[0004] The embodiments of the present disclosure provide a shooting method, system, intelligent terminal and storage medium, which are based on the front-end processing module identifying the shooting scene under the current shooting function, determining the multi-camera strategy corresponding to the identified shooting scene, and obtaining the corresponding image data to transmit to the back-end processing module for processing to generate a video image.
[0005] The present disclosure provides a photographing system, including:
[0006] A front-end processing module and a back-end processing module;
[0007] The front-end processing module is configured to identify the shooting scene under the current shooting function according to the captured image obtained under the current shooting function; determine the corresponding multi-shooting strategy according to the identified shooting scene; obtain transmission data according to the determined multi-shooting strategy, and transmit it to the back-end processing module;
[0008] The back-end processing module is configured to generate a video image according to the received transmission data;
[0009] The multi-camera strategy includes one or more of the following settings:
[0010] The switch for scene depth data, the switch for early start of the camera, the switch for acquiring local area images of the camera, and the switch for transmitting partial frames of images captured by the camera.
[0011] The present disclosure also provides a shooting method, including:
[0012] The front-end processing module identifies the shooting scene under the current shooting function according to the shooting image obtained under the current shooting function;
[0013] According to the identified shooting scene, a corresponding multi-camera strategy is determined, and transmission data is obtained according to the determined multi-camera strategy and transmitted to a back-end processing module;
[0014] The back-end processing module generates a video image according to the received transmission data;
[0015] The multi-camera strategy includes one or more of the following settings:
[0016] The switch for scene depth data, the switch for early start of the camera, the switch for acquiring local area images of the camera, and the switch for transmitting partial frames of images captured by the camera.
[0017] The present disclosure also provides a smart terminal, including:
[0018] one or more processors;
[0019] a storage device for storing one or more programs,
[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the shooting method as described in any embodiment of the present disclosure.
[0021] An embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the shooting method as described in any embodiment of the present disclosure is implemented.
[0022] It can be seen that according to the shooting scheme provided by the embodiment of the present disclosure, the front-end processing module performs differentiated shooting image processing and / or transmission according to different shooting scenes, which can effectively share the data processing pressure of the back-end processing module in the multi-camera scheme, reduce the data transmission volume between the front-end and back-end processing modules, significantly reduce the shooting processing performance of the smart terminal, and ensure the optimal user experience under the relevant shooting functions.
[0023] Other aspects will be apparent upon reading and understanding the drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the solutions of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.
[0025] Figure 1 A schematic diagram of a principle for realizing a virtualization function;
[0026] Figure 2This is a schematic diagram of a main camera shooting interface;
[0027] Figure 3 This is a schematic diagram of an ultra-wide-angle lens shooting interface;
[0028] Figure 4 A schematic diagram of an achievable SAT function;
[0029] Figure 5 This is a schematic diagram of the structure of a shooting system in an embodiment of the present disclosure;
[0030] Figure 6 is a schematic diagram of another shooting system structure in an embodiment of the present disclosure;
[0031] Figure 7 This is a flow chart of a shooting method in an embodiment of the present disclosure;
[0032] Figure 8 is a flow chart of another shooting method in an embodiment of the present disclosure;
[0033] Fig. 9 This is a flow chart of another shooting method in an embodiment of the present disclosure.
[0034] The realization of the purpose, functional features and advantages of the present application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0036] It should be noted that all directional indications in the embodiments of the present disclosure (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0037] In addition, in the present disclosure, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0038] In the present disclosure, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present application scheme can be understood according to specific circumstances.
[0039] In addition, the technical solutions between the various embodiments of the present disclosure can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0040] Before describing the specific embodiments, the abbreviations of the relevant terms involved in the present disclosure are explained as follows:
[0041]
[0042]
[0043] In some feasible dual-camera solutions, the shooting functions mainly include image blur and smooth switching (zoom), among which the smooth switching function is also called the spatial alignment transform (SAT) function, which performs blur processing or spatial alignment based on the image data of the primary and auxiliary cameras. Among them, for the image blur mode, the image data of the auxiliary camera (also called the secondary camera) is used to obtain the depth information of the current scene through the calculation and analysis of the main and auxiliary frames; and the SAT function is mainly to perform spatial alignment when the sensor is switched, to avoid the image jump during switching, and to ensure smooth switching.
[0044] Before further describing the embodiments of the present disclosure in detail, the relevant aspects of the shooting system are introduced as follows:
[0045] Blur:
[0046] In some feasible solutions, the image blurring solution is based on depth map for image optimization. There are two ways to obtain the depth map: one camera and two cameras. Based on one camera, the depth information of the shooting area can be obtained through AI or TOF lens. Some feasible mainstream depth information acquisition solutions will use dual cameras to build a depth map (DepthMap), thereby inferring the front and back relationship of the scene and controlling the degree of blur. Dual cameras can use the difference in the viewing angles of the two cameras to calculate the distance of each pixel from the focal plane and calculate the depth information based on the focal plane distance.
[0047] The principle of the blur function is as follows Figure 1 As shown in the figure, for the image blur processing, theoretically, the farther away from the focal plane, the larger the confusion circle and the blurrier the image. In other words, the closer to the focal plane, the clearer the image. The ideal blur effect is that although everything outside the depth of field is blurred, the degree of blur is different. The closer to the focal plane, the clearer it is, and the farther away from the focal plane, the blurrier it is.
[0048] SAT:
[0049] Generally, the camera operation page of a multi-camera phone will have focal length switching options of 0.6x, 1x, 2x or 5x, 10x. The smaller the number, the wider the shot, and the larger the number, the farther the shot. These roughly correspond to the focal lengths of an ultra-wide-angle lens (also called an ultra-wide-angle camera, or ultra-wide-angle for short), a main camera lens (also called a main camera, or main camera for short), and a telephoto lens (also called a telephoto camera, or telephoto for short):
[0050] Main camera
[0051] The main camera of most mobile phones is generally a wide-angle lens equivalent to about 28mm. Because this focal length is close to the field of view of "what the human eye sees", what you see is what you shoot, and it is also the camera we use most frequently.
[0052] like Figure 2 As shown, open the operation page of the mobile phone camera, you can see the 1X value displayed on the page. 1X is the focal length of the main camera. The image quality is the best among all focal lengths. It is suitable for taking portraits, architecture, scenery, documentaries, etc.
[0053] Ultra wide angle (Wide)
[0054] The ultra-wide-angle lens can provide a wider field of view than the main camera. It should be noted that the focal length of the ultra-wide angle is displayed differently on some mobile phone cameras. For example, it is displayed as 0.5X on the mobile phone of manufacturer 1, 0.6X on the mobile phone of manufacturer 2, and directly displayed as the text wide angle on the mobile phone of manufacturer 3. Figure 3 shown.
[0055] Compared with the main camera, the ultra-wide-angle camera can capture a wider picture, which is suitable for shooting scenery and architecture, and can obtain an impactful sense of picture.
[0056] In addition, the ultra-wide angle has a large viewing range, and a wider picture can be obtained at the same position, which brings great convenience to the re-cropping in the later stage. When shooting scenery, using an ultra-wide angle lens can not only allow more scenery elements to be captured, but also reflect the depth and sense of space of the picture as long as the composition is used, making the entire scenery photo look magnificent.
[0057] In addition, ultra-wide-angle lenses have the inherent characteristic of lens distortion, which will make things on the edge of the photo appear elongated and magnified. You can make full use of its distortion effect to shoot from above, which can create the visual impact of "near is big and far is small". When shooting buildings, you can use the characteristic of lens distortion to make the buildings look more magnificent.
[0058] Telephoto
[0059] A focal length above 1x is usually called a telephoto lens. The higher the coefficient in front of x, the farther you can shoot. For example, the 5x optical zoom on the phone of manufacturer 3 allows you to switch from the original main camera to the telephoto lens when switching the focal length to 5X. The telephoto lens can take higher-quality photos from a distance, shoot objects farther away, or zoom in on objects in the picture without causing a drop in image quality like digital zoom. When it is inconvenient to move around and want to frame in a messy building, it is difficult to capture the subject with the main camera of the phone alone. At this time, using the telephoto lens of the phone can make ordinary photos more layered.
[0060] Some of the available SAT functions are: Figure 4 As shown, multiple fixed-focus lens relays are used, including:
[0061] Main camera 2: 40MP, ultra-sensitive, equivalent focal length 27mm, F1.6;
[0062] Ultra wide angle (wide) 3: 20MP, ultra wide angle, equivalent focal length 16mm, F2.2;
[0063] Telephoto 1: 8MP, periscope telephoto, equivalent focal length 125mm, F3.4;
[0064] The focal length changes continuously between 16-27mm and 27-125mm, and by integrating digital zoom, it ultimately achieves 16-125mm continuous zoom.
[0065] Figure 4 The shooting system shown also includes: a flash 4 and a TOF 5.
[0066] It should be noted that the lens and the camera are equivalent concepts in the embodiments of the present disclosure. The main camera / main camera lens / main camera refers to a specific lens / camera, which is not the same as the "main lens / main camera" in the exemplary embodiment. Among them, the main lens / main camera is a relative concept and does not correspond to a certain lens / camera. It is the main lens / camera under the current shooting function. The main lens / camera may be different under different shooting functions. The opposite of the main lens / main camera is the auxiliary lens / auxiliary camera.
[0067] It can be seen that each lens has different characteristics. In a multi-camera shooting system, multiple cameras are used to shoot together to achieve a variety of shooting functions. In some feasible dual-camera or more camera shooting schemes, for each camera that is started, the front-end processing module obtains the captured image (for example, RAW image data) of each camera respectively and transmits it to the back-end processing module through the data interface (for example, MIPI / PCIe interface) in the smart terminal system for related blur processing or SAT processing to achieve related functions. With the continuous improvement of the hardware configuration of each camera, the amount of image data directly generated continues to increase. The transmission of image data from multiple cameras to the back-end processing module will directly bring huge processing pressure to the back-end processing module, occupying a large amount of transmission bandwidth within the system, resulting in huge consumption of processing resources of the smart terminal system and rising power consumption, which makes the user experience poor experience such as video image display freeze, zoom screen jump and / or terminal heating during use.
[0068] The disclosed embodiments provide a shooting solution, which can perform differentiated front-end processing according to the recognition results of the shooting scenes when executing the multi-camera function. On the premise of meeting the requirements of the relevant multi-camera functions, it can effectively reduce the computing pressure of the back-end processing module, reduce the occupancy of the terminal's internal transmission bandwidth by the data transmission between the front-end and back-end processing modules, and improve the processing performance of the terminal as a whole.
[0069] The present disclosure provides a shooting system. Figure 5 As shown, including:
[0070] A front-end processing module 510 and a back-end processing module 520;
[0071] The front-end processing module 510 is configured to identify the shooting scene under the current shooting function according to the captured image obtained under the current shooting function; determine the corresponding multi-shooting strategy according to the identified shooting scene; obtain transmission data according to the determined multi-shooting strategy, and transmit it to the back-end processing module;
[0072] The backend processing module 520 is configured to generate a video image according to the received transmission data;
[0073] The multi-camera strategy includes one or more of the following settings (setting items):
[0074] The switch for scene depth data, the switch for early start of the camera, the switch for acquiring local area images of the camera, and the switch for transmitting partial frames of images captured by the camera.
[0075] In some exemplary embodiments, the video image is used for video recording or shooting preview.
[0076] In some exemplary embodiments, the setting item - scene depth data switch includes an attribute: switch state.
[0077] In some exemplary embodiments, the setting item - the early start switch of the camera includes attributes: camera identification and switch status.
[0078] For example, camera identification: camera 3, switch status: on; the corresponding setting item is: the advance start switch of camera 3 is on. More examples are not recorded here one by one.
[0079] In some exemplary embodiments, the setting item - local area image acquisition switch of the camera includes attributes: camera identification, local area range and switch status.
[0080] For example, the camera identifier is: camera 1, local area range: upper 1 / 3 of the image, switch status: on, and the corresponding setting item is: the upper 1 / 3 local area image acquisition switch of camera 1 is on. For another example, the camera identifier is: camera 1, local area range: target area, i.e., the local area including the target, switch status: on, and the corresponding setting item is: the local area image acquisition switch including the target of camera 1 is on. More examples are not listed here one by one.
[0081] In some exemplary embodiments, the setting item - camera image partial frame transmission switch includes attributes: camera identifier and switch state. For example, camera identifier: camera 1, switch state: on; the corresponding setting item is: camera 1 image partial frame transmission switch on. More examples are not listed here one by one.
[0082] In some exemplary embodiments, the setting item - camera captured image partial frame transmission switch also includes an attribute: transmission frame interval. For example, camera identifier: camera 1, transmission frame interval: every 2 frames, switch state: on; the corresponding setting item is: the camera 1 captured image transmission switch is on every 2 frames. More examples are not listed here one by one.
[0083] If the transmission frame interval of the setting item is not set, it means that the captured image is transmitted according to the default transmission frame interval. The default transmission frame interval can be flexibly set, for example, the default transmission frame interval is 1 frame.
[0084] It should be noted that, in the absence of conflicts, a multi-camera strategy may include multiple setting items, which may be multiple setting items for the same camera or multiple setting items for different cameras. According to the camera identifier of the setting item, the data processing scheme of the corresponding camera is controlled accordingly. It is not limited to controlling only the relevant aspects of the second camera, and is flexibly determined according to the specific cameras involved in the multi-camera function, and is not limited to a specific camera.
[0085] In some exemplary embodiments, the front-end processing module 510 includes: an ISP, a DSP, an MCU, an NPU, a driver module, or a front-end execution node in a system workflow framework.
[0086] In some exemplary embodiments, the backend processing module 520 includes: an AP or a CPU.
[0087] It can be understood that the front-end processing module is a hardware module or a hardware + software module that can run in parallel with the back-end processing module in the intelligent terminal system. In some exemplary embodiments, the front-end processing module is a hardware module or a hardware + software module with an independent processor.
[0088] In some exemplary embodiments, the system further includes: an interface module 530, which is arranged between the front-end processing module 510 and the back-end processing module 520, and is used for transmitting data between the front-end processing module 510 and the back-end processing module 520, including business data and control data.
[0089] In some exemplary embodiments, the interface module 530 is a MICI interface, or a PCIe interface.
[0090] In some exemplary embodiments, the backend processing module 520 is further configured to perform shooting preview according to the video image, or to perform recording according to the video image.
[0091] It should be noted that, depending on the working mode (shooting preview or recording), the video image generated by the back-end processing module 520 based on the received transmission data may be the same or different, and is flexibly determined according to the specific functional definition of preview or recording of the shooting system, and is not limited to a specific video image generation scheme. More detailed aspects are not further discussed in the embodiments of this application.
[0092] In some exemplary embodiments, the transmission data includes: an image captured by the first camera.
[0093] In some exemplary embodiments, when the current shooting function is blurring, the shooting scene includes: a single-target scene or a multi-target scene;
[0094] Accordingly, the multi-camera strategy includes: turning on the scene depth data switch;
[0095] Correspondingly, the transmission data also includes: scene depth data; the scene depth data is calculated based on the image captured by the first camera and the image captured by the second camera.
[0096] The scene depth data switch is used to indicate whether the front-end processing module generates scene depth data. If it is turned on, it indicates generation, and if it is turned off, it indicates no generation.
[0097] It should be noted that in the disclosed embodiment, the first camera is the main camera under the current shooting function, and the second camera is the auxiliary camera under the current shooting function, which refers to two different cameras that are started and working under the multi-camera solution; the first camera and the second camera are not limited to specific cameras. For example, under shooting function 1, the first camera is the main camera, and the second camera is an ultra-wide-angle camera; under shooting function 2, the first camera is an ultra-wide-angle camera, and the second camera is the main camera; under shooting function 3, the first camera is the main camera, and the second camera is a telephoto camera; under shooting function 4, the first camera is a telephoto camera, and the second camera is not turned on; more examples are not recorded here one by one.
[0098] In some exemplary embodiments, the first camera includes: a main camera, a wide-angle camera, or a telephoto camera; or other types of cameras.
[0099] In some exemplary embodiments, the second camera includes: a main camera, a wide-angle camera, or a telephoto camera; or other types of cameras.
[0100] It should also be noted that the camera involved in the embodiments of the present disclosure may have more than one lens / camera of each type, for example, including two main cameras, etc.; the camera involved in the embodiments of the present disclosure may also include other types of lenses / cameras, and parameters such as the focal range of each type of lens / camera are not limited to the aspects of the aforementioned recorded examples.
[0101] In some exemplary embodiments, the targets include: people, animals, food, flowers, potted plants, etc., and are not limited to specific types of targets. The specific target recognition scheme can be implemented according to the relevant scheme and will not be discussed in detail in this application.
[0102] It can be understood that in some exemplary embodiments, when performing blur function shooting, the front-end processing module identifies the shooting scene based on the captured image of the first camera or the second camera, and determines whether it is a single-target scene or a multi-target scene, or there is no obvious target. In the case where the front-end processing module identifies that the current shooting scene is a single-target scene or a multi-target scene, the determined multi-camera strategy includes a setting item: scene depth data switch, and is set to on, that is, the determined multi-camera strategy includes: the scene depth data switch is turned on, and the front-end processing module calculates the scene depth data based on the captured image of the first camera and the captured image of the second camera.
[0103] In some exemplary embodiments, the scene depth data is a depth map; or other forms of depth data, not limited to a specific form.
[0104] In some exemplary embodiments, the front-end processing module calculates scene depth data according to the image captured by the first camera and the image captured by the second camera, including:
[0105] Depending on whether the shooting scene is a single-target scene or a multi-target scene, corresponding scene depth data is calculated based on the images shot by the first camera and the images shot by the second camera, respectively, so as to be used for blurring the single target or blurring the multiple targets.
[0106] The detailed aspects of calculating the scene depth for a single-target scene or a multi-target scene are not further discussed in the embodiments of the present application.
[0107] In some exemplary embodiments, the front-end processing module performs shooting scene recognition based on the image captured by the first camera or the second camera, including:
[0108] Identify based on nearby historical frame data captured by the first camera or the second camera;
[0109] or,
[0110] The recognition is performed according to the current frame data captured by the first camera or the second camera.
[0111] According to the embodiment, under the blur function, according to the identified shooting scene, the front-end processing module transmits the calculated scene depth data and the image captured by the first camera to the back-end processing module, and the back-end processing module further uses the scene depth data to perform blur processing based on the image captured by the first camera to generate a video image. Compared with directly transmitting the images captured by the first camera and the second camera, the amount of transmitted data is significantly reduced, which naturally reduces the occupation of the interface bandwidth in the terminal system, effectively shares the calculation pressure of the back-end processing module, and improves the terminal processing performance.
[0112] In some exemplary embodiments, when the current shooting function is blurring, the shooting scene includes: a single-target scene or a multi-target scene;
[0113] Accordingly, the multi-camera strategy includes: turning on a switch for acquiring an image of a target area of the second camera;
[0114] Correspondingly, the transmission data also includes: a first extracted image; the first extracted image is a local area image containing the single target or the multiple targets extracted from the image captured by the second camera.
[0115] The target area image acquisition switch of the second camera is turned on to indicate whether the front-end processing module generates local image data of the target area captured by the second camera. If it is turned on, it indicates generation, and if it is turned off, it indicates no generation. Compared with the complete captured image of the second camera, the first extracted image is a local area image containing the single target or the multiple targets.
[0116] It can be understood that in some exemplary embodiments, when performing blur function shooting, the front-end processing module identifies the shooting scene according to the shooting image of the first camera or the second camera, and determines whether it is a single target scene or a multi-target scene, or there is no obvious target. In the case where the front-end processing module identifies that the current shooting scene is a single target scene or a multi-target scene, the determined multi-shooting strategy includes setting items: the local area image acquisition switch of the camera, and the attributes are set as: camera identification-second camera identification, local area range-target area, switch state-on; that is, the determined multi-shooting strategy includes: the target area image acquisition switch of the second camera is turned on, then the front-end processing module extracts the local area image containing the single target or the multi-target from the shooting image of the second camera, recorded as the first extracted image; and transmits the first extracted image and the shooting image of the first camera as the transmission data to the back-end processing module. Correspondingly, the back-end processing module further calculates the scene depth data and performs blur processing based on the local area image containing only the target and the complete shooting image of the first camera, and generates a video image for preview or recording.
[0117] According to this embodiment, under the blur function, it is necessary to implement blur processing of non-target areas in the image. Therefore, for the image captured by the second camera, only the local area containing the target can be focused on. According to the identified shooting scene, the front-end processing module transmits the local area image including the single target or the multiple targets in the image captured by the second camera to the back-end processing module, and the back-end processing module further uses the local area image and the image captured by the first camera to calculate the scene depth data, and performs blur processing to generate a video image. Compared with directly transmitting the images captured by the first camera and the second camera, the amount of transmitted data is significantly reduced, the occupation of the interface bandwidth in the terminal system is reduced, the calculation pressure of the back-end processing module is effectively shared, and the terminal processing performance is improved.
[0118] In some exemplary embodiments, when the current shooting function is spatial alignment transformation, and the shooting scene is a scene in which the area ratio of the region of interest in the captured image is less than the first ratio threshold or the area ratio of the region of interest in the captured image is greater than the second ratio threshold, the multi-camera strategy includes: turning on the advance start switch of the second camera;
[0119] Correspondingly, the transmission data also includes: an image captured by the second camera.
[0120] It can be understood that in some exemplary embodiments, when the current shooting function is spatial alignment transformation, and the shooting scene is a scene in which the area ratio of the region of interest in the captured image is less than the first ratio threshold or the area ratio of the region of interest in the captured image is greater than the second ratio threshold, the determined multi-camera strategy includes a setting item: the early start switch of the camera. Among them, the setting item of the early start switch of the camera has the property set as: camera identification - second camera, switch state - on. That is, the determined multi-camera strategy includes: the early start switch of the second camera is turned on.
[0121] It should be noted that the area ratio of the region of interest represents the area ratio of the region of interest compared to the complete image. The advance start switch of the second camera is turned on, which means that although the conditions for switching to the second camera have not been met and the switch has not been made, the second camera is started in advance. The "advance" mentioned above means that compared with some feasible solutions, the corresponding second camera is started (turned on) when the switching conditions are met.
[0122] In some exemplary embodiments, the first proportion threshold is a set value.
[0123] In some exemplary embodiments, the second proportion threshold is a set value;
[0124] In some exemplary embodiments, the first proportion threshold is smaller than the second proportion threshold.
[0125] If the area ratio of the region of interest is less than the first ratio threshold, it means that the area ratio of the region of interest in the current captured image is small, and it is predicted that the user may need to further adjust the focal length to increase the area ratio of the region of interest; if the area ratio of the region of interest is greater than the second ratio threshold, it means that the area ratio of the region of interest in the current captured image is large, and it is predicted that the user may need to further adjust the focal length to reduce the area ratio of the region of interest. Therefore, in these two shooting scenarios, the second camera is started in advance, the captured image of the second camera is obtained and transmitted to the back-end processing module, and the back-end processing module performs spatial alignment transformation in advance, so that when the camera switching conditions are met, the video image that has completed SAT processing can be quickly provided for preview or recording, which significantly shortens the reaction time of lens switching, alleviates the jump phenomenon of video images when the camera is switched, and improves the user experience of smooth switching.
[0126] For example, under the SAT function, at time T1, the main camera lens (as the first camera) is being used to shoot a billboard across the road. Based on the image captured by the main camera lens, a scene in which the area ratio of the region of interest in the captured image is less than the first ratio threshold is identified, and it is predicted that the user is likely to need to further adjust the focal length to increase the area ratio of the region of interest. That is, it is predicted that the user's operation of further adjusting the focal length will cause the first camera to need to switch to telephoto. Therefore, the second camera (telephoto) is started in advance, and the captured image of the second camera is obtained and transmitted to the back-end processing module, which performs SAT processing in advance. At time T2, when the user performs a focusing operation and meets the lens switching conditions and switches to the method of using the telephoto lens (as the first camera), since the captured image based on the second camera started in advance has been pre-processed with SAT, the video image that has completed SAT processing can be used for preview or recording within a relatively low latency, which significantly improves the user experience of the SAT function.
[0127] It can be understood that in the above embodiment, the second camera is the non-current first camera with the highest probability of being switched as the first camera (main camera) under the current SAT function. Which camera is determined as the second camera that needs to be started in advance is not discussed in detail in the embodiment of the present application. In some exemplary embodiments, the data can be obtained from the back-end processing module.
[0128] In the above embodiment, according to the multi-camera strategy determined by the shooting scene, the front-end processing module starts the second camera to be switched in advance and transmits the captured image to the back-end processing module, so that the back-end processing module can perform SAT processing based on the captured image of the second camera in advance, effectively reducing the processing delay after switching and improving the smooth switching effect.
[0129] The transmitted image captured by the second camera may be a complete image, or may be determined as a local area image according to other configuration items described below. The transmitted image captured by the second camera may be image data transmitted frame by frame, or may be determined as image data transmitted at intervals according to other configuration items described below. In the absence of conflict, it is not limited to specific aspects.
[0130] In some exemplary embodiments, when the current shooting function is spatial alignment transformation and the shooting scene is a scene in which the shooting object in the included area of interest remains stable, the multi-camera strategy includes: the image acquisition switch of the area of interest of the second camera is turned on and the partial frame transmission switch of the image shot by the second camera is turned on.
[0131] Among them, a scene in which the subject in the region of interest (ROI) remains stable indicates that the subject is relatively stable. Although there may be changes in the size and / or position of the region of interest, the subject contained in the region of interest remains basically unchanged. In some exemplary embodiments, whether the subject in the region of interest remains stable is determined based on multiple frames of images captured by the camera close to the current time. The specific determination scheme is not discussed in detail in this application.
[0132] The region of interest is a local area including a dynamic object, or a local area including a static object. For example, in the SAT function, when a boat on a lake is photographed, the region of interest is a local area including the boat; when a temple on a distant hillside is photographed, the region of interest is a local area including the temple.
[0133] Correspondingly, when the image acquisition switch of the region of interest of the second camera is turned on, the transmission data further includes: a second extracted image; the second extracted image is a local area image containing the region of interest extracted from the image captured by the second camera;
[0134] When the partial frame transmission switch of the captured image of the second camera is turned on, according to the determined frame transmission interval, the transmission data corresponding to the transmission time point also includes: a complete captured image or a partial captured image of the second camera.
[0135] It can be understood that in some exemplary embodiments, when the current shooting function is spatial alignment transformation, and the shooting scene is a scene in which the shooting object in the included area of interest remains stable, the determined multi-shooting strategy includes setting items: the local area image acquisition switch of the camera and the partial frame transmission switch of the camera shot image. Among them, the setting item of the local area image acquisition switch of the camera, the attribute is set to: camera identification - the identification of the second camera, local area range - area of interest, switch state - on; the setting item of the partial frame transmission switch of the camera shot image, the attribute is set to: camera identification - second camera identification, switch state - on. That is, the determined multi-shooting strategy includes: the image acquisition switch of the area of interest of the second camera is turned on and the partial frame transmission switch of the shot image of the second camera is turned on.
[0136] In some exemplary embodiments, since the current shooting scene includes an area of interest, the user is more concerned about the alignment of the area of interest in the video image under the spatial alignment transformation (smooth zoom) function, and pays relatively less attention to the non-interest area (background area). Therefore, the focus can be on the local area image of the area of interest in the captured image. Therefore, the multi-camera strategy includes: the image acquisition switch of the area of interest of the second camera is turned on, and the front-end processing module obtains the second extracted image according to the setting item and transmits it to the back-end processing module.
[0137] In order to further reduce the data processing pressure of the back-end processing module, there is no need to transmit all the captured images of the second camera frame by frame. For the complete captured images including the non-interest area (background area) or the local area image of the non-interest area (background area), the interval transmission method can be adopted, and the images are transmitted to the back-end processing module according to the determined frame transmission interval. Therefore, the multi-camera strategy also includes: the partial frame transmission switch of the captured image of the second camera is turned on, and the front-end processing module obtains the complete captured image or the local area image of the non-interest area (background area) of the second camera according to the setting item and transmits it to the back-end processing module according to the determined frame transmission interval.
[0138] For example, if the frame transmission interval is 1 frame, the front-end processing module transmits 1 frame of the captured image of the second camera to the back-end processing module every 1 frame.
[0139] In some exemplary embodiments, when the complete captured image of the second camera is transmitted according to the frame transmission interval, it indicates that the complete captured image has included the region of interest; therefore, at the transmission time point corresponding to the frame transmission interval, the transmission data corresponds to the captured image of the first camera and the captured image of the second camera, and at the transmission time point corresponding to the non-frame transmission interval, the transmission data corresponds to the captured image of the first camera and the second extracted image. For example, when the frame transmission interval is 1 frame, at the transmission time points corresponding to the 1st, 3rd, 5th, 7th… frames, the transmission data corresponds to the captured images of the 1st, 3rd, 5th, 7th… frames of the first camera and the captured images of the 1st, 3rd, 5th, 7th… frames of the second camera; at the transmission time points corresponding to the 2nd, 4th, 6th, 8th… frames, the transmission data corresponds to the captured images of the 2nd, 4th, 6th, 8th… frames of the first camera and the region of interest images of the 2nd, 4th, 6th, 8th… frames of the second camera.
[0140] In some exemplary embodiments, when a local captured image of the second camera is transmitted according to a frame transmission interval, the local captured image is a local image of a non-region of interest; therefore, at a transmission time point corresponding to the frame transmission interval, the transmitted image corresponds to the captured image of the first camera, the second extracted image and the third extracted image, and at a transmission time point corresponding to a non-frame transmission interval, the transmitted image corresponds to the captured image of the first camera and the second extracted image; wherein the third extracted image is a local area image extracted from the captured image of the second camera that does not include the region of interest.
[0141] It can be understood that in the above embodiment, according to the multi-camera strategy determined by the shooting scene, the front-end processing module effectively reduces the amount of image data transmitted to the back-end processing module. On the basic premise that the back-end processing module performs SAT processing, the processing pressure of the back-end processing module is reduced and the processing performance of the terminal is improved.
[0142] It should be noted that the specific shooting scenes recorded in the embodiments of the present disclosure include:
[0143] A single-target scene under the blur function, a multi-target scene under the blur function, a scene in which the proportion of the area of interest in the image captured under the SAT function is less than a first proportion threshold, a scene in which the proportion of the area of interest in the image captured under the SAT function is greater than a second proportion threshold, and a scene in which the captured object in the area of interest contained in the SAT function remains stable.
[0144] It can be seen that the shooting scenes in the above examples are specific scenes in the multi-camera scheme that involve using the output data of multiple cameras to collaboratively realize the corresponding shooting functions, wherein the shooting functions involved are shooting functions implemented based on the multi-camera scheme, including blur functions and SAT functions. The scheme provided by the front-end processing module of the present disclosure embodiment for performing differentiated multi-camera data processing according to the shooting scene recognition results is not limited to these specific shooting functions and / or shooting scenes used for examples. For more shooting functions based on multi-camera schemes, according to the embodiments recorded in the embodiments of the present disclosure, a similar analysis process can be used to define relevant shooting scenes and determine corresponding multi-camera strategies, so as to reduce the amount of transmitted data, reduce the computing power requirements of the back-end processing module, and / or improve the effect of collaborative work of multiple cameras while satisfying the multi-camera function.
[0145] In some exemplary embodiments, the video image includes a first video image; the first video image is video data generated based on the image data captured by the camera that is not started in advance in the received transmission data. That is, the first video image is a video image generated based on the image data captured by one or more cameras that are officially used under the current shooting function, and the one or more cameras do not include the camera that is started in advance.
[0146] It should be noted that which camera or cameras are used officially under the current shooting function is determined based on the configuration data from the back-end processing module, or based on the configuration data on the front-end processing module, and is not limited to specific situations.
[0147] The back-end processing module is also configured to use the first video image in a shooting preview or video recording mode.
[0148] In some exemplary embodiments, the video image further includes a preliminary video image; the preliminary video image is video data generated according to received transmission data, and the transmission data includes image data captured by a camera activated in advance.
[0149] It is understood that, in the case where the received transmission data includes the image data taken by the camera started in advance, the back-end processing module can generate the first video image and the preliminary video image. In the case where the received transmission data does not include the image data taken by the camera started in advance, the back-end processing module only generates the first video image.
[0150] In some exemplary embodiments, the back-end processing module is further configured to perform spatial alignment transformation in advance to generate a preliminary video image based on the captured image of the second camera started in advance; when the camera switching conditions are met, the preliminary video image is used in the shooting preview or recording mode.
[0151] In some exemplary embodiments, the front-end processing module is also configured to determine whether the transmission data is transmitted to the back-end processing module through one channel of data or is transmitted to the back-end processing module separately through multiple channels of data according to the amount of image data captured by non-first cameras in the transmission data.
[0152] It can be seen that in various shooting scenarios, the transmitted data includes images captured by the first camera and other data. These other data are collectively referred to as image data not captured by the first camera, for example, scene depth data, image data captured by the second camera, first extracted data or second extracted data, etc.
[0153] According to the data volume of the image data captured by the non-first camera, the corresponding transmission mode is determined:
[0154] When the data volume of the image data captured by the non-first camera is greater than a first data volume threshold, transmitting multiple channels of data to the back-end processing module;
[0155] When the data volume of the image data captured by the non-first camera is less than or equal to the first data volume threshold, the data is transmitted to the back-end processing module through one channel.
[0156] In some exemplary embodiments, separating multiple data transmission paths to the back-end processing module includes:
[0157] The image data captured by the first camera is transmitted to the back-end processing module via the first data path;
[0158] The image data captured by the non-first camera is transmitted to the back-end processing module via the second data path.
[0159] The present disclosure also provides a shooting system. Figure 6 As shown, including:
[0160] Front-end processing module 610, back-end processing module 620 and MIPI / PCIe interface module 630;
[0161] The front-end processing module 610 includes: a scene analysis submodule 6101 , a multi-camera image differentiation processing submodule 6102 , and a data packaging submodule 6103 ; the back-end processing module 620 includes: a multi-camera image processing submodule 6201 .
[0162] The scene analysis submodule 6101 is configured to perform scene analysis and determine the shooting scene according to the shooting parameters sent by the backend processing module and the images shot by one or more cameras;
[0163] The shooting parameters include: shooting mode and / or shooting function information;
[0164] The multi-camera image differentiation processing submodule 6102 is configured to determine a corresponding multi-camera strategy according to the identified shooting scene, and obtain transmission data according to the determined multi-camera strategy;
[0165] The data packaging submodule 6103 is configured to complete the transmission data packaging and transmit it to the backend processing module 620 through the MIPI / PCIe interface module 630;
[0166] The multi-camera image processing submodule 6201 is configured to generate a video image according to the received transmission data.
[0167] The present disclosure also provides a shooting method. Figure 7 As shown, including:
[0168] Step 710, the front-end processing module identifies the shooting scene under the current shooting function according to the shooting image obtained under the current shooting function;
[0169] Step 720, determining a corresponding multi-camera strategy according to the identified shooting scene, and acquiring transmission data according to the determined multi-camera strategy, and transmitting the data to a back-end processing module;
[0170] Step 730, the backend processing module generates a video image according to the received transmission data;
[0171] The multi-camera strategy includes one or more of the following settings:
[0172] The switch for scene depth data, the switch for early start of the camera, the switch for acquiring local area images of the camera, and the switch for transmitting partial frames of images captured by the camera.
[0173] In some exemplary embodiments, when the current shooting function is blurring, the shooting scene includes: a single-target scene or a multi-target scene;
[0174] The multi-camera strategy includes: turning on the scene depth data switch;
[0175] or,
[0176] The multi-camera strategy includes: a target area image acquisition switch of the second camera is turned on.
[0177] In some exemplary embodiments, the transmission data includes: an image captured by a first camera;
[0178] When the scene depth data switch is turned on, the transmission data further includes: scene depth data; the scene depth data is calculated based on the image captured by the first camera and the image captured by the second camera;
[0179] When the target area image acquisition switch of the second camera is turned on, the transmission data also includes: a first extracted image; the first extracted image is a local area image containing the single target or the multiple targets extracted from the captured image of the second camera.
[0180] In some exemplary embodiments, when the current shooting function is spatial alignment transformation, and the shooting scene is a scene in which the proportion of the area of interest in the captured image is less than the first proportion threshold or the proportion of the area of interest in the captured image is greater than the second proportion threshold, the multi-camera strategy includes: turning on the advance start switch of the second camera;
[0181] and / or;
[0182] When the current shooting function is spatial alignment transformation and the shooting scene is a scene in which the shooting object in the included area of interest remains stable, the multi-camera strategy includes: turning on the image acquisition switch of the area of interest of the second camera and turning on the partial frame transmission switch of the image shot by the second camera.
[0183] In some exemplary embodiments, the transmission data includes: an image captured by a first camera;
[0184] When the advance start switch of the second camera is turned on, the transmission data further includes: an image captured by the second camera;
[0185] When the image acquisition switch of the region of interest of the second camera is turned on, the transmission data further includes: a second extracted image; the second extracted image is a local area image containing the region of interest extracted from the image captured by the second camera;
[0186] When the partial frame transmission switch of the captured image of the second camera is turned on, according to the determined frame transmission interval, the transmission data corresponding to the transmission time point also includes: a complete captured image or a partial captured image of the second camera.
[0187] In some exemplary embodiments, step 730 includes:
[0188] The first video data is generated according to the image data captured by the camera that is not started in advance in the received transmission data.
[0189] In some exemplary embodiments, step 730 includes:
[0190] Generate first video data according to the captured image data of the camera not started in advance in the received transmission data;
[0191] Based on the image captured by the second camera started in advance, a spatial alignment transformation is performed in advance to generate a preliminary video image.
[0192] In some exemplary embodiments, the method further comprises:
[0193] Step 740: Use the first video image to shoot, preview or record.
[0194] In some exemplary embodiments, step 740 further includes: when a camera switching condition is met, using the prepared video image for shooting preview or recording.
[0195] The present disclosure also provides a shooting method. Figure 8 As shown, including:
[0196] Step 700, obtaining system operation load information;
[0197] Step 701, judging whether multi-camera operation optimization is required according to the load information, if so, executing steps 710-720; otherwise, executing step 750;
[0198] Step 750, according to the acquired camera start instruction, start the corresponding camera, obtain the complete captured image and transmit it to the back-end processing module.
[0199] Among them, when the current system operation load is less than the load threshold, it is determined that multi-camera operation optimization is not required; when the current system operation load is greater than or equal to the load threshold, it is determined that multi-camera operation optimization is required.
[0200] Among them, the operating load information includes: processor temperature, processor usage and / or memory usage, etc. The corresponding load information can be obtained according to different system load evaluation schemes, and the current system operating load can be determined. More detailed aspects are not further discussed in the embodiments of this application.
[0201] It can be understood that the embodiments of the present disclosure can flexibly determine whether the front-end processing module is required to perform differentiated processing for the shooting scene according to the operating load of the system. When the current system operating load is small, there is no need to perform differentiated processing, and the full amount of data is transmitted to the back-end processing module for corresponding function execution, at the cost of occupying more system computing power and internal interface transmission bandwidth, to provide a better multi-camera shooting function; when the current system operating load is not small, differentiated processing is performed to reduce the amount of data transmission, so as to reduce the occupation of the back-end processing module system computing power, reduce the occupation of the internal interface transmission bandwidth, take into account the overall performance of the terminal, and provide a better multi-camera shooting function.
[0202] The present disclosure also provides a shooting method. Fig. 9 As shown, including:
[0203] Step 910, the backend processing module sets shooting parameters;
[0204] Step 920, the front-end processing module obtains shooting parameters;
[0205] Step 930, the camera captures and outputs the captured image;
[0206] Step 940, the front-end processing module determines the shooting scene according to the shooting parameters and the shot image;
[0207] Step 950: The front-end processing module determines a multi-camera strategy according to the shooting scene;
[0208] Step 960: The front-end processing module performs differential processing according to the determined multi-camera strategy to obtain transmission data;
[0209] Step 970, the front-end processing module sends the transmission data to the back-end processing module;
[0210] Step 980: The backend processing module generates a video image for shooting preview or recording.
[0211] The shooting parameters include one or more of the following:
[0212] Shooting modes, shooting functions.
[0213] The shooting modes include: taking photos, recording videos, or shooting preview, or other modes.
[0214] The shooting functions include: blur function, spatial transformation SAT function, or other multi-camera functions.
[0215] The captured image output in step 930 is a RAW image.
[0216] The shooting scenes determined in step 940 include one or more of the following:
[0217] A single-target scene under the blur function, a multi-target scene under the blur function, a scene in which the proportion of the area of interest in the image captured under the SAT function is less than a first proportion threshold, a scene in which the proportion of the area of interest in the image captured under the SAT function is greater than a second proportion threshold, a scene in which the captured object in the area of interest contained in the SAT function remains stable; or other scenes defined under other multi-camera functions.
[0218] Among them, in step 950, the multi-shooting strategy corresponding to each shooting scene is determined according to the preset multi-shooting strategy configuration data.
[0219] The present disclosure also provides a smart terminal, including:
[0220] one or more processors;
[0221] a storage device for storing one or more programs,
[0222] When the one or more programs are executed by the one or more processors, the one or more processors implement the shooting method as described in any embodiment of the present disclosure.
[0223] An embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the shooting method as described in any embodiment of the present disclosure is implemented.
[0224] It can be seen that the shooting scheme provided by the embodiment of the present disclosure determines differentiated multi-camera strategies based on the front-end processing module's recognition of the shooting scene under the current shooting function, and correspondingly generates and transmits the data required for different multi-camera schemes to the back-end processing module in a differentiated manner. Compared with the multi-camera scheme that simply transmits the full amount of multi-camera shooting data without distinguishing the shooting scenes, it can effectively reduce the occupancy of the transmission bandwidth between the internal interfaces of the terminal system, reduce the computing power pressure of the back-end processing module, and improve the overall terminal system performance.
[0225] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0226] The above description is only a preferred embodiment of the present application scheme, and does not limit the patent scope of the present application. All equivalent structural changes made based on the concept of the present application scheme and the contents of the present application specification and drawings, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A shooting system, characterized in that: include: A front-end processing module and a back-end processing module; The front-end processing module is configured to identify the shooting scene under the current shooting function according to the captured image obtained under the current shooting function; determine the corresponding multi-shooting strategy according to the identified shooting scene; obtain transmission data according to the determined multi-shooting strategy, and transmit it to the back-end processing module; The back-end processing module is configured to generate a video image according to the received transmission data; and is further configured to perform a spatial alignment transformation in advance according to the captured image of the second camera started in advance to generate a preliminary video image; and when the camera switching condition is met, the preliminary video image is used in the shooting preview or recording mode; The multi-camera strategy includes one or more of the following settings: A scene depth data switch, a camera early start switch, a camera local area image acquisition switch, and a camera image partial frame transmission switch; When the current shooting function is blurring, the shooting scene includes: a single-target scene or a multi-target scene; the multi-shooting strategy includes: a scene depth data switch is turned on, and a target area image acquisition switch of the second camera is turned on; wherein the turned-on target area image acquisition switch of the second camera is used to enable the front-end processing module to extract a local area image containing the single target or the multiple targets from the captured image of the second camera to form the transmission data; When the current shooting function is spatial alignment transformation, and the shooting scene is a scene in which the proportion of the region of interest in the shot image is less than the first proportion threshold or the proportion of the region of interest in the shot image is greater than the second proportion threshold, the multi-camera strategy includes: turning on the advance start switch of the second camera; When the current shooting function is spatial alignment transformation and the shooting scene is a scene in which the shooting object in the included area of interest remains stable, the multi-camera strategy includes: the image acquisition switch of the area of interest of the second camera is turned on and the partial frame transmission switch of the image shot by the second camera is turned on; wherein the turned-on image acquisition switch of the area of interest of the second camera is used to enable the front-end processing module to extract the local area image including the area of interest from the image shot by the second camera to constitute the transmission data.
2. The system according to claim 1, characterized in that The transmission data includes: images captured by the first camera; When the scene depth data switch is turned on, the transmission data further includes: scene depth data; the scene depth data is calculated based on the image captured by the first camera and the image captured by the second camera; When the target area image acquisition switch of the second camera is turned on, the transmission data further includes: a first extracted image; the first extracted image is a local area image containing the single target or the multiple targets extracted from the captured image of the second camera; When the advance start switch of the second camera is turned on, the transmission data further includes: an image captured by the second camera; When the switch for acquiring the image of the region of interest of the second camera is turned on, the transmission data further includes: a second extracted image; the second extracted image is a local area image containing the region of interest extracted from the image captured by the second camera; When the partial frame transmission switch of the captured image of the second camera is turned on, according to the determined frame transmission interval, the transmission data corresponding to the transmission time point also includes: a complete captured image or a partial captured image of the second camera.
3. The system according to claim 1 or 2, characterized in that: The front-end processing module is also configured to determine, based on the amount of image data other than the first camera captured in the transmission data, whether the transmission data is transmitted to the back-end processing module via one data path or is transmitted to the back-end processing module in multiple data paths.
4. A shooting method, characterized in that: include: The front-end processing module identifies the shooting scene under the current shooting function according to the shooting image obtained under the current shooting function; According to the identified shooting scene, a corresponding multi-camera strategy is determined, and transmission data is obtained according to the determined multi-camera strategy and transmitted to a back-end processing module; The back-end processing module generates a video image according to the received transmission data; and is further configured to perform a spatial alignment transformation in advance according to the captured image of the second camera started in advance to generate a preliminary video image; and when the camera switching condition is met, the preliminary video image is used in the shooting preview or recording mode; The multi-camera strategy includes one or more of the following settings: A scene depth data switch, a camera early start switch, a camera local area image acquisition switch, and a camera image partial frame transmission switch; When the current shooting function is blurring, the shooting scene includes: a single-target scene or a multi-target scene; the multi-shooting strategy includes: a scene depth data switch is turned on, and a target area image acquisition switch of the second camera is turned on; wherein the turned-on target area image acquisition switch of the second camera is used to enable the front-end processing module to extract a local area image containing the single target or the multiple targets from the captured image of the second camera to form the transmission data; When the current shooting function is spatial alignment transformation, and the shooting scene is a scene in which the proportion of the region of interest in the shot image is less than the first proportion threshold or the proportion of the region of interest in the shot image is greater than the second proportion threshold, the multi-camera strategy includes: turning on the advance start switch of the second camera; When the current shooting function is spatial alignment transformation and the shooting scene is a scene in which the shooting object in the included area of interest remains stable, the multi-camera strategy includes: the image acquisition switch of the area of interest of the second camera is turned on and the partial frame transmission switch of the image shot by the second camera is turned on; wherein the turned-on image acquisition switch of the area of interest of the second camera is used to enable the front-end processing module to extract the local area image including the area of interest from the image shot by the second camera to constitute the transmission data.
5. An intelligent terminal, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the shooting method as claimed in claim 4.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the shooting method as claimed in claim 4 is implemented.
