Photography method, device, equipment and medium for multi-camera equipment

Abstract

The present invention provides a photography method, device, equipment and medium for a multi-camera device, wherein the photography method for the multi-camera device comprises: sending a first photography request to a default camera according to a user application request to report first image data; when a camera switching instruction is received, sending a second photography request to a designated camera, wherein the designated camera is any camera in the multi-camera device except the default camera, to report second image data; performing frame matching on the second image data with image data that has not been displayed in the first image data, and stopping sending the first photography request when the current frame in the second image data is synchronized with the current frame in the first image data. The present invention can solve the problem of high power consumption caused by camera switching in the background technology.

Description

Technical Field

[0001] The present invention relates to the field of photography, and in particular to a photography method, device, equipment and medium for a multi-camera device. Background Art

[0002] In traditional cameras, optical zoom is one of the basic functions. This zoom is achieved by operating the zoom mechanism on the traditional camera head to change the relative displacement of the optical components in the optical system.

[0003] At present, mobile phones have become a common photography tool for people, but it is difficult for mobile phones to achieve the complex and precise optical zoom function mentioned above. The optical zoom function of mobile phones is achieved by switching between multiple different fixed focal length modules. However, directly switching the display screen between multiple focal length modules will make the overall visual effect of the picture abrupt. Therefore, in order to solve this problem, the digital zoom is usually used to connect the field of view of the picture before switching between two adjacent focal length modules to ensure that the picture switching between the front and rear focal length modules is smooth and smooth, and the automatic exposure (Auto Exposure, AE), automatic focus (Auto Focus, AF), automatic white balance (Auto White Balance, AWB) and other related effects between the front and rear focal length modules are synchronized to reduce the abrupt changes in the picture effect caused by hardware differences, so as to achieve a function similar to the optical zoom of traditional cameras. This technology is usually called hybrid optical zoom.

[0004] Mobile phones have gradually evolved from a single camera to three or even four cameras. The variability of pixel size and the diversity of field of view not only increase the demand for camera hardware, but also the demand for software solutions that can support efficient switching between multiple cameras. In order to obtain a clearer vision or a wider viewing angle, the built-in camera of the mobile phone will support wide-angle and telephoto cameras to adapt to more application scenarios. Therefore, when adjusting the shooting effect, the stable and smooth switching performance in the focal length will also affect the camera's shooting effect. In addition, in the process of switching cameras, discontinuity of effects such as clarity, brightness, white balance, and restart of data channels will also cause phenomena such as mobile phone freezes.

[0005] The current work plans for switching between multiple cameras are:

[0006] 1) Multi-channel switching solution requires multiple digital camera interfaces (DCAMs) and does not support dynamic switching control between DCAMs and camera serial interfaces (CSIs). Each DCAM is bound to each CSI one by one. All DCAMs and sensors remain in continuous working state after power-on until power-off. For example, a Chinese patent with publication number "CN112073642A" discloses a video recording method and device, storage medium, and terminal for a multi-camera device. Although it can solve the problem of unsmooth images caused by camera switching, it sacrifices power consumption and increases hardware costs.

[0007] 2) Single-channel switching solution supports dynamic switching control of DCAM and CSI, but only one DCAM is in working state during each switching process. That is, during the switching process, the DCAM must be disconnected from the CSI in the foreground display before it can connect to another CSI to transfer its data to the foreground display. This solution is suitable for products with one or more DCAMs and has low power consumption, but the switching speed experience is much inferior to the previous solution, and due to the inherent characteristics of the solution itself, it is easy to show switching jams.

[0008] Therefore, in order to ensure that the problem of unsmooth images caused by camera switching can be solved, the present invention proposes a photography method, device, equipment and medium for a multi-camera device to reduce power consumption during multi-camera switching. Summary of the invention

[0009] The present invention provides a photography method, device, equipment and medium for a multi-camera device to solve the problem of high power consumption during the switching process of multiple cameras.

[0010] In a first aspect, the present invention provides a photography method for a multi-camera device, wherein the multi-camera device includes N cameras, where N is a positive integer greater than or equal to 2, the N cameras include a default camera, and any one of the N cameras except the default camera can be used as a designated camera in a camera switching process, comprising: sending a first photography request to the default camera according to a user application request, so that the default camera reports first image data according to the first photography request, and the multi-camera device is used to display an image according to the first image data, wherein the first photography request includes any one of a preview request, a photo request, and a video recording request; when a camera switching instruction is received, sending a second photography request to the designated camera, so that the designated camera reports second image data according to the second photography request, and the second photography request includes any one of a preview request, a photo request, and a video recording request; performing frame matching on the second image data and graphic data that has not yet been displayed in the first image data, and when a current frame in the second image data is synchronized with a current frame in the first image data, stopping sending the first photography request to the default camera, and the multi-camera device is used to display an image according to the second image data; The image data that has not been displayed in the first image data is recycled, and the default camera enters a stop working state.

[0011] The beneficial effect is that the present invention can solve the problem of high power consumption caused by all DCAMs and Sensors in the preview, photo or video state maintaining a continuous working state after power-on until power-off in the background technology by sending a second photography request to the designated camera only when receiving a camera switching instruction, reporting the second image data according to the second photography request, and after recovering the image data that has not been displayed in the first image data. And when the current frame in the second image data is synchronized with the current frame in the first image data, the first photography request is stopped from being sent to the default camera, and the multi-camera device is used to display the image according to the second image data, which can solve the problem of unsmooth picture caused by camera switching.

[0012] Optionally, when the current frame in the second image data is synchronized with the current frame in the first image data, it includes: obtaining the acquisition time of the first image data and the acquisition time of the second image data; aligning the display effect of the current frame in the second image data with the current frame in the first image data according to the image frame number in the second photography request, the acquisition time of the first image data, the acquisition time of the second image data, the image frame number of the image data that has not yet been displayed in the first image data, and the statistical information of the display effect of the second image data and the first image data that changes with time. Its beneficial effect is that the present invention not only needs to align the frame number of the image data that has not yet been displayed in the first image data and the second image data, but also needs to align the display effects of these two parts of the image data to further ensure the smoothness and naturalness of the picture when switching cameras.

[0013] Further optionally, the frame matching of the second image data with the image data in the first image data that has not yet been displayed includes: determining the type of request frames to be re-sent to the designated camera and the number of request frames based on the second photography request and the image data in the first image data that has not yet been displayed, so that the second image data and the image data in the first image data that has not yet been displayed maintain the same number of frames, and the type of the request frame is consistent with the first photography request.

[0014] Optionally, the display effect includes at least one of automatic exposure, automatic focus and automatic white balance.

[0015] Optionally, after stopping sending the first photography request to the default camera, the method further includes: continuing to align the request queue of the second photography request with the request queue of the user application request. The beneficial effect is that the present invention provides for aligning the request queue of the second photography request with the request queue of the user application request, further ensuring a smooth switching effect between the designated camera and the default camera.

[0016] Optionally, the default camera and the designated camera have different focal lengths. After the second photography request is sent to the designated camera, the method includes: obtaining the calibration data of the designated camera on the calibration plane and the basic parameters of the corresponding image sensor, the calibration data including: a zoom ratio critical value, a horizontal error and a vertical error relative to the default camera on the calibration plane, a calibration plane width, and a calibration plane height, and the basic parameters of the image sensor include an image sensor width and an image sensor height; according to the calibration data of the designated camera on the calibration plane and the basic parameters of the image sensors corresponding to the designated cameras, the center distance offset of the designated camera relative to the default camera is corrected with the default camera as a reference; according to the correction result of the center distance offset of the designated camera relative to the default camera and the basic parameters of the image sensor of the designated camera, the zoom ratio critical value of the designated camera is corrected; according to the corrected zoom ratio critical value, the focal length corresponding to the designated camera is corrected to obtain the corrected focal length of the designated camera. The beneficial effect is that because the setting positions of each camera on a multi-camera device are different, when switching cameras, it is necessary to calibrate the focal lengths of the designated camera and the default camera to prevent abrupt images and unsmooth switching caused by the camera switching process.

[0017] Optionally, the photography method of the multi-camera device further includes: when N is greater than or equal to 3, and when the calibrated focal length of the designated camera does not meet the set requirements, stop sending the second photography request to the designated camera, and send a third photography request to other cameras, so that the other cameras report the third image data according to the received third photography request, the other cameras are any one of the multi-camera device except the default camera and the designated camera, and the third photography request includes any one of a preview request, a photo request and a video recording request. Its beneficial effect is that when the designated camera does not meet the requirements, it is switched.

[0018] In a second aspect, the present invention provides a photographing device of a multi-camera device, wherein the multi-camera device includes N cameras, where N is a positive integer greater than or equal to 2, and is used to execute the photographing method of the multi-camera device as described in any one of the first aspects, wherein the N cameras include a default camera, and any one of the N cameras except the default camera can be used as a designated camera in a camera switching process, including: a default photographing module, a switching module, a matching module, a termination module and a recycling module; the default photographing module is used to send a first photographing request to the default camera according to a user application request, so that the default camera reports first image data according to the first photographing request, and the multi-camera device is used to display an image according to the first image data, and the first photographing request includes any one of a preview request, a photo request and a video recording request. The switching module is used to send a second photography request to the designated camera when receiving a camera switching instruction, so that the designated camera reports the second image data according to the second photography request, and the second photography request includes any one of a preview request, a photo request and a video recording request; the matching module is used to perform frame matching between the second image data and the image data that has not yet been displayed in the first image data, and when the current frame in the second image data is synchronized with the current frame in the first image data, the termination module controls the default photography module to stop sending the first photography request to the default camera, and the multi-camera device is used to display an image according to the second image data; the recycling module is used to recycle the image data that has not yet been displayed in the first image data, and is also used to control the default camera to enter a stopped working state.

[0019] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, executes the steps of the photography method of a multi-camera device as described in any one of the first aspects.

[0020] In a fourth aspect, the present invention provides a terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, the steps of the photography method of a multi-camera device as described in any one of the first aspects are executed.

[0021] For the beneficial effects of the second to fourth aspects, reference may be made to the description of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of an embodiment of a photography method for a multi-camera device provided by the present invention;

[0023] Figure 2 A flowchart of a partial structural embodiment of a photographing device of a multi-camera device provided by the present invention;

[0024] Figure 3 A schematic diagram of a partial structural embodiment of a photographing device of another multi-camera device provided by the present invention;

[0025] Figure 4 A schematic structural diagram of an embodiment of a photographing device of a multi-camera device provided by the present invention;

[0026] Figure 5 A schematic diagram of a terminal embodiment provided by the present invention. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a kind", "the", "above" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a kind of "or" relationship.

[0028] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0029] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0030] The present invention provides a photography method, device, equipment and medium of a multi-camera device to solve the problem of high power consumption during multi-camera switching. The multi-camera device includes N cameras, N is a positive integer greater than or equal to 2, the N cameras include a default camera, and any one of the N cameras except the default camera can be used as a designated camera during camera switching.

[0031] The present invention provides a photography method for a multi-camera device, the process of which is as follows: Figure 1 As shown, including:

[0032] S101: sending a first photography request to the default camera according to a user application request, so that the default camera reports first image data according to the first photography request, and the multi-camera device is used to display an image according to the first image data, wherein the first photography request includes any one of a preview request, a photo request, and a video recording request;

[0033] S102: When a camera switching instruction is received, a second photographing request is sent to the designated camera, so that the designated camera reports second image data according to the second photographing request, where the second photographing request includes any one of a preview request, a photo request, and a video recording request;

[0034] S103: performing frame matching on the second image data and image data that has not been displayed in the first image data, and when a current frame in the second image data is synchronized with a current frame in the first image data, stopping sending the first photography request to the default camera, and the multi-camera device is used to display an image according to the second image data;

[0035] S104: Recycling the image data that has not been displayed in the first image data, and the default camera enters a stop working state.

[0036] When the multi-camera device starts to display images according to the second image data, the remaining first image data no longer displayed on the screen of the multi-camera device needs to be recycled, that is, the image data that has not been displayed in the first image data, and after completion, the default camera can be put into a shutdown state. And the default camera-related components can be selectively controlled to enter a shutdown state, and the default camera-related components include but are not limited to stopping the data flow path connected to the default camera.

[0037] The present invention can solve the problem of high power consumption caused by all DCAMs and sensors in the background technology remaining in a continuous working state until power off after power on by sending a second photography request to the designated camera only when receiving a camera switching instruction, reporting the second image data according to the second photography request, and making the data path or hardware corresponding to the first photography request enter a stopped working state after recovering the image data that has not been displayed in the first image data. And when the current frame in the second image data is synchronized with the current frame in the first image data, stop sending the first photography request to the default camera, and the multi-camera device is used to display the image according to the second image data, which can solve the problem of unsmooth picture caused by camera switching.

[0038] The multi-camera device includes N cameras, where N is a positive integer greater than or equal to 2, and the focal lengths of the N cameras are respectively [R1, R2), [R2, R3), [R3, R4) ... [R n ,R n+1 ], where R1, R2, R3, R4, R n , R n+1 are the zoom ratio critical values ​​of each camera, and 0<R1<R2<R3<R4<R n <R n+1 For example, the multiple designated cameras include: an ultra-wide-angle camera, a wide-angle camera, and a telephoto camera. The focal length of the ultra-wide-angle camera is [R1, R2), the focal length of the wide-angle camera is [R2, R3), and the focal length of the telephoto camera is [R3, R4]. Among them, R1<R2<R3<R4.

[0039] In some embodiments, the focal length of the camera is related to the field of view angle, and the larger the focal length of the camera, the smaller the field of view angle. Take a multi-camera device including an ultra-wide-angle camera, a wide-angle camera, and a telephoto camera as an example. From the above description, it can be seen that the focal length [R1, R2) of the ultra-wide-angle camera SW is smaller than the focal length [R2, R3) of the wide-angle camera W, the focal length [R2, R3) of the wide-angle camera W is smaller than the focal length [R3, R4] of the telephoto camera T, the field of view angle of the ultra-wide-angle camera SW is larger than the field of view angle of the wide-angle camera W, and the field of view angle of the wide-angle camera W is larger than the field of view angle of the telephoto camera T. Optionally, as Figure 2 As shown, in the multi-camera device, the multi-camera device includes a switch module ( Figure 2 (not shown), two digital camera interfaces (Digital Camera Interface, DCAM) with the same capabilities and three cameras with different focal lengths (ultra-wide-angle camera, wide-angle camera and telephoto camera), each camera includes a sensor (Senor), a physical layer interface (Physics Layer) and a camera interface (Camera Serial Interface, CSI), and their connection relationship is common knowledge in the technical field and will not be repeated here. Among them, the wide-angle camera includes a first sensor Sensor1, a first physical layer interface PHY1 and a first camera serial port CSI1, the ultra-wide-angle camera includes a second sensor Sensor2, a second physical layer interface PHY2 and a second camera serial port CSI2, and the telephoto camera includes a third sensor Sensor3, a third physical layer interface PHY3 and a third camera serial port CSI3. Because the switching of the two digital camera interfaces is involved later, in order to better distinguish them, the two digital camera interfaces are named the first digital camera interface DCAM1 and the second digital camera interface DCAM2, respectively, and by default, the first digital camera interface DCAM1 is connected to the first camera serial port CSI1. The present application is not limited to the cameras connected by default. The first sensor Sensor1, the second sensor Sensor2 and the third sensor Sensor3 are all image sensors.

[0040] When performing dynamic switching, the first digital camera interface DCAM1 may stop receiving data from the first camera serial port CSI1 and connect to the second camera interface CSI2 or the third camera interface CSI3. When the multi-camera device includes more cameras, the first digital camera interface DCAM1 may connect to any one except the first camera serial port CSI1. Figure 2The other devices in the system have no effect. Only the first digital camera interface DCAM1 and the second digital camera interface DCAM2 need to switch parameters and control timing in time. For inputs from sensors of different sources (for example, the first sensor Sensor1, the second sensor Sensor2 and the third sensor Sensor3), the first digital camera interface DCAM1 and the second digital camera interface DCAM2 are responsible for saving their respective parameter contexts. And the switching support occurs at any time when the first digital camera interface DCAM1 and the second digital camera interface DCAM2 are working. When the first digital camera interface DCAM1 is disconnected from the currently connected first camera serial port CSI1, the hardware will wait for the first camera serial port CSI1 to output a frame of data before disconnecting; when the first digital camera interface DCAM1 is connected to the second camera serial port CSI2, the hardware will also wait for the second camera serial port CSI2 to output a frame of data before connecting. For the first digital camera interface DCAM1 and the second digital camera interface DCAM2, it is two consecutive frames with a slightly longer interval.

[0041] In the multi-camera device of the present invention, the first digital camera interface DCAM1 or the second digital camera interface DCAM2 is also connected to an image signal processor (ISP). The first digital camera interface DCAM1 or the second digital camera interface DCAM2 is also connected to a control module. The connection relationship between the first digital camera interface DCAM1 or the second digital camera interface DCAM2 and the ISP and the control module is as follows: Figure 3 As shown, the control module 301 is used to issue control instructions and send different instructions to the digital camera interface 302 or the image signal processor 303 to inform the driver to receive or disconnect the image sensor of the corresponding camera and its input and output size and other parameters. The digital camera interface 302 is Figure 2 The first digital camera interface DCAM1 or the second digital camera interface DCAM2.

[0042] based on Figure 2 and Figure 3 The schematic diagram of the multi-camera device provided takes the preview request as an example to describe the specific execution steps of the present invention as follows:

[0043] Step 1: By default (with the wide-angle camera responsible for preview display as the initial working state), after the preview starts, before there is no camera switching action, only the first digital camera interface DCAM1 and the first camera serial port CSI1 are always kept in the on state, and continue to provide the data required for the preview request, while the connection between the second digital camera interface DCAM2, the second camera serial port CSI2, and the third camera serial port CSI3 will be disconnected after the preview is completed. More specifically, after the preview is completed, the preview frame request is only sent to the wide-angle camera, and the first camera interface CSI1 is responsible for reporting the preview frame in response to the request. At this time, the second camera serial port CSI2 and the third camera serial port CSI3 are in a disconnected state, and only the parameter context needs to be saved.

[0044] Step 2: If the camera is switched to an ultra-wide-angle camera, then when the camera switching starts, the previous preview frame request of the first digital camera interface DCAM1 has not been fully returned, the second sensor Sensor2 starts the flow, and the second camera serial port CSI2 is connected to the second digital camera interface DCAM2, and the preview frame request issued by the user application is also sent to the second digital camera interface DCAM2. Here, it is necessary to determine the number of request frames and the requested frame type that need to be reissued to the second digital camera interface DCAM2 based on the request status of the frames that have been returned. In this embodiment, the frame type is preview frame until the same number of frames is maintained. This step is called request synchronization, and its purpose is to align as much as possible with the number of requests that have not been returned by the first digital camera interface DCAM1, and to align the display effects. The display effects include automatic exposure, automatic focus and automatic white balance, so that the control module can switch the display at any time after the display effect alignment is completed.

[0045] Step 3: synchronize the display effect of the image output of the second digital camera interface DCAM2 with that of the first digital camera interface DCAM1. After several frames have passed and it is confirmed that the effects are close to being consistent, notify the control module that the effect synchronization is complete and proceed to the next step.

[0046] Step 4: When the control module receives the signal indicating that the effect synchronization is completed, it stops sending requests to the first digital camera interface DCAM1 and continues sending requests only to the second digital camera interface DCAM2. Although there are still some preview frames that have not been returned by the first digital camera interface DCAM1, the foreground preview screen can be output by the second digital camera interface DCAM2 for display.

[0047] Step 5: At this time, the request queue of the second digital camera interface DCAM2 and the request queue issued by the user application may not be consistent. It is necessary to continue to align the request queue of the second digital camera interface DCAM2 and the request queue issued by the user application. This is the second request synchronization process. Generally, this process is very fast. Basically, it is only necessary to make the request of the second digital camera interface DCAM2 consistent with the request issued by the user application in the next request process. This is because the previous tracking of the reissue request is very close to the current actual request.

[0048] Step 6: When all frames of the first digital camera interface DCAM1 are recovered, the first digital camera interface DCAM1 can be disconnected from the first camera interface CSI1, and then the first sensor Sensor is shut down. The current switching process is completed and the next switching can be performed.

[0049] According to the above steps and rules, in coordination with the driver and ISP algorithm related modules, the application of this method, compared with the method of only one request in the background technology, can display the preview screen after switching in advance by at least one buffer rotation cycle, so that the switching experience is improved. Compared with the method of keeping multiple modules in working state at all times, the present invention can also take into account the power consumption problem in the non-switching state. Compared with the multi-channel switching method in the background technology, the dual-channel switching method in the present invention will only start two DCAMs and the corresponding sensors senosr to work simultaneously when switching is required, and will shut down one of the DCAMs and the corresponding sensor after the switching is completed, so that it maintains the lowest power consumption in the non-switching state. In the non-switching state, the working state of the dual-channel switching mode is consistent with the single-channel switching mode, and in the switching state, for systems with three cameras or more, it can save at least one hardware requirement and software and hardware working power consumption relative to the multi-channel switching mode.

[0050] In some embodiments, when the current frame in the second image data is synchronized with the current frame in the first image data, it includes: obtaining the acquisition time of the first image data and the acquisition time of the second image data; aligning the display effect of the current frame in the second image data with the current frame in the first image data according to the image frame number in the second photography request, the acquisition time of the first image data, the acquisition time of the second image data, the image frame number of the image data that has not yet been displayed in the first image data, and the statistical information of the display effect of the second image data and the first image data that changes over time. The present invention not only needs to align the frame numbers of the image data that has not yet been displayed in the first image data and the second image data, but also needs to align the display effects of these two parts of the image data to further ensure the smoothness and naturalness of the picture when switching cameras.

[0051] In some embodiments, the frame matching of the second image data with the image data in the first image data that has not been displayed includes: determining the type of request frames to be reissued to the designated camera and the number of request frames according to the second photography request and the image data in the first image data that has not been displayed, so that the second image data and the image data in the first image data that has not been displayed maintain the same number of frames, and the type of request frames is consistent with the first photography request. The present invention makes the designated camera request and the user application request consistent by the type of request frames and the number of request frames reissued to the designated camera.

[0052] In some embodiments, the display effect includes at least one of automatic exposure, automatic focus, and automatic white balance.

[0053] In some embodiments, after stopping sending the first photography request to the default camera, the method further includes: continuing to align the request queue of the second photography request with the request queue of the user application request. The present invention provides aligning the request queue of the second photography request with the request queue of the user application request, further ensuring a smooth switching effect between the designated camera and the default camera.

[0054] In some embodiments, the default camera and the designated camera have different focal lengths. After the second photography request is sent to the designated camera, the method includes: obtaining the calibration data of the designated camera on the calibration plane and the basic parameters of the corresponding image sensor, the calibration data including: a zoom ratio critical value, a horizontal error and a vertical error relative to the default camera on the calibration plane, a calibration plane width, and a calibration plane height, and the basic parameters of the image sensor include an image sensor width and an image sensor height; according to the calibration data of the designated camera on the calibration plane and the basic parameters of the image sensors corresponding to the designated cameras, the center distance offset of the designated camera relative to the default camera is corrected with the default camera as a reference; according to the correction result of the center distance offset of the designated camera relative to the default camera and the basic parameters of the image sensor of the designated camera, the zoom ratio critical value of the designated camera is corrected; according to the corrected zoom ratio critical value, the focal length corresponding to the designated camera is corrected to obtain the corrected focal length of the designated camera. Because the setting positions of each camera on a multi-camera device are different, when switching cameras, it is necessary to calibrate the focal lengths of the designated camera and the default camera to prevent the problems of abrupt images and unsmooth switching during camera switching.

[0055] In some embodiments, the photography method of the multi-camera device further includes: when N is greater than or equal to 3, and when the calibrated focal length of the designated camera does not meet the set requirements, stop sending the second photography request to the designated camera, and send a third photography request to other cameras, so that the other cameras report the third image data according to the received third photography request, the other cameras are any one of the multi-camera device except the default camera and the designated camera, and the third photography request includes any one of a preview request, a photo request, and a video recording request. When the designated camera does not meet the requirements, switch it. And when the other cameras do not meet the requirements, continue to perform the camera switching operation until the switched camera meets the focal length requirements.

[0056] The present invention proposes a photography method for a multi-camera device, which is applicable to an imaging system composed of multiple cameras with different focal lengths (different field of view angles), and can realize arbitrary switching of cameras during multi-camera preview, video recording, and photo shooting to preview or shoot different focal length scenes. In the present invention, when a user switches different cameras, the control layer of the multi-camera will perform frame synchronization on the image data of different cameras reported by the bottom layer, so that the transition process of switching is more natural and smooth. The present invention does not require all cameras at the bottom layer to keep working at all times. Only when switching occurs, the relevant hardware will enter the working state according to the actual switching requirements issued by the upper layer, and after completing the acquisition of the requested output image data, the relevant hardware will exit the working state again, so that the system power consumption requirements in the preview, video recording or photo shooting application scenarios during the switching and non-switching processes are kept at the lowest level; in order to properly control the switching timing, the present invention introduces the concept of two synchronizations: one is effect synchronization, and the other is request synchronization. The effect synchronization refers to the synchronization of the first image data and the second image data in visual effects during switching, so as to ensure that there is no excessive exposure difference between the images before and after switching. In some embodiments, the visual effect includes exposure and white balance. The request synchronization refers to frame matching, that is, matching between the designated camera and the user application request, including frame number matching and frame type matching. In some embodiments, the designated camera supplements the corresponding type of image data according to the frame type of the image data required by the application in the terminal device.

[0057] Based on the photography method of the multi-camera device provided in the above embodiment, the present invention provides a photography device of a multi-camera device, wherein the multi-camera device includes N cameras, where N is a positive integer greater than or equal to 2, and is used to execute the photography method of the multi-camera device as described in any one of the above items, wherein the N cameras include a default camera, and any one of the N cameras except the default camera can be used as a designated camera in the camera switching process, and its structure is as follows: Figure 4As shown, it includes: a default photography module 401, a switching module 402, a matching module 403, a termination module 404 and a recovery module 405; the default photography module 401 is used to send a first photography request to the default camera according to a user application request, so that the default camera reports the first image data according to the first photography request, The multi-camera device is used to display an image according to the first image data, and the first photography request includes any one of a preview request, a photography request and a video recording request; the switching module 402 is used to receive a camera switching instruction, and when the switching module 402 receives the camera switching instruction, sends a second photography request to the designated camera, so that the designated camera reports the second image data according to the second photography request, and the second photography request includes any one of a preview request, a photography request and a video recording request; the matching module 403 is used to frame match the second image data with the image data that has not yet been displayed in the first image data, and when the current frame in the second image data is synchronized with the current frame in the first image data, the termination module 404 controls the default photography module 401 to stop sending the first photography request to the default camera, and the multi-camera device is used to display an image according to the second image data; the recycling module 405 is used to recycle the image data that has not yet been displayed in the first image data, and is also used to control the default camera to enter a stopped working state.

[0058] All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding unit module, and will not be repeated here.

[0059] In some other embodiments of the present application, the present application embodiment discloses a terminal, which can be used to refer to the photographing device of the multi-camera device in the above embodiment, such as Figure 5 As shown, the terminal 500 may include: one or more processors 501; a memory 502; a display 503; one or more applications (not shown); and one or more computer programs 504, and the above components may be connected via one or more communication buses 505. The one or more computer programs 504 are stored in the above memory 502 and are configured to be executed by the one or more processors 501, and the one or more computer programs 504 include instructions, which may be used to execute the various steps in the corresponding embodiment.

[0060] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0061] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional unit.

[0062] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0063] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A photography method of a multi-camera device, wherein the multi-camera device includes N cameras, where N is a positive integer greater than or equal to 2, characterized in that: The N cameras include a default camera, and any one of the N cameras except the default camera can be used as a designated camera in the camera switching process, include: sending a first photographing request to the default camera according to a user application request, so that the default camera reports first image data according to the first photographing request, and the multi-camera device is used to display an image according to the first image data, wherein the first photographing request includes any one of a preview request, a photo request, and a video recording request; When a camera switching instruction is received, a second photography request is sent to the designated camera, so that the designated camera reports second image data according to the second photography request, where the second photography request includes any one of a preview request, a photo request, and a video recording request; Performing frame matching on the second image data and image data that has not been displayed in the first image data, and when a current frame in the second image data is synchronized with a current frame in the first image data, stopping sending the first photography request to the default camera, and the multi-camera device is used to display an image according to the second image data; The image data that has not been displayed in the first image data is recycled, and the default camera enters a stop working state.

2. The photography method of the multi-camera device according to claim 1, characterized in that: When the current frame in the second image data is synchronized with the current frame in the first image data, the method comprises: Acquire the acquisition time of the first image data and the acquisition time of the second image data; The display effects of the current frame in the second image data are aligned with those of the current frame in the first image data according to the image frame number in the second photography request, the acquisition time of the first image data, the acquisition time of the second image data, the image frame number of the image data that has not yet been displayed in the first image data, and the statistical information of the display effects of the second image data and the first image data that change over time.

3. The photography method of the multi-camera device according to claim 1, characterized in that: The frame matching of the second image data with image data in the first image data that has not been displayed comprises: Based on the second photography request and the image data that has not yet been displayed in the first image data, determine the type of request frame to be reissued to the designated camera and the number of request frames, so that the second image data and the image data that has not yet been displayed in the first image data maintain the same number of frames, and the type of the request frame is consistent with the first photography request.

4. The photography method of the multi-camera device according to claim 2, characterized in that: The display effect includes at least one of automatic exposure, automatic focus and automatic white balance.

5. The photography method of the multi-camera device according to claim 1, characterized in that: After stopping sending the first photography request to the default camera, the method further includes: continuing to align the request queue of the second photography request with the request queue of the user application request.

6. The photography method of the multi-camera device according to claim 1, characterized in that: The default camera and the designated camera have different focal lengths. After the second photography request is sent to the designated camera, the method includes: Obtaining calibration data of the designated camera on the calibration plane and basic parameters of the corresponding image sensor, wherein the calibration data includes: a zoom factor critical value, a horizontal error and a vertical error relative to the default camera on the calibration plane, a calibration plane width, and a calibration plane height, and the basic parameters of the image sensor include an image sensor width and an image sensor height; According to the calibration data of the designated camera on the calibration plane and the basic parameters of the image sensors respectively corresponding to the designated cameras, and taking the default camera as a reference, correcting the center distance offset of the designated camera relative to the default camera; Correcting the zoom magnification critical value of the designated camera according to the correction result of the center distance offset of the designated camera relative to the default camera and the basic parameters of the image sensor of the designated camera; The focal length corresponding to the designated camera is corrected according to the corrected zoom ratio critical value to obtain the corrected focal length of the designated camera.

7. The photography method of the multi-camera device according to claim 6, characterized in that: Also includes: When N is greater than or equal to 3, and when the calibrated focal length of the designated camera does not meet the set requirements, stop sending the second photography request to the designated camera, and send the third photography request to other cameras, so that the other cameras report the third image data according to the received third photography request. The other cameras are any one of the cameras in the multi-camera device except the default camera and the designated camera, and the third photography request includes any one of a preview request, a photo request and a video recording request.

8. A photographing device of a multi-camera device, the multi-camera device comprising N cameras, wherein N is a positive integer greater than or equal to 2, characterized in that: The method for executing the photography method of the multi-camera device according to any one of claims 1 to 7, wherein the N cameras include a default camera, and any one of the N cameras except the default camera can be used as a designated camera in the camera switching process. Includes: default photography module, switching module, matching module, termination module and recycling module; The default photography module is used to send a first photography request to the default camera according to a user application request, so that the default camera reports first image data according to the first photography request, and the multi-camera device is used to display an image according to the first image data, wherein the first photography request includes any one of a preview request, a photo request, and a video recording request; The switching module is used to send a second photography request to the designated camera when receiving the camera switching instruction, so that the designated camera reports the second image data according to the second photography request, wherein the second photography request includes any one of a preview request, a photo request and a video recording request; The matching module is used to perform frame matching between the second image data and image data that has not been displayed in the first image data, and when a current frame in the second image data is synchronized with a current frame in the first image data, the termination module controls the default photography module to stop sending the first photography request to the default camera, and the multi-camera device is used to display an image according to the second image data; The recycling module is used to recycle the image data that has not been displayed in the first image data, and is also used to control the default camera to enter a stop working state.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the photography method of a multi-camera device as claimed in any one of claims 1 to 7 are executed.

10. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the photography method of the multi-camera device according to any one of claims 1 to 7 are executed.

Images