Photosensitive sensor rollback method and device, electronic equipment and medium

Abstract

The invention discloses a photosensitive sensor rollback method and device, electronic equipment and a medium, and belongs to the technical field of camera shooting. According to the method, when a first photosensitive sensor participating in display needs to be returned, N candidate photosensitive sensors capable of imaging clearly are selected from M second photosensitive sensors, N is a positive integer larger than or equal to 2, M is a positive integer larger than or equal to N, the imaging offset degree between the N candidate photosensitive sensors and the first photosensitive sensor is calculated, and the imaging offset degree between the N candidate photosensitive sensors and the first photosensitive sensor is calculated. Based on the imaging offset degree between the N candidate photosensitive sensors and the first photosensitive sensor, selecting a candidate photosensitive sensor of which the imaging offset degree meets a preset imaging offset condition from the N candidate photosensitive sensors as a target photosensitive sensor, and switching the photosensitive sensor participating in display from the first photosensitive sensor to the target photosensitive sensor.

Description

Technical Field

[0001] The present application belongs to the field of camera technology, and specifically relates to a method, device, electronic device and medium for retracting a photosensitive sensor. Background Art

[0002] With the development of camera technology, in order to meet higher shooting needs, an electronic device is usually equipped with multiple photosensitive sensors. For example, a mobile phone can be equipped with multiple cameras. In actual applications, users often move electronic devices to shoot. At this time, the photosensitive sensor involved in the display cannot form a clear image, and then the photosensitive sensor needs to be rolled back, that is, switched to a photosensitive sensor that can form a clear image to participate in the shooting and display. However, during the photosensitive sensor rollback switching process, the image jump problem occurs, and the user experience is poor.

[0003] Therefore, how to retract the photosensor to improve image jumps has become a problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a photosensitive sensor fallback method, device, electronic device and medium, which can solve the problem of how to improve image jumps.

[0005] In a first aspect, an embodiment of the present application provides a method for retracting a photosensitive sensor, the method comprising:

[0006] When the first photosensor involved in display needs to be retracted, N candidate photosensors capable of clear imaging are selected from the M second photosensors, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N;

[0007] Calculating the imaging offset between the N candidate photosensors and the first photosensor;

[0008] Based on the imaging offset between the N candidate photosensors and the first photosensor, selecting a candidate photosensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensors as a target photosensor;

[0009] The photosensor participating in the display is switched from the first photosensor to the target photosensor.

[0010] In a second aspect, an embodiment of the present application provides a device, including:

[0011] A candidate selection module, configured to select N candidate photosensors capable of clear imaging from the M second photosensors when the first photosensor involved in display needs to be retracted, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N;

[0012] An offset calculation module, used to calculate the imaging offset between the N candidate photosensors and the first photosensor;

[0013] a target selection module, configured to select, based on an imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition from among the N candidate photosensitive sensors as a target photosensitive sensor;

[0014] A switching module is used to switch the photosensor involved in display from the first photosensor to the target photosensor.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0018] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0019] In an embodiment of the present application, when the first photosensitive sensor participating in the display needs to be retracted, N candidate photosensitive sensors that can clearly image are selected from the M second photosensitive sensors, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. Since the embodiment of the present application will further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor. The target photosensitive sensor with improved image jump situation can be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the retraction process of the photosensitive sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of an imaging viewing angle provided in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of an application scenario of a photosensitive sensor retreat method provided in an embodiment of the present application;

[0022] Figure 3 is a flow chart of a method for retracting a photosensor provided in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of imaging offset calculation provided by an embodiment of the present application;

[0024] Figure 5 is a flow chart of a method for retracting a photosensor provided by another embodiment of the present application;

[0025] Figure 6 A flowchart of a method for retracting a photosensor provided by another embodiment of the present application;

[0026] Figure 7 It is a structural schematic diagram of a photosensitive sensor retraction device provided in an embodiment of the present application;

[0027] Figure 8 is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0028] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0030] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] First, the terms involved in the embodiments of the present application are explained.

[0032] The photosensitive sensor is a key component in an imaging device (such as a camera), responsible for converting the received light signals into electrical signals and then generating digital images.

[0033] Focal length refers to the distance from the optical center of the lens (also known as the optical rear principal point) to the focus of the imaging surface. It can also be understood as the distance between the position where the light converges to form a focus after passing through the lens and the optical center of the lens.

[0034] Object distance refers to the distance from the subject to the front of the lens.

[0035] Fallback refers to the process of switching the light sensor of one camera device to the light sensor of another camera device in a system with multiple camera devices in order to adapt to different shooting requirements or scene changes.

[0036] Imaging shift refers to the phenomenon that the image shifts relative to the expected position due to the movement of the lens, sensor or subject. The imaging shift degree is a measurement value used to characterize the degree of imaging shift.

[0037] As mentioned in the background technology, in actual applications, users often move electronic devices to take pictures. At this time, the photosensitive sensors involved in the display cannot form clear images, and the photosensitive sensors need to be retracted, that is, switched to photosensitive sensors that can form clear images to participate in the shooting and display. At present, the photosensitive sensor retraction is usually selected based on the relationship between the object distance of the subject and the focal length of the photosensitive sensor. It only considers whether the switched photosensitive sensor can form a clear image, but does not consider whether the switching process will cause image jumps. In this regard, the inventors of the present application have found that due to the different placement positions of the camera devices, there will be certain differences in the viewing angles of the imaging of each photosensitive sensor. For example, Figure 1 As shown in the imaging perspective diagram, points I, II, and III are the imaging center points of the three photosensitive sensors. Among them, the photosensitive sensor with a short focal length has a larger imaging perspective, but due to the difference in the placement of the camera device, the long focal length photosensitive sensor cannot completely cover the imaging content of the short focal length photosensitive sensor, and the imaging centers of the two photosensitive sensors are deviated, which easily causes image jumps during the photosensitive sensor back-off switching process.

[0038] In view of this, the embodiments of the present application provide a photosensitive sensor fallback method, device, electronic device and medium. When the first photosensitive sensor participating in the display needs to fall back, N candidate photosensitive sensors that can clearly image are selected from M second photosensitive sensors, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. Since the embodiments of the present application will further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is strongly related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor. The target photosensitive sensor with improved image jump situation can be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the photosensitive sensor fallback process.

[0039] In the following, in conjunction with the accompanying drawings, the photosensor retraction method, device, electronic device and medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0040] Figure 2 Schematic diagram of an application scenario of a method for retracting a photosensitive sensor provided in an embodiment of the present application. Figure 2As shown, the application scenario may include: an electronic device 20, wherein the electronic device 20 is configured with multiple camera devices (such as cameras), and each camera device includes one or more photosensitive sensors. Based on this, it can be understood that the electronic device 20 is configured with multiple photosensitive sensors. The multiple photosensitive sensors include at least one first photosensitive sensor and M second photosensitive sensors, wherein M is a positive integer greater than or equal to 2. In addition, the electronic device 20 may also include a camera controller. Exemplarily, the camera controller is used to manage and control various components of the camera, including photosensitive sensors, lenses, image processors, etc., and is also used to control switching between different photosensitive sensors, and set and adjust the parameters of the photosensitive sensors. The electronic device 20 may include: a mobile device or a non-mobile device, such as a smart phone, a tablet computer, a smart camera, a drone, a smart watch, a security camera, an industrial camera, a medical imaging device, a drone-mounted camera, and any one or more devices. The electronic device is equipped with multiple camera devices (such as cameras).

[0041] based on Figure 2 The electronic device 20 shown includes multiple cameras. In order to adapt to movement, different shooting requirements or scene changes, it is necessary to switch the photosensor from one camera to the photosensor of another camera, that is, it is necessary to perform photosensor rollback. In order to effectively solve the image jump problem during the photosensitive sensor retreat process, according to the method provided in the embodiment of the present application, when the first photosensitive sensor participating in the display needs to retreat, the electronic device 20 selects N candidate photosensitive sensors that can clearly image from M second photosensitive sensors, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. Since the embodiment of the present application will further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor, and the target photosensitive sensor with improved image jump situation can be selected, so that the photosensitive sensor participating in the display can be switched from the first photosensitive sensor to the target photosensitive sensor by sending a control signal to the camera device controller, which can effectively improve the image jump problem during the photosensitive sensor retreat process.

[0042] It is worth noting that in the embodiment of the present application, the photosensor retraction method provided in the embodiment of the present application can be implemented when the processor of the electronic device 20 executes a program or instruction to determine the target photosensor, and the control signal is sent to the camera device controller, and the camera device controller controls the photosensor retraction according to the control signal. However, in some embodiments, the camera device controller may also have similar functions. For example: when the camera device controller executes a program or instruction, the photosensor retraction method provided in the embodiment of the present application is implemented to determine the target photosensor, and then control the photosensor retraction. The embodiment of the present application does not limit this.

[0043] It should be noted that the application scenarios described in the above embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. A person of ordinary skill in the art will know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. The photosensitive sensor retraction method provided in the embodiments of the present application can be applied to application scenarios where the photosensitive sensor needs to be retracted during shooting.

[0044] According to the above application scenarios, the following Figure 3 , a method for retracting a light-sensitive sensor provided in an embodiment of the present application is exemplarily described. The execution subject of the light-sensitive sensor retracting method may be an electronic device. It should be noted that the execution subject does not constitute a limitation on the present application.

[0045] Figure 3 FIG. 1 is a flow chart of a method for retracting a photosensitive sensor according to an exemplary embodiment of the present application. Figure 3 As shown, the photosensor retreat method may include the following steps:

[0046] Step 310: When the first photosensor involved in display needs to be retracted, N candidate photosensors capable of clear imaging are selected from the M second photosensors, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N.

[0047] The first photosensor may be any photosensor in the electronic device of the multi-camera device. In the electronic device of the multi-camera device, there is a photosensor that is being used and providing image data. The photosensor is the photosensor participating in the display. It can also be understood that the image data of the photosensor is being displayed to the user or processed by the system. In this step, the photosensor participating in the display is the first photosensor.

[0048] The second photosensitive sensor and the first photosensitive sensor belong to the same electronic device and are other photosensitive sensors in the electronic device except the first photosensitive sensor. In this electronic device, the first photosensitive sensor participating in the display is being used and providing image data, while the second photosensitive sensor may be in a standby state, a sleep state, a background processing state, an auxiliary state, a fault state, or other states. In the embodiments of the present application, when the first photosensitive sensor is participating in the display, the state of the second photosensitive sensor is not limited, and the state of the second photosensitive sensor may depend on the design of the electronic device system and the current requirements, as long as the efficient operation of the electronic device system and the user requirements are satisfied.

[0049] The candidate photosensitive sensor is a photosensitive sensor selected from M second photosensitive sensors. The candidate photosensitive sensor is used as a candidate for the target photosensitive sensor for fallback switching.

[0050] Exemplarily, step 310 may include:

[0051] Based on the object distance of the target object to be photographed and the focal lengths of the M second photosensitive sensors, select, from the M second photosensitive sensors, the second photosensitive sensors with focal lengths less than the object distance as candidate photosensitive sensors, and the number of candidate photosensitive sensors may be N.

[0052] The target object to be photographed refers to the key object that needs to be captured in photography or image processing and is usually located at the focus of the image frame. The target object to be photographed may be a person, an environment, or an object.

[0053] According to the Gaussian imaging theorem, when the object distance is less than the focal length of the lens, a clear image cannot be obtained on the photosensitive sensor. Therefore, when the user uses a photosensitive sensor with a long focal length to photograph a nearby object, it will cause the image to be out of focus, and at this time, a fallback of the photosensitive sensor is required. The primary principle of the fallback is to ensure that the photosensitive sensor after the fallback switching can form a clear image. Being able to form a clear image means that the photosensitive sensor meets the clear imaging conditions. Assume that the focal length of the photosensitive sensor participating in the display is P, the distance between the target object to be photographed and the lens is P1 (P < P1), and the focal lengths of the M second photosensitive sensors are N2…Nm+1 respectively. Among the second photosensitive sensors corresponding to N2…Nm+1, the second photosensitive sensors with their corresponding focal lengths P less than the object distance P1 can form a clear image, and the clear imaging condition is also that the focal length is less than the object distance. Thus, among the M second photosensitive sensors, N second photosensitive sensors with focal lengths P less than the object distance P1 are marked as candidate photosensitive sensors.

[0054] It can be understood that after selecting the candidate photosensors that can form clear images, the embodiment of the present application further selects the target photosensor from the candidate photosensors based on the imaging offset. Therefore, the N candidate photosensors selected in step 310 should be at least 2 in order to continue the selection. Therefore, N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N.

[0055] If only one candidate photosensor that can produce clear images is selected from M photosensors, subsequent processing steps can be set according to actual application needs. For example, a screened candidate photosensor can be directly switched as the target photosensor, or a prompt of an image jump can be given to the user so that the user can adjust the photography operation, or the imaging offset and / or other parameters corresponding to the candidate photosensor can be continued to be calculated so as to adjust the candidate photosensor so that the adjusted photosensor can be switched as the target photosensor. The present application does not limit this.

[0056] Exemplarily, it is assumed that the electronic device is a mobile phone, and the mobile phone has m cameras, each camera has a photosensitive sensor, for example: the mobile phone has photosensitive sensor 1, photosensitive sensor 2... and photosensitive sensor m. Assuming that the first photosensitive sensor involved in the display is photosensitive sensor 1, because the user moves the mobile phone, the object distance of the target subject is less than the focal length of photosensitive sensor 1, and the target subject cannot be clearly displayed. At this time, photosensitive sensor 1 needs to be retreated. According to step 310, at least two photosensitive sensors that can clearly form images can be selected from photosensitive sensors 2-m based on the comparison of object distance and focal length as candidate photosensitive sensors.

[0057] Step 320: Calculate the imaging offset between the N candidate photosensors and the first photosensor.

[0058] Through the screening in step 310, N candidate photosensors capable of clear imaging will be obtained. In order to improve the smoothness of the fallback switching process and solve the problem of image mutation during the fallback switching process in the prior art, the step 320 will further calculate the imaging offset between the N candidate photosensors and the first photosensor.

[0059] Imaging offset is a measurement used to characterize the degree of imaging offset. Imaging offset can be measured by a variety of measurement values, including pixel offset, reference point (such as corner point) distance, etc. Different measurement values ​​have their own advantages, and the appropriate measurement method can be selected according to the specific application scenario.

[0060] It should be noted that in the embodiments of the present application, the algorithms and measurement values ​​used by each of the N candidate photosensors to calculate the imaging offset may be the same or different. For example, in some embodiments, one or more candidate photosensors measure the imaging offset by the distance of reference points at the same position in different images, and another one or more candidate photosensors measure the imaging offset by the pixel offset of different images. For another example, in some embodiments, the N candidate photosensors all measure the imaging offset by the distance of reference points at the same position in different images.

[0061] The following is an exemplary description of the embodiment of the present application by measuring the imaging offset by the distance of the reference point at the same position of different images. Specifically, the imaging offset between the N candidate photosensors and the first photosensor is calculated, including:

[0062] Performing field of view angle alignment and cropping on an image captured by any of the candidate photosensors to obtain a first image;

[0063] Performing field-of-view angle alignment and cropping on the image captured by the first photosensor to obtain a second image;

[0064] For any of the candidate photosensors, calculate the distance between the first reference point of the first image captured by any of the candidate photosensors and the second reference point of the second image captured by the first photosensor to obtain the imaging offset between any of the candidate photosensors and the first photosensor, the first reference point being a point at a first specified position in the first image, and the second reference point being a point at the first specified position in the second image.

[0065] The first reference point and the second reference point are points at the same designated position in the respective images, and are used to measure the imaging offset between the two images. For example, the first reference point and the second reference point can be points that are easy to calculate, such as the upper left corner point, lower left corner point, upper right corner point, and lower right corner point of the first image and the second image.

[0066] For example: Figure 4 As shown in the imaging offset calculation diagram, point A is the upper left corner point of image 1 with point O as the imaging center, and point C is the upper left corner point of another image 2 with point B as the imaging center. The second reference point in image 1 is point A, and the first reference point in image 2 is point C. The imaging offset between image 1 and image 2 can be measured by the distance between point A and point C.

[0067] The first image is an image captured by the candidate photosensor and cropped after the field of view is aligned. The second image is an image captured by the first photosensor and cropped after the field of view is aligned. Figure 3As shown, the first image may be an image 2 with point B as the imaging center, and the second image may be another image 1 with point O as the imaging center.

[0068] Field of view angle alignment and cropping refers to the process of adjusting and cropping images captured by different photosensors in a multi-camera system so that their field of view angles are aligned and cropped to ensure that the images have consistent viewing angles and positions when they are stitched or fused.

[0069] Using the photosensor 1-m of the previous example, combined with Figure 4 As shown in the calculation diagram, assuming that the N candidate photosensors that are selected to produce clear images are photosensors 2 to m, the imaging offset of photosensor 2 can be calculated by the following steps:

[0070] The distance between the upper left corner point C of the image 2 of the photosensitive sensor 2 after the field of view angle is aligned and cropped and the upper left corner point A of the image 1 of the photosensitive sensor 1 after the field of view angle is aligned and cropped is calculated to obtain the imaging deviation degree of the photosensitive sensor 2.

[0071] The calculation of the imaging offset of other candidate photosensors is similar to the calculation of the imaging offset of the photosensor 2 described above, and will not be repeated here.

[0072] In the above embodiment, the image captured by the first photosensitive sensor and the image captured by any candidate photosensitive sensor are firstly cropped after the field of view angle is aligned, and then the distance between the first reference point and the second reference point at the same position of the two images is calculated. This enables the imaging offset of the images of the two photosensitive sensors to be measured when the field of view angle is aligned and cropped, thereby ensuring that the calculated imaging offset is the offset between two images with relatively consistent viewing angles and positions when spliced ​​or fused, and the measurement is more precise, so as to more accurately select the target photosensitive sensor whose imaging offset meets the conditions.

[0073] In order to improve the calculation of the imaging offset of each candidate photosensitive sensor, in some embodiments of the present application, a coordinate system is established by taking the imaging center of the first photosensitive sensor as the coordinate origin to calculate the coordinates of the reference point. Specifically, for any of the candidate photosensitive sensors, the distance between the first reference point of the first image captured by any of the candidate photosensitive sensors and the second reference point of the second image captured by the first photosensitive sensor is calculated to obtain the imaging offset between any of the candidate photosensitive sensors and the first photosensitive sensor, including:

[0074] For any of the candidate photosensors, calculate the first coordinates of a first reference point of a first image captured by any of the candidate photosensors, wherein the first coordinates are coordinates in a coordinate system established based on an imaging center of the first photosensor as a coordinate origin;

[0075] Determine a second coordinate of a second reference point of a second image captured by the first light-sensitive sensor, wherein the second coordinate is a coordinate in the coordinate system;

[0076] The distance between the first coordinate and the second coordinate is calculated to obtain the imaging offset corresponding to any of the candidate fall-back photosensors.

[0077] Following the previous example, the candidate photosensors selected are photosensor 2 to photosensor m. According to this embodiment, the imaging offset of photosensor 2 can be calculated by the following steps:

[0078] Calculate the coordinates of the upper left corner point C of the image 2 of the photosensitive sensor 2 after the field of view angle alignment and cropping, where the coordinates of point C are the coordinates of a coordinate system established with the imaging center point O of the photosensitive sensor 1 as the coordinate origin;

[0079] Calculate the coordinates of the upper left corner point A of the image 1 of the photosensitive sensor 1 after the field of view angle alignment and cropping, where the coordinates of point A are the coordinates in the above coordinate system;

[0080] The distance between point A and point C is calculated to obtain the imaging deviation of the photosensitive sensor 2 .

[0081] The imaging offset of the photosensitive sensor m can be calculated by the following steps:

[0082] Calculate the upper left corner point of the image 3 after the field of view angle alignment and cropping of the photosensitive sensor m (assuming it is point E, in order to more clearly and briefly illustrate the calculation principle, Figure 3 The coordinates of point E) are not shown above, and the coordinates of point E are the coordinates in the above-mentioned coordinate system;

[0083] Directly obtain the coordinates of point A calculated previously;

[0084] The distance between point A and point E is calculated to obtain the imaging offset of the photosensitive sensor m.

[0085] It can be seen that since this embodiment establishes a coordinate system based on the imaging center of the first photosensitive sensor as the coordinate origin, the calculation of the imaging offset of multiple candidate photosensitive sensors only requires the calculation of the coordinates of the reference point in the image of the first photosensitive sensor once. Compared with the method of establishing a coordinate system based on the imaging center of the candidate photosensitive sensor, it avoids repeated calculations caused by different imaging centers of different candidate photosensitive sensors, thereby improving the calculation efficiency of the imaging offset of multiple candidate photosensitive sensors.

[0086] Furthermore, in some embodiments of the present application, calculating, for any of the candidate photosensors, a first coordinate of a first reference point of a first image captured by any of the candidate photosensors includes:

[0087] Calculate the first coordinates of the first reference point of the first image captured by any of the candidate photosensors based on the resolution, current zoom factor and base factor of any of the candidate photosensors;

[0088] The determining the second coordinates of the second reference point of the second image captured by the first photosensor includes:

[0089] Based on the resolution of the first photosensitive sensor, the current zoom magnification and the basic magnification, second coordinates of a second reference point of a second image captured by the first photosensitive sensor are calculated.

[0090] Resolution refers to the number of pixels in an image, usually expressed as the number of pixels in width and height. Current zoom ratio refers to the current zoom in or zoom out ratio, usually used to adjust the field of view. Base ratio refers to the default zoom ratio of the photosensitive sensor when it is not zoomed.

[0091] Following the previous example, Figure 3 The imaging offset calculation diagram shown in the figure establishes a two-dimensional coordinate system with the imaging center O of the first photosensor 1 participating in the display as the coordinate origin. The offsets of the imaging centers B of the candidate photosensor 2 to the candidate photosensor m from the imaging center O of the first photosensor 1 are (x2, y2)…(xm, ym) respectively. Points A and C are the coordinates of the upper left corners of the two images after the field of view is aligned and cropped based on the current zoom ratio. The calculation process of the coordinates of point A and point C is as follows:

[0092] Among them, the coordinates of A with O as the origin are: It is abbreviated as (xa, ya). The coordinates of point B are (xm, ym).

[0093] The coordinates of point C with point B as the origin are Abbreviated as (xc, yc).

[0094] Since the resolution of the photosensitive sensor, the current zoom ratio and the base ratio are key parameters that affect the geometric transformation and scaling of the image, this embodiment uses them to calculate the coordinates of the midpoint of the image after the field of view angle is aligned and cropped. The calculation is simple and accurate, and can effectively improve the calculation efficiency.

[0095] In some embodiments of the present application, calculating the distance between the first coordinate and the second coordinate includes:

[0096] Based on the first coordinate, the second coordinate and the coordinate of the coordinate origin (such as in combination with the law of cosines), the distance between the first coordinate and the second coordinate is calculated.

[0097] Following the previous example, Figure 3The imaging offset calculation diagram shown in FIG. 1 shows that the imaging offset D of any candidate photosensor m is Figure 3 The length of AC in is calculated as follows:

[0098] The length of AC can be calculated using the law of cosines:

[0099]

[0100] The AO length is:

[0101]

[0102] The length of CO is:

[0103]

[0104] ∠AOC is:

[0105]

[0106] This embodiment calculates the distance between the first coordinate and the second coordinate based on the triangle formed by the first reference point, the second reference point and the coordinate origin, and the calculation is simple and efficient.

[0107] Step 330: Based on the imaging offset between the N candidate photosensors and the first photosensor, select a candidate photosensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensors as a target photosensor.

[0108] The preset imaging offset condition is used to constrain the imaging offset degree of the candidate photosensitive sensor finally selected, so that the selected candidate photosensitive sensor meets the image jump improvement requirements. The condition content of the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications, and this application does not limit this.

[0109] In one embodiment, based on the imaging offset between the N candidate photosensors and the first photosensor, selecting a candidate photosensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensors as a target photosensor includes:

[0110] Based on the imaging offset between the N candidate photosensors and the first photosensor, a candidate photosensor with the smallest imaging offset is selected from the N candidate photosensors as a target photosensor.

[0111] For example, N candidate photosensors may be sorted according to the magnitude of the imaging offset, and the photosensor with the smallest imaging offset may be selected as the target photosensor through the sorting.

[0112] In this embodiment, the preset imaging offset condition is that the imaging offset is the smallest. By selecting the candidate photosensor with the smallest imaging offset as the target photosensor, the image jump during the fallback switching process of the photosensor can be effectively reduced.

[0113] It is understandable that in practical applications, in addition to improving the image jump problem by selecting a candidate photosensor with the smallest imaging offset, selecting a candidate photosensor with a relatively small imaging offset can also achieve a certain improvement effect on the image jump.

[0114] Step 340: Switch the photosensor involved in display from the first photosensor to the target photosensor.

[0115] For example, it can be Figure 2 When the processor of the electronic device 20 shown executes a program or instruction, it implements the photosensor fallback method provided in the embodiment of the present application, and sends a control signal to the camera device controller of the electronic device 10. The camera device controller controls the photosensor participating in the display to switch from the first photosensor to the target photosensor according to the control signal.

[0116] In an embodiment of the present application, when the first photosensitive sensor participating in the display needs to be retracted, N candidate photosensitive sensors that can clearly image are selected from the M second photosensitive sensors, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. Since the embodiment of the present application will further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor. The target photosensitive sensor with improved image jump situation can be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the retraction process of the photosensitive sensor.

[0117] In addition, in some embodiments of the present application, in order to solve the image jump problem caused by only one frame transition in the prior art during the fallback process, the appropriate number of transition frames is further calculated, thereby providing users with better switching smoothness during the fallback process. Specifically, Figure 5 is a flow chart of a photosensitive sensor retreat method according to another exemplary embodiment of the present application. Figure 5 For the details of the implementation of steps 510-540 shown in FIG. Figure 3 The relevant description of the embodiment shown will not be repeated here. Figure 5 As shown, the photosensor fallback method, before switching the photosensor involved in display from the first photosensor to the target photosensor in step 340, further includes:

[0118] Step 3302: Based on a preset transition frame number calculation rule, the number of transition frames corresponding to the process of the first photosensor switching to the target photosensor is calculated, and the number of transition frames is greater than 1.

[0119] It should be noted that the preset transition frame number calculation rule is a rule for calculating the number of transition frames in the photosensitive sensor switching process with the goal of achieving smoothness of the image in the photosensitive sensor switching process, provided that the number of transition frames is greater than 1. The specific algorithm of the preset transition frame number calculation rule is not limited, as long as multiple frames are calculated and the image in the switching process is smooth.

[0120] For example, in some embodiments of the present application, the processing of step 3302 may include:

[0121] Based on the imaging offset corresponding to the target photosensor and a preset inter-frame pixel offset threshold, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated.

[0122] Among them, the preset inter-frame pixel offset threshold is used to constrain the threshold of pixel offset between one frame and another frame, such as, it can be the pixel offset threshold between adjacent frames, it can be the pixel offset threshold between interval frames, etc., and the present application does not limit this. By dividing the imaging offset corresponding to the target photosensor by the preset inter-frame pixel offset threshold, the number of transition frames with smooth image transition can be obtained. It can be understood that the smaller the threshold, the more transition frames there are, and the smoother the image transition. The larger the threshold, the fewer transition frames there are, and the smoothness of the image transition is relatively reduced. Therefore, the value of the threshold can be set according to the need to improve the smoothness of image transition in actual applications. For example: the size of the preset inter-frame pixel offset threshold can be determined based on usage experience, and can be determined based on the statistical results of user feedback on image smoothness under different thresholds.

[0123] For example, assuming that the imaging offset corresponding to the target photosensor is 0.5 pixels and the preset inter-frame pixel offset threshold is 0.1 pixels, the number of transition frames is 0.5 divided by 0.1, which is 5 frames.

[0124] In order to solve the problem that excessive inter-frame offset may cause image jumps, this embodiment calculates the number of transition frames by presetting an inter-frame pixel offset threshold, thereby effectively controlling the pixel offset of adjacent frames and making the image transition smooth.

[0125] For another example, in some embodiments of the present application, the processing of step 3302 may include:

[0126] Based on the frame rate of the first photosensor and the frame rate of the target photosensor, a reference frame rate is determined, and based on the reference frame rate and a preset switching duration threshold, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated.

[0127] Among them, the frame rate is the number of image frames that the photosensitive sensor can capture and process per second. The frame rates of different photosensitive sensors can be the same or different. Therefore, the reference frame rate can be determined based on the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor. The reference frame rate can be the frame rate of the first photosensitive sensor, the frame rate of the target photosensitive sensor, or a value between the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor. The preset switching time threshold is the threshold of the duration of the photosensitive sensor switching process. By dividing the preset switching time threshold by the reference frame rate, the number of transition frames with smooth image transition can be obtained. It can be understood that if the switching time is too long, the user will wait too long, and if the switching time is too short, it will cause image mutations. Both situations will lead to poor user experience. Therefore, the value of the preset switching time threshold can be set according to the need to improve the smoothness of image transition in actual applications. For example: the size of the preset switching time can be determined based on usage experience, and can be determined based on the statistical results of user feedback on image smoothness under different thresholds. In this embodiment, the preset switching duration threshold may be divided by the reference frame rate to obtain the number of transition frames.

[0128] Exemplarily, assuming that the reference frame rate is 30 frames per second and the preset inter-frame pixel offset threshold is 0.1 seconds, the number of transition frames is 0.1 seconds divided by 30 frames per second, which is 3 frames.

[0129] This embodiment aims to solve the problem that a switching time that is too short will cause image jumps, and a switching time that is too long will cause waiting. By calculating the number of transition frames through a preset switching time threshold, the switching time and the number of transition frames can be effectively controlled to make the image transition smooth.

[0130] After calculating the number of transition frames, in step 340, the photosensor involved in the display may be switched from the first photosensor to the target photosensor based on the calculated number of transition frames, so that the image transition is smooth and the user experience is better.

[0131] Below, the processing process of a light sensor fallback method provided in an embodiment of the present application is exemplarily described by taking the application of the light sensor fallback method provided in an embodiment of the present application to a mobile terminal such as a mobile phone. The mobile terminal may include multiple cameras, each with a light sensor. Figure 6FIG. 1 is a flow chart of a method for retracting a photosensitive sensor according to another exemplary embodiment of the present application. Figure 6 As shown, the photosensor retreat method may include the following steps:

[0132] Step 610: The mobile terminal detects the object distance of the target subject currently being photographed.

[0133] Step 620: Determine whether the object distance is less than the focal length of the current main camera of the mobile terminal.

[0134] Step 630: If not, determine not to perform photosensor rollback.

[0135] Step 640: If yes, it is determined that the photosensor needs to be rolled back, and multiple candidate photosensors that can form clear images are selected from multiple cameras on the mobile terminal based on the focal length.

[0136] Step 650: Calculate the imaging offsets of multiple candidate photosensors, and select the candidate photosensor with the smallest imaging offset as the target photosensor.

[0137] Step 660: Calculate the number of transition frames based on the imaging offset of the target photosensor and a preset pixel threshold between adjacent frames.

[0138] Step 670: During the switching process, a transition frame corresponding to the number of transition frames is used to switch the current main camera of the mobile terminal to the camera to which the target photosensor belongs.

[0139] Parts of the specific implementation of the above steps 610 to 670 that are not described in detail can be referred to the aforementioned Figure 3 The relevant descriptions of steps 310 to 340 of the illustrated embodiment are not repeated here.

[0140] In this embodiment, candidate photosensors that can form clear images are first selected by comparing the object distance and the focal length of each photosensor, and then the imaging offset between the candidate photosensors and the main camera photosensor involved in the display is calculated to determine the target photosensor with the smallest imaging offset. Then, the main camera is switched using transition frames corresponding to the number of transition frames calculated based on a preset adjacent frame minus pixel threshold, thereby providing the user with a switching process with the best image smoothness experience during the main camera switching process.

[0141] The photosensitive sensor retraction method provided in the embodiment of the present application can be executed by a photosensitive sensor retraction device. In the embodiment of the present application, the photosensitive sensor retraction device executing the photosensitive sensor retraction method is taken as an example to illustrate the photosensitive sensor retraction device provided in the embodiment of the present application.

[0142] Figure 7FIG. 1 is a schematic diagram showing the structure of a photosensitive sensor retraction device provided in an embodiment of the present application. Figure 7 As shown, the photosensitive sensor retraction device 700 provided in the embodiment of the present application may include:

[0143] The candidate selection module 710 is used to select N candidate photosensors that can clearly image from M second photosensors when the first photosensor involved in the display needs to be retreated, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N.

[0144] The offset calculation module 720 is used to calculate the imaging offset between the N candidate photosensors and the first photosensor.

[0145] The target selection module 730 is used to select a candidate photosensitive sensor whose imaging offset meets a preset imaging offset condition from the N candidate photosensitive sensors as a target photosensitive sensor based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor.

[0146] The switching module 740 is used to switch the photosensor involved in display from the first photosensor to the target photosensor.

[0147] When the first photosensor participating in the display needs to be retired, the photosensitive sensor retirement device 700 selects N candidate photosensors that can clearly image from the M second photosensors, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. Since the embodiment of the present application will further calculate the imaging offset between the N candidate photosensors and the first photosensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensors and the first photosensor, a candidate photosensor whose imaging offset meets the preset imaging offset condition is selected from the N candidate photosensors as the target photosensor, and the target photosensor with improved image jump situation can be selected, thereby switching the photosensor participating in the display from the first photosensor to the target photosensor, which can effectively improve the image jump problem during the photosensitive sensor retirement process.

[0148] The above-mentioned light-sensitive sensor retracting device 700 is described in detail below, as shown below:

[0149] In some embodiments, the offset calculation module is used to perform field of view angle alignment and cropping on the image captured by any of the candidate photosensors to obtain a first image, perform field of view angle alignment and cropping on the image captured by the first photosensor to obtain a second image, and for any of the candidate photosensors, calculate the distance between a first reference point of the first image captured by any of the candidate photosensors and a second reference point of the second image captured by the first photosensor to obtain the imaging offset between any of the candidate photosensors and the first photosensor, the first reference point being a point at a first specified position in the first image, and the second reference point being a point at the first specified position in the second image.

[0150] In some embodiments, the offset calculation module includes:

[0151] A first coordinate calculation unit is used to calculate the first coordinates of the first reference point of the first image captured by any of the candidate photosensors, wherein the first coordinates are coordinates in a coordinate system established based on the imaging center of the first photosensor as the coordinate origin.

[0152] The second coordinate calculation unit is used to determine the second coordinates of the second reference point of the second image captured by the first photosensitive sensor, wherein the second coordinates are the coordinates in the coordinate system.

[0153] The distance calculation unit is used to calculate the distance between the first coordinate and the second coordinate to obtain the imaging offset corresponding to any of the candidate fallback photosensors.

[0154] In some of the embodiments, the first coordinate calculation unit is used to calculate the first coordinates of the first reference point of the first image captured by any of the candidate photosensors based on the resolution, current zoom factor and base factor of any of the candidate photosensors.

[0155] The second coordinate calculation unit is used to calculate the second coordinates of the second reference point of the second image captured by the first photosensitive sensor based on the resolution, current zoom factor and basic factor of the first photosensitive sensor.

[0156] In some embodiments, the distance calculation unit is used to calculate the distance between the first coordinate and the second coordinate based on the first coordinate, the second coordinate and the coordinate of the coordinate origin.

[0157] In some of the embodiments, the target selection module is used to select a candidate photosensitive sensor with the smallest imaging offset from the N candidate photosensitive sensors as a target photosensitive sensor based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor.

[0158] In some embodiments, the photosensor fallback device also includes: a transition frame calculation module, which is used to calculate the number of transition frames corresponding to the process of switching the first photosensor to the target photosensor based on a preset transition frame number calculation rule before switching the photosensor involved in the display from the first photosensor to the target photosensor, and the number of transition frames is greater than 1.

[0159] In some embodiments, the transition frame calculation module is used to calculate the number of transition frames corresponding to the process of switching from the first photosensitive sensor to the target photosensitive sensor based on the imaging offset corresponding to the target photosensitive sensor and a preset inter-frame pixel offset threshold.

[0160] In some embodiments, the transition frame calculation module is used to determine a reference frame rate based on the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor, and calculate the number of transition frames corresponding to the process of switching from the first photosensitive sensor to the target photosensitive sensor based on the reference frame rate and a preset switching duration threshold.

[0161] The photosensitive sensor retraction device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0162] The photosensitive sensor device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0163] The photosensitive sensor device provided in the embodiment of the present application can achieve Figures 2 to 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0164] Alternatively, if Figure 8 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned photosensitive sensor method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0165] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0166] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0167] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910 and other components.

[0168] Those skilled in the art will appreciate that the electronic device 900 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0169] The processor 910 is configured to:

[0170] When the first photosensor involved in display needs to be retracted, N candidate photosensors capable of clear imaging are selected from the M second photosensors, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N;

[0171] Calculating the imaging offset between the N candidate photosensors and the first photosensor;

[0172] Based on the imaging offset between the N candidate photosensors and the first photosensor, selecting a candidate photosensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensors as a target photosensor;

[0173] The photosensor participating in the display is switched from the first photosensor to the target photosensor.

[0174] In this embodiment, when the first photosensitive sensor participating in the display needs to be retracted, N candidate photosensitive sensors that can clearly image are selected from the M second photosensitive sensors, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. Since the embodiment of the present application will further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor, and the target photosensitive sensor with improved image jump situation can be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the retraction process of the photosensitive sensor.

[0175] In some embodiments, the processor 910 is further configured to:

[0176] Perform field of view angle alignment and cropping on the image captured by any of the candidate photosensors to obtain a first image, perform field of view angle alignment and cropping on the image captured by the first photosensor to obtain a second image, and for any of the candidate photosensors, calculate the distance between a first reference point of the first image captured by any of the candidate photosensors and a second reference point of the second image captured by the first photosensor to obtain an imaging offset between any of the candidate photosensors and the first photosensor, the first reference point being a point at a first specified position in the first image, and the second reference point being a point at the first specified position in the second image.

[0177] In the above embodiment, the image captured by the first photosensitive sensor and the image captured by any candidate photosensitive sensor are firstly cropped after the field of view angle is aligned, and then the distance between the first reference point and the second reference point at the same position of the two images is calculated. This enables the imaging offset of the images of the two photosensitive sensors to be measured when the field of view angle is aligned and cropped, thereby ensuring that the calculated imaging offset is the offset between two images with relatively consistent viewing angles and positions when spliced ​​or fused, and the measurement is more precise, so as to more accurately select the target photosensitive sensor whose imaging offset meets the conditions.

[0178] In some embodiments, the processor 910 is further configured to:

[0179] For any of the candidate photosensors, calculate the first coordinates of a first reference point of a first image captured by any of the candidate photosensors, wherein the first coordinates are coordinates in a coordinate system established based on an imaging center of the first photosensor as a coordinate origin;

[0180] Determine a second coordinate of a second reference point of a second image captured by the first light-sensitive sensor, wherein the second coordinate is a coordinate in the coordinate system;

[0181] The distance between the first coordinate and the second coordinate is calculated to obtain the imaging offset corresponding to any of the candidate fall-back photosensors.

[0182] Since this embodiment establishes a coordinate system based on the imaging center of the first photosensitive sensor as the coordinate origin, the calculation of the imaging offsets of multiple candidate photosensitive sensors only requires the calculation of the coordinates of the reference point in the image of the first photosensitive sensor once. Compared with the method of establishing a coordinate system based on the imaging center of the candidate photosensitive sensors, repeated calculations caused by different imaging centers of different candidate photosensitive sensors are avoided, thereby improving the calculation efficiency of the imaging offsets of multiple candidate photosensitive sensors.

[0183] In some embodiments, the processor 910 is further configured to:

[0184] Calculate the first coordinates of the first reference point of the first image captured by any of the candidate photosensors based on the resolution, current zoom factor and base factor of any of the candidate photosensors;

[0185] Based on the resolution of the first photosensitive sensor, the current zoom magnification and the basic magnification, second coordinates of a second reference point of a second image captured by the first photosensitive sensor are calculated.

[0186] Since the resolution of the photosensitive sensor, the current zoom ratio and the base ratio are key parameters that affect the geometric transformation and scaling of the image, this embodiment uses them to calculate the coordinates of the midpoint of the image after the field of view angle is aligned and cropped. The calculation is simple and accurate, and can effectively improve the calculation efficiency.

[0187] In some embodiments, the processor 910 is further configured to:

[0188] Based on the first coordinate, the second coordinate, and the coordinate of the coordinate origin, a distance between the first coordinate and the second coordinate is calculated.

[0189] This embodiment uses the first coordinate, the second coordinate and the coordinate of the coordinate origin to calculate the distance between the first coordinate and the second coordinate, and the calculation is simple and efficient.

[0190] In some embodiments, the processor 910 is further configured to:

[0191] Based on the imaging offset between the N candidate photosensors and the first photosensor, a candidate photosensor with the smallest imaging offset is selected from the N candidate photosensors as a target photosensor.

[0192] By selecting the candidate photosensor with the smallest imaging offset as the target photosensor, image jumps during the photosensor fallback switching process can be effectively reduced.

[0193] In some embodiments, the processor 910 is further configured to:

[0194] Before switching the photosensor involved in display from the first photosensor to the target photosensor, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated based on a preset transition frame number calculation rule, and the number of transition frames is greater than 1.

[0195] This embodiment aims at the image jump problem caused by only one frame transition during the fallback process in the prior art, and further calculates the appropriate number of transition frames, thereby providing users with better switching fluency during the fallback process.

[0196] In some embodiments, the processor 910 is further configured to:

[0197] Based on the imaging offset corresponding to the target photosensor and a preset inter-frame pixel offset threshold, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated.

[0198] In order to solve the problem that excessive inter-frame offset may cause image jumps, this embodiment calculates the number of transition frames by presetting an inter-frame pixel offset threshold, thereby effectively controlling the pixel offset of adjacent frames and making the image transition smooth.

[0199] In some embodiments, the processor 910 is further configured to:

[0200] Based on the frame rate of the first photosensor and the frame rate of the target photosensor, a reference frame rate is determined, and based on the reference frame rate and a preset switching duration threshold, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated.

[0201] This embodiment aims to solve the problem that a switching time that is too short will cause image jumps, and a switching time that is too long will cause waiting. By calculating the number of transition frames through a preset switching time threshold, the switching time and the number of transition frames can be effectively controlled to make the image transition smooth.

[0202] It should be understood that in the embodiment of the present application, the input unit 904 may include a graphics processor (Graphics Processing Unit, GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and at least one of other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0203] The memory 909 can be used to store software programs and various data. The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 909 may include a volatile memory or a non-volatile memory, or the memory 909 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 909 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0204] The processor 910 may include one or more processing units; optionally, the processor 910 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 910.

[0205] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned photosensitive sensor fallback method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0206] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0207] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned photosensitive sensor method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0208] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0209] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned photosensitive sensor backoff method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0210] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0211] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0212] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for retracting a photosensitive sensor, characterized in that: include: When the first photosensor involved in display needs to be retracted, N candidate photosensors capable of clear imaging are selected from the M second photosensors, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N; Calculating the imaging offset between the N candidate photosensors and the first photosensor; Based on the imaging offset between the N candidate photosensors and the first photosensor, selecting a candidate photosensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensors as a target photosensor; The photosensor participating in the display is switched from the first photosensor to the target photosensor.

2. The method according to claim 1, characterized in that The calculating the imaging offset between the N candidate photosensors and the first photosensor includes: Performing field of view angle alignment and cropping on an image captured by any of the candidate photosensors to obtain a first image; Performing field-of-view angle alignment and cropping on the image captured by the first photosensor to obtain a second image; For any of the candidate photosensors, calculate the distance between a first reference point of the first image captured by any of the candidate photosensors and a second reference point of the second image captured by the first photosensor to obtain an imaging offset between any of the candidate photosensors and the first photosensor, the first reference point being a point at a first specified position in the first image, and the second reference point being a point at the first specified position in the second image.

3. The method according to claim 2, characterized in that The step of calculating, for any of the candidate photosensors, a distance between a first reference point of a first image acquired by any of the candidate photosensors and a second reference point of a second image acquired by the first photosensor to obtain an imaging offset between any of the candidate photosensors and the first photosensor, includes: For any of the candidate photosensors, calculate the first coordinates of a first reference point of a first image captured by any of the candidate photosensors, wherein the first coordinates are coordinates in a coordinate system established based on an imaging center of the first photosensor as a coordinate origin; Determine a second coordinate of a second reference point of a second image captured by the first light-sensitive sensor, wherein the second coordinate is a coordinate in the coordinate system; The distance between the first coordinate and the second coordinate is calculated to obtain the imaging offset corresponding to any of the candidate fall-back photosensors.

4. The method according to claim 3, characterized in that The step of calculating, for any of the candidate photosensors, a first coordinate of a first reference point of a first image captured by any of the candidate photosensors comprises: Calculate the first coordinates of the first reference point of the first image captured by any of the candidate photosensors based on the resolution, current zoom factor and base factor of any of the candidate photosensors; The determining the second coordinates of the second reference point of the second image captured by the first photosensor includes: Based on the resolution of the first photosensitive sensor, the current zoom magnification and the basic magnification, second coordinates of a second reference point of a second image captured by the first photosensitive sensor are calculated.

5. The method according to claim 3, characterized in that: The calculating the distance between the first coordinate and the second coordinate comprises: Based on the first coordinate, the second coordinate, and the coordinate of the coordinate origin, a distance between the first coordinate and the second coordinate is calculated.

6. The method according to claim 1, characterized in that The step of selecting, based on the imaging offset between the N candidate photosensors and the first photosensor, a candidate photosensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensors as a target photosensor comprises: Based on the imaging offset between the N candidate photosensors and the first photosensor, a candidate photosensor with the smallest imaging offset is selected from the N candidate photosensors as a target photosensor.

7. The method according to any one of claims 1 to 6, characterized in that: Before switching the photosensor involved in display from the first photosensor to the target photosensor, the method further includes: Based on a preset transition frame number calculation rule, the number of transition frames corresponding to the process of switching the first photosensor to the target photosensor is calculated, and the number of transition frames is greater than 1.

8. The method according to claim 7, characterized in that The calculating, based on a preset transition frame number calculation rule, the number of transition frames corresponding to the process in which the first photosensor is switched to the target photosensor includes: Based on the imaging offset corresponding to the target photosensor and a preset inter-frame pixel offset threshold, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated.

9. The method according to claim 7, characterized in that: The calculating, based on a preset transition frame number calculation rule, the number of transition frames corresponding to the process in which the first photosensor is switched to the target photosensor includes: Based on the frame rate of the first photosensor and the frame rate of the target photosensor, a reference frame rate is determined, and based on the reference frame rate and a preset switching duration threshold, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated.

10. A photosensitive sensor retraction device, characterized in that: include: A candidate selection module, configured to select N candidate photosensors capable of clear imaging from the M second photosensors when the first photosensor involved in display needs to be retracted, wherein N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N; An offset calculation module, used to calculate the imaging offset between the N candidate photosensors and the first photosensor; a target selection module, configured to select, based on an imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition from among the N candidate photosensitive sensors as a target photosensitive sensor; A switching module is used to switch the photosensor involved in display from the first photosensor to the target photosensor.

11. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the photosensor fallback method as described in any one of claims 1 to 9 are implemented.

Images