Photosensitive sensor rollback method, device, electronic equipment and medium

Abstract

This application discloses a method, apparatus, electronic device, and medium for photosensor rollback, belonging to the field of imaging technology. The method, when a first photosensor participating in the display needs to roll back, selects N candidate photosensors capable of clear imaging from M second photosensors, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N. It calculates the imaging offset between the N candidate photosensors and the first photosensor. Based on the imaging offset between the N candidate photosensors and the first photosensor, it selects a candidate photosensor whose imaging offset satisfies a preset imaging offset condition as the target photosensor, switching the photosensor participating in the display from the first photosensor to the target photosensor.

Description

Technical Field

[0001] This application belongs to the field of camera technology, specifically relating to methods, devices, electronic devices, and media for retracting photosensors. Background Technology

[0002] With the development of camera technology, multiple light sensors are typically installed on electronic devices to meet higher shooting demands. For example, a mobile phone can have multiple cameras. In practical applications, users often move their electronic devices to take pictures. This can cause the light sensor used for display to fail to capture a clear image, necessitating a sensor fallback—switching to a light sensor capable of producing a clear image. However, currently, this sensor fallback switching process results in image abruptness, leading to a poor user experience.

[0003] Therefore, how to perform photosensitive sensor rollback to improve image abruptness has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a photosensitive sensor rollback method, apparatus, electronic device, and medium that can solve the problem of how to improve image jumps.

[0005] In a first aspect, embodiments of this application provide a photosensitive sensor rollback method, the method comprising:

[0006] When the first photosensitive sensor participating in the display needs to be deactivated, N candidate photosensitive sensors that can clearly image are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N.

[0007] Calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor;

[0008] Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor;

[0009] The photosensitive sensor used for display is switched from the first photosensitive sensor to the target photosensitive sensor.

[0010] Secondly, embodiments of this application provide an apparatus, comprising:

[0011] The candidate selection module is used to select N candidate photosensitive sensors that can clearly image from M second photosensitive sensors when the first photosensitive sensor participating in the display needs to be backed up, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N.

[0012] The offset calculation module is used to calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor;

[0013] The target selection module is used to select, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition as the target photosensitive sensor.

[0014] A switching module is used to switch the photosensor participating in the display from the first photosensor to the target photosensor.

[0015] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0017] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0018] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0019] In this embodiment, when the first photosensitive sensor participating in the display needs to revert, N candidate photosensitive sensors capable of clear imaging are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Since this embodiment further calculates the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected as the target photosensitive sensor. This allows the target photosensitive sensor that improves the image jump situation to be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the photosensitive sensor reverting process. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of an imaging perspective provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating an application scenario of a photosensitive sensor rollback method provided in an embodiment of this application;

[0022] Figure 3 This is a flowchart of a photosensitive sensor rollback method provided in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an imaging offset calculation provided in an embodiment of this application;

[0024] Figure 5 This is a flowchart of another embodiment of the present application providing a photosensitive sensor rollback method;

[0025] Figure 6 This is a flowchart of a photosensitive sensor rollback method provided in another embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of a photosensitive sensor retraction device provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0030] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0031] First, the terms and concepts involved in the embodiments of this application will be explained.

[0032] A light sensor is a key component in a camera device (such as a webcam), responsible for converting received light signals into electrical signals, and then generating digital images.

[0033] Focal length refers to the distance from the optical center (also known as the optical rear principal point) of the lens to the focal point of the image plane. It can also be understood as the distance between the point where light rays converge to a focal point after passing through the lens and the optical center of the lens.

[0034] Object distance refers to the distance from the subject to the front of the lens.

[0035] In a multi-camera system, back-off refers to the process of switching from the photosensitive sensor of one camera device to the photosensitive sensor of another camera device in order to adapt to different shooting needs or scene changes.

[0036] Image shift refers to the phenomenon where the image shifts relative to the expected position due to the movement of the lens, sensor, or subject. Image shift degree is a measure used to characterize the degree of image shift.

[0037] As mentioned in the background technology, in practical applications, users often move electronic devices to take pictures. This can cause the photosensor used for display to fail to capture a clear image, necessitating a photosensor backoff process—switching to a photosensor capable of capturing a clear image. Currently, photosensor backoff is typically selected based on the relationship between the object distance and the focal length of the photosensor, considering only whether the switched photosensor can capture a clear image, but without considering whether the switching process will cause image jumps. In response, the inventors of this application discovered that different placement positions of the camera device can cause variations in the viewing angles of the respective photosensors. Figure 1 As shown in the schematic diagram of the imaging angle, points I, II, and III are the imaging center points of the three photosensitive sensors. Among them, the photosensitive sensor with a shorter focal length has a wider imaging angle, but due to the difference in the placement of the camera device, the photosensitive sensor with a longer focal length cannot completely cover the imaging content of the photosensitive sensor with a shorter focal length, and there is a deviation between the imaging centers of the two photosensitive sensors, which can easily cause image jumps during the photosensitive sensor back-back switching process.

[0038] In view of this, embodiments of this application provide a photosensitive sensor rollback method, apparatus, electronic device, and medium. When a first photosensitive sensor participating in the display needs to roll back, N candidate photosensitive sensors capable of clear imaging are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Since embodiments of this application further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is strongly correlated with image jumps, a preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected as the target photosensitive sensor. This allows the selection of a target photosensitive sensor that improves the image jump situation, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, effectively improving the image jump problem during the photosensitive sensor rollback process.

[0039] The photosensitive sensor rollback method, apparatus, electronic device, and medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0040] Figure 2 This is a schematic diagram illustrating an application scenario of a photosensitive sensor rollback method provided in an embodiment of this application. For example... Figure 2As shown, this application scenario may include: an electronic device 20, wherein the electronic device 20 is configured with multiple camera devices (such as cameras), each camera device including one or more photosensors. Based on this, it can be understood that the electronic device 20 is configured with multiple photosensors. These multiple photosensors include at least one first photosensor and M second photosensors, where M is a positive integer greater than or equal to 2. Additionally, the electronic device 20 may also include a camera device controller. Exemplarily, the camera device controller is used to manage and control various components of the camera device, including photosensors, lenses, image processors, etc., and is also used to control the switching between different photosensors and to set and adjust the parameters of the photosensors. The electronic device 20 may include: mobile or non-mobile devices, such as smartphones, tablets, smart cameras, drones, smartwatches, security cameras, industrial cameras, medical imaging equipment, drone-borne cameras, etc. The electronic device is equipped with multiple camera devices (such as cameras).

[0041] based on Figure 2 The electronic device 20 shown includes multiple camera devices. In order to adapt to movement, different shooting needs or scene changes, it is necessary to switch the light sensor of one camera device to the light sensor of another camera device, that is, it is necessary to perform light sensor rollback. To effectively address the image jump problem during the photosensitive sensor rollback process, according to the method provided in this application embodiment, when the first photosensitive sensor participating in the display needs to roll back, the electronic device 20 selects N candidate photosensitive sensors capable of clear imaging from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Since this application embodiment further calculates the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected as the target photosensitive sensor. This allows the selection of a target photosensitive sensor that improves the image jump situation. Thus, by sending a control signal to the camera device controller, the photosensitive sensor participating in the display can be switched from the first photosensitive sensor to the target photosensitive sensor, effectively improving the image jump problem during the photosensitive sensor rollback process.

[0042] It is worth noting that in this embodiment, the photosensitive sensor back-off method provided in this embodiment can be implemented by the processor of the electronic device 20 when executing a program or instruction to determine the target photosensitive sensor, and a control signal can be sent to the camera device controller, which can then control the photosensitive sensor back-off according to the control signal. However, in some embodiments, the camera device controller may also have similar functions, such as implementing the photosensitive sensor back-off method provided in this embodiment to determine the target photosensitive sensor and then controlling the photosensitive sensor back-off when executing a program or instruction. This embodiment does not limit this aspect.

[0043] It should be noted that the application scenarios described in the above embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems. The photosensitive sensor retraction method provided in the embodiments of this application can be applied to application scenarios where photosensitive sensor retraction is required during the shooting process.

[0044] Based on the above application scenarios, the following will combine... Figure 3 This application provides an exemplary description of a photosensitive sensor rollback method. The execution subject of this photosensitive sensor rollback method can be an electronic device. It should be noted that this execution subject does not constitute a limitation on this application.

[0045] Figure 3 This is a flowchart illustrating a photosensitive sensor rollback method according to an exemplary embodiment of this application. Figure 3 As shown, the photosensitive sensor rollback method may include the following steps:

[0046] Step 310: When the first photosensitive sensor participating in the display needs to retreat, select N candidate photosensitive sensors that can clearly image from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N.

[0047] The first photosensor can be any one of the photosensors in a multi-camera electronic device. In a multi-camera electronic device, there will be a photosensor that is being used and providing image data; this photosensor is the one participating in the display, or its image data is being displayed to the user or processed by the system. In this step, the photosensor participating in the display is the first photosensor.

[0048] The second photosensitive sensor and the first photosensitive sensor belong to the same electronic device and are the other photosensitive sensors in the electronic device except the first photosensitive sensor. In this electronic device, the first photosensitive sensor participating in the display is being used and provides image data, while the second photosensitive sensor may be in a standby state, a sleep state, a background processing state, an auxiliary state, a fault state or other states. In the embodiments of the present application, when the first photosensitive sensor is participating in the display, the state of the second photosensitive sensor is not limited, and the state of the second photosensitive sensor may depend on the design of the electronic device system and the current requirements, as long as the efficient operation of the electronic device system and the user requirements are satisfied.

[0049] The candidate photosensitive sensor is a photosensitive sensor selected from M second photosensitive sensors. The candidate photosensitive sensor is used as a candidate for the target photosensitive sensor for fallback switching.

[0050] Exemplarily, step 310 may include:

[0051] Based on the object distance of the target object to be photographed and the focal lengths of the M second photosensitive sensors, select, from the M second photosensitive sensors, the second photosensitive sensors with focal lengths less than the object distance as candidate photosensitive sensors, and the number of candidate photosensitive sensors may be N.

[0052] The target object to be photographed refers to the key object that needs to be captured in photography or image processing and is usually located at the focus of the image frame. The target object to be photographed may be a person, an environment or an object.

[0053] According to the Gaussian imaging theorem, when the object distance is less than the focal length of the lens, a clear image cannot be obtained on the photosensitive sensor. Therefore, when the user uses a photosensitive sensor with a long focal length to photograph a nearby object, it will cause unclear imaging, and at this time, a fallback of the photosensitive sensor is required. The primary principle of the fallback is to ensure that the photosensitive sensor after the fallback switching can form a clear image. Being able to form a clear image means that the photosensitive sensor meets the clear imaging condition. Assume that the focal length of the photosensitive sensor participating in the display is P, the distance between the target object to be photographed and the lens is P1 (P < P1), and the focal lengths of the M second photosensitive sensors are N2…Nm+1 respectively. Among the second photosensitive sensors corresponding to N2…Nm+1, the second photosensitive sensors with their corresponding focal lengths P less than the object distance P1 can form a clear image, and the clear imaging condition is also that the focal length is less than the object distance. Therefore, among the M second photosensitive sensors, N second photosensitive sensors with focal lengths P less than the object distance P1 are marked as candidate photosensitive sensors.

[0054] It is understood that after selecting candidate photosensitive sensors capable of clear imaging, this application embodiment further selects a target photosensitive sensor from the candidate photosensitive sensors based on the imaging offset. Therefore, the N candidate photosensitive sensors selected in step 310 should be at least 2 in order to continue the selection. Thus, N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to N.

[0055] If only one candidate photosensitive sensor capable of producing a clear image is selected from M photosensitive sensors, subsequent processing steps can be set according to the actual application needs. For example, the selected candidate photosensitive sensor can be directly used as the target photosensitive sensor for switching, or an image jump prompt can be given to the user so that the user can adjust the shooting operation, or the imaging offset and / or other parameters corresponding to the candidate photosensitive sensor can be calculated to adjust the candidate photosensitive sensor so that the adjusted photosensitive sensor can be used as the target photosensitive sensor for switching. This application does not limit this.

[0056] For example, suppose the electronic device is a mobile phone with m cameras, each camera having a photosensor. For instance, the phone may have photosensor 1, photosensor 2, ..., and photosensor m. Assuming photosensor 1 is the first photosensor involved in the display, if the user moves the phone, causing the distance to the target object to be less than the focal length of photosensor 1, the target object cannot be clearly displayed. In this case, photosensor 1 needs to retract. According to step 310, at least two photosensors capable of clear imaging can be selected from photosensors 2-m as candidate photosensors based on a comparison of object distance and focal length.

[0057] Step 320: Calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor.

[0058] After screening in step 310, N candidate photosensitive sensors that can produce clear images will be obtained. In order to improve the smoothness of the rollback switching process and solve the problem of image abrupt changes in the rollback switching process in the prior art, step 320 will further calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor.

[0059] Image offset is a metric used to characterize the degree of image shift. Image offset can be measured by various metrics, including pixel offset and distance to reference points (such as corner points). Different metrics have their advantages, and the appropriate metric should be selected based on the specific application scenario.

[0060] It should be noted that in the embodiments of this application, the algorithms and metrics used by the N candidate photosensitive sensors to calculate the imaging offset can be the same or different. For example, in some embodiments, one or more candidate photosensitive sensors measure the imaging offset by the distance between reference points at the same location in different images, while other one or more candidate photosensitive sensors measure the imaging offset by the pixel offset in different images. As another example, in some embodiments, all N candidate photosensitive sensors measure the imaging offset by the distance between reference points at the same location in different images.

[0061] The following exemplifies an embodiment of this application by measuring the imaging offset using the distance between reference points at the same location in different images. Specifically, calculating the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor includes:

[0062] The image acquired by any of the candidate photosensitive sensors is cropped with field-of-view alignment to obtain a first image;

[0063] The image acquired by the first photosensitive sensor is cropped with field-of-view alignment to obtain the second image;

[0064] For any of the candidate photosensitive sensors, the distance between a first reference point in the first image acquired by the candidate photosensitive sensor and a second reference point in the second image acquired by the first photosensitive sensor is calculated to obtain the imaging offset between the candidate photosensitive sensor and the first photosensitive sensor. The first reference point is a point at a first specified position in the first image, and the second reference point is a point at the first specified position in the second image.

[0065] The first reference point and the second reference point are points at the same specified location in their respective images, used to measure the imaging offset between the two images. For example, the first reference point and the second reference point can be points that are easy to calculate, such as the top left corner, bottom left corner, top right corner, and bottom right corner of the first and second images.

[0066] For example: Figure 4 The diagram shown illustrates the calculation of image offset. Point A is the upper left corner of image 1 with point O as the imaging center, and point C is the upper left corner of another image 2 with point B as the imaging center. The second reference point in image 1 is point A, and the first reference point in image 2 is point C. The imaging offset between image 1 and image 2 can be measured by the distance between point A and point C.

[0067] The first image is an image captured by the candidate photosensitive sensor and cropped after field-of-view alignment. The second image is an image captured by the first photosensitive sensor and cropped after field-of-view alignment. For example: Figure 3As shown, the first image can be image 2 with point B as the imaging center, and the second image can be another image 1 with point O as the imaging center.

[0068] Field-of-view alignment cropping refers to the process in a multi-camera system where images acquired by different photosensitive sensors are adjusted and cropped to align their field of view, ensuring that the images have a consistent viewpoint and position when stitched or merged.

[0069] Using the same photosensor 1-m as before, combined with Figure 4 The calculation diagram shown assumes that the N candidate photosensitive sensors selected for clear imaging are photosensitive sensor 2 to photosensitive sensor m. The imaging offset of photosensitive sensor 2 can be calculated through the following steps:

[0070] The imaging offset of the photosensitive sensor 2 is obtained by calculating the distance between the top left corner point C of the image 2 after field-angle alignment and cropping of the photosensitive sensor 2 and the top left corner point A of the image 1 after field-angle alignment and cropping of the photosensitive sensor 1.

[0071] The calculation of the imaging offset of other candidate photosensitive sensors is similar to that of photosensitive sensor 2, and will not be repeated here.

[0072] In the above embodiments, the image acquired by the first photosensitive sensor and the image acquired by any candidate photosensitive sensor are first cropped with field-of-view alignment. Then, the distance between the first reference point and the second reference point at the same position in the two images is calculated. This allows the imaging offset of the images from the two photosensitive sensors to be measured under the condition of field-of-view alignment cropping. This ensures that the calculated imaging offset is the offset between two images with relatively consistent viewpoints and positions when stitching or fusion, making the measurement more accurate and allowing for more accurate selection of the target photosensitive sensor that meets the imaging offset conditions.

[0073] To facilitate the calculation of imaging offset for each candidate photosensitive sensor, in some embodiments of this application, a coordinate system is established with the imaging center of the first photosensitive sensor as the origin to calculate the coordinates of the reference point. Specifically, for any candidate photosensitive sensor, calculating the distance between a first reference point in the first image acquired by the candidate photosensitive sensor and a second reference point in the second image acquired by the first photosensitive sensor, to obtain the imaging offset between the candidate photosensitive sensor and the first photosensitive sensor, includes:

[0074] For any of the candidate photosensitive sensors, calculate the first coordinates of the first reference point of the first image acquired by the candidate photosensitive sensor, wherein the first coordinates are coordinates in a coordinate system established with the imaging center of the first photosensitive sensor as the origin.

[0075] Determine the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor, wherein the second coordinates are coordinates in the coordinate system;

[0076] Calculate the distance between the first coordinate and the second coordinate to obtain the imaging offset corresponding to any of the candidate back-away photosensitive sensors.

[0077] Following the previous example, the selected candidate photosensitive sensors are photosensitive sensors 2 to m. According to this embodiment, the imaging offset of photosensitive sensor 2 can be calculated through the following steps:

[0078] Calculate the coordinates of the upper left corner point C of the image 2 after field-of-view alignment and cropping of the photosensitive sensor 2. The coordinates of point C are in a coordinate system established with the imaging center point O of the photosensitive sensor 1 as the origin.

[0079] Calculate the coordinates of the top left corner point A of the image 1 after field-of-view alignment and cropping of the photosensitive sensor 1. The coordinates of point A are the coordinates in the above coordinate system.

[0080] Calculate the distance between point A and point C to obtain the imaging offset of photosensitive sensor 2.

[0081] The imaging offset of the photosensitive sensor m can be calculated using the following steps:

[0082] Calculate the top left corner point (let's call it point E) of the image 3 after field-of-view alignment and cropping from the photosensitive sensor m. To illustrate the calculation principle more clearly and concisely... Figure 3 The coordinates of point E are not shown above. The coordinates of point E are in the coordinate system described above.

[0083] Directly obtain the coordinates of point A calculated earlier;

[0084] Calculate the distance between point A and point E to obtain the imaging offset of the photosensitive sensor m.

[0085] As can be seen, since this embodiment establishes a coordinate system based on the imaging center of the first photosensitive sensor as the origin, the calculation of the imaging offset of multiple candidate photosensitive sensors only requires calculating the coordinates of the reference point in the image of the first photosensitive sensor once. Compared with the method of establishing a coordinate system based on the imaging center of the candidate photosensitive sensors, this avoids repeated calculations caused by different imaging centers of different candidate photosensitive sensors, and improves the calculation efficiency of the imaging offset of multiple candidate photosensitive sensors.

[0086] Furthermore, in some embodiments of this application, calculating the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors includes:

[0087] Based on the resolution, current zoom ratio, and base zoom ratio of any of the candidate photosensitive sensors, calculate the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors;

[0088] Determining the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor includes:

[0089] Based on the resolution of the first photosensitive sensor, the current zoom level, and the base zoom level, calculate the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor.

[0090] Resolution refers to the number of pixels in an image, usually expressed as the number of pixels in width and height. Current zoom level refers to the current magnification or reduction factor, typically used to adjust the field of view. Base zoom level refers to the default zoom level of the image sensor when it is not zoomed.

[0091] Following the previous example, such as Figure 3 The diagram illustrating the imaging offset calculation establishes a two-dimensional coordinate system with the imaging center O of the first photosensitive sensor 1 as the origin. The offsets of the imaging centers B of candidate photosensitive sensors 2 and m from the imaging center O of the first photosensitive sensor 1 are (x2, y2)...(xm, ym). Points A and C are the coordinates of the upper left corners of the two images after field-of-view alignment and cropping based on the current zoom magnification. The calculation process for the coordinates of points A and C is as follows:

[0092] Where, the coordinates of A with O as the origin are: It is abbreviated as (xa, ya). The coordinates of point B are (xm, ym).

[0093] The coordinates of point C, with point B as the origin, are: It is abbreviated as (xc, yc).

[0094] Since the resolution, current zoom ratio, and base zoom ratio of the photosensitive sensor are key parameters affecting the geometric transformation and scaling of the image, this embodiment uses them to calculate the coordinates of the midpoint of the image after field-of-view alignment and cropping. The calculation is simple, accurate, and can effectively improve computational efficiency.

[0095] In some embodiments of this application, calculating the distance between the first coordinate and the second coordinate includes:

[0096] Calculate the distance between the first coordinate and the second coordinate based on the first coordinate, the second coordinate, and the coordinate of the origin (e.g., using the law of cosines).

[0097] Following the previous example, such as Figure 3The diagram showing the imaging offset calculation illustrates that the magnitude of the imaging offset D for any candidate photosensitive sensor m is... Figure 3 The length of AC in the equation is calculated as follows:

[0098] The length of AC can be calculated using the law of cosines:

[0099]

[0100] The length of AO is:

[0101]

[0102] The length of CO is:

[0103]

[0104] ∠AOC is:

[0105]

[0106] This embodiment calculates the distance between the first and second coordinates based on a triangle formed by the first reference point, the second reference point, and the origin of the coordinate system. The calculation is simple and efficient.

[0107] Step 330: Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, select the candidate photosensitive sensor whose imaging offset satisfies the preset imaging offset condition from the N candidate photosensitive sensors as the target photosensitive sensor.

[0108] The preset imaging offset condition is used to constrain the imaging offset of the finally selected candidate photosensitive sensor, so that the selected candidate photosensitive sensor meets the requirements for improving image abruptness. The content of the preset imaging offset condition can be set according to the image abruptness improvement requirements in actual applications, and this application does not limit it.

[0109] In one embodiment, selecting a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition from the N candidate photosensitive sensors as the target photosensitive sensor based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor includes:

[0110] Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor with the smallest imaging offset is selected as the target photosensitive sensor from the N candidate photosensitive sensors.

[0111] For example, N candidate photosensitive sensors can be sorted according to the magnitude of their imaging offset, and the one with the smallest imaging offset can be selected as the target photosensitive sensor.

[0112] In this embodiment, the preset imaging offset condition is the minimum imaging offset. By selecting the candidate photosensitive sensor with the minimum imaging offset as the target photosensitive sensor, image jumps during the photosensitive sensor back-off switching process can be effectively reduced.

[0113] Understandably, in practical applications, in addition to selecting the candidate photosensitive sensor with the smallest imaging offset to improve the image jump problem, selecting a candidate photosensitive sensor with a relatively small imaging offset can also achieve a certain degree of improvement in image jump.

[0114] Step 340: Switch the photosensor participating in the display from the first photosensor to the target photosensor.

[0115] For example, it can be made by Figure 2 When the processor of the electronic device 20 shown executes the program or instructions, it implements the photosensitive sensor rollback method provided in this application embodiment and sends a control signal to the camera device controller of the electronic device 10. The camera device controller controls the photosensitive sensor participating in the display to switch from the first photosensitive sensor to the target photosensitive sensor according to the control signal.

[0116] In this embodiment, when the first photosensitive sensor participating in the display needs to revert, N candidate photosensitive sensors capable of clear imaging are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Since this embodiment further calculates the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected as the target photosensitive sensor. This allows the target photosensitive sensor that improves the image jump situation to be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the photosensitive sensor reverting process.

[0117] Furthermore, in some embodiments of this application, to address the image jump issue caused by using only one frame for transition during the fallback process in existing technologies, a suitable number of transition frames is calculated, thereby providing users with better switching smoothness during the fallback process. Specifically, Figure 5 This is a flowchart illustrating a photosensitive sensor rollback method according to another exemplary embodiment of this application. Figure 5 For details not described in the specific implementation of steps 510-540 shown, please refer to [the relevant documentation]. Figure 3 The relevant descriptions of the illustrated embodiments will not be repeated here. Figure 5 As shown, the photosensitive sensor rollback method, before switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor in step 340, further includes:

[0118] Step 3302: Based on the preset transition frame number calculation rule, calculate the number of transition frames corresponding to the process of the first photosensitive sensor switching to the target photosensitive sensor, wherein the number of transition frames is greater than 1.

[0119] It should be noted that the preset transition frame count calculation rule aims to achieve smooth image transitions during sensor switching. It calculates the number of transition frames during sensor switching, provided the number of transition frames is greater than one. The specific algorithm for the preset transition frame count calculation rule is not limited, as long as multiple frames are calculated and the image transition is smooth.

[0120] For example, in some embodiments of this application, the process of step 3302 may include:

[0121] Based on the imaging offset corresponding to the target photosensitive sensor and the preset inter-frame pixel offset threshold, the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor is calculated.

[0122] The preset inter-frame pixel offset threshold is used to constrain the pixel offset between one frame and another. For example, it could be a pixel offset threshold between adjacent frames or between frames with intervals, etc., and this application does not limit this. By dividing the imaging offset corresponding to the target photosensitive sensor by the preset inter-frame pixel offset threshold, the number of transition frames for smooth image transition can be obtained. It is understood that the smaller the threshold, the more transition frames there are, and the smoother the image transition; the larger the threshold, the fewer transition frames there are, and the relatively lower the smoothness of the image transition. Therefore, the value of the threshold can be set according to the actual application requirements for improving the smoothness of the image transition. For example, the size of the preset inter-frame pixel offset threshold can be determined based on usage experience or statistical results of user feedback on image smoothness under different thresholds.

[0123] For example, assuming the imaging offset corresponding to the target photosensitive sensor is 0.5 pixels and the preset inter-frame pixel offset threshold is 0.1 pixels, the number of transition frames is the result of 0.5 divided by 0.1, which is 5 frames.

[0124] This embodiment addresses the problem of image abruptness caused by excessive inter-frame offset by calculating the number of transition frames through a preset inter-frame pixel offset threshold, thereby effectively controlling the pixel offset between adjacent frames and making the image transition smooth.

[0125] For example, in some embodiments of this application, the process of step 3302 may include:

[0126] Based on the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor, a reference frame rate is determined, and based on the reference frame rate and a preset switching duration threshold, the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor is calculated.

[0127] The frame rate is the number of image frames that a photosensor can capture and process per second. Different photosensors can have the same or different frame rates. Therefore, a reference frame rate can be determined based on the frame rates of the first and target photosensors. The reference frame rate can be the frame rate of the first photoensor, the frame rate of the target photoensor, or a value between the two. The preset switching duration threshold is a threshold for the duration of the photosensor switching process. By dividing the preset switching duration threshold by the reference frame rate, the number of transition frames for a smooth image transition can be obtained. It is understandable that if the switching duration is too long, it will lead to excessive user waiting time; if the switching duration is too short, it will cause abrupt image changes. Both situations result in a poor user experience. Therefore, the value of the preset switching duration threshold can be set according to the actual application requirements for improving the smoothness of image transitions. For example, the preset switching duration can be determined based on user experience or statistical results of user feedback on image smoothness at different thresholds. In this embodiment, the number of transition frames can be obtained by dividing the preset switching duration threshold by the reference frame rate.

[0128] For example, assuming a reference frame rate of 30 frames per second and a preset inter-frame pixel offset threshold of 0.1 seconds, the number of transition frames is the result of dividing 0.1 seconds by 30 frames per second, which is 3 frames.

[0129] This embodiment addresses the issues of image jumps caused by excessively short switching durations and waiting caused by excessively long switching durations. By pre-setting a switching duration threshold to calculate the number of transition frames, the switching duration and the number of transition frames can be effectively controlled, resulting in smooth image transitions.

[0130] After calculating the number of transition frames, in step 340, the photosensitive sensor participating in the display can be switched from the first photosensitive sensor to the target photosensitive sensor based on the calculated number of transition frames, so that the image transition is smooth and the user experience is better.

[0131] The following example illustrates the processing procedure of the photosensitive sensor rollback method provided in this application, using a mobile device such as a smartphone as an example. The mobile device may include multiple cameras, each with its own photosensitive sensor. Figure 6This is a flowchart illustrating a photosensitive sensor rollback method according to yet another exemplary embodiment of this application. For example... Figure 6 As shown, the photosensitive sensor rollback method may include the following steps:

[0132] Step 610: The mobile device detects the distance to the target object being photographed.

[0133] Step 620: Determine if the object distance is less than the focal length of the mobile device's current main camera.

[0134] Step 630: If not, then confirm that the photosensor rollback will not be performed.

[0135] Step 640: If so, determine that photosensitive sensor rollback is required, and select multiple candidate photosensitive sensors that can produce clear images from multiple cameras on the mobile device based on focal length.

[0136] Step 650: Calculate the imaging offset of multiple candidate photosensitive sensors and select the candidate photosensitive sensor with the smallest imaging offset as the target photosensitive sensor.

[0137] Step 660: Calculate the number of transition frames based on the imaging offset of the target photosensitive sensor and the preset inter-frame pixel threshold.

[0138] Step 670: During the switching process, use the transition frames corresponding to the number of transition frames to switch the current main camera of the mobile device to the camera belonging to the target light sensor.

[0139] For details not described in the specific implementation of steps 610-670 above, please refer to the foregoing. Figure 3 The relevant descriptions of steps 310-340 in the illustrated embodiment will not be repeated here.

[0140] In this embodiment, candidate photosensitive sensors capable of clear imaging are first selected by comparing the object distance and the focal length of each photosensitive sensor. Then, the imaging offset between the candidate photosensitive sensors and the main camera photosensitive sensor involved in the display is calculated to determine the target photosensitive sensor with the smallest imaging offset. After that, the main camera is switched using the number of transition frames corresponding to the number of transition frames calculated based on the preset threshold of pixel subtraction between adjacent frames. Thus, during the main camera switching process, the user is provided with the best image smoothness experience during the switching process.

[0141] The photosensitive sensor rollback method provided in this application can be executed by a photosensitive sensor rollback device. This application uses the photosensitive sensor rollback device executing the photosensitive sensor rollback method as an example to illustrate the photosensitive sensor rollback device provided in this application.

[0142] Figure 7A schematic diagram of a photosensitive sensor retraction device provided in an embodiment of this application is shown. Figure 7 As shown, the photosensitive sensor rollback device 700 provided in this application embodiment may include:

[0143] The candidate selection module 710 is used to select N candidate photosensitive sensors that can clearly image from M second photosensitive sensors when the first photosensitive sensor participating in the display needs to be withdrawn, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N.

[0144] The offset calculation module 720 is used to calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor.

[0145] The target selection module 730 is used to select, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition as the target photosensitive sensor.

[0146] The switching module 740 is used to switch the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor.

[0147] When the first photosensitive sensor participating in the display needs to be retracted, the photosensitive sensor retraction device 700 selects N candidate photosensitive sensors that can clearly image from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Since this embodiment further calculates the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected as the target photosensitive sensor. The target photosensitive sensor that improves the image jump situation can be selected, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the photosensitive sensor retraction process.

[0148] The photosensitive sensor retraction device 700 described above will be explained in detail below:

[0149] In some embodiments, the offset calculation module is used to perform field-angle aligned cropping on the image acquired by any of the candidate photosensitive sensors to obtain a first image, perform field-angle aligned cropping on the image acquired by the first photosensitive sensor to obtain a second image, and, for any of the candidate photosensitive sensors, calculate the distance between a first reference point of the first image acquired by any of the candidate photosensitive sensors and a second reference point of the second image acquired by the first photosensitive sensor to obtain the imaging offset between any of the candidate photosensitive sensors and the first photosensitive sensor, wherein the first reference point is a point at a first specified position in the first image, and the second reference point is a point at the first specified position in the second image.

[0150] In some embodiments, the offset calculation module includes:

[0151] The first coordinate calculation unit is used to calculate the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors, wherein the first coordinates are coordinates in a coordinate system established with the imaging center of the first photosensitive sensor as the origin.

[0152] The second coordinate calculation unit is used to determine the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor, wherein the second coordinates are coordinates in the coordinate system.

[0153] The distance calculation unit is used to calculate the distance between the first coordinate and the second coordinate to obtain the imaging offset corresponding to any of the candidate back-away photosensitive sensors.

[0154] In some embodiments, the first coordinate calculation unit is used to calculate the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors based on the resolution, current zoom ratio, and base zoom ratio of any of the candidate photosensitive sensors.

[0155] The second coordinate calculation unit is used to calculate the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor based on the resolution of the first photosensitive sensor, the current zoom ratio, and the base zoom ratio.

[0156] In some embodiments, the distance calculation unit is used to calculate the distance between the first coordinate and the second coordinate based on the first coordinate, the second coordinate, and the coordinate of the origin.

[0157] In some embodiments, the target selection module is used to select the candidate photosensitive sensor with the smallest imaging offset from the N candidate photosensitive sensors as the target photosensitive sensor based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor.

[0158] In some embodiments, the photosensitive sensor rollback device further includes: a transition frame calculation module, used to calculate the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor based on a preset transition frame number calculation rule before switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, wherein the number of transition frames is greater than 1.

[0159] In some embodiments, the transition frame calculation module is used to calculate the number of transition frames corresponding to the process of the first photosensitive sensor switching to the target photosensitive sensor based on the imaging offset corresponding to the target photosensitive sensor and a preset inter-frame pixel offset threshold.

[0160] In some embodiments, the transition frame calculation module is used to determine a reference frame rate based on the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor, and to calculate the number of transition frames corresponding to the process of the first photosensitive sensor switching to the target photosensitive sensor based on the reference frame rate and a preset switching duration threshold.

[0161] The photosensitive sensor fallback device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0162] The photosensor device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0163] The photosensitive sensor device provided in this application embodiment can achieve... Figures 2 to 6 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0164] Optionally, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various steps of the above-described photosensor method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0165] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0166] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0167] The electronic device 900 includes, but is not limited to, components such as: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910.

[0168] Those skilled in the art will understand that the electronic device 900 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 910 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0169] The processor 910 is used for:

[0170] When the first photosensitive sensor participating in the display needs to be deactivated, N candidate photosensitive sensors that can clearly image are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N.

[0171] Calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor;

[0172] Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor;

[0173] The photosensitive sensor used for display is switched from the first photosensitive sensor to the target photosensitive sensor.

[0174] In this embodiment, when the first photosensitive sensor participating in the display needs to revert, N candidate photosensitive sensors capable of clear imaging are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Since this embodiment further calculates the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, and the degree of imaging offset is related to the image jump situation, the preset imaging offset condition can be set according to the image jump improvement requirements in actual applications. Based on this, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor whose imaging offset meets the preset imaging offset condition is selected as the target photosensitive sensor. This can select the target photosensitive sensor that improves the image jump situation, thereby switching the photosensitive sensor participating in the display from the first photosensitive sensor to the target photosensitive sensor, which can effectively improve the image jump problem during the photosensitive sensor reverting process.

[0175] In some embodiments, the processor 910 is further configured to:

[0176] A first image is obtained by aligning and cropping the image acquired by any of the candidate photosensitive sensors. A second image is obtained by aligning and cropping the image acquired by the first photosensitive sensor. For any of the candidate photosensitive sensors, the distance between a first reference point of the first image acquired by the candidate photosensitive sensor and a second reference point of the second image acquired by the first photosensitive sensor is calculated to obtain the imaging offset between the candidate photosensitive sensor and the first photosensitive sensor. The first reference point is a point at a first specified position in the first image, and the second reference point is a point at the first specified position in the second image.

[0177] In the above embodiments, the image acquired by the first photosensitive sensor and the image acquired by any candidate photosensitive sensor are first cropped with field-of-view alignment. Then, the distance between the first reference point and the second reference point at the same position in the two images is calculated. This allows the imaging offset of the images from the two photosensitive sensors to be measured under the condition of field-of-view alignment cropping. This ensures that the calculated imaging offset is the offset between two images with relatively consistent viewpoints and positions when stitching or fusion, making the measurement more accurate and allowing for more accurate selection of the target photosensitive sensor that meets the imaging offset conditions.

[0178] In some embodiments, the processor 910 is further configured to:

[0179] For any of the candidate photosensitive sensors, calculate the first coordinates of the first reference point of the first image acquired by the candidate photosensitive sensor, wherein the first coordinates are coordinates in a coordinate system established with the imaging center of the first photosensitive sensor as the origin.

[0180] Determine the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor, wherein the second coordinates are coordinates in the coordinate system;

[0181] Calculate the distance between the first coordinate and the second coordinate to obtain the imaging offset corresponding to any of the candidate back-away photosensitive sensors.

[0182] Since this embodiment establishes a coordinate system based on the imaging center of the first photosensitive sensor as the origin, the calculation of the imaging offset of multiple candidate photosensitive sensors only requires calculating the coordinates of the reference point in the image of the first photosensitive sensor once. Compared with the method of establishing a coordinate system based on the imaging center of the candidate photosensitive sensors, this avoids repeated calculations caused by different imaging centers of different candidate photosensitive sensors, and improves the calculation efficiency of the imaging offset of multiple candidate photosensitive sensors.

[0183] In some embodiments, the processor 910 is further configured to:

[0184] Based on the resolution, current zoom ratio, and base zoom ratio of any of the candidate photosensitive sensors, calculate the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors;

[0185] Based on the resolution of the first photosensitive sensor, the current zoom level, and the base zoom level, calculate the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor.

[0186] Since the resolution, current zoom ratio, and base zoom ratio of the photosensitive sensor are key parameters affecting the geometric transformation and scaling of the image, this embodiment uses them to calculate the coordinates of the midpoint of the image after field-of-view alignment and cropping. The calculation is simple, accurate, and can effectively improve computational efficiency.

[0187] In some embodiments, the processor 910 is further configured to:

[0188] Calculate the distance between the first coordinate and the second coordinate based on the first coordinate, the second coordinate, and the coordinate of the origin.

[0189] This embodiment uses the first coordinate, the second coordinate, and the coordinates of the origin to calculate the distance between the first coordinate and the second coordinate, which is simple and efficient.

[0190] In some embodiments, the processor 910 is further configured to:

[0191] Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor with the smallest imaging offset is selected as the target photosensitive sensor from the N candidate photosensitive sensors.

[0192] By selecting the candidate photosensitive sensor with the smallest imaging offset as the target photosensitive sensor, image jumps during the photosensitive sensor back-off switching process can be effectively reduced.

[0193] In some embodiments, the processor 910 is further configured to:

[0194] Before switching the photosensor participating in the display from the first photosensor to the target photosensor, the number of transition frames corresponding to the process of switching from the first photosensor to the target photosensor is calculated based on a preset transition frame number calculation rule, and the number of transition frames is greater than 1.

[0195] This embodiment addresses the issue of image abruptness caused by using only one frame for transition during the rollback process in existing technologies. It further calculates the appropriate number of transition frames, thereby providing users with better switching smoothness during the rollback process.

[0196] In some embodiments, the processor 910 is further configured to:

[0197] Based on the imaging offset corresponding to the target photosensitive sensor and the preset inter-frame pixel offset threshold, the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor is calculated.

[0198] This embodiment addresses the problem of image abruptness caused by excessive inter-frame offset by calculating the number of transition frames through a preset inter-frame pixel offset threshold, thereby effectively controlling the pixel offset between adjacent frames and making the image transition smooth.

[0199] In some embodiments, the processor 910 is further configured to:

[0200] Based on the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor, a reference frame rate is determined, and based on the reference frame rate and a preset switching duration threshold, the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor is calculated.

[0201] This embodiment addresses the issues of image jumps caused by excessively short switching durations and waiting caused by excessively long switching durations. By pre-setting a switching duration threshold to calculate the number of transition frames, the switching duration and the number of transition frames can be effectively controlled, resulting in smooth image transitions.

[0202] It should be understood that, in this embodiment, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042. The GPU 9041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0203] The memory 909 can be used to store software programs and various data. The memory 909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 909 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0204] Processor 910 may include one or more processing units; optionally, processor 910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 910.

[0205] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described photosensitive sensor rollback method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0206] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0207] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described photosensor method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0208] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0209] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described photosensor rollback method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0210] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0211] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0212] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for retracting a photosensor, characterized in that, include: When the first photosensitive sensor participating in the display needs to be deactivated, N candidate photosensitive sensors that can clearly image are selected from M second photosensitive sensors, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. Calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor; Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition is selected from the N candidate photosensitive sensors as the target photosensitive sensor; The photosensitive sensor used for display is switched from the first photosensitive sensor to the target photosensitive sensor.

2. The method according to claim 1, characterized in that, The calculation of the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor includes: The image acquired by any of the candidate photosensitive sensors is cropped with field-of-view alignment to obtain a first image; The image acquired by the first photosensitive sensor is cropped with field-of-view alignment to obtain the second image; For any of the candidate photosensitive sensors, the distance between a first reference point in the first image acquired by the candidate photosensitive sensor and a second reference point in the second image acquired by the first photosensitive sensor is calculated to obtain the imaging offset between the candidate photosensitive sensor and the first photosensitive sensor. The first reference point is a point at a first specified position in the first image, and the second reference point is a point at the first specified position in the second image.

3. The method according to claim 2, characterized in that, For any of the candidate photosensitive sensors, the step of calculating the distance between a first reference point in a first image acquired by the candidate photosensitive sensor and a second reference point in a second image acquired by the first photosensitive sensor, to obtain the imaging offset between the candidate photosensitive sensor and the first photosensitive sensor, includes: For any of the candidate photosensitive sensors, calculate the first coordinates of the first reference point of the first image acquired by the candidate photosensitive sensor, wherein the first coordinates are coordinates in a coordinate system established with the imaging center of the first photosensitive sensor as the origin. Determine the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor, wherein the second coordinates are coordinates in the coordinate system; Calculate the distance between the first coordinate and the second coordinate to obtain the imaging offset corresponding to any of the candidate photosensitive sensors.

4. The method according to claim 3, characterized in that, The step of calculating the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors includes: Based on the resolution, current zoom ratio, and base zoom ratio of any of the candidate photosensitive sensors, calculate the first coordinates of the first reference point of the first image acquired by any of the candidate photosensitive sensors; Determining the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor includes: Based on the resolution of the first photosensitive sensor, the current zoom level, and the base zoom level, calculate the second coordinates of the second reference point of the second image acquired by the first photosensitive sensor.

5. The method according to claim 3, characterized in that, The calculation of the distance between the first coordinate and the second coordinate includes: Calculate the distance between the first coordinate and the second coordinate based on the first coordinate, the second coordinate, and the coordinate of the origin.

6. The method according to claim 1, characterized in that, The step of selecting a candidate photosensitive sensor as the target photosensitive sensor from the N candidate photosensitive sensors based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor includes: Based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, the candidate photosensitive sensor with the smallest imaging offset is selected as the target photosensitive sensor from the N candidate photosensitive sensors.

7. The method according to any one of claims 1-6, characterized in that, Before switching the photosensor participating in the display from the first photosensor to the target photosensor, the method further includes: Based on a preset transition frame number calculation rule, the number of transition frames corresponding to the process of the first photosensitive sensor switching to the target photosensitive sensor is calculated, and the number of transition frames is greater than 1.

8. The method according to claim 7, characterized in that, The calculation of the number of transition frames during the switching process from the first photosensitive sensor to the target photosensitive sensor, based on a preset transition frame number calculation rule, includes: Based on the imaging offset corresponding to the target photosensitive sensor and the preset inter-frame pixel offset threshold, the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor is calculated.

9. The method according to claim 7, characterized in that, The calculation of the number of transition frames during the switching process from the first photosensitive sensor to the target photosensitive sensor, based on a preset transition frame number calculation rule, includes: Based on the frame rate of the first photosensitive sensor and the frame rate of the target photosensitive sensor, a reference frame rate is determined, and based on the reference frame rate and a preset switching duration threshold, the number of transition frames corresponding to the process of switching the first photosensitive sensor to the target photosensitive sensor is calculated.

10. A photosensitive sensor retraction device, characterized in that, include: The candidate selection module is used to select N candidate photosensitive sensors that can clearly image from M second photosensitive sensors when the first photosensitive sensor participating in the display needs to be backed up, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to N. The offset calculation module is used to calculate the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor; The target selection module is used to select, based on the imaging offset between the N candidate photosensitive sensors and the first photosensitive sensor, a candidate photosensitive sensor whose imaging offset satisfies a preset imaging offset condition as the target photosensitive sensor. A switching module is used to switch the photosensor participating in the display from the first photosensor to the target photosensor.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the photosensitive sensor rollback method as described in any one of claims 1-9.

Images