Drop protection method, electronic device, program product, and readable medium

Abstract

The present application provides a drop protection method, an electronic device, a program product, and a computer readable storage medium. The electronic device comprises a camera module, and the camera module comprises a rotatable reflection prism. The drop protection method comprises: receiving and responding to a first operation from a user, and controlling a camera module to capture images or perform focus tracking to record a video, the reflection prism being in a rotating state when the camera module captures images or performs focus tracking to record a video; when the reflection prism is in the rotating state, detecting the operating state of an electronic device; and when it is detected that the electronic device is in a state of being dropped, controlling the reflection prism to stop rotating. According to the drop protection method, the reflection prism is in the rotating state when the camera module captures images or performs focus tracking to record a video, and if it is detected that the electronic device is in the state of being dropped when the reflection prism is in the rotating state, the reflection prism is controlled to stop rotating, so that the reflection prism can be protected from damage caused by shaking and impact.

Description

Fall protection method, electronic device, program product, and readable medium Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a fall protection method, electronic equipment, program product, and computer-readable storage medium. Background Art

[0002] The camera module of the electronic device can capture images or record focus videos.

[0003] If the electronic device accidentally falls while its camera module is capturing images or recording focus-tracking videos, the reflective prism in the camera module may be shaken and damaged by impact.

[0004] Summary of the Invention

[0005] The present application provides a drop protection method, an electronic device, a computer program product, and a computer-readable storage medium, the purpose of which is to prevent the reflective prism in the camera module from shaking and being damaged by impact when the electronic device falls.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] In a first aspect, the present application provides a drop protection method, which is applied to an electronic device, wherein the electronic device includes a camera module, and the camera module includes a rotatable reflective prism. The drop protection method includes: receiving and responding to a first operation of the user, controlling the camera module to capture images or track focus recordings, wherein the reflective prism is in a rotating state during the camera module capturing images or tracking focus recordings; when the reflective prism is in a rotating state, detecting the operating state of the electronic device; and when it is detected that the electronic device is in a falling state, controlling the reflective prism to stop rotating.

[0008] In one embodiment, a user activates a camera application of an electronic device, and the electronic device displays a camera preview interface in a photo mode, wherein the camera preview interface displays a first object, and the user performs a first operation on the first object to control the camera module to capture a close-up of the first object. Alternatively, a user activates a camera application of an electronic device, and the electronic device displays a camera preview interface or a camera capture interface in a video recording mode, wherein the camera preview interface or the camera capture interface may also display the first object, and the user performs a first operation on the first object to control the camera module to perform focus tracking recording on the first object.

[0009] In the drop protection method provided in the first aspect, when the camera module is shooting images or tracking focus recording, the reflective prism is in a rotating state. If the electronic device is detected to be in a falling state while the reflective prism is in a rotating state, the reflective prism is controlled to stop rotating, which can prevent the reflective prism from shaking and being damaged by impact.

[0010] In a possible embodiment, after controlling the reflective prism to stop rotating, the method further includes: locking the position of the reflective prism. By locking the position of the reflective prism, the reflective prism can be fixed, further preventing the reflective prism from shaking and being damaged by impact.

[0011] In a possible implementation, locking the position of the reflective prism includes: locking the position of the reflective prism by a brake component, or locking the position of the reflective prism by a driving component of the reflective prism.

[0012] In one possible embodiment, the electronic device includes a first controller, and the reflective prism is in a rotating state during the process of the camera module shooting images or tracking focus recording, including: when the camera module shoots images or tracking focus recording, the first controller controls the reflective prism to be in a rotating state; when the reflective prism is in the rotating state, the operating state of the electronic device is detected; when it is detected that the electronic device is in a falling state, the reflective prism is controlled to stop rotating, including: when the reflective prism is in a rotating state, the first controller detects the operating state of the electronic device; and when it is detected that the electronic device is in a falling state, the reflective prism is controlled to stop rotating.

[0013] In the above possible implementation manner, the first controller can be understood as a Scan IC. The first controller independently detects whether the electronic device is in a falling state, and determines that the electronic device is in a falling state, controls the reflective prism to stop rotating, and also has the advantage of not occupying the application processor resources of the electronic device.

[0014] In one possible embodiment, the electronic device also includes an acceleration sensor and / or a gyroscope sensor, and the first controller detects the operating status of the electronic device when the reflective prism is in a rotating state, including: the first controller detects the operating status of the electronic device based on the detection data of the acceleration sensor and / or the gyroscope sensor when the reflective prism is in a rotating state.

[0015] In one possible embodiment, the first controller detects the operating status of the electronic device when the reflective prism is in a rotating state, including: the first controller detects the operating status of the electronic device based on a message from an application processor of the electronic device when the reflective prism is in a rotating state, and the message from the application processor is used to indicate whether the electronic device is in a falling state.

[0016] In one possible embodiment, when it is detected that the electronic device is in a falling state, it also includes: obtaining abnormal state coding information of the driving module of the reflective prism; determining that the electronic device is in a falling state based on the abnormal state coding information of the driving module of the reflective prism; and stopping obtaining the target position of the reflective prism.

[0017] In the above possible implementation, upon detecting that the electronic device is in a dropped state, abnormal state coding information of the driving module of the reflective prism is obtained; and based on the abnormal state coding information of the driving module of the reflective prism, it is determined that the electronic device is in a dropped state. This allows for further accurate determination of whether the electronic device is in a dropped state based on the abnormal state coding information of the driving module of the reflective prism. Furthermore, after determining that the electronic device is in a dropped state based on the abnormal state coding information, obtaining the target position of the reflective prism is stopped, thereby avoiding wasting resources by continuing to obtain the target position of the reflective prism while the reflective prism is not rotating.

[0018] In a possible implementation, obtaining abnormal state coding information of the driving module of the reflective prism includes: calling an abnormal dump interface of the driving module of the reflective prism to obtain abnormal state coding information of the driving module of the reflective prism.

[0019] In one possible embodiment, after controlling the reflective prism to stop rotating, it also includes: displaying a first interface, the first interface including a first control; receiving a second operation of the user on the first control; determining the target position of the reflective prism in response to the second operation, and controlling the reflective prism to rotate to the target position.

[0020] In the above possible implementation manner, the user inputs a second operation to the first control, indicating that the electronic device is already in a non-falling state. The electronic device responds to the second operation, determines the target position of the reflective prism, and controls the reflective prism to rotate to the target position to restore the rotation of the reflective prism.

[0021] In one possible embodiment, after controlling the reflective prism to stop rotating, it also includes: detecting that the electronic device is in a non-falling state based on detection data of the acceleration sensor and / or the gyroscope sensor; determining the target position of the reflective prism; and controlling the reflective prism to rotate to the target position. In this way, when the electronic device is in a non-falling state, the rotation of the reflective prism can be automatically detected and restored without user operation.

[0022] In a possible implementation, before controlling the reflective prism to rotate to the target position, the method further includes: unlocking the position of the reflective prism.

[0023] In a possible implementation, before determining the target position of the reflective prism, the method further includes: restoring and obtaining the target position of the reflective prism.

[0024] In a second aspect, the present application provides an electronic device comprising: one or more processors, a memory, and a display screen; the memory and the display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the fall protection method as described in any one of the first aspects.

[0025] In a third aspect, the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the fall protection method as described in any one of the first aspects.

[0026] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the fall protection method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0028] FIG2 is a software framework diagram of an electronic device provided in an embodiment of the present application;

[0029] FIG3 is a timing diagram of the operation of the fall detection method provided in an embodiment of the present application;

[0030] FIG4 is a flowchart of registering two interfaces of the Scan driver, OIS driver, and AF driver to the anomaly detection module according to an embodiment of the present application;

[0031] FIG5 is a signaling interaction diagram of a fall detection method provided in an embodiment of the present application;

[0032] FIG6 is another operating timing diagram of the fall detection method provided in an embodiment of the present application;

[0033] FIG7 is another signaling interaction diagram of the fall detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0035] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0036] The "multiple" involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first" and "second" are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance or order.

[0037] FIG1 is a composition example of an electronic device provided in an embodiment of the present application.

[0038] The electronic device may be a mobile phone, a tablet computer, a personal digital assistant (PDA), a desktop, a laptop, a notebook computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a wearable device, etc.

[0039] As shown in FIG1 , the electronic device 100 may include a processor 110 , an internal memory 120 , a camera 130 , a display screen 140 , a mobile communication module 150 , a wireless communication module 160 , an audio module 170 , a sensor module 180 , and a camera module 190 , etc.

[0040] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0041] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0042] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0043] The internal memory 120 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0044] In some embodiments, the internal memory 120 stores instructions for executing the fall protection method.

[0045] The electronic device can implement some shooting functions through the ISP, camera 130, video codec, GPU, display 140 and application processor, etc. The shooting function can be understood as a regular shooting function.

[0046] The ISP processes data fed back by the camera 130. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene.

[0047] The camera 130 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 130, where N is a positive integer greater than 1.

[0048] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency, the DSP performs a Fourier transform on the frequency energy.

[0049] Video codecs are used to compress or decompress digital video. Electronic devices may support one or more video codecs. This allows them to play or record videos in a variety of encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0050] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications in electronic devices, such as image recognition, face recognition, speech recognition, and text comprehension.

[0051] The electronic device implements its display functionality through a GPU, display screen 140, and an application processor. A GPU is a microprocessor for image processing that connects display screen 140 and the application processor. The GPU performs mathematical and geometric calculations for image rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0052] Display screen 140 is used to display images, video interfaces, and the like. Display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device can include one or N display screens 140, where N is a positive integer greater than one.

[0053] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0054] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0055] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 110 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0056] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0057] The electronic device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0058] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0059] The speaker 170A, also called a "speaker," is used to convert audio electrical signals into sound signals. The electronic device can listen to music or make hands-free calls through the speaker 170A.

[0060] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be heard by placing the receiver 170B close to the human ear.

[0061] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, and identification of sound sources, and realize directional recording function, etc.

[0062] The headphone jack 170D is used to connect a wired headphone and can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0063] In the sensor module 180, the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 140. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device including at least two parallel plates with conductive material. When force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 140, the electronic device detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0064] Touch sensor 180B, also known as a "touch-sensitive device," can be disposed on display screen 140. Touch sensor 180B and display screen 140 form a touch screen, also known as a "touch screen." Touch sensor 180B is configured to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via display screen 140. In other embodiments, touch sensor 180B can also be disposed on the surface of the electronic device, at a location different from that of display screen 140.

[0065] The acceleration sensor 180C can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity and can also be used to identify the posture of the electronic device.

[0066] The gyro sensor 180D can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 180D.

[0067] In some embodiments, the electronic device implements other shooting functions through the ISP, camera module 190, video codec, GPU, display 140, and application processor. Other shooting functions may refer to: in the camera's photo mode, automatically zooming in or out of the object centered on the object in response to user input on the object being shot; or continuously tracking moving objects in the camera's video mode, implementing focus tracking operations in response to user input on the object being shot, and automatically tracking and shooting the moving object selected by the user.

[0068] In some embodiments, the camera module 190 includes a lens sensor, an OIS IC, and a Scan IC. The camera module 190 may also include an AF IC (not shown). In some embodiments, the camera module 190 is a telephoto module.

[0069] The ISP in processor 110 also processes data fed back by the camera module 190. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the sensor. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization for image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene.

[0070] The lens of the camera module 190 includes a reflective prism, which is located in the incident light path of the sensor and is used to reflect light to the sensor. In some embodiments, the reflective prism supports rotational movement in the xz plane, exemplarily, the range of rotational movement is +-24 degrees, and also supports rotational movement in the yz plane, exemplarily, the range of rotational movement is +-7. The rotation of the reflective prism can also be understood as the rotation of the camera module lens. The rotation of the camera module lens can achieve the object being in the center of the picture during the zooming in or out process.

[0071] The Scan IC can be understood as a controller that controls the rotation of the reflective prism. For example, the Scan IC can be a chip with logic control capabilities. The Scan motor receives control commands from the Scan IC and drives the rotation of the reflective prism. In some embodiments, the Scan motor includes two motors: one that drives the rotation of the reflective prism in the xz plane and the other that drives the rotation of the reflective prism in the yz plane. The xz plane and the yz plane are two intersecting planes.

[0072] In some embodiments, a brake component can be used to lock and unlock the camera module lens position. Before the reflective prism rotates, the brake component can be unlocked. Once the reflective prism rotates, the brake component is locked, allowing the camera module lens to quickly stabilize once it reaches its position. In some embodiments, the brake component includes a brake IC and an actuator. The brake IC receives control commands from the scan IC to control the actuator's locking or unlocking. The brake IC can be understood as a controller, such as a chip with logic control capabilities.

[0073] In some embodiments, an Optical Image Stabilizer (OIS) IC (Optical Image Stabilizer) is used for image stabilization. The OIS IC can control the OIS motor for image stabilization. In some embodiments, the OIS motor does not perform image stabilization while the Scan motor is rotating the reflective prism. Therefore, the Scan IC can control the OIS motor to not operate during the Scan motor's operation and to operate after the Scan motor has finished operating. The OIS IC and OIS motor can be collectively referred to as the image stabilization component.

[0074] In some embodiments, the AF motor is used for focusing, and the OIS IC can control the AF motor for focusing after performing algorithmic calculations and compensation. The AF IC and AF motor can be referred to as a focusing component.

[0075] In some embodiments, the Scan IC and OIS IC are configured with exception interrupt pins. The exception interrupt pins of the OIS IC and Scan IC are connected to the AP. The exception interrupt pins of the OIS IC and Scan IC can be connected in parallel to the AP or individually to the AP.

[0076] In some embodiments, the Scan IC, OIS IC and the AP in the processor 110 communicate via the I2C bus. The AP can read the abnormal status codes of the Scan IC and OIS IC via the I2C bus, and can also read the interrupt signals of the Scan IC and OIS IC via the I2C bus.

[0077] In some embodiments, the brake IC is connected to the scan IC via an I2C bus, and the AP can read the abnormal status code of the brake IC via the scan IC.

[0078] In some embodiments, the AF IC is connected to the OIS IC via an I2C bus, and the AP can read the abnormal status code of the AF IC through the OIS IC.

[0079] 1 , the reflective prism in the lens of the camera module 190 can rotate. During the rotation of the reflective prism, if the electronic device accidentally falls, the reflective prism may be shaken and damaged.

[0080] Based on this, the present application provides a drop protection solution, which can lock the reflective prism when it is detected that the electronic device is in a drop state to prevent damage to the reflective prism.

[0081] The following describes how the electronic device controls the reflective prism to lock the electronic device when the electronic device is in a falling state, in conjunction with some hardware components of the electronic device shown in FIG. 1 .

[0082] In some embodiments, the accelerometer 180C and the gyroscope 180D can be connected to the Scan IC via a bus, such as a Serial Peripheral Interface (SPI) bus. The accelerometer 180C can send detection data to the Scan IC via the bus, and the gyroscope 180D can also send detection data to the Scan IC via the bus.

[0083] In some embodiments, the Scan IC can determine whether the electronic device is in a falling state based on detection data from the accelerometer sensor 180C and / or the gyroscope sensor 180D. When the Scan IC determines that the electronic device is in a falling state, it can control the reflective prism to stop rotating and lock the reflective prism using a brake component to protect the reflective prism. Furthermore, the Scan IC independently detects whether the electronic device is in a falling state, and when it determines that the electronic device is in a falling state, controls the reflective prism to stop rotating and locks the reflective prism using a brake component, thereby not occupying the electronic device's application processor (AP) resources.

[0084] In other embodiments, the Scan IC supports drop detection but does not support generating instructions to stop the reflective prism from rotating or to lock the reflective prism using a brake component. If the Scan IC determines that the electronic device has dropped, based on detection data from the acceleration sensor 180C and the gyroscope sensor 180D, it reports an interrupt signal to the AP in the processor 110. The AP can then generate a control instruction and send it to the Scan IC. In response to the control instruction, the Scan IC stops the reflective prism from rotating and also locks the reflective prism using a brake component.

[0085] In one embodiment, the Scan IC is configured with an abnormal interrupt pin, and the Scan IC can report an interrupt signal to the AP via the abnormal interrupt pin.

[0086] The way in which the Scan IC determines whether the electronic device is in a falling state through the detection data of the acceleration sensor 180C and / or the gyroscope sensor 180D can be referred to conventional technology and is not specifically limited in the embodiments of the present application.

[0087] In other embodiments, the camera module 190 of the electronic device is not equipped with a brake component. The Scan IC determines that the electronic device is in a falling state through the detection data of the acceleration sensor 180C and the gyroscope sensor 180D. After controlling the reflection prism to stop rotating, the driving motor of the reflection prism can be powered on to fix the reflection prism in a position; or, after controlling the reflection prism to stop rotating, the driving motor of the reflection prism is used to drive the reflection prism to return to the center position, or after the reflection prism returns to the center position, the driving motor of the reflection prism can be powered on to fix the reflection prism in the center position to prevent it from falling damage.

[0088] In other embodiments, the Scan IC may also control the reflective prism to stop rotating and lock the reflective prism through a brake component based on the falling state detected by the electronic device, or control the reflective prism to stop rotating and fix the reflective prism in one position by powering on the driving motor of the reflective prism, or control the reflective prism to stop rotating and pull the reflective prism back to the center position, and also fix the reflective prism in the center position by powering on the driving motor of the reflective prism.

[0089] Scan IC detects the drop status based on the electronic device as follows:

[0090] The sensor hub in processor 110 can determine whether the electronic device is in a dropped state based on detection data from acceleration sensor 180C and / or gyroscope sensor 180D. If the sensor hub determines that the electronic device is in a dropped state, it sends a message to the access point (AP) to inform the AP of the dropped state. The AP can then send a message to the Scan IC to inform the AP of the dropped state. The Scan IC can then perform the aforementioned control operations based on the message. Alternatively, the AP can generate control instructions to instruct the Scan IC to perform the aforementioned control operations.

[0091] It should be noted that the OIS IC can also detect whether the electronic device is in a falling state, and based on the electronic device being in a falling state, control the components performing the anti-shake function to stop running, thereby preventing the components performing the anti-shake function from being damaged due to the falling of the electronic device.

[0092] In some embodiments, the accelerometer 180C and the gyroscope 180D can be connected to the OIS IC via a bus, such as a serial peripheral interface (SPI) bus. The accelerometer 180C can send detection data to the OIS IC via the bus, and the gyroscope 180D can also send detection data to the OIS IC via the bus.

[0093] In some embodiments, the OIS IC can determine whether the electronic device has been dropped based on detection data from the acceleration sensor 180C and the gyroscope sensor 180D. If the OIS IC determines that the electronic device has been dropped, it can suspend anti-shake control, lock the anti-shake prism using a brake component, or employ other methods to protect the anti-shake prism.

[0094] In other embodiments, the OIS IC supports drop detection but does not generate instructions to lock the anti-shake prism via a brake component. If the OIS IC determines that the electronic device has dropped based on detection data from the accelerometer 180C and gyroscope 180D, it will report an interrupt signal to the AP in the processor 110. The AP can then generate a control instruction and send it to the OIS IC. In response to the control instruction, the OIS IC locks the anti-shake prism via a brake component.

[0095] In one embodiment, the OIS IC is configured with an abnormal interrupt pin, and the OIS IC can report an interrupt signal to the AP via the abnormal interrupt pin.

[0096] The way in which the OIS IC determines whether the electronic device is in a falling state through the detection data of the acceleration sensor 180C and the gyro sensor 180D can also be referred to conventional technology.

[0097] In other embodiments, the camera module 190 of the electronic device is not equipped with a brake component. The OIS IC determines that the electronic device is in a falling state through the detection data of the acceleration sensor 180C and the gyroscope sensor 180D. In this case, the anti-shake prism can be fixed in a position by powering on the drive motor of the anti-shake prism to prevent damage from falling.

[0098] In other embodiments, the OIS IC may also lock the anti-shake prism through a brake component based on the falling state detected by the electronic device, or fix the anti-shake prism in a position by powering on and controlling the driving motor of the anti-shake prism.

[0099] The OIS IC detects the drop status based on the electronic device in the following way:

[0100] The sensor hub in processor 110 can determine whether the electronic device has been dropped based on detection data from accelerometer 180C and gyroscope 180D. If the sensor hub determines that the electronic device has been dropped, it sends a message to the access point (AP) to notify the AP of the drop. The AP can then send a message to the OIS IC to notify the AP of the drop. The OIS IC can then perform the aforementioned control operations based on the message. Alternatively, the AP can generate control instructions to instruct the OIS IC to perform the aforementioned control operations.

[0101] The electronic device also runs an operating system on top of the hardware components shown in Figure 1. For example operating system, operating system, Operating system, etc. Applications such as camera applications can be installed and run on the operating system.

[0102] FIG2 is a schematic diagram of the software structure of the electronic device provided in an embodiment of the present application.

[0103] The layered architecture divides the operating system of an electronic device into several layers, each with a clear role and division of labor. Layers communicate with each other via software interfaces. In some embodiments, the operating system of the electronic device is the Android system. The Android system can be divided into five layers: from top to bottom: the application (APP) layer, the application framework layer (FWK), the system library, the hardware abstraction layer (HAL), and the kernel layer.

[0104] The application layer may include a series of application packages. As shown in Figure 2, an application package may include applications such as camera and call.

[0105] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0106] As shown in FIG2 , the application framework layer may include window management services, content providers, telephony managers, view systems, resource managers, and camera services.

[0107] The window management service is used to manage window programs. The window management service can implement window addition, deletion, display and hiding controls, etc. The content provider is used to store and obtain data and make this data accessible to applications. The data can include video, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including answering, hanging up, etc.). The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0108] In some embodiments, the camera service may also be referred to as a camera framework, which is used to receive recording requests, image capture requests, and other requests from the camera application, while maintaining the business logic of the internal flow of recording requests, image capture requests, and other requests, and sending the final result of the request to the camera application.

[0109] The Android Runtime includes core libraries and a virtual machine. The Android Runtime is responsible for scheduling and management of the Android system. In some embodiments of the present application, an application cold start will be executed within the Android Runtime. The Android Runtime can then obtain the application's optimization file status parameters. The Android Runtime can then use the optimization file status parameters to determine whether the optimization file has become outdated due to a system upgrade and return the determination result to the application management module.

[0110] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0111] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0112] The system library may include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0113] The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as static image files. The media library supports a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D drawing.

[0114] The hardware abstraction layer is a software located between the operating system kernel and the hardware circuit. It is usually used to abstract the hardware to enable interaction between the operating system and the hardware circuit at the logical layer.

[0115] In some embodiments, the hardware abstraction layer includes a camera Hal, which includes an image link Pipeline and CamxHal3. The camera Hal is mainly responsible for the construction, linking, interactive control, etc. of all modules in the image link. CamxHal3 is used to interact with the upper-layer module to open or close the camera Hal, and to transmit data to the camera Hal.

[0116] In some embodiments, the image link pipeline includes: a module control module, an ISP control module and an algorithm module; the module module is responsible for handling camera module management, and the module module includes a sensor control module, a scan control module, an OIS control module, and an actuator control module.

[0117] In some embodiments, the sensor control module is used to connect to the image sensor. The sensor control module can control the operation of the sensor through the sensor driver and sensor firmware. The sensor control module is also called a sensor node.

[0118] In some embodiments, the Scan control module is used to connect to the Scan motor to realize the rotation of the reflective prism. The Scan control module can control the operation of the Scan motor through the Scan driver and Scan Firmware.

[0119] In some embodiments, the OIS control module is used to implement anti-shake processing during the rotation of the camera module. The OIS control module can control the operation of the OIS motor through the OIS driver and OIS firmware. In other embodiments, the Scan control module can control the operation of the OIS motor through the Scan driver, Scan firmware, and OIS firmware.

[0120] In some embodiments, the actuator control module is used for AF focusing during the rotation of the camera module. The actuator control module can control the operation of the AF motor through AF driver, OIS firmware and AF firmware.

[0121] The ISP control module is used to receive image data sent by the sensor, process the image data, and then send it to the algorithm module. In some embodiments, the sensor refers to the sensor of the camera module 190.

[0122] The algorithm module is used to receive image data sent by the ISP control module, analyze the image data, obtain the operation strategy of the lower-level modules, and feed it back to the modules in the module control module. In some embodiments, the algorithm module obtains the target position of the reflective prism and feeds it back to the Scan control module.

[0123] The kernel layer is the layer between hardware and software. It receives commands from the control layer and translates them into actual device operation instructions. These include power-on and power-off control, and the generation and issuance of device mode register instructions. Register control commands from the driver layer are passed to the firmware layer via the device bus. The kernel layer includes at least the sensor driver, scan driver, OIS driver, and AF driver.

[0124] The firmware layer refers to the software system integrated into the components of the camera module, including sensor firmware, scan firmware, OIS firmware, AF firmware, and brake firmware. In some embodiments, the firmware layer may not include brake firmware.

[0125] In some embodiments, Sensor Firmware is a software system running inside the Sensor, responsible for sensor exposure and image data output. Scan Firmware is a software system running inside the Scan IC, used to control the rotation of the camera module lens, and is responsible for converting the code sent down from the upper layer into a control signal, thereby driving the camera module lens to rotate and achieve the focus tracking and framing target. Brake Firmware is a software system running inside the brake IC, used to lock the position of the camera module, so that the camera module can quickly stabilize after reaching the target position. OIS Firmware is a software system running inside the OIS IC, used for anti-shake. AF Firmware is a software system running inside the AF IC, used for focus tracking, responsible for converting the target position sent down from the upper layer into motor current, and driving the AF motor to focus.

[0126] It should be noted that although the embodiments of the present application are based on The system is used as an example, but the basic principles are also applicable to Electronic devices with other operating systems.

[0127] As mentioned above: During the process of the camera module 190 taking images or tracking focus recording, if the electronic device determines that it is in a falling state, it can control the reflective prism of the camera module 190 lens to stop rotating, and can also lock the reflective prism through the brake component to achieve the purpose of protecting the reflective prism.

[0128] After the reflective prism stops rotating, the electronic device is released from the falling state, that is, it is no longer in the falling state. The electronic device needs to control the reflective prism to resume rotating to support the camera module 190 to capture images or track focus recording.

[0129] There are two ways for the electronic device to control the reflection prism to resume rotation: manual reset by the user and automatic reset by the Scan IC of the electronic device. The following describes these two ways respectively in conjunction with the module of the electronic device shown in FIG. 2 .

[0130] The following content introduces two methods of manual reset by the user and automatic reset by the Scan IC of the electronic device. The example in which the Scan IC determines that the electronic device is in a fallen state and generates a control instruction to control the reflective prism to stop rotating and control the brake component to lock the reflective prism is used for explanation.

[0131] FIG3 shows the overall operating sequence of the modules of the electronic device corresponding to the user manual reset mode.

[0132] As shown in Figure 3, the accelerometer and gyroscope start running and obtain detection data. The accelerometer performs 1-1. Reporting detection data to the Scan IC firmware Scan Firmware, and the gyroscope performs 1-2. Reporting detection data to the Scan IC firmware Scan Firmware.

[0133] Based on the detection data from the acceleration sensor and the gyroscope sensor, the Scan Firmware executes 2-1 to detect that the electronic device is in a falling state, executes 2-2 to control the reflection prism to stop rotating, executes 2-3 to lock the reflection prism through the brake firmware, and executes 2-4 to report an interrupt signal to the abnormality detection module to inform it that the electronic device has an abnormality.

[0134] After the exception detection module receives the interrupt signal reported by the Scan Firmware, it can, based on the interrupt signal, call back the exception dump interface of each driver module to obtain the exception status code of each driver module. For example, Figure 3 shows the exception detection model executing 3-1. Calling back the exception dump interface registered by the Scan driver to obtain the Scan exception status code; the exception detection model executing 3-2. Calling back the exception dump interface registered by the OIS driver to obtain the OIS exception status code; and the exception detection model executing 3-1. Calling back the exception dump interface registered by the AF driver to obtain the AF exception status code. In some embodiments, the exception detection module can also call back the exception dump interface of the Scan driver based on the interrupt signal to obtain the exception status code of the Scan driver.

[0135] It should be noted that multiple drivers in the kernel layer, such as the Scan driver, OIS driver, and AF driver, can register the following two interfaces with the anomaly detection module when the kernel driver is loaded:

[0136] The exception information parsing and dump interface, referred to as the exception dump interface, is used to call back each module to complete exception analysis when an exception occurs and provide module diagnostic results.

[0137] Exception reset interface, used to support exception processing and actions within the module.

[0138] Figure 4 shows a flowchart of how the Scan driver, OIS driver, and AF driver register two interfaces with the anomaly detection module. As shown in Figure 4, when the Scan driver, OIS driver, and AF driver are loaded, they execute the following steps S401 to S403; wherein:

[0139] S401. The Scan driver registers an exception dump interface and an exception reset interface with the exception detection module.

[0140] S402: The OIS driver registers an exception dump interface and an exception reset interface with the exception detection module.

[0141] S403: The AF driver registers the exception dump interface and the exception reset interface with the exception detection module.

[0142] It's also important to note that Scan Firmware is configured with an anomaly check process, which it uses to detect anomalies during its operation. Scan Firmware performs an anomaly check and, upon detecting anomalies, encodes them. Different anomalies can use different codes.

[0143] In some embodiments, Scan Firmware can encode the electronic device being in a dropped state to obtain an abnormal state code, which can uniquely indicate that the electronic device is in a dropped state. The abnormal state code generally refers to abnormal state coding information, and is referred to as the abnormal state code in the following content.

[0144] The Scan driver reads the exception status code from the Scan Firmware. The Scan Firmware returns the exception status code to the exception detection module.

[0145] Based on the abnormal status code of the driver module, the abnormality detection module executes 4-1. Determine that the electronic device is in a falling state, and executes 4-2. Notify the Scan driver to suspend the issuance of all instructions, which usually refers to the control instructions issued to the Scan Firmware and control the reflective prism.

[0146] The anomaly detection module also executes 5. reporting an event to the CRM to indicate that the electronic device has fallen, which is an abnormal state. The CRM executes 6. reporting the event to the image link Pipeline through V4L2 and CSL.

[0147] The image link pipeline further executes step 7. Sending abnormal status information to the algorithm module to inform it that the electronic device is in a dropped state. In some embodiments, after receiving the abnormal status information, the algorithm module may also stop calculating the target position of the reflective prism based on the abnormal status information.

[0148] The algorithm module executes 8. Writing the abnormal status information to the Metadata Pool.

[0149] The Metadata Pool may execute 9. Report abnormal status information to the camera application. After receiving the abnormal status information, the camera application may execute 10. Display the abnormal status information and a prompt message. The abnormal status information may be used to notify the user that the electronic device has fallen. The prompt message may include a user-operable control that may be used to control an instruction to release the falling state.

[0150] The camera application receives the user's operation and executes 11. In response to the user's operation, it sends a drop cancellation instruction to the Metadata Pool. The Metadata Pool executes 12. It sends a drop cancellation instruction to the Scan control module.

[0151] The Scan control module executes step 13. Sending a drop release command to the Scan driver via the CSL and V4L2 buses. After receiving the drop release command, the Scan driver executes step 14. Sending a drop release command to the Scan firmware.

[0152] Scan Firmware responds to the drop release instruction and executes 15. Controls the brake Firmware to unlock the reflective prism.

[0153] In some embodiments, after the Scan driver receives the drop release instruction, the Scan driver may resume sending instructions to the Scan Firmware to control the rotation of the reflective prism. The Scan Firmware may also control the rotation of the reflective prism after the reflective prism is unlocked based on the instruction.

[0154] It should be noted that in some embodiments, the target position of the reflective prism obtained by the algorithm module, i.e., the code, can be written into the metadata pool so that other modules can monitor data changes in the metadata pool. The Scan control module monitors the metadata pool. The Scan control module detects the code written into the metadata pool, verifies the code, and then issues a control command to the Scan driver. The control command carries the code. After receiving the control command, the Scan driver can generate an instruction to control the rotation of the reflective prism based on the code.

[0155] FIG5 shows a signaling interaction diagram of the fall detection solution provided in an embodiment of the present application.

[0156] As shown in FIG5 , the fall detection method provided in the embodiment of the present application includes:

[0157] S501: The sensor reports detection data to the Scan Firmware.

[0158] The sensor may refer to an acceleration sensor 180C and / or a gyro sensor 180D of the electronic device.

[0159] In some embodiments, the sensor may be configured in a normally-on mode. After the electronic device is turned on, the sensor starts running and performs continuous or periodic detection to obtain detection data.

[0160] In other embodiments, the electronic device activates a camera application to capture images or focus-track videos, and the camera module 190 is simultaneously activated and operated. The sensor can be activated and operated to continuously or periodically detect and obtain detection data.

[0161] In some embodiments, the sensor may be connected to the Scan IC via a bus, and based on this, the detection data of the sensor may be reported to the Scan Firmware via the bus.

[0162] S502 : Scan Firmware determines that the electronic device is in a dropped state based on the detection data.

[0163] Scan Firmware receives detection data from the sensor and can determine whether the electronic device is in a falling state based on the detection data. The way in which Scan Firmware determines whether the electronic device is in a falling state based on the detection data can be referred to conventional technology, and the embodiments of the present application are not limited thereto.

[0164] It is understandable that the falling state may refer to the electronic device falling from the air.

[0165] If the Scan Firmware determines based on the detection data that the electronic device is in a dropped state, then step S503 is executed. If the Scan Firmware determines based on the detection data that the electronic device is in a dropped state, then the process may be exited. The Scan Firmware waits for new detection data to be received and then determines whether the electronic device is in a dropped state based on the new detection data.

[0166] S503, Scan Firmware controls the reflection prism to stop rotating.

[0167] As mentioned above, when the camera module 190 is operating to capture images or focus-track videos, the reflective prism is driven to rotate. The Scan Firmware determines that the electronic device is in a dropped state and controls the reflective prism to stop rotating in order to prevent the reflective prism from being damaged.

[0168] S504. Scan Firmware controls brake locking through brake Firmware.

[0169] Brake Firmware controls the brake components to perform brake locking.

[0170] In some embodiments, the camera module 190 of the electronic device is not equipped with a brake component, and the Scan Firmware does not execute step S504. The Scan Firmware may control the driving motor of the reflective prism to fix the reflective prism in a certain position; alternatively, the driving motor of the reflective prism may be used to drive the reflective prism back to the center position; or, after the reflective prism returns to the center position, the driving motor of the reflective prism may be controlled to control the driving motor to fix the reflective prism in the center position to prevent it from falling and being damaged.

[0171] S505. Scan Firmware reports an interrupt signal to the abnormality detection module.

[0172] Step S505 is not limited to the execution order shown in FIG. 5 . Scan Firmware determines based on the detection data that the electronic device is in a falling state, and may report an interrupt signal to the abnormality detection module.

[0173] S506: The abnormality detection module reads the abnormality status code from the Scan driver.

[0174] The abnormality detection module receives the interrupt signal and can determine that an abnormal state occurs in the electronic device based on the interrupt signal.

[0175] In some embodiments, the Scan Firmware executes a module diagnostic abnormality process to perform an abnormality check. After obtaining an abnormal phenomenon, the abnormal phenomenon may be encoded, and different abnormal phenomena may use different encodings. Based on the detection data of the sensor, the Scan Firmware determines that the electronic device is in a dropped state. The Scan Firmware then encodes the dropped state to obtain an abnormal state code for the dropped state, referred to as the abnormal state code. The Scan Firmware returns the abnormal state code to the abnormality detection module, so that the abnormality detection module can read the abnormal state code from the Scan driver.

[0176] In other embodiments, the drop detection module, which often cannot identify the abnormal component using an interrupt signal, can trigger multiple driver modules to execute the module abnormality diagnosis process. In some embodiments, the abnormality detection module calls back the abnormality dump interface of each driver module to obtain the abnormal status code of each driver module. The method for multiple driver modules to execute the module abnormality diagnosis process is similar to the method for scanning firmware described above and will not be repeated here.

[0177] S507: The abnormality detection module determines that the electronic device is in a dropped state based on the abnormal state code.

[0178] The abnormal state code may indicate the abnormal state type of the electronic device. Based on this, the abnormality detection module may determine that the electronic device is in a dropped state based on the abnormal state code.

[0179] S508 , the Scan driver suspends issuing instructions for controlling the rotation of the reflective prism based on the abnormal state code.

[0180] As mentioned above, the Scan driver receives a control instruction sent by the Scan control module. The control instruction carries a code. The Scan driver can generate an instruction to control the rotation of the reflective prism based on the code.

[0181] The electronic device is in a dropped state and the reflective prism cannot rotate. Therefore, the Scan driver obtains an abnormal state code and can suspend issuing instructions to control the rotation of the reflective prism.

[0182] In some embodiments, the Scan driver may cache the code carried in the control instructions issued by the Scan control module.

[0183] S509: The anomaly detection module reports the event to the image link Pipeline.

[0184] In some embodiments, the anomaly detection module reports an event to the camera request manager (CRM), indicating that the electronic device is in a dropped state. The CRM receives the event and reports it to the image link pipeline via the V4L2 and CSL buses.

[0185] The Camera Request Manager (CRM) is a module that manages camera application requests. Requests issued by modules in the Hardware Abstraction Layer (HAL) are first transmitted to the CRM, which matches them according to the sensor's frame output timing and then issues configurations to the corresponding modules using an N+1 or N+2 mechanism. Sensor frame data interruptions can also be reported to the HAL module via the CRM.

[0186] In some embodiments, the anomaly detection module may report events to the image link pipeline, or report events to the image link pipeline through other modules.

[0187] S510: The image link Pipeline sends abnormal status information to the algorithm module based on the event.

[0188] The image link pipeline sends abnormal status information to the algorithm module based on events to inform the algorithm module that the electronic device is in a falling state and the reflective prism has stopped rotating.

[0189] S511: The algorithm module writes abnormal status information to the Metadata Pool.

[0190] The algorithm module receives the abnormal status information and can write the abnormal status information into the Metadata Pool.

[0191] In some embodiments, after receiving abnormal status information, the algorithm module may suspend calculating the target position of the reflective prism.

[0192] S512. The Metadata Pool reports abnormal status information to the camera application.

[0193] The data written to the Metadata Pool can be reported to the camera application. Based on this, after the algorithm module writes abnormal status information to the Metadata Pool, the Metadata Pool reports the abnormal status information to the camera application.

[0194] S513: The camera application displays abnormal status information and prompt information.

[0195] The camera application receives the abnormal state information and can determine that the electronic device is in a falling state and the reflective prism of the camera module 190 has stopped rotating. Based on this, the camera application can display the abnormal state information and prompt information.

[0196] The abnormal status information may indicate that the electronic device is in a falling state and the reflective prism of the camera module 190 has stopped rotating; the prompt information may be used to prompt the user whether to release the falling state, and the prompt information may include a user-operable control that can be used to generate an instruction to control the release of the falling state.

[0197] In some embodiments, the camera application may display abnormal status information and prompt information in the form of pop-up windows.

[0198] S514: The camera application receives the user operation and, in response to the user operation, sends a drop state release instruction to the Metadata Pool.

[0199] The user inputs an operation to release the drop state. The camera application receives the user's operation and, in response to the user's operation, sends a drop state release instruction to the Metadata Pool.

[0200] It should be noted that the user input operation to release the falling state means that the electronic device has been held by the user and is not in the falling state. Therefore, the camera application receives the user operation and, in response to the user operation, sends a falling state release instruction to the Metadata Pool.

[0201] S515: The Metadata Pool sends a drop state release instruction to the Scan control module.

[0202] The Metadata Pool is written with the instruction to release the drop state. The Scan control module can monitor the Metadata Pool writing the instruction to release the drop state.

[0203] S516. The Scan control module sends a drop state release instruction to the Scan driver.

[0204] The Scan control module monitors the Metadata Pool and sends a command to release the drop status to the Scan driver.

[0205] S517: The Scan driver sends a drop state release instruction to the Scan Firmware.

[0206] The Scan driver receives the drop state release instruction and can send the drop state release instruction to the Scan Firmware.

[0207] In some embodiments, after the Scan driver receives the drop release instruction, the Scan driver may resume sending instructions to the Scan Firmware to control the rotation of the reflective prism. The Scan Firmware may also control the rotation of the reflective prism after the reflective prism is unlocked based on the instruction.

[0208] S518, Scan Firmware controls brake unlocking through brake firmware.

[0209] After receiving the drop release instruction, the Scan driver can send the drop release instruction to the Scan Firmware. In response to the drop release instruction, the Scan Firmware controls the brake firmware to unlock the reflective prism.

[0210] In some embodiments, after the camera application sends a drop state release instruction to the metadata pool, the algorithm module can also monitor the drop state release instruction and resume calculating the target position of the reflective prism. In some embodiments, the drop state release instruction is in metadata format.

[0211] FIG6 shows the overall operating sequence of the modules of the electronic device corresponding to the automatic reset of the Scan IC of the electronic device.

[0212] The automatic reset of the Scan IC of the electronic device can be understood as the electronic device self-detecting whether it is in a non-drop state, and if it is determined to be in a non-drop state, it automatically unlocks the reflective prism and resumes the rotation of the reflective prism without user input. This is described in detail below with reference to FIG6.

[0213] As shown in Figure 6, the accelerometer and gyroscope start running and obtain detection data. The accelerometer performs 1-1. Reporting the detection data to the Scan IC firmware Scan Firmware, and the gyroscope performs 1-2. Reporting the detection data to the Scan IC firmware Scan Firmware.

[0214] Based on the detection data from the acceleration sensor and the gyroscope sensor, the Scan Firmware executes 2-1 to detect that the electronic device is in a falling state, executes 2-2 to control the reflection prism to stop rotating, executes 2-3 to lock the reflection prism through the brake firmware, and executes 2-4 to report an interrupt signal to the abnormality detection module to inform it that the electronic device has an abnormality.

[0215] After the exception detection module receives the interrupt signal reported by the Scan Firmware, it can, based on the interrupt signal, call back the exception dump interface of each driver module to obtain the exception status code of each driver module. For example, Figure 3 shows the exception detection model executing 3-1. Calling back the exception dump interface registered by the Scan driver to obtain the Scan exception status code; the exception detection model executing 3-2. Calling back the exception dump interface registered by the OIS driver to obtain the OIS exception status code; and the exception detection model executing 3-1. Calling back the exception dump interface registered by the AF driver to obtain the AF exception status code. In some embodiments, the exception detection module can also call back the exception dump interface of the Scan driver based on the interrupt signal to obtain the exception status code of the Scan driver.

[0216] It should be noted that multiple drivers in the kernel layer, such as the Scan driver, OIS driver, and AF driver, can register the exception dump interface and the exception reset interface to the exception detection module when the kernel driver is loaded. Please refer to the embodiment corresponding to Figure 5 above, and no further details will be given here.

[0217] Based on the abnormal status code of the driver module, the abnormality detection module executes 4-1. Determine that the electronic device is in a falling state, and executes 4-2. Notify the Scan driver to suspend the issuance of all instructions, which usually refers to the control instructions issued to the Scan Firmware and control the reflective prism.

[0218] The abnormality detection module further executes 5. reporting a first event to the CRM to indicate that the electronic device is in an abnormal state of falling, and the CRM executes 6. reporting the first event to the image link Pipeline through V4L2 and CSL.

[0219] The image link pipeline further executes step 7. Sending abnormal status information to the algorithm module to inform it that the electronic device is in a dropped state. In some embodiments, after receiving the abnormal status information, the algorithm module may also stop calculating the rotation strategy of the reflective prism based on the abnormal status information.

[0220] The accelerometer and gyroscope periodically report detection data to the Scan Firmware, which continuously monitors the electronic device's status based on this detection data. If the Scan Firmware determines that the electronic device is no longer in a dropped state, executing step 8, unlocking the reflective prism using the brake firmware, is successful.

[0221] In some embodiments, the Scan driver may start a timer and periodically poll the Scan Firmware via a bus such as I2C to execute 9. Check whether the drop state of the electronic device is released. If the Scan driver determines that the drop state of the electronic device is released, it may send a code to the Scan Firmware.

[0222] As previously mentioned, in some embodiments, the target position of the reflective prism, i.e., the code, obtained by the algorithm module can be written into the metadata pool so that other modules can monitor data changes in the data pool. The Scan control module monitors the metadata pool. The Scan control module detects the code written into the metadata pool, verifies the code, and then issues a control command to the Scan driver. The control command carries the code. After receiving the control command, the Scan driver can generate an instruction to control the rotation of the reflective prism based on the code and issue it to the Scan firmware.

[0223] If the Scan driver determines that the electronic device has been released from its falling state, it can send a code to the Scan Firmware. It can be understood that if the Scan driver determines that the electronic device has been released from its falling state, it will generate an instruction to control the rotation of the reflective prism based on the code and send it to the Scan Firmware.

[0224] The Scan driver may also execute 10. Notify the abnormality detection module that the falling state is released.

[0225] The anomaly detection module further executes 11. reporting a second event to the CRM to indicate that the falling state of the electronic device is released, and the CRM executes 12. reporting the second event to the image link Pipeline through V4L2 and CSL.

[0226] The image link pipeline further executes step 13. Sending abnormal state release information to the algorithm module to inform it that the electronic device is no longer in a dropped state. In some embodiments, after receiving the abnormal state information, the algorithm module may also restore the target position of the reflective prism based on the abnormal state release information.

[0227] The algorithm module calculates the target position of the reflective prism, i.e., the code, and executes step 14. This code is written to the metadata pool. The Scan control module executes step 15. This monitors the metadata pool. If the Scan control module detects the code being written to the metadata pool, it executes step 16. After verifying the code, it issues a control command to the Scan driver, which includes the code. After receiving the control command, the Scan driver generates instructions for controlling the reflective prism's rotation based on the code and issues them to the Scan firmware.

[0228] FIG7 shows a signaling interaction diagram of a fall detection solution provided in another embodiment of the present application.

[0229] As shown in FIG7 , the fall detection method provided in the embodiment of the present application includes:

[0230] S701: The sensor reports detection data to the Scan Firmware.

[0231] S702 : Scan Firmware determines that the electronic device is in a dropped state based on the detection data.

[0232] S703, Scan Firmware controls the reflection prism to stop rotating.

[0233] In an embodiment of the present application, the Scan Firmware determines that the electronic device is in a falling state based on the detection data, and then controls the reflective prism to stop rotating, thereby avoiding damage to the reflective prism.

[0234] S704, Scan Firmware controls brake lock through brake firmware.

[0235] S707. Scan Firmware reports an interrupt signal to the abnormality detection module.

[0236] S706: The abnormality detection module reads the abnormality status code from the Scan driver.

[0237] S707: The abnormality detection module determines that the electronic device is in a dropped state based on the abnormal state code.

[0238] S708. The Scan driver suspends issuing instructions for controlling the rotation of the reflective prism based on the abnormal state code.

[0239] The contents of steps S701 to S708 may refer to the contents of steps S501 to S508 in the aforementioned embodiment, and will not be repeated here.

[0240] S709: The anomaly detection module reports the first event to the image link Pipeline.

[0241] In some embodiments, the anomaly detection module reports a first event to a camera request manager (CRM), where the first event indicates that the electronic device is in a falling state. Upon receiving the first event, the CRM reports the first event to the image link pipeline via the V4L2 and CSL buses.

[0242] In some embodiments, the anomaly detection module may report the event to the image link pipeline, or report the first event to the image link pipeline through other modules.

[0243] S710: The image link Pipeline sends abnormal status information to the algorithm module based on the first event.

[0244] Based on the first event, the image link Pipeline sends abnormal status information to the algorithm module to inform the algorithm module that the electronic device is in a falling state and the reflective prism stops rotating.

[0245] In some embodiments, after receiving the abnormal state information, the algorithm module may also stop calculating the target position of the reflective prism based on the abnormal state information.

[0246] S711. Scan Firmware determines, based on the detection data, that the electronic device is not in a dropped state.

[0247] Sensors operate continuously or periodically to generate detection data. This data is reported to the Scan Firmware. Based on this data, the Scan Firmware continuously determines the status of the electronic device. When the electronic device is no longer dropped, the Scan Firmware can determine based on this data that the device is no longer dropped.

[0248] S712, Scan Firmware controls brake unlocking through brake firmware.

[0249] Scan Firmware controls the brake firmware to unlock the reflective prism.

[0250] S713: The Scan driver determines that the electronic device is not in a dropped state, and sends a first code to the Scan Firmware.

[0251] In some embodiments, the Scan driver may start a timer and periodically poll the Scan Firmware via a bus such as I2C to check whether the drop state of the electronic device has been released. If the Scan driver determines that the drop state of the electronic device has been released, it may send a first code to the Scan Firmware.

[0252] The first code can be understood as the code cached by the Scan driver after it pauses issuing instructions to control the rotation of the reflective prism.

[0253] If the Scan driver determines that the falling state of the electronic device is released, it can send a first code to the Scan Firmware. It can be understood that if the Scan driver determines that the falling state of the electronic device is released, it will generate an instruction to control the rotation of the reflective prism based on the first code and send it to the Scan Firmware.

[0254] S714. Scan Firmware controls the rotation of the reflective prism based on the first code.

[0255] The first code indicates the target position of the reflecting prism. Therefore, the Scan Firmware controls the rotation of the reflecting prism based on the first code.

[0256] S715. The Scan driver sends a message to the abnormality detection module to notify that the falling state is released.

[0257] If the Scan driver determines that the drop state of the electronic device has been released, it can send a message to the abnormality detection module to notify that the drop state has been released.

[0258] Step S713 and step S715 are not limited to the execution order shown in FIG. 7 , and can be executed in parallel, or step S715 can be executed first and then step S713 .

[0259] S716: The anomaly detection module reports the second event to the image link Pipeline.

[0260] The abnormality detection module receives the message that the electronic device is no longer in the dropped state and can report the second event to the image link pipeline. The first event indicates that the electronic device is no longer in the dropped state.

[0261] In some embodiments, the anomaly detection module reports the second event to a camera request manager (CRM). Upon receiving the second event, the CRM reports the second event to the image link pipeline via the V4L2 and CSL buses.

[0262] S717: The image link Pipeline sends abnormal state release information to the algorithm module based on the second event.

[0263] Based on the second event, the image link Pipeline sends abnormal state release information to the algorithm module to inform the algorithm module that the electronic device is not in a falling state.

[0264] S718. The algorithm module writes the second code into the Metadata Pool.

[0265] After receiving the abnormal state release information, the algorithm module can resume calculating the target position of the reflective prism, that is, the code. The algorithm module calculates the code and writes it to the metadata pool. To distinguish it from the code mentioned in step S713, the newly calculated code by the algorithm module is called the second code.

[0266] S719, Scan control module monitors data in the Metadata Pool.

[0267] S720: The Scan control module monitors that the Metadata Pool writes the second code, verifies the second code, and then issues a control command to the Scan driver, where the control command carries the second code.

[0268] In some embodiments, before the Scan control module sends the second code to the Scan driver, it verifies the second code, which can be understood as verifying whether the target position of the reflective prism indicated by the second code is a legal position. If the verification passes, the second code is sent to the Scan driver.

[0269] In some embodiments, after the algorithm module obtains the target position of the reflective prism, it may also send it directly to the Scan control module.

[0270] S721. The Scan driver sends a control instruction to the Scan Firmware based on the second code.

[0271] The Scan driver sends a control instruction to the Scan Firmware, where the control instruction carries the second code.

[0272] S722. Scan Firmware controls the rotation of the reflective prism based on the second code.

[0273] The second code indicates the target position of the reflecting prism. Therefore, the Scan Firmware controls the rotation of the reflecting prism based on the second code.

[0274] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or processor executes one or more steps in any of the above methods.

[0275] The computer readable storage medium may be a non-transitory computer readable storage medium, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0276] Another embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute one or more steps in any of the above methods.

Claims

1. A fall protection method, characterized in that: Applied to an electronic device, the electronic device includes a camera module, the camera module includes a rotatable reflective prism, and the fall protection method includes: receiving and responding to a first operation of a user, controlling the camera module to capture an image or to focus and record a video, wherein the reflective prism is in a rotating state during the process of the camera module capturing an image or focusing and recording a video; When the reflective prism is in a rotating state, detecting the operating state of the electronic device; When it is detected that the electronic device is in a falling state, the reflective prism is controlled to stop rotating.

2. The fall protection method according to claim 1, characterized in that: After the controlling the reflective prism to stop rotating, the method further includes: locking the position of the reflective prism.

3. The fall protection method according to claim 2, characterized in that: The step of locking the position of the reflecting prism comprises: The position of the reflecting prism is locked by a brake component, or the position of the reflecting prism is locked by a driving component of the reflecting prism.

4. The fall protection method according to any one of claims 1 to 3, characterized in that: The electronic device comprises a first controller, During the process of the camera module shooting images or tracking focus recording, the reflective prism is in a rotating state, including: during the process of the camera module shooting images or tracking focus recording, the first controller controls the reflective prism to be in a rotating state; When the reflective prism is in a rotating state, detecting the operating state of the electronic device; when detecting that the electronic device is in a falling state, controlling the reflective prism to stop rotating, comprises: The first controller detects the operating state of the electronic device when the reflective prism is in a rotating state; and controls the reflective prism to stop rotating when it is detected that the electronic device is in a falling state.

5. The fall protection method according to claim 4, characterized in that: The electronic device further includes an acceleration sensor and / or a gyroscope sensor, and the first controller detects the operating state of the electronic device when the reflective prism is in a rotating state, including: When the reflective prism is in a rotating state, the first controller detects the operating state of the electronic device based on detection data of the acceleration sensor and / or the gyroscope sensor.

6. The fall protection method according to claim 4, characterized in that: The first controller detects the operating state of the electronic device when the reflective prism is in a rotating state, including: When the reflective prism is in a rotating state, the first controller detects the operating state of the electronic device based on a message from an application processor of the electronic device, where the message from the application processor is used to indicate whether the electronic device is in a falling state.

7. The fall protection method according to any one of claims 1 to 6, characterized in that: When it is detected that the electronic device is in a falling state, the method further includes: Acquiring abnormal state coding information of a driving module of the reflective prism; Determining that the electronic device is in a falling state based on the abnormal state coding information of the driving module of the reflective prism; Stop to obtain the target position of the reflecting prism.

8. The fall protection method according to claim 7, characterized in that: The step of obtaining the abnormal state code of the driving module of the reflective prism includes: The abnormal dump interface of the driving module of the reflecting prism is called to obtain the abnormal state coding information of the driving module of the reflecting prism.

9. The fall protection method according to any one of claims 1 to 8, characterized in that: After controlling the reflection prism to stop rotating, the method further includes: Display a first interface, wherein the first interface includes a first control; receiving a second operation of the user on the first control; In response to the second operation, a target position of the reflecting prism is determined, and the reflecting prism is controlled to rotate to the target position.

10. The fall protection method according to any one of claims 1 to 8, characterized in that: After controlling the reflection prism to stop rotating, the method further includes: Detecting that the electronic device is in a non-falling state based on detection data of an acceleration sensor and / or a gyroscope sensor; determining a target position of the reflecting prism; The reflecting prism is controlled to rotate to the target position.

11. The fall protection method according to claim 9 or 10, characterized in that: Before controlling the reflective prism to rotate to the target position, the method further includes: Unlock the position of the reflecting prism.

12. The fall protection method according to claim 9 or 10, characterized in that: Before determining the target position of the reflecting prism, the method further includes: The target position of the reflecting prism is recovered.

13. An electronic device, characterized in that: include: One or more processors, memories, camera modules and display screens; the camera modules include a rotatable reflective prism; The memory, the camera module and the display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the fall protection method as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the fall protection method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the fall protection method according to any one of claims 1 to 12.

Images