A dToF device initialization method and electronic device
Through the interaction between the hardware abstraction layer and the kernel layer, the dToF device is detected and re-initialized, which solves the problem of firmware loss caused by power-down abnormalities, ensures that the dToF device returns to its working state before the electronic device is turned on, and improves the effect of autofocus.
Patent Information
- Application Number
- CN202310610474.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-05-26
AI Technical Summary
After the electronic device is turned on, the power failure is abnormal due to the flash fragment connection, resulting in loss of firmware information and inability to resume work, affecting the camera's autofocus effect.
Through the interaction between the hardware abstraction layer and the kernel layer, the power-down exception of the dToF device is detected and initialized, and the kernel layer reinitializes the dToF device to ensure that its working state is restored before the next power-on.
After the dToF device is powered down and reconnected, it can continue to provide the function of measuring depth data, improve user experience, solve the problem of firmware loss caused by powered down abnormalities, and improve the effect of autofocus.
Smart Images

Figure CN117707627B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of sensor technology, and in particular to a dToF device initialization method and electronic device. Background Art
[0002] The auto focus (AF) function is one of the functions of the camera of an electronic device. When a user uses an electronic device to take a photo, if the electronic device is equipped with a camera with an auto focus function, the camera can automatically focus on the object within the framing range to achieve clear imaging, which can enhance the user experience. The specific implementation steps of auto focus are: obtaining the object distance of the object, moving the lens according to the object distance, so that the focus is located on the image sensor of the camera. In practical applications, a dToF device can be configured for the camera, and the object distance can be obtained using the dToF device.
[0003] The dToF device can emit light pulses. After the light pulses are irradiated on the object, the surface of the object will reflect the light pulses. At this time, the light receiving unit of the dToF device can receive the reflected light pulses. The time interval between the reflected light pulse and the emitted light pulse can be obtained by measurement, and this time interval is the flight time of light. The object distance can be obtained by calculation based on the flight time of light and the speed of light. In order for the dToF device to realize the above functions, the dToF device needs to be configured based on the dToF firmware when the electronic device is turned on.
[0004] However, since the connection between the dToF device and the electronic device is generally a spring connection, when the electronic device falls or encounters problems such as problems, the dToF device spring is easily disconnected, causing the dToF device to lose power after the electronic device is turned on. In this case, the configuration information saved in the dToF device will be lost. Therefore, even if the spring can be reconnected after disconnection, the dToF device cannot provide the corresponding function, which ultimately leads to a poor autofocus effect of the camera. Summary of the invention
[0005] The embodiments of the present application provide a dToF device initialization method and an electronic device to solve the problem that the dToF device cannot resume operation after a power failure occurs.
[0006] In a first aspect, an embodiment of the present application provides a dToF device initialization method, wherein the dToF device is applied to an electronic device, and when the electronic device is turned on, the kernel layer initializes the dToF device; the method comprises: after the dToF device is initialized, the hardware abstraction layer determines, based on interaction with the kernel layer, that a power-off exception occurs in the dToF device, and the power-off exception comprises that the pins of the dToF device are disconnected from the electronic device and then reconnected; in response to the power-off exception occurring in the dToF device, the hardware abstraction layer generates a first instruction; the hardware abstraction layer sends the first instruction to the kernel layer, so that the kernel layer reinitializes the dToF device based on the first instruction.
[0007] After the dToF device has a power failure exception, the firmware information is lost and the function of measuring depth data cannot be continued. The dToF device can only work normally after the dToF device is initialized again the next time the electronic device is turned on. The method provided in the embodiment of the present application can reinitialize the dToF device in time after the dToF device has a power failure exception, so that the dToF device can continue to provide the function of measuring depth data after power failure and reconnection. In this way, the dToF device can be restored to a working state before the electronic device is turned on and initialized the dToF device next time, which can improve the user experience when using the electronic device and solve the problem of firmware loss caused by power failure exception.
[0008] In one achievable manner, the hardware abstraction layer includes a first thread, which is a resident thread. The hardware abstraction layer determines that a power-off anomaly occurs in the dToF device based on interaction with the kernel layer, including: the first thread of the hardware abstraction layer receives an abnormal event reported by the kernel layer, and the abnormal event is created by the kernel layer when a power-off anomaly occurs in the dToF device; the first thread modifies the first variable recorded by the hardware abstraction layer to a first value, and the first value is used to indicate that the dToF device has not completed initialization. In this way, the hardware abstraction layer can be notified of a power-off anomaly in the dToF device in a timely manner.
[0009] In one achievable manner, after the hardware abstraction layer sends the first instruction to the kernel layer, it also includes: the first thread receives the first return value reported by the kernel layer, the first return value is generated by the kernel layer after the dToF device completes initialization; in response to receiving the first return value, the first thread modifies the first variable to a second value, the second value is used to indicate that the dToF device has completed initialization. In this way, the first variable can be updated so that the first variable indicates the latest operating status of the dToF device.
[0010] In an achievable manner, the method further includes: in response to an instruction to turn on the camera, the hardware abstraction layer creates and runs a second thread; the second thread queries the first variable at a preset time interval; if the first variable is a second value, the second thread generates a second instruction; the second thread sends the second instruction to the kernel layer, so that the kernel layer configures and enables the dToF device based on the second instruction. In this way, the dToF device can start ranging.
[0011] In one achievable manner, the hardware abstraction layer determines that the dToF device has a power-off anomaly based on the interaction with the kernel layer, including: in response to the instruction to turn on the camera, the hardware abstraction layer creates and runs a third thread; the third thread queries the status file in the kernel layer to obtain the second variable; wherein, when the dToF device has a power-off anomaly, the kernel layer modifies the second variable from the default value to the third value; if the second variable is the third value, the hardware abstraction layer determines that the dToF device has a power-off anomaly after powering on. In this way, the hardware abstraction layer can obtain the latest operating status of the dToF device after turning on the camera.
[0012] In one achievable manner, the hardware abstraction layer determines that the dToF device has a power failure exception based on the interaction with the kernel layer, and further includes: if the second variable is a default value, the third thread generates a third instruction; the third thread sends the third instruction to the kernel layer, so that the kernel layer configures and enables the dToF device based on the third instruction. In this way, the dToF device can start ranging.
[0013] In one achievable manner, after the hardware abstraction layer sends the first instruction to the kernel layer, it also includes: a third thread receives a second return value reported by the kernel layer, the second return value is generated by the kernel layer after the dToF device completes initialization; in response to the received second return value, the third thread generates a third instruction; the third thread sends the third instruction to the kernel layer. In this way, the dToF device can start ranging.
[0014] In a second aspect, an embodiment of the present application provides a dToF device initialization method, in which the dToF device is applied to an electronic device, and the method includes: in response to the electronic device being powered on, the kernel layer initializes the dToF device; the kernel layer interacts with the hardware abstraction layer so that the hardware abstraction layer determines that the dToF device has a power-off exception, and the power-off exception includes the pins of the dToF device being disconnected from the electronic device and then reconnected; in response to a first instruction issued by the hardware abstraction layer, the kernel layer reinitializes the dToF device based on the first instruction, and the first instruction is generated by the hardware abstraction layer in response to the power-off exception of the dToF device.
[0015] After the dToF device has a power failure exception, the firmware information is lost and the function of measuring depth data cannot be continued. The dToF device can only work normally after the dToF device is initialized again the next time the electronic device is turned on. The method provided in the embodiment of the present application can reinitialize the dToF device in time after the dToF device has a power failure exception, so that the dToF device can continue to provide the function of measuring depth data after power failure and reconnection. In this way, the dToF device can be restored to a working state before the electronic device is turned on and initialized the dToF device next time, which can improve the user experience when using the electronic device and solve the problem of firmware loss caused by power failure exception.
[0016] In an achievable manner, the kernel layer interacts with the hardware abstraction layer, including: when the electronic device is dormant, the kernel layer queries the status register of the dToF device; if an error code is stored in the status register, the kernel layer creates an exception event, and the error code is used to indicate that the dToF device has a power-off exception; the kernel layer reports the exception event to the hardware abstraction layer. The kernel layer interacts with the hardware abstraction layer, and also includes: in response to a second instruction issued by the hardware abstraction layer, the kernel layer queries the status register; if an error code is stored in the status register, the kernel layer creates an exception event; the kernel layer reports the exception event to the hardware abstraction layer; if a normal value is stored in the status register, the kernel layer configures and enables the dToF device based on the second instruction, and the normal value is used to indicate that the dToF device does not have a power-off exception. In response to a first instruction issued by the hardware abstraction layer, after the kernel layer reinitializes the dToF device based on the first instruction, it also includes: after the dToF device completes initialization, the kernel layer reports a first return value to the hardware abstraction layer.
[0017] In this way, the steps of reinitializing the dToF device can be completed during the sleep process of the electronic device. Reinitialization can be performed regardless of whether the camera of the electronic device is turned on. After the camera is turned on, depth data can be obtained in time, which can shorten the time the user waits for autofocus after turning on the camera and improve the autofocus effect.
[0018] In an implementable manner, the kernel layer interacts with the hardware abstraction layer, and further includes: when the electronic device is dormant, the kernel layer queries the status register of the dToF device; if an error code is stored in the status register, the kernel layer modifies the second variable from the default value to the third value; wherein the second variable is saved in the status file, and the status file is used for querying by the hardware abstraction layer. In response to the first instruction issued by the hardware abstraction layer, after the kernel layer reinitializes the dToF device based on the first instruction, it also includes: after the dToF device completes initialization, the kernel layer modifies the second variable from the third value to the default value; the kernel layer reports the second return value to the hardware abstraction layer. After the kernel layer reports the second return value to the hardware abstraction layer, it also includes: in response to the third instruction issued by the hardware abstraction layer, the kernel layer queries the status register; if an error code is stored in the status register, the kernel layer modifies the second variable from the default value to the third value; if a normal value is stored in the status register, the kernel layer configures and enables the dToF device based on the third instruction, and the normal value is used to indicate that there is no power-off abnormality in the dToF device.
[0019] It can be seen that the embodiment of the present application can use the sleep state of the electronic device as a trigger condition to query whether the dToF device has a power-off anomaly. After that, after turning on the camera, the step of reinitializing the dToF device can not only detect the anomaly of the dToF device in time, but also will not increase the power consumption of the electronic device in the sleep state.
[0020] In one achievable method, the kernel layer reinitializes the dToF device based on the first instruction, including: the kernel layer releases the memory applied for when the dToF device is initialized during the power-on phase of the electronic device; the kernel layer resets the reference count modified when the dToF device is initialized during the power-on phase of the electronic device to zero; the kernel layer reloads the firmware for the dToF device; and the kernel layer resends calibration data to the dToF device.
[0021] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor and a memory; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes a dToF device initialization method as proposed in any implementation method in the first aspect, or the electronic device executes a dToF device initialization method as proposed in any implementation method in the second aspect.
[0022] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores instructions. When the instructions are executed on an electronic device, the electronic device executes a dToF device initialization method as proposed in any implementation method in the first aspect, or the electronic device executes a dToF device initialization method as proposed in any implementation method in the second aspect.
[0023] In the fifth aspect, an embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the dToF device initialization method proposed in any implementation method in the first aspect, or the electronic device executes the dToF device initialization method proposed in any implementation method in the second aspect.
[0024] It can be understood that the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A schematic diagram of the operation of a dToF device provided in an embodiment of the present application;
[0026] Figure 2 A schematic diagram of the structure of a dToF device provided in an embodiment of the present application;
[0027] Figure 3 This is a schematic diagram of the application of dToF devices;
[0028] Figure 4 A pin diagram of a dToF device;
[0029] Figure 5 A schematic diagram of the connection between the dToF device and the electronic device;
[0030] Figure 6 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;
[0031] Figure 7 Schematic diagram of the dToF software architecture provided for an embodiment of the present application;
[0032] Figure 8 A schematic diagram of a process for initializing a dToF device at the kernel layer when starting up provided in an embodiment of the present application;
[0033] Fig. 9 A schematic diagram of the working process of the dToF device provided in an embodiment of the present application;
[0034] Fig.10 A schematic diagram of a first flow chart of a dToF device initialization method provided in an embodiment of the present application;
[0035] Fig.11 A timing diagram of a dToF device initialization method provided in an embodiment of the present application;
[0036] Fig.12A first flow chart of the interaction between the first kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0037] Fig.13 A first timing diagram of the interaction between the first kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0038] Fig.14 A second flow chart of the first interaction between the kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0039] Fig.15 A third flow chart of the first interaction between the kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0040] Fig.16 A second timing diagram of the first type of interaction between the kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0041] Fig.17 A second flow chart of the dToF device initialization method provided in an embodiment of the present application;
[0042] Fig.18 A first flow chart of the second interaction between the kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0043] Fig.19 A first timing diagram of the interaction between the second kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0044] Fig. 20 A second flow chart of the interaction between the second kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0045] Fig.21 A second timing diagram of the interaction between the second kernel layer and the hardware abstraction layer provided in an embodiment of the present application;
[0046] Fig. 22 A third flow chart of the dToF device initialization method provided in an embodiment of the present application;
[0047] Fig.23 A schematic diagram of the structure of a dToF device initialization device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] The technical solutions in the embodiments of the present application will be described clearly below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, other embodiments obtained by ordinary technicians in this field without making creative work all belong to the protection scope of the present application.
[0049] The terms "first", "second", and "third" in the specification and claims of this application are used to distinguish different objects, rather than to limit a specific order. In the description of this application, unless otherwise specified, "at least one" means one, two, and more than two.
[0050] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0051] The terms used in the implementation method part of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application. The embodiments of this application will be described in detail below in conjunction with the accompanying drawings.
[0052] In order to facilitate technical personnel to understand the technical solutions of the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained below.
[0053] 1. Auto focus (AF) refers to the process in which an imaging instrument such as a camera uses the principle of light reflection from the object to receive the light reflected from the object, calculates the object distance (also known as depth data) of the object based on the received light signal, and automatically moves the lens to complete focusing according to the object distance to achieve clear imaging.
[0054] 2. Direct time-of-flight (dToF) device: a type of time-of-flight (ToF) sensor.
[0055] Figure 1 A schematic diagram of the operation of a dToF device provided in an embodiment of the present application.
[0056] like Figure 1 As shown, the dToF device can emit light pulses. After the light pulses are irradiated on the object, the surface of the object (target reflection surface) will reflect the light pulses. At this time, the light receiving unit of the dToF device can receive the reflected light pulses. Furthermore, the time interval between the reflected light pulse and the emitted light pulse can be obtained by measurement, and this time interval is the flight time of light. The object distance can be obtained by calculation based on the flight time of light and the speed of light. In practical applications, the imaging instrument can obtain the object distance by being equipped with a dToF sensor.
[0057] Figure 2 A schematic diagram of the structure of a dToF device provided in an embodiment of the present application.
[0058] like Figure 2 As shown, the dToF device 200 includes at least a dToF driver IC 201, a vertical-cavity surface-emitting laser (VCSEL) 202, a single photon avalanche diode (SPAD) 203, and a time digital converter (TDC) 204. The vertical-cavity surface laser emitter 202 can be used to emit light pulses into the scene, the single-photon avalanche diode 203 can receive light pulses reflected from the object, and the time digital converter 204 can record the flight time of each received light pulse, that is, the time interval between the emitted light pulse and the received light pulse.
[0059] 3. dToF firmware: For the above-mentioned dToF devices, when they are applied to electronic devices such as smartphones, they need to be configured based on the dToF firmware. After configuration, the dToF device can realize the corresponding functions. The dToF firmware can be a binary code that undertakes the underlying work of the dToF device. The dToF firmware is generally stored in the memory of the electronic device.
[0060] The auto focus (AF) function is one of the functions of the camera of an electronic device. When a user uses an electronic device to take a photo, if the electronic device is equipped with a camera with an auto focus function, the camera can automatically focus on the object within the framing range to achieve clear imaging, which can enhance the user experience. The specific implementation steps of auto focus are: obtaining the object distance of the object, moving the lens according to the object distance, so that the focus is located on the image sensor of the camera. In practical applications, a dToF device can be configured for the camera, and the object distance can be obtained using the dToF device.
[0061] The dToF device can emit light pulses. After the light pulses are irradiated on the object, the surface of the object will reflect the light pulses. At this time, the light receiving unit of the dToF device can receive the reflected light pulses. The time interval between the reflected light pulse and the emitted light pulse can be obtained by measurement, and this time interval is the flight time of light. The object distance can be obtained by calculation based on the flight time of light and the speed of light. In order for the dToF device to realize the above functions, the dToF device needs to be configured based on the dToF firmware when the electronic device is turned on.
[0062] However, since the connection between the dToF device and the electronic device is generally a spring connection, when the electronic device falls or encounters problems such as problems, the dToF device spring is easily disconnected, causing the dToF device to lose power after the electronic device is turned on. In this case, the configuration information saved in the dToF device will be lost. Therefore, even if the spring can be reconnected after disconnection, the dToF device cannot provide the corresponding function, which ultimately leads to a poor autofocus effect of the camera.
[0063] Figure 3 Schematic diagram of the application of dToF devices.
[0064] like Figure 3 As shown, the dToF device can be applied to the rear camera of a smartphone, and when the user turns on the camera, the dToF device can start ranging. Since the ranging time of the dToF device is very short, ranging based on dToF can achieve fast focusing, which can improve the user experience. In practical applications, not only can the dToF device be applied to the rear camera for fast focusing, but the dToF device can also be applied to the front panel of a smartphone. For example, the dToF device can be arranged in the front camera, so that automatic focusing of the front shooting can be achieved, or fast face recognition can be achieved. In practical applications, the dToF firmware can be written based on the desired functions, and the embodiments of the present application do not limit the specific content of the dToF firmware.
[0065] Figure 4 A pin diagram of a dToF device.
[0066] Figure 5 Schematic diagram of the connection between dToF device and electronic equipment.
[0067] like Figure 4 and Figure 5 As shown, the packaged dToF device 200 includes a plurality of pins, each of which is, for example, a VDD power supply, an EN pin and an INT pin for controlling the dToF device, an SCL pin and an SDA pin for data transmission, etc. Each pin includes a spring extending outward. The electronic device 100 includes a general input / output port, which is located on the motherboard of the electronic device, and each pin of the dToF device 200 can be connected to the general input / output port. After the general input / output port is packaged, it is a metal protrusion, and the springs extending outward from each pin of the dToF device 200 can be connected to the metal protrusion one by one. In this way, the dToF device can be connected to the electronic device 100.
[0068] It should be noted that Figure 5It is only a pin diagram of the dtof device 200. The vertical cavity surface laser emitter 202, the single photon avalanche diode 203 and the time digital converter 204 are not shown. Figure 5 Shown in.
[0069] The spring connection is an unstable connection method. When the electronic device falls or is subjected to external force, the spring is easily misaligned with the metal protrusion, or the spring is deformed and separated from the metal protrusion. At this time, the dToF device 200 loses power and is disconnected from the electronic device 100, and the firmware information is lost. After the external force disappears, the spring can be reset and the dToF device 200 can be reconnected to the metal protrusion, but the firmware information cannot be restored and the dToF device 200 cannot continue to work.
[0070] It is understandable that the distance measuring device configured for the camera of an electronic device is not limited to the dToF device 200, but may also include other distance measuring devices, such as a structured light sensor or a binocular sensor. When the dToF device 200 loses power, other means may also be activated for autofocus to ensure that the camera does not lose the autofocus function after the dToF device 200 loses power. The difference between the dToF device 200 and other focusing means lies in the focus time, focus accuracy, and the difference in visual effects after autofocus.
[0071] Figure 6 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application.
[0072] like Figure 6 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0073] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0074] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.
[0075] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory.
[0076] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.
[0077] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.
[0078] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.
[0079] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0080] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0081] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.
[0082] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.
[0083] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module.
[0084] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
[0085] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0086] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0087] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a file transfer function, an album function, etc.), etc. The data storage area can store data (such as files, photos, videos, etc.) created during the use of the electronic device 100, etc.
[0088] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0089] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. When a touch operation is applied to the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.
[0090] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B.
[0091] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
[0092] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a folding screen mobile phone, the electronic device 100 can detect the opening and closing of the folding screen mobile phone based on the magnetic sensor 180D.
[0093] The acceleration sensor 180E can detect the magnitude of acceleration in various directions (generally three axes) of the electronic device 100. When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected.
[0094] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser.
[0095] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100.
[0096] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.
[0097] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.
[0098] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy.
[0099] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194.
[0100] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of a human body's vocal part. The bone conduction sensor 180M can also contact a human body's pulse to receive a blood pressure pulse signal. In some embodiments, the bone conduction sensor 180M can also be disposed in an earphone to form a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of a vibrating bone block of a vocal part obtained by the bone conduction sensor 180M to implement a voice function.
[0101] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.
[0102] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Motor 191 can also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194.
[0103] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0104] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.
[0105] It will be apparent to those skilled in the art that some of the specific details presented above regarding the electronic device 100 may not be necessary for practicing a specific embodiment or its equivalent. Similarly, other electronic devices may include a greater number of modules, components, etc. Where appropriate, some modules may be implemented as software or hardware. Therefore, it should be understood that the above description is not intended to be exhaustive or to limit the present application to the precise form described herein. On the contrary, it will be apparent to those of ordinary skill in the art that, based on the above teachings, many modifications and variations are possible.
[0106] Before introducing the dToF device initialization method according to an embodiment of the present application, an exemplary introduction is first given to the dToF software architecture and the steps of initializing the dToF device 200 when the electronic device is powered on.
[0107] Figure 7 Schematic diagram of the dToF software architecture provided for an embodiment of the present application.
[0108] like Figure 7 As shown, the dToF software architecture may include an application layer (APP), a framework layer (framework), a hardware abstraction layer (HAL), a kernel layer (kernel) and a hardware layer (hardware). The application layer specifically includes the camera APP, and the application layer can be used to provide an interface for interacting with the outside world to achieve human-computer interaction. The framework layer includes a camera service (CameraService). The hardware abstraction layer is the hardware abstraction layer of Android, which is an encapsulation of the Linux kernel driver. The hardware abstraction layer specifically encapsulates the underlying hardware driver, which can shield the implementation details of the underlying hardware. Therefore, the role of the hardware abstraction layer is to provide a general interface for calling the driver to the framework layer.
[0109] Furthermore, the hardware abstraction layer includes an autofocus module (Auto Focus Node), which is used to control the dToF workflow and can call platform algorithms, such as AF algorithms. The hardware abstraction layer also includes camera peripheral device services (Non-CameraService, NCS), a driver interface layer (TOFSensorIntf), and tools for providing functions such as registering NAC services and buffer processing. The hardware abstraction layer also includes a driver control module sensorHAL, which is a dynamic library and a standard interface implemented according to the Android sensor HAL, which directly controls the driver of the kernel layer. The hardware abstraction layer can also include a camera abstract interface cameraHAL3 (not shown in the figure).
[0110] Furthermore, the kernel layer includes a dToF driver, which is used to interact with hardware via the I2C protocol. The hardware layer includes a dToF sensor 200 .
[0111] It should be noted that the above dToF software architecture is only an exemplary description. In actual applications, the dToF software architecture and the specifically implemented functions are not limited to the above content.
[0112] The following is an exemplary introduction to the specific implementation steps of applying the dToF device 200 to the autofocus scenario with reference to the accompanying drawings.
[0113] In an embodiment of the present application, in order to enable the dToF device 200 to realize functions such as autofocus, it is necessary to initialize the dToF device 200 when the electronic device 100 is turned on. The initialization step includes powering on the dToF device 200, configuring the dToF device 200 based on the dToF firmware, and other initialization operations.
[0114] First, the power-on process of the electronic device 100 of the embodiment of the present application may include: the main power chip of the electronic device 100 works to power the CPU of the electronic device 100, and then the main power chip can power the crystal oscillator and output the clock signal CLK to the CPU. After that, the main power chip can send a reset signal to reset the CPU, and then the CPU can reset the font library. Further, the CPU can send a power maintenance signal to the main power chip so that the main power chip maintains normal power output. After that, the CPU can perform a self-test.
[0115] Furthermore, the CPU can enable the display circuit to start working, and can send display data to the display module, and the display screen can display the LOGO icon. After that, the CPU can read the hard disk system program and load the operating system. In addition, the CPU can turn on the touch, audio, WIFI, camera and other modules. Furthermore, the CPU can start the RF part such as RF IC, power amplifier and antenna switch. After that, the baseband CPU can read the SIM card information, the signal receiving channel can start working, and select the operator according to the SIM card information. After that, the signal transmission passband can start working, confirm the connection with the operator's base station, enter the system, and the boot process ends.
[0116] The above-mentioned power-on process of the electronic device 100 is only an exemplary description. The actual power-on process can be determined according to the specific operation conditions of the electronic device 100, and the embodiment of the present application does not make any specific limitation on this.
[0117] In the embodiment of the present application, the step of initializing the dToF device 200 may occur when the CPU turns on touch, audio, WIFI, camera and other modules.
[0118] Figure 8 A schematic diagram of the process of initializing the dToF device by the kernel layer 304 at startup provided in an embodiment of the present application.
[0119] like Figure 8 As shown, when the electronic device 100 is turned on, the kernel layer 304 can complete the operation of initializing the dToF device 200, which specifically includes the following steps S101-S102.
[0120] S101 : The core layer 304 powers on the dToF device 200 using a preset power configuration.
[0121] Specifically, the step of the core layer 304 powering on the dToF device 200 may include: the dToF driver 3041 calls the power module in the electronic device 100, so that the power module powers on the dToF device 200 based on the preset power configuration. The preset power configuration includes a preset voltage value, and the preset voltage value can be any value in the range of 3.6V to 4.2V, preferably 3.6V, 3.8V or 4.2V, which is not specifically limited in the embodiment of the present application.
[0122] S102 : The core layer 304 configures registers of the dToF device 200 based on the dToF firmware.
[0123] Specifically, the steps of configuring the dToF device 200 based on the dToF firmware by the kernel layer 304 may include: the dToF driver 3041 reads the dToF firmware from the memory of the electronic device 100, and then parses the dToF firmware to obtain the value of the register. The dToF driver 3041 can then configure each register of the dToF device 200 through the I2C protocol, and the above steps may also be referred to as firmware loading.
[0124] Fig. 9 A schematic diagram of the workflow of the dToF device provided in an embodiment of the present application.
[0125] Combine the following Fig. 9 The working process of the dToF device 200 is introduced as an example.
[0126] After the user clicks the camera icon, the camera APP 3011 located in the application layer 301 can execute the following steps S201 and S202.
[0127] S201: Respond to user interaction behavior.
[0128] S202: Notify the camera service (CameraService) 3021 located in the framework layer (framework) 302 to start.
[0129] Afterwards, the camera service is started and executes the following step S203.
[0130] S203: Notify the camera abstract interface (cameraHAL3) 3031 located in the hardware abstraction layer (HAL) 303 to start the camera opening process.
[0131] Further, after step S203, the hardware abstraction layer 303 may specifically perform the following steps:
[0132] S204: the camera abstract interface 3031 initializes the auto focus module (Auto Focus Node) 3032 .
[0133] At this point, the auto focus module 3032 may complete initialization and start working, and the AF thread 31 is started.
[0134] S205: The camera abstract interface 3031 initializes the camera peripheral device service (Non-Camera Service, NCS) 3033.
[0135] At this point, the camera peripheral device service 3033 can create a thread, complete initialization and start working.
[0136] Further, the hardware abstraction layer 303 may continue to perform the following steps:
[0137] S206 : The auto focus module 3032 transmits an instruction to turn on dToF to the camera peripheral device service 3033 .
[0138] After receiving the instruction to turn on dToF, the camera peripheral device service 3033 can perform the following steps:
[0139] S207: The camera peripheral device service 3033 transmits instructions for configuring and enabling dToF to the driver control module (sensorHAL) 3034.
[0140] In this step, the camera peripheral device service 3033 eventually passes instructions for configuring and enabling dToF to the driver interface layer (TOFSensorIntf), and then the driver interface layer passes the instructions to the driver control module 3034.
[0141] Further, the hardware abstraction layer 303 may continue to perform the following steps:
[0142] S208 : The drive control module 3034 transmits instructions for configuring and enabling dToF to the dToF driver 3041 .
[0143] Furthermore, the dToF driver 3041 located in the kernel layer 304 may perform the following steps:
[0144] S209: The dToF driver 3041 configures and enables the dToF device 200 located at the hardware layer (hardware) 305 based on the instruction.
[0145] In this step, the specific configuration step may be that the dToF driver 3041 configures the register of the dToF device 200 based on the preset sampling rate, laser frequency, etc. Further, the dToF device 200 is enabled. In this way, the dToF device 200 can start ranging and obtain depth data.
[0146] Furthermore, the dToF device 200 may transmit the depth data to the dToF driver 3041. The dToF driver 3041 finally transmits the depth data to the autofocus module 3032 through the driver control module 3034 and the camera peripheral device service 3033. Afterwards, the autofocus module 3032 may call the AF algorithm to use the depth data for autofocus.
[0147] In an embodiment of the present application, after the camera is turned on, the rear camera mode can be entered by default. At this time, the depth data can be applied to automatically focus on the rear viewfinder.
[0148] After the electronic device 100 is turned on, the dToF device 200 completes initialization. At this time, if the electronic device 100 falls, the shrapnel of the dToF device 200 may be disconnected from the electronic device 100, and the dToF device 200 has a power-off abnormality after power-on. After power failure, the various registers of the dToF device 200 will be reset, the firmware information will be lost, and the dToF device 200 will no longer be able to provide functions such as ranging. It is understandable that even if the external force on the electronic device 100 disappears and the shrapnel of the dToF device 200 is reset, the dToF device 200 still needs to be reinitialized for the dToF device 200 to work properly.
[0149] However, after the power failure and reconnection, the software level of the electronic device 100 is not aware of the power failure of the dToF device 200. Therefore, after the user turns on the camera, the AF thread 31 (autofocus thread) will still issue an instruction to turn on dToF. At this time, the dToF device 200 cannot perform measurements, and the AF thread 31 cannot obtain depth data.
[0150] In order to solve the above problems, an embodiment of the present application provides a dToF device initialization method, which can re-initialize the dToF device after a power failure occurs in the dToF device. In this way, the dToF device 200 can be restored to work before the electronic device 100 is turned on and initialized the dToF device 200 next time.
[0151] The method provided in the embodiments of the present application can be applied to devices with depth data acquisition functions, such as smart phones, tablet computers, personal computers, workstation equipment, large-screen devices (for example, smart screens, smart TVs, etc.), wearable devices (for example, smart bracelets, smart watches), handheld game consoles, home game consoles, virtual reality devices, augmented reality devices, mixed reality devices, etc., in-vehicle smart terminals and other devices.
[0152] Continue to see Figure 7 From the perspective of software architecture, in order to solve the problem of reinitialization caused by power failure of the dToF device 200, the embodiment of the present application improves the hardware abstraction layer 303 and the kernel layer 304, so that interaction can be added between the hardware abstraction layer 303 and the kernel layer 304, and finally reinitialization is achieved. The specific improvement of the hardware abstraction layer 303 lies in the camera peripheral device service 3033, the driver interface layer and the driver control module 3034 of the hardware abstraction layer 303, and the specific improvement of the kernel layer 304 lies in the dToF driver 3041. Furthermore, the embodiment of the present application will make a detailed introduction to the improvement of the software architecture in combination with the specific dToF device initialization solution.
[0153] Fig.10 A first flow chart of the dToF device initialization method provided in an embodiment of the present application.
[0154] Fig.11 A timing diagram of the dToF device initialization method provided in an embodiment of the present application.
[0155] Combination Figure 7-Figure 11 The dToF device initialization method provided in the embodiment of the present application may include the following steps S300.
[0156] S300 : In response to the electronic device 100 being powered on, the core layer 304 initializes the dToF device 200 .
[0157] The dToF device 200 provided in the embodiment of the present application can be used to measure depth data. For example, the dToF device 200 can be applied to the rear camera of the electronic device so that when the rear camera is working, it can automatically focus based on the depth data, improve the focusing speed, and improve the three-dimensional sense of the subject in the photo. For another example, the dToF device 200 can be applied to the front camera of the electronic device so that when the electronic device performs face recognition, the time of face recognition is shortened and the accuracy of face recognition is improved. The embodiment of the present application does not limit the specific application method of the dToF device 200 in the electronic device 100.
[0158] Furthermore, the kernel layer 304 may initialize the dToF device 200 when the electronic device 100 is turned on, and the initialization step may include powering on the dToF device 200 and loading firmware on the dToF device 200. The specific power-on and firmware loading process can refer to the above introduction to steps S101-S102, which will not be repeated here.
[0159] Furthermore, the dToF device initialization method provided in the embodiment of the present application may include the following steps S400-S800.
[0160] S400: The kernel layer 304 interacts with the hardware abstraction layer 303 so that the hardware abstraction layer 303 determines that a power failure exception occurs in the dToF device 200.
[0161] Among them, the power-off anomaly is caused by the falling of the electronic device 100, etc., and the power-off anomaly is manifested in that the pins of the dToF device 200 are disconnected from the electronic device 100 and then reconnected. Specifically, the pins of the dToF device 200 are connected to the universal input / output port on the mainboard of the electronic device 100. The packaging form of the universal input / output port is a metal protrusion. The packaging of the pins of the dToF device 200 includes a spring extending outward, and the spring can be connected to the metal protrusion one by one. When the electronic device 100 falls, the spring is easily misaligned with the metal protrusion, or the spring is deformed and separated from the metal protrusion. At this time, the dToF device 200 loses power and is disconnected from the electronic device 100. After the fall, the spring can be reset, and the dToF device 200 can be reconnected to the metal protrusion.
[0162] It can be understood that the kernel layer 304 is used to interact directly with the hardware. Therefore, the dToF driver 3041 in the kernel layer 304 can obtain the operating status of the dToF device 200. Afterwards, the dToF driver 3041 can interact with the hardware abstraction layer 303 so that the hardware abstraction layer 303 can know whether the dToF device 200 has a power failure exception. In the embodiment of the present application, the interaction between the kernel layer 304 and the hardware abstraction layer 303 can be specifically reflected in the interaction between the dToF driver 3041 and the camera peripheral device service 3033.
[0163] S500 : After the dToF device 200 completes initialization, the hardware abstraction layer 303 determines that a power-off exception occurs in the dToF device 200 based on interaction with the kernel layer 304 .
[0164] It is understandable that the hardware abstraction layer 303 can respond to a power-off exception in the dToF device 200 and send instructions to the kernel layer 304 so that the kernel layer 304 re-initializes the dToF device 200.
[0165] S600 : In response to a power failure exception in the dToF device 200 , the hardware abstraction layer 303 generates a first instruction 21 .
[0166] In an embodiment of the present application, after a power-off exception occurs in the dToF device 200, the camera peripheral device service 3033 of the hardware abstraction layer 303 can generate a first instruction 21, and use the first instruction 21 to instruct the kernel layer 304 to reinitialize the dToF device 200.
[0167] S700 : The hardware abstraction layer 303 sends the first instruction 21 to the kernel layer 304 , so that the kernel layer 304 reinitializes the dToF device 200 based on the first instruction 21 .
[0168] In an embodiment of the present application, the camera peripheral device service 3033 of the hardware abstraction layer 303 can directly send the first instruction 21 to the dToF driver 3041 of the kernel layer 304, or it can send the first instruction 21 to the dToF driver 3041 of the kernel layer 304 through the driver control module 3034. The specific sending process will be described in detail below and will not be repeated here.
[0169] S800 : In response to the first instruction 21 issued by the hardware abstraction layer 303 , the kernel layer 304 reinitializes the dToF device 200 based on the first instruction 21 .
[0170] Among them, the dToF driver 3041 of the core layer 304 can reinitialize the dToF device 200 based on the first instruction 21, and the reinitialization steps include the following S801-S804.
[0171] S801: The kernel layer 304 releases the memory requested when the dToF device 200 is initialized during the startup phase of the electronic device 100.
[0172] It is understandable that when the dToF driver 3041 located in the kernel layer 304 initializes the dToF device 200 during the startup phase of the electronic device 100, it will apply to the system for memory to save the calculation data generated during the initialization process or the data exchanged with the hardware layer. After the dToF device 200 has a power failure exception, since the dToF firmware information has been lost, the dToF device 200 can no longer provide the function of depth data measurement and no longer needs to use memory. If the occupied memory is not released, this part of the memory space cannot be continued to be used by the dToF driver 3041, or other applications of the electronic device cannot use this part of the memory space, which will cause a waste of memory space.
[0173] Therefore, in the process of reinitializing the dToF device 200 in the embodiment of the present application, the memory space occupied when the dToF device 200 is initialized during the power-on phase can be first released to improve the utilization rate of the memory space and avoid wasting memory space.
[0174] S802: The kernel layer 304 resets the reference count modified when the dToF device 200 is initialized during the power-on phase of the electronic device to zero.
[0175] Among them, reference counting is a technical means of managing memory. When the dToF driver 3041 located in the kernel layer 304 initializes the dToF device 200 during the boot phase, an object call will be made. The reference count records the number of times each object is referenced by the user, or records how many users are referencing the object. As the object is referenced by the user, the reference count will increase or decrease. When the reference count is zero, it means that the memory occupied by the object can be released.
[0176] In an embodiment of the present application, the reference count modified when initializing the dToF device 200 during the power-on phase of the electronic device is reset to zero, and the object can be restored to an unreferenced state to avoid interference with the process of reinitializing the dToF device 200 caused by the modified reference count.
[0177] S803: The kernel layer 304 reloads the firmware for the dToF device.
[0178] Among them, the dToF driver 3041 located in the kernel layer 304 accesses the memory of the electronic device 100 and reads out the dToF firmware. After that, the dToF driver 3041 can parse the firmware to obtain the values of multiple registers. Further, the dToF driver 3041 can configure each register of the dToF device 200 through the I2C protocol based on the values of the parsed registers to complete the loading of the dToF firmware.
[0179] S804: The core layer 304 re-sends the calibration data to the dToF device 200 .
[0180] The calibration data is a parameter used to adjust and calibrate the measurement accuracy of the dToF device 200. For example, the calibration data may be the size of the crosstalk between the dToF device 200 and the battery cover of the electronic device 100. The step of calibrating using the calibration data may, for example, be configuring the registers of the dToF device 200 based on the calibration data.
[0181] Furthermore, the specific implementation method of sending calibration data may include: the dToF driver 3041 accesses the memory of the electronic device 100 to obtain the calibration data, and then writes the calibration data into the register of the dToF device 200 through the I2C protocol, so that the dToF device 200 can use the calibration data for calibration before performing depth data measurement.
[0182] It should be noted that the steps of reinitializing the dToF device 200 are not limited to the above steps S801-S804, and the specific steps of reinitializing the dToF device 200 can be adjusted based on actual needs.
[0183] In summary, after releasing memory, modifying reference counts, reloading firmware, and sending calibration data, the dToF device 200 can be restored to a normal working state, and the dToF device 200 can complete the task of measuring depth data.
[0184] It can be seen from the above technical solution that after the dToF device 200 has a power failure exception, the firmware information is lost and the function of measuring depth data cannot be continued. The dToF device 200 can only work normally after the dToF device 200 is initialized again when the electronic device is turned on next time. The embodiment of the present application provides a dToF device initialization method, which can reinitialize the dToF device 200 in time after the dToF device 200 has a power failure exception, so that the dToF device 200 can continue to provide the function of measuring depth data after power failure and reconnection. In this way, the dToF device 200 can be restored to a working state before the electronic device 100 is turned on and initialized to the dToF device 200 next time, which can improve the user experience when using the electronic device 100 and solve the problem of firmware loss caused by power failure exception.
[0185] In the embodiment of the present application, the execution timing of steps S300-S800 may occur after a power failure occurs in the dToF device 200. The specific triggering conditions of steps S300-S800 may be, for example, the electronic device 100 goes into sleep mode, the electronic device 100 ends sleep mode, the electronic device 100 locks the screen, the electronic device 100 returns to the main screen interface, the user turns on the camera, the facial recognition fails, the user unlocks the device with a password, or the user unlocks the device with a fingerprint. In actual applications, different triggering conditions may be selected based on the specific application scenario of the dToF device 200.
[0186] The following uses the dToF device 200 for camera autofocus as a specific application scenario to introduce the sub-steps of steps S300-S800 in detail.
[0187] Fig.12The first flowchart of the first interaction between the kernel layer 304 and the hardware abstraction layer 303 provided in the embodiment of the present application is shown.
[0188] Fig.13 This is a first timing diagram of the interaction between the first kernel layer 304 and the hardware abstraction layer 303 provided in an embodiment of the present application.
[0189] like Fig.12 and Fig.13 As shown, in the embodiment of the present application, step S400 may include the following steps S401-S403.
[0190] S401 : When the electronic device 100 is in sleep mode, the kernel layer 304 queries the status register of the dToF device 200 .
[0191] Among them, sleep is an operating mode of the electronic device 100. Sleep can be specifically understood as the electronic device 100 causing some components or peripherals to enter a low power consumption mode, so that the power consumption of the electronic device 100 can be reduced. For different electronic devices 100, the sleep triggering conditions and sleep levels are different, and the embodiments of the present application do not specifically limit this. Exemplarily, in the embodiments of the present application, the power button of the electronic device 100 can be used as a condition for triggering sleep and wake-up. For example, when the screen of the electronic device 100 is on, the user presses the power button, the screen goes out, and the electronic device 100 is triggered to sleep. The user presses the power button again, the screen is lit, and the electronic device 100 is awakened. For another example, if the electronic device 100 remains in the off-screen state for more than a preset period of time, the electronic device 100 can be triggered to sleep.
[0192] It is understandable that triggering the electronic device 100 to sleep is a frequently occurring event, and after the electronic device 100 falls, the user may press the power button multiple times to check whether the electronic device 100 has some abnormalities. In the embodiment of the present application, the electronic device 100 sleeps as a trigger condition for the kernel layer 304 to query the operating status of the dToF device 200, and whether the dToF device 200 has a power-off abnormality can be discovered in time. Therefore, by using the electronic device 100 sleep as a trigger condition for detecting whether the dToF device 200 has an abnormality, the latest operating status of the dToF device 200 can be discovered in time.
[0193] In the embodiment of the present application, the operation of the core layer 304 querying the status register of the dToF device 200 is specifically performed by the dToF driver 3041 located in the core layer 304 .
[0194] Further, the status register is one of the multiple registers in the dToF driver chip 201, which is used to indicate the operating status of the dToF device 200. After the dToF device 200 completes the loading of the dToF firmware, the dToF driver chip 201 can modify the status register from the error code to the normal value. Among them, the normal value is used to indicate that the dToF device is in a normal operating state and there is no power-off exception. The error code is used to indicate that the dToF device 200 has a power-off exception after initialization in the boot phase. When the dToF device 200 has a power-off exception, each register of the dToF driver chip 201 will be reset, for example, the register based on the dToF firmware configuration is reset to the initial value, and the status register will also be reset from the normal value to the error code. Therefore, in the embodiment of the present application, after the dToF driver 3041 located in the kernel layer 304 queries the status register of the dToF device 200, it can be determined whether the dToF device 200 has a power-off exception.
[0195] It is understandable that the specific data of the error code and the normal value are determined by the manufacturer of the dToF device 200, and the embodiment of the present application does not make any specific limitations on this.
[0196] S402: If an error code is stored in the status register, the kernel layer 304 creates an exception event, and the error code is used to indicate that the dToF device 200 has a power-off exception.
[0197] It is understandable that if an error code is stored in the status register, it means that a power-off exception has occurred in the dToF device 200, so the kernel layer 304 needs to report the exception to the hardware abstraction layer 303 so that the hardware abstraction layer 303 can complete some steps related to reinitializing the dToF device 200.
[0198] S403: The kernel layer 304 reports the abnormal event to the hardware abstraction layer 303.
[0199] In order to realize the interaction between the kernel layer 304 and the hardware abstraction layer 303 in the embodiment of the present application, a method of reporting abnormal events is adopted. Specifically, the dToF driver 3041 located in the kernel layer 304 can generate an abnormal event (event) after querying the status register. An abnormal event is a kernel object that can be created by calling the CreateEvent function. The embodiment of the present application does not limit the specific implementation method of creating an abnormal event in the kernel layer 304.
[0200] Continue to see Fig.12 and Fig.13, the specific steps for the kernel layer 304 to report the abnormal event may be: the dToF driver 3041 calls the interface of the first framework CamCfg V4L2 device 3042. Among them, the first framework 3042 is a framework obtained by encapsulating the target interface in the Linux V4L2 framework 3043. Since the Linux V4L2 framework 3043 includes many interfaces, directly calling the interface in the Linux V4L2 framework 3043 may increase the difficulty of calling. Therefore, in the embodiment of the present application, the target interface in the Linux V4L2 framework 3043 can be encapsulated to form the first framework 3042. In this way, when it is necessary to call the target interface, the interface in the first framework 3042 can be called first, and then the interface of the first framework 3042 can call the target interface in the LinuxV4L2 framework 3043. With the call of the target interface, the abnormal event can be reported to the camera peripheral device service 3033 located in the hardware abstraction layer 303.
[0201] In some implementations, the dToF driver 3041 may directly call the target interface in the Linux V4L2 framework.
[0202] It should be noted that CamCfg V4L2 device is only an exemplary name of the first framework in software implementation and does not impose any limitation on the embodiments of the present application.
[0203] For further information, see Fig.12 and Fig.13 , step S500 may include the following sub-steps S501-S502.
[0204] S501: The first thread 32 of the hardware abstraction layer 303 receives an abnormal event reported by the kernel layer 304. The abnormal event is created by the kernel layer 304 when a power failure occurs in the dToF device.
[0205] The first thread 32 is a resident thread in the hardware abstraction layer 303 , and specifically may be a thread of the camera peripheral device service 3033 .
[0206] It should be noted that Fig. 9 The thread of the camera peripheral device service 3033 shown in the figure is the second thread 33, which may be created or run when the camera abstract interface 3031 initializes the camera peripheral device service 3033 during the camera opening stage. Different from the second thread 33, the embodiment of the present application provides a resident thread, which may be created when the electronic device 100 initializes the dToF device 200. The resident thread will not sleep due to the sleep of the electronic device 100, but will remain in the running state. In this way, the camera peripheral device service 3033 can receive the abnormal events reported by the kernel layer 304 in a timely manner.
[0207] S502: The first thread 32 modifies the first variable recorded in the hardware abstraction layer 303 to a first value, where the first value is used to indicate that the dToF device 200 has not completed initialization.
[0208] Among them, the first variable is a static variable of the hardware abstraction layer 303, which can be maintained by the camera peripheral device service 3033. The first variable is used to indicate whether the dToF device 200 has completed initialization. In this way, the second thread 33 of the camera peripheral device service 3033 can know the latest status of the dToF device 200, which can avoid the second thread 33 from issuing an instruction to enable the dToF device 200 when the dToF device 200 has not completed initialization. The embodiment of the present application does not specifically limit the actual value of the first numerical value.
[0209] It is understandable that steps S501-S502 may also be performed when the electronic device 100 is in a sleep state.
[0210] Furthermore, with the execution of steps S501-S502, the hardware abstraction layer 303 can determine that the dToF device 200 has a power-off anomaly, and therefore can execute step S600 to instruct the kernel layer 304 to reinitialize the dToF device.
[0211] Continue to see Fig.13 In step S600, the first instruction 21 may specifically include instructions for instructing the kernel layer 304 to execute steps S801-S803, and may also include instructions for instructing the kernel layer 304 to execute step S804. Among them, for the instruction for instructing the kernel layer 304 to execute steps S801-S803, the camera peripheral device service 3033 may send it to the drive control module 3034, and finally the drive control module 3034 sends it to the dToF driver 3041 located in the kernel layer 304. For the instruction for instructing the kernel layer 304 to execute step S804, the camera peripheral device service 3033 may directly send it to the dToF driver 3041. The above-mentioned step of issuing the first instruction 21 is only an exemplary description, and the specific issuing process may be determined by the actual situation, and the embodiment of the present application does not specifically limit this.
[0212] After the hardware abstraction layer 303 issues the first instruction, the kernel layer 304 may execute step S800 to reinitialize the dToF device 200. It is understandable that after the dToF driver 3041 reinitializes the dToF device 200, the dToF driver chip 201 may modify the status register from the error code to the normal value.
[0213] Fig.14 A second flow chart of the first type of interaction between the kernel layer and the hardware abstraction layer provided in an embodiment of the present application.
[0214] Further, such as Fig.13 and Fig.14 As shown, after step S800, the dToF driver 3041 of the core layer 304 can continue to execute the following step S404.
[0215] S404 : After the dToF device 200 completes initialization, the kernel layer 304 reports a first return value to the hardware abstraction layer 303 .
[0216] It is understandable that the first return value is generated by the kernel layer 304 after the dToF device 200 is initialized. The first return value can be generated by the dToF driver 3041 and reported to the first thread 32 of the camera peripheral device service 3033 to notify the camera peripheral device service 3033 that the dToF device 200 has been initialized.
[0217] For further information, see Fig.13 and Fig.14 In the embodiment of the present application, the hardware abstraction layer 303 may also perform the following steps S503-S504.
[0218] S503 : The first thread 32 receives the first return value reported by the kernel layer 304 .
[0219] S504: In response to receiving the first return value, the first thread 32 modifies the first variable to a second value;
[0220] The second value is used to indicate that the dToF device 200 has completed initialization. The embodiment of the present application does not specifically limit the actual value of the first value.
[0221] In this way, the second thread 33 of the camera peripheral device service 3033 can obtain the latest status of the dToF device in a timely manner.
[0222] It can be seen that in the embodiment of the present application, the step of reinitializing the dToF device 200 can be completed when the electronic device 100 is in a dormant state. In this way, after the user turns on the camera, the dToF device 200 can directly measure, and can measure the depth data in time, which can shorten the autofocus time.
[0223] The following is an introduction to the steps after turning on the camera with the help of the attached pictures.
[0224] Fig.15 The third flow chart of the first interaction between the kernel layer 304 and the hardware abstraction layer 303 provided in the embodiment of the present application is as follows.
[0225] Fig.16This is a second timing diagram of the interaction between the first kernel layer 304 and the hardware abstraction layer 303 provided in an embodiment of the present application.
[0226] like Fig.15 and Fig.16 As shown, the dToF device initialization method provided in the embodiment of the present application also includes the following steps S901-S904.
[0227] S901 : In response to an instruction to turn on the camera, the hardware abstraction layer 303 creates and runs the second thread 33 .
[0228] The instruction to open the camera may correspond to the instruction transmitted to the hardware abstraction layer 303 in step S203, or may be other instructions in steps S201-S206.
[0229] It is understandable that step S901 and the steps thereafter may occur in Fig. 9 The position corresponding to the “first time” shown in .
[0230] Further, the second thread 33 is another thread in the camera peripheral device service 3033 that is different from the first thread 32 . For example, the camera peripheral device service 3033 may be created based on the initialization of the camera abstract interface 3031 .
[0231] S902: The second thread 33 queries the first variable at a preset time interval.
[0232] S903 : If the first variable is a second value, the second thread 33 generates a second instruction 22 .
[0233] In the embodiment of the present application, after the camera is turned on, the second thread 33 can query the first variable maintained by the first thread 32 to obtain the latest operating status of the dToF device 200. If the first variable is the second value, it means that the dToF device 200 is in the state of having completed initialization. Therefore, the second thread 33 can generate a second instruction 22 to enable the kernel layer 304 to configure and enable the dToF device 200.
[0234] In step S903, the second thread 33 may query the first variable according to the preset time. In this way, if the step of reinitializing the dToF device 200 is not completed when the camera is turned on, the second thread 33 may wait for the initialization to be completed before issuing the second instruction 22. The preset time interval may be, for example, 10ms, 20ms, or 50ms, etc., which is not specifically limited in the embodiment of the present application.
[0235] S904 : The second thread 33 sends the second instruction 22 to the kernel layer 304 , so that the kernel layer 304 configures and enables the dToF device based on the second instruction 22 .
[0236] In an embodiment of the present application, the second instruction 22 can be sent by the second thread 33 to the driver interface layer located inside the camera peripheral device service 3033, and then the driver interface layer passes the instruction to the driver control module 3034, and then the driver control module 3034 sends it to the dToF driver 3041 located in the kernel layer 304.
[0237] Continue to see Fig.15 and Fig.16 , step S400 may also include the following steps S405-S408.
[0238] S405 : In response to the second instruction 22 issued by the hardware abstraction layer 303 , the kernel layer 304 queries the status register.
[0239] Among them, step S405 and the steps thereafter may correspond to Fig. 9 The position of the "second time" is shown.
[0240] It can be seen that in the embodiment of the present application, after the camera is turned on, in the workflow of the dToF device 200, the kernel layer 304 can be used to query the status register again to determine the latest operating status of the dToF device 200.
[0241] It is understandable that in the embodiment of the present application, the step of discovering that the dToF device 200 has a power failure anomaly and reinitializing the dToF device 200 can occur during the sleep process of the electronic device 100. If the dToF device 200 has a power failure anomaly again after reinitialization, and the electronic device 100 does not sleep until the camera is turned on, then the dToF device 200 is still in a state where it cannot work normally. By executing step S405, it can be timely known that the dToF device 200 has a power failure anomaly, and the dToF device 200 can be reinitialized in time so that the dToF device 200 can work normally.
[0242] In other implementations, after the electronic device 100 is turned on, the dToF device 200 has a power failure exception, and before the camera is turned on, the electronic device 100 is not in sleep mode. Then, executing step S405 can also reinitialize the dToF device 200 in time, so that the dToF device 200 can measure depth data during the operation of the camera, and the camera can use the depth data for automatic focus, thereby improving the user experience.
[0243] S406: If an error code is stored in the status register, the kernel layer 304 creates an abnormal event.
[0244] S407 : The kernel layer 304 reports the abnormal event to the hardware abstraction layer 303 .
[0245] The specific reporting process can refer to step S403 and will not be described in detail here.
[0246] S408: If a normal value is stored in the status register, the kernel layer 304 configures and enables the dToF device 200 based on the second instruction 22, and the normal value is used to indicate that there is no power-off abnormality in the dToF device 200.
[0247] Among them, steps S405-S408 are executed by the dToF driver 3041 of the core layer 304.
[0248] In an embodiment of the present application, the second thread 33 can send the second instruction 22 to the driver control module 3034 located in the hardware abstraction layer 303, and then the driver control module 3034 sends it to the dToF driver 3041 located in the kernel layer 304.
[0249] Further, the step of configuring the dToF device 200 based on the second instruction 22 by the kernel layer 304 may specifically include: the dToF driver 3041 located in the kernel layer 304 parses the second instruction 22 to obtain the configuration information included in the second instruction 22, and then configures the register of the dToF device 200 based on the configuration information. The configuration information is, for example, the sampling rate and laser frequency of the dToF device 200, which is not specifically limited in the embodiment of the present application.
[0250] The step in which the kernel layer 304 enables the dToF device 200 based on the second instruction 22 may specifically include: configuring the registers of the dToF device 200 based on the second instruction 22 so that the dToF device 200 starts ranging.
[0251] In this way, the dToF device 200 can start depth measurement. After the dToF device 200 measures the depth data, the depth data can be fed back to the dToF driver 3041, and then reported to the AF thread 31 through the driver control module 3034 and the camera peripheral device service 3033 in sequence, so that the AF thread 31 calls the AF algorithm and performs autofocus based on the depth data. Among them, the AF algorithm is an algorithm for realizing autofocus, for example, it can be a dToF depth data algorithm, etc., which is not specifically limited in the embodiments of the present application.
[0252] It can be seen that the second instruction 22 is not only used to enable the kernel layer 304 to configure and enable the dToF device 200, but also to enable the kernel layer 304 to know the latest operating status of the dToF device 200 before configuring and enabling the dToF device 200 to avoid enabling failure.
[0253] It should be noted that in step S407, the kernel layer 304 reports the abnormal event to the hardware abstraction layer 303. Since the first thread 32 is a resident thread, the first thread 32 can promptly trigger the process of reinitializing the dToF device 200 after receiving the abnormal event, specifically including executing steps S501-S502. At this time, the first thread 32 can issue the first instruction 21, and then the kernel layer 304 can execute step S800, respond to the first instruction 21 and reinitialize the dToF device 200.
[0254] Furthermore, after the dToF device 200 is initialized, the kernel layer 304 can report the first return value to the hardware abstraction layer 303, so that the hardware abstraction layer 303 can issue the second instruction 22 for configuring and enabling the dToF device 200. It can be seen that even if a power failure abnormality is found in the dToF device 200 during the operation of the camera, the method provided in the embodiment of the present application can reinitialize the dToF device 200 in time, so that the dToF device 200 can be restored to a state where it can operate normally in time, thereby improving the user experience when using the camera.
[0255] It can be seen from the above technical solution that the dToF device initialization method provided in the embodiment of the present application can start the step of reinitializing the dToF device 200 during the sleep process of the electronic device. Reinitialization can be performed regardless of whether the electronic device 100 turns on the camera. After turning on the camera, the AF thread 31 can obtain depth data in time, which can shorten the time the user waits for autofocus after turning on the camera, thereby improving the autofocus effect.
[0256] Fig.17 A second flow chart of the dToF device initialization method provided in an embodiment of the present application.
[0257] Combine the following Fig.17 An exemplary introduction is given to each path when initializing the dToF device 200 and performing depth measurement using the dToF device 200. Fig.17 The abnormal event (event) reporting path, the reinitialization path (recovery processing path) and the depth data acquisition path are shown.
[0258] like Fig.17 As shown, for the kernel layer 304, the dToF driver 3041 generates and reports an abnormal event, and the reporting process specifically includes step ①: reporting the abnormal event to the first framework CamCfg V4L2 Dvice 3042, and the first framework 3042 then calls V4L2-core in the Linux V4L2 framework 3043 to complete step ②.
[0259] Further, for the hardware abstraction layer 303, the camera peripheral device service 3033 includes a resident thread (first thread), and the resident thread subscribes to the abnormal event through step ③, so that the abnormal event reported by the bottom layer (kernel layer 304) through the V4L2 framework can be obtained.
[0260] Further, the resident thread first updates the first variable, which is used to indicate the initialization state of the dToF device 200. Among them, "ready" is used to indicate that the dToF device 200 has completed initialization, and "recovery" is used to indicate that the dToF device 200 has not completed initialization.
[0261] Execute step ④ to trigger the reinitialization process (recovery process), and after step ④, include step ⑤-1 to clean up the software status, which step corresponds to steps S801-S802, and the path in this step points to the driver control module 3034 located in the hardware abstraction layer 303. Further, the driver control module 3034 can send each instruction in step ⑤-1 to the kernel layer 304 for execution.
[0262] Step ⑤-1 also includes step ⑤-2 to reconfigure the device, which corresponds to step S803 and is specifically implemented by calling the first function of the driver control module 3034. Further, the driver control module 3034 can send each instruction in step ⑤-2 to the kernel layer 304 for execution.
[0263] Step ⑤-2 is followed by step ⑤-3 to re-issue the calibration data, which corresponds to step S804. In this step, the path can directly point to the dToF driver 3041 located in the core layer 304 (the path is not shown in the figure).
[0264] After step ⑤-3 is completed, the resident thread can update the first variable.
[0265] Further, the autofocus module 3032 executes step ⑥ to turn on the camera, calls up the AF thread 31, the AF thread 31 calls the first method of the camera peripheral device service 3033 to query the state of the first variable, and then calls the second function and the third function of the drive control module 3034 to execute step ⑦ to obtain depth data.
[0266] Furthermore, the driver control module 3034 may execute step ⑧, and send each instruction in step ⑦ to the kernel layer 304 for execution.
[0267] It should be noted that, for the sake of simplicity in the representation of the pathway, Fig.17 In the figure, the abnormal event (event) reporting path, the reinitialization path (recovery processing path) and the multiplexing path in the depth data acquisition path are omitted.
[0268] It should be noted that the specific implementation process of calling the first method, the first function, the second function and the third function can be determined based on actual conditions, and the embodiments of the present application do not make specific limitations on this.
[0269] Fig.18 This is a first flow chart of the interaction between the second kernel layer 304 and the hardware abstraction layer 303 provided in an embodiment of the present application.
[0270] Fig.19 This is the first timing diagram of the interaction between the second kernel layer 304 and the hardware abstraction layer 303 provided in the embodiment of the present application.
[0271] like Fig.18 and Fig.19 As shown, in the embodiment of the present application, step S400 may include the following steps S409-S410.
[0272] S409 : When the electronic device 100 is in sleep mode, the kernel layer 304 queries the status register of the dToF device 200 .
[0273] In the embodiment of the present application, step S409 is performed by the dToF driver 3041 located in the kernel layer 304. By querying the status register of the dToF device 200, it can be known whether the dToF device 200 has a power-off abnormality after the electronic device 100 completes initialization during the startup phase.
[0274] S410: If an error code is stored in the status register, the kernel layer 304 modifies the second variable from the default value to a third value.
[0275] In the embodiment of the present application, step S409 is executed by the dToF driver 3041 located in the core layer 304 .
[0276] The second variable is stored in a state file, which is used for querying by the hardware abstraction layer 303. It is understandable that the state file is located in the kernel layer 304, and the specific way in which the hardware abstraction layer 303 interacts with the kernel layer 304 is that the hardware abstraction layer 303 accesses the state file maintained by the kernel layer 304.
[0277] Furthermore, the status file may specifically be a device node, and the status file may be created by the kernel layer 304 when the dToF device 200 is initialized during the boot phase, and the default value may be written when the status file is created, and the default value is used to indicate that the dToF device 200 has completed initialization, and the third value is used to indicate that the dToF device 200 has not completed initialization. In this way, the hardware abstraction layer 303 can know the latest operating status of the dToF device 200 by querying the status file.
[0278] The embodiment of the present application does not limit the specific time for creating the status file, and can be flexibly adjusted based on actual conditions.
[0279] Continue to see Fig.18 and Fig.19 In the embodiment of the present application, step S500 may also include the following steps S505-S507.
[0280] S505 : In response to the instruction to turn on the camera, the hardware abstraction layer 303 creates and runs the third thread 34 .
[0281] The instruction to open the camera may correspond to the instruction transmitted to the hardware abstraction layer 303 in step S203, or may be other instructions in steps S201-S206. Step S505 and subsequent steps may occur in Fig. 9 Further, the third thread 34 is specifically the camera peripheral device service 3033 which can be created based on the initialization of the camera abstract interface 3031.
[0282] S506: The third thread 34 queries the status file in the kernel layer 304 to obtain the second variable; wherein, when a power failure exception occurs in the dToF device 200, the kernel layer 304 modifies the second variable from the default value to the third value.
[0283] S507: If the second variable is the third value, the hardware abstraction layer 303 determines that the dToF device 200 has a power-off exception after startup.
[0284] It can be seen that the method provided in the embodiment of the present application can first determine the operating status of the dToF device 200 before configuring and enabling the dToF device 200 to avoid enabling failure.
[0285] After step S507 , steps S600 - S700 may be executed: the kernel layer 304 generates a first instruction 21 , and sends the first instruction 21 to the kernel layer 304 , so that the kernel layer 304 reinitializes the dToF device 200 .
[0286] Further, the first instruction 21 may specifically include an instruction for instructing the kernel layer 304 to execute steps S801-S803, and may also include an instruction for instructing the kernel layer 304 to execute step S804. Among them, for the instruction for instructing the kernel layer 304 to execute steps S801-S803, the camera peripheral device service 3033 may send it to the drive control module 3034, and finally the drive control module 3034 sends it to the dToF driver 3041 located in the kernel layer 304. For the instruction for instructing the kernel layer 304 to execute step S804, the camera peripheral device service 3033 may directly send it to the dToF driver 3041. The above-mentioned step of issuing the first instruction 21 is only an exemplary description, and the specific issuing process may be determined by the actual situation, and the embodiment of the present application does not specifically limit this.
[0287] Further, after the hardware abstraction layer 303 issues the first instruction 21, the kernel layer 304 may execute step S800 to reinitialize the dToF device 200. It is understandable that after the dToF driver 3041 reinitializes the dToF device 200, the dToF driver chip 201 may modify the status register from the error code to the normal value.
[0288] Fig. 20 A second flow chart of the interaction between the second kernel layer 304 and the hardware abstraction layer 303 provided in the embodiment of the present application.
[0289] Fig.21 This is a second timing diagram of the interaction between the second kernel layer 304 and the hardware abstraction layer 303 provided in an embodiment of the present application.
[0290] Combination Figure 18-Figure 21 , after steps S801-S804, the kernel layer 304 may further perform the following steps S905-S906.
[0291] S905: After the dToF device 200 completes initialization, the core layer 304 modifies the second variable from the third value to the default value.
[0292] In this way, the status information recorded in the status file can be updated. When the hardware abstraction layer 303 subsequently queries the status file, the latest operating status of the dToF device 200 can be known. The timing of the subsequent query of the status file can be, for example, the next time the camera is turned on.
[0293] In the embodiment of the present application, the step of specifically modifying the second variable may be performed by the dToF driver 3041 located in the core layer 304 .
[0294] S906 : The kernel layer 304 reports the second return value to the hardware abstraction layer 303 .
[0295] It is understandable that the second return value is generated by the kernel layer 304 after the dToF device 200 is initialized. The second return value can be reported by the dToF driver 3041 to the third thread 34 of the camera peripheral device service 3033 to notify the camera peripheral device service 3033 that the dToF device 200 has been initialized.
[0296] Furthermore, in the embodiment of the present application, step S700 also includes the following steps S907-S909.
[0297] S907: The third thread 34 receives the second return value reported by the kernel layer 304, where the second return value is generated by the kernel layer 304 after the dToF device completes initialization.
[0298] In this way, the third thread 34 can know that the dToF device 200 has completed reinitialization.
[0299] S908 : In response to the received second return value, the third thread 34 generates a third instruction 23 .
[0300] S909 : The third thread 34 sends the third instruction 23 to the kernel layer 304 .
[0301] In this way, the core layer 304 can configure and enable the dToF device 200 based on the third instruction 23 .
[0302] Continue to see Fig.18 and Fig.19 , after step S507, the following steps S508-S509 are also included.
[0303] S508 : If the second variable is a default value, the third thread 34 generates a third instruction 23 .
[0304] S509 : The third thread 34 sends the third instruction 23 to the kernel layer 304 , so that the kernel layer 304 configures and enables the dToF device 200 based on the third instruction 23 .
[0305] It can be understood that if the second variable is the default value, it means that when the kernel layer 304 queries the status of the dToF device 200, the dToF device 200 has not experienced a power-off exception or the power-off exception has been resolved. Therefore, the third thread 34 can directly configure and enable the dToF device 200 at this time.
[0306] In an embodiment of the present application, the third thread 34 can send the third instruction 23 to the driver interface layer located inside the camera peripheral device service 3033, and then the driver interface layer passes the instruction to the driver control module 3034, and then the driver control module 3034 sends it to the dToF driver 3041 located in the kernel layer 304.
[0307] Continue to see Figure 18-Figure 21 , after step S509 or step S909, the following steps S910-S912 may also be included.
[0308] S910: In response to the third instruction 23 issued by the hardware abstraction layer 303, the kernel layer 304 queries the status register.
[0309] Among them, steps S910-S911 correspond to Fig. 9 The position of the "second time" is shown.
[0310] It is understandable that in some implementations, since the step of querying whether the dToF device 200 has a power-off anomaly by the kernel layer 304 occurs when the electronic device 100 is in sleep mode, if the dToF device 200 has a power-off anomaly, and after the power-off anomaly, the electronic device 100 remains awake until the camera is turned on and does not sleep, then the kernel layer 304 does not know that the dToF device 200 has a power-off anomaly, and the second variable is not updated, so the hardware abstraction layer 303 does not know that the dToF device 200 is still in a state where it cannot work normally. In the dToF device workflow after the camera is turned on, the embodiment of the present application uses the kernel layer 304 to query the status register again to determine the latest operating status of the dToF device 200.
[0311] It can be seen that the third instruction 23 is not only used to enable the kernel layer 304 to configure and enable the dToF device 200, but also to enable the kernel layer 304 to know the latest operating status of the dToF device 200 before configuring and enabling the dToF device 200 to avoid enabling failure.
[0312] S911: If an error code is stored in the status register, the kernel layer 304 modifies the second variable from the default value to a third value.
[0313] Among them, step S911 can be executed by the dToF driver 3041 of the core layer 304.
[0314] In this way, the second variable in the status file corresponds to the latest operating status of the dToF device 200 .
[0315] In the embodiment of the present application, the method of interaction between the kernel layer 304 and the hardware abstraction layer 303 depends on the hardware abstraction layer 303 actively querying the status file. Therefore, after turning on the camera, if the kernel layer 304 learns that the dToF device 200 has a power-off exception, it can modify the second variable and wait until the camera is turned on next time, and then reinitialize the dToF device 200 based on the instruction of the hardware abstraction layer 303 to avoid too many processes after turning on the camera, affecting the autofocus of other ranging sensors.
[0316] In other implementations, after the camera is turned on, if the kernel layer 304 learns that the dToF device 200 has a power-off exception, it can modify the second variable and actively report the power-off exception of the dToF device 200 to the hardware abstraction layer 303, so that the hardware abstraction layer 303 instructs the dToF device 200 to be reinitialized during the process of turning on the camera this time. The embodiments of the present application do not make specific limitations on this.
[0317] S912 : If a normal value is stored in the status register, the kernel layer 304 configures and enables the dToF device 200 based on the third instruction 23 .
[0318] The normal value is used to indicate that there is no power-off abnormality in the dToF device 200. Step S912 is executed by the dToF driver 3041 of the core layer 304.
[0319] Further, the steps of configuring and enabling the dToF device 200 based on the third instruction 23 by the kernel layer 304 may specifically include: the dToF driver 3041 located in the kernel layer 304 parses the third instruction 23, obtains the configuration information included in the third instruction 23, and then configures the register of the dToF device 200 based on the configuration information. The configuration information is, for example, the sampling rate and laser frequency of the dToF device 200, which is not specifically limited in the embodiment of the present application.
[0320] The step in which the kernel layer 304 enables the dToF device 200 based on the third instruction 23 may specifically include: configuring the registers of the dToF device 200 based on the third instruction 23 so that the dToF device 200 starts ranging.
[0321] In this way, the dToF device 200 can start depth measurement. After the dToF device 200 measures the depth data, the depth data can be fed back to the dToF driver 3041, and then reported to the AF thread 31 through the driver control module 3034 and the camera peripheral device service 3033 in sequence, so that the AF thread 31 calls the AF algorithm and performs autofocus based on the depth data. Among them, the AF algorithm is an algorithm for realizing autofocus, for example, it can be a dToF depth data algorithm, etc., which is not specifically limited in the embodiments of the present application.
[0322] It can be seen from the above technical solutions that the dToF device initialization method provided in the embodiment of the present application can use the electronic device sleep 100 as a trigger condition to query whether the dToF device 200 has a power-off anomaly. After that, after turning on the camera, the step of reinitializing the dToF device 200 can not only detect the anomaly of the dToF device 200 in time, but also will not increase the power consumption of the electronic device 100 in the sleep state.
[0323] Fig. 22A third flow chart of the dToF device initialization method provided in an embodiment of the present application.
[0324] Combine the following Fig. 22 An exemplary introduction is given to each path when initializing the dToF device 200 and performing depth measurement using the dToF device 200. Fig. 22 The device status update path, the reinitialization path (recovery processing path) and the deep data acquisition path are shown in FIG.
[0325] like Fig. 22 As shown, for the kernel layer 304, the dToF driver 3041 updates the input / status located in the device node of the file system so that the device node stores the latest device status, completing step ①.
[0326] The auto focus module 3032 executes step ② to turn on the camera and call up the AF thread 31. The AF thread 31 calls the first method of the camera peripheral device service 3033 to execute step ③ to trigger the re-initialization process (recovery process).
[0327] Further, step ③ includes step ③-1: calling the fourth function located in the driver control module 3034 to check the device status. Step ③-1 specifically includes step ③-1-1: reading the device node.
[0328] Step ③-1 also includes step ③-2 for cleaning the software status, which corresponds to steps S801-S802, and the path in this step points to the driver control module 3034 located in the hardware abstraction layer 303. Further, the driver control module 3034 can send each instruction in step ③-2 to the kernel layer 304 for execution.
[0329] Step ③-2 also includes step ③-3 to reconfigure the device, which corresponds to step S803 and is specifically implemented by calling the first function of the driver control module 3034. Further, the driver control module 3034 can send each instruction in step ③-3 to the kernel layer 304 for execution.
[0330] Step ③-3 is followed by step ③-4 to re-issue the calibration data, which corresponds to step S804. In this step, the path can directly point to the dToF driver 3041 located in the core layer 304 (the path is not shown in the figure).
[0331] Further, the AF thread 31 calls the first method of the camera peripheral device service 3033, and then calls the second function and the third function of the drive control module 3034 to execute step ④ to obtain depth data.
[0332] Furthermore, the driver control module 3034 may execute step ⑤, and send each instruction in step ④ to the kernel layer 304 for execution.
[0333] It should be noted that, for the sake of simplicity in the representation of the pathway, Fig. 22 In the figure, the device status update path, the reinitialization path (recovery processing path) and the multiplexing path in the depth data acquisition path are omitted.
[0334] It should be noted that the specific implementation process of calling the first method, the first function, the second function, the third function and the fourth function can be determined based on the actual situation, and the embodiment of the present application does not specifically limit this. It should be noted that in step S500: the dToF device 200 completes initialization, and the hardware abstraction layer 303 interacts with the kernel layer 304 to determine that the dToF device 200 has a power-off exception. It can be understood that the completion of the initialization of the dToF device 200 referred to here may correspond to the first initialization of the dToF device 200 when the electronic device 100 is turned on, and may also correspond to the reinitialization of the dToF device 200 after a power-off exception occurs.
[0335] It should be noted that the execution of the dToF device initialization method provided in the embodiment of the present application includes multiple function calls, and each function call can have a return value (return). The above timing diagram only exemplarily shows several of the return values, such as the first return value, the second return value and the depth data, and not all return values are shown.
[0336] Fig.23 A schematic diagram of the structure of a dToF device initialization device provided in an embodiment of the present application.
[0337] like Fig.23As shown, the dToF device initialization device provided in the embodiment of the present application may include: a display screen 1001, a memory 1002, a processor 1003 and a communication module 1004. The above-mentioned devices may be connected via one or more communication buses 1005. The display screen 1001 may include a display panel 10011 and a touch sensor 10012, wherein the display panel 10011 is used to display an image, and the touch sensor 10012 may pass the detected touch operation to the application processor to determine the type of touch event, and provide visual output related to the touch operation through the display panel 10011. The processor 1003 may include one or more processing units, for example: the processor 1003 may include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor, etc. Among them, different processing units may be independent devices or integrated in one or more processors. The memory 1002 is coupled to the processor 1003 for storing various software programs and / or computer instructions, and the memory 1002 may include a volatile memory and / or a non-volatile memory. When the processor 1003 executes computer instructions, the dToF device initialization device can perform various functions or steps performed by the electronic device 100 in the above method embodiment.
[0338] In some embodiments, the dToF device is applied to an electronic device, and when the electronic device is turned on, the kernel layer 304 initializes the dToF device. When the software program and / or multiple sets of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: after the dToF device is initialized, the hardware abstraction layer 303 determines that the dToF device has a power-off anomaly based on the interaction with the kernel layer 304, and the power-off anomaly includes the pins of the dToF device being disconnected from the electronic device and then reconnected; in response to the power-off anomaly of the dToF device, the hardware abstraction layer 303 generates a first instruction; the hardware abstraction layer 303 sends the first instruction to the kernel layer 304, so that the kernel layer 304 reinitializes the dToF device based on the first instruction.
[0339] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: the first thread 32 of the hardware abstraction layer 303 receives an abnormal event reported by the kernel layer 304, and the abnormal event is created by the kernel layer 304 when a power-off abnormality occurs in the dToF device; the first thread modifies the first variable recorded in the hardware abstraction layer 303 to a first value, and the first value is used to indicate that the dToF device has not completed initialization.
[0340] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: the first thread receives a first return value reported by the kernel layer 304, and the first return value is generated by the kernel layer 304 after the dToF device completes initialization; in response to receiving the first return value, the first thread modifies the first variable to a second value, and the second value is used to indicate that the dToF device has completed initialization.
[0341] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: in response to an instruction to turn on the camera, the hardware abstraction layer 303 creates and runs a second thread; the second thread queries the first variable at a preset time interval; if the first variable is a second value, the second thread generates a second instruction; the second thread sends the second instruction to the kernel layer 304, so that the kernel layer 304 configures and enables the dToF device based on the second instruction.
[0342] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: in response to an instruction to turn on the camera, the hardware abstraction layer 303 creates and runs a third thread; the third thread queries the status file in the kernel layer 304 to obtain a second variable; wherein, when the dToF device has a power-off exception, the kernel layer 304 modifies the second variable from the default value to a third value; if the second variable is the third value, the hardware abstraction layer 303 determines that the dToF device has a power-off exception after startup.
[0343] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: if the second variable is a default value, the third thread generates a third instruction; the third thread sends the third instruction to the kernel layer 304, so that the kernel layer 304 configures and enables the dToF device based on the third instruction.
[0344] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: the third thread receives the second return value reported by the kernel layer 304, and the second return value is generated by the kernel layer 304 after the dToF device completes initialization; in response to the received second return value, the third thread generates a third instruction; the third thread sends the third instruction to the kernel layer 304.
[0345] In some embodiments, the dToF device is applied to an electronic device. When the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: in response to the electronic device being powered on, the kernel layer 304 initializes the dToF device; the kernel layer 304 interacts with the hardware abstraction layer 303 so that the hardware abstraction layer 303 determines that the dToF device has a power-off anomaly, and the power-off anomaly includes the pins of the dToF device being disconnected from the electronic device and then reconnected; in response to the first instruction issued by the hardware abstraction layer 303, the kernel layer 304 reinitializes the dToF device based on the first instruction, and the first instruction is generated by the hardware abstraction layer 303 in response to the dToF device having a power-off anomaly.
[0346] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: when the electronic device is in sleep mode, the kernel layer 304 queries the status register of the dToF device; if an error code is stored in the status register, the kernel layer 304 creates an exception event, and the error code is used to indicate that the dToF device has a power-off exception; the kernel layer 304 reports the exception event to the hardware abstraction layer 303.
[0347] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: in response to the second instruction issued by the hardware abstraction layer 303, the kernel layer 304 queries the status register; if an error code is stored in the status register, the kernel layer 304 creates an exception event; the kernel layer 304 reports the exception event to the hardware abstraction layer 303; if a normal value is stored in the status register, the kernel layer 304 configures and enables the dToF device based on the second instruction, and the normal value is used to indicate that there is no power-off exception in the dToF device.
[0348] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: after the dToF device completes initialization, the kernel layer 304 reports a first return value to the hardware abstraction layer 303.
[0349] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: when the electronic device is in sleep mode, the kernel layer 304 queries the status register of the dToF device; if an error code is stored in the status register, the kernel layer 304 changes the second variable from the default value to a third value; wherein the second variable is saved in a status file, and the status file is used for query by the hardware abstraction layer 303.
[0350] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: after the dToF device completes initialization, the kernel layer 304 changes the second variable from the third value to the default value; the kernel layer 304 reports the second return value to the hardware abstraction layer 303.
[0351] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: in response to the third instruction issued by the hardware abstraction layer 303, the kernel layer 304 queries the status register; if an error code is stored in the status register, the kernel layer 304 modifies the second variable from the default value to a third value; if a normal value is stored in the status register, the kernel layer 304 configures and enables the dToF device based on the third instruction, and the normal value is used to indicate that there is no power-off abnormality in the dToF device.
[0352] In some embodiments, when the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device is used to perform the following method steps: the kernel layer 304 releases the memory requested when the dToF device is initialized during the power-on phase of the electronic device; the kernel layer 304 resets the reference count modified when the dToF device is initialized during the power-on phase of the electronic device to zero; the kernel layer 304 reloads the firmware for the dToF device; the kernel layer 304 re-sends the calibration data to the dToF device.
[0353] The above specific implementation methods further explain in detail the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above are only specific implementation methods of the embodiments of the present application and are not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.
Claims
1. A dToF device initialization method, characterized in that: The dToF device is applied in an electronic device, and when the electronic device is turned on, the kernel layer initializes the dToF device; The method comprises: After the dToF device completes initialization, in response to the electronic device going into sleep mode, and / or in response to an instruction to turn on the camera, after the kernel layer determines that an error code is stored in the status register of the dToF device, the hardware abstraction layer determines that a power-off exception occurs in the dToF device based on interaction with the kernel layer; wherein the hardware abstraction layer interacts with the kernel layer through an abnormal event reported by the kernel layer, and / or by querying a status file in the kernel layer, the abnormal event is created by the kernel layer when a power-off exception occurs in the dToF device, and the status file is created by the kernel layer and is used to indicate the operating status of the dToF device; the power-off exception includes a pin of the dToF device being disconnected from the electronic device and then reconnected; In response to a power failure exception occurring in the dToF device, the hardware abstraction layer generates a first instruction; The hardware abstraction layer sends the first instruction to the kernel layer, so that the kernel layer reinitializes the dToF device based on the first instruction.
2. The dToF device initialization method according to claim 1, characterized in that: The hardware abstraction layer includes a first thread, which is a resident thread. The hardware abstraction layer determines that the dToF device has a power failure exception based on interaction with the kernel layer, including: The first thread of the hardware abstraction layer receives the abnormal event reported by the kernel layer; The first thread modifies the first variable recorded in the hardware abstraction layer to a first value, where the first value is used to indicate that the dToF device has not completed initialization.
3. The dToF device initialization method according to claim 2, characterized in that: After the hardware abstraction layer sends the first instruction to the kernel layer, the method further includes: The first thread receives a first return value reported by the kernel layer, where the first return value is generated by the kernel layer after the dToF device completes initialization; In response to receiving the first return value, the first thread modifies the first variable to a second value, where the second value is used to indicate that the dToF device has completed initialization.
4. The dToF device initialization method according to claim 3, characterized in that: The method further comprises: In response to the instruction to open the camera, the hardware abstraction layer creates and runs a second thread; The second thread queries the first variable at a preset time interval; If the first variable is the second value, the second thread generates a second instruction; The second thread sends the second instruction to the kernel layer, so that the kernel layer configures and enables the dToF device based on the second instruction.
5. The dToF device initialization method according to claim 1, characterized in that: The hardware abstraction layer determines that the dToF device has a power failure exception based on the interaction with the kernel layer, including: In response to the instruction to open the camera, the hardware abstraction layer creates and runs a third thread; The third thread queries the status file in the kernel layer to obtain a second variable; wherein, when the dToF device has a power failure exception, the kernel layer modifies the second variable from a default value to a third value; If the second variable is a third value, the hardware abstraction layer determines that the dToF device has a power-off exception after startup.
6. The dToF device initialization method according to claim 5, characterized in that: The hardware abstraction layer determines that the dToF device has a power failure exception based on the interaction with the kernel layer, and further includes: If the second variable is the default value, the third thread generates a third instruction; The third thread sends the third instruction to the kernel layer, so that the kernel layer configures and enables the dToF device based on the third instruction.
7. The dToF device initialization method according to claim 6, characterized in that: After the hardware abstraction layer sends the first instruction to the kernel layer, the method further includes: The third thread receives a second return value reported by the kernel layer, where the second return value is generated by the kernel layer after the dToF device completes initialization; In response to the received second return value, the third thread generates the third instruction; The third thread sends the third instruction to the kernel layer.
8. A dToF device initialization method, characterized in that: The dToF device is applied in an electronic device, and the method comprises: In response to the electronic device being powered on, the kernel layer initializes the dToF device; In response to the electronic device going into sleep mode, and / or in response to an instruction to turn on the camera, after determining that an error code is stored in a status register of the dToF device, the kernel layer interacts with the hardware abstraction layer so that the hardware abstraction layer determines that a power-off exception occurs in the dToF device; wherein the kernel layer interacts with the hardware abstraction layer by reporting an abnormal event to the hardware abstraction layer, and / or through a status file for the hardware abstraction layer to query, the abnormal event is created by the kernel layer when a power-off exception occurs in the dToF device, and the status file is created by the kernel layer and is used to indicate the operating status of the dToF device; the power-off exception includes a pin of the dToF device being disconnected from the electronic device and then reconnected; In response to a first instruction issued by the hardware abstraction layer, the kernel layer reinitializes the dToF device based on the first instruction, where the first instruction is generated by the hardware abstraction layer in response to a power-off exception in the dToF device.
9. The dToF device initialization method according to claim 8, characterized in that: The kernel layer interacts with the hardware abstraction layer, including: When the electronic device is in sleep mode, the kernel layer queries the status register; If an error code is stored in the status register, the kernel layer creates the abnormal event, and the error code is used to indicate that the dToF device has a power-off abnormality; The kernel layer reports the abnormal event to the hardware abstraction layer.
10. The dToF device initialization method according to claim 9, characterized in that: The kernel layer interacts with the hardware abstraction layer and also includes: In response to a second instruction issued by the hardware abstraction layer, the kernel layer queries the status register; If the error code is stored in the status register, the kernel layer creates the exception event; The kernel layer reports the abnormal event to the hardware abstraction layer; If a normal value is stored in the status register, the kernel layer configures and enables the dToF device based on the second instruction, and the normal value is used to indicate that there is no power-off abnormality in the dToF device.
11. The dToF device initialization method according to claim 10, characterized in that: After the kernel layer reinitializes the dToF device based on the first instruction issued by the hardware abstraction layer in response to the first instruction, the method further includes: After the dToF device completes initialization, the kernel layer reports a first return value to the hardware abstraction layer.
12. The dToF device initialization method according to claim 8, characterized in that: The kernel layer interacts with the hardware abstraction layer and also includes: When the electronic device is in sleep mode, the kernel layer queries the status register; If the error code is stored in the status register, the kernel layer modifies the second variable from a default value to a third value; wherein the second variable is stored in the status file.
13. The dToF device initialization method according to claim 12, characterized in that: After the kernel layer reinitializes the dToF device based on the first instruction issued by the hardware abstraction layer in response to the first instruction, the method further includes: After the dToF device completes initialization, the kernel layer modifies the second variable from the third value to the default value; The kernel layer reports a second return value to the hardware abstraction layer.
14. The dToF device initialization method according to claim 13, characterized in that: After the kernel layer reports the second return value to the hardware abstraction layer, the method further includes: In response to a third instruction issued by the hardware abstraction layer, the kernel layer queries the status register; If the error code is stored in the status register, the kernel layer modifies the second variable from the default value to the third value; If a normal value is stored in the status register, the kernel layer configures and enables the dToF device based on the third instruction, and the normal value is used to indicate that there is no power-off abnormality in the dToF device.
15. The dToF device initialization method according to any one of claims 8 to 14, characterized in that: The kernel layer reinitializes the dToF device based on the first instruction, including: The kernel layer releases the memory applied for when the dToF device is initialized during the startup phase of the electronic device; The kernel layer resets the reference count modified when the dToF device is initialized during the power-on phase of the electronic device to zero; The kernel layer reloads firmware for the dToF device; The core layer re-sends the calibration data to the dToF device.
16. An electronic device, characterized in that: include: Processor and memory; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the dToF device initialization method according to any one of claims 1 to 15.
Citation Information
Patent Citations
Initializing control method and device for terminal device, and terminal device
CN106528460A
4G module stable working method based on Android system
CN109766197A
Conductive elastic connecting piece and electronic equipment
CN115706340A
Method and device for detecting abnormal core in graphics processor and electronic equipment
CN115827355A
Rapid photographing method, electronic equipment and computer readable storage medium
CN115967851A
