Data processing method and related device

Abstract

The embodiments of the present application provide a data processing method and related devices, which relate to the field of terminal technology. The method includes: obtaining first anti-shake data reported by a sensor through a second chip; when the first anti-shake data is not within the data range that the first chip can recognize, transmitting second anti-shake data to the first chip through the second chip, and the second anti-shake data is within the data range that the first chip can recognize. In this way, when the first anti-shake data is not within the recognizable data range, the first anti-shake data is modified to the second anti-shake data within the recognizable data range. The electronic device can recognize the second anti-shake data, so as to perform subsequent anti-shake processing, image output and display, and other processes. This method can reduce the situation where the electronic device display appears to be stuck or even stuck due to the anti-shake data not being within the recognizable data range.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a data processing method and related devices. Background Art

[0002] Electronic devices such as mobile phones are provided with cameras to facilitate users to preview and shoot pictures.

[0003] However, when the electronic device is previewing in the video recording mode, the preview screen displayed by the electronic device may freeze, the content corresponding to the preview screen remains unchanged, and the human-computer interaction experience is poor. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and related devices, which are applied to the field of terminal technology. When the anti-shake data reported by the sensor is not within the recognizable data range, the anti-shake data is modified so that the electronic device can recognize the modified anti-shake data. In this way, when the anti-shake data is not within the recognizable data range, the electronic device can recognize the modified anti-shake data, so as to perform subsequent anti-shake processing, image output and display, and other processes. This method can reduce the situation where the electronic device display appears stuck or even stuck due to the anti-shake data not being within the recognizable data range.

[0005] In a first aspect, an embodiment of the present application proposes a data processing method, which is applied to an electronic device. The electronic device includes a first chip, a second chip, and a sensor. The method includes: obtaining first anti-shake data reported by the sensor through the second chip; when the first anti-shake data is not within the data range that the first chip can recognize, transmitting second anti-shake data to the first chip through the second chip, and the second anti-shake data is within the data range that the first chip can recognize.

[0006] The first anti-shake data may be data collected by an anti-shake related sensor, for example, gyroscope data collected by a gyroscope sensor, acceleration data collected by an acceleration sensor, Hall data collected by a Hall sensor, and the like.

[0007] The first anti-shake data may be data reported when a sensor fails, data with a transmission error when the sensor reports, etc., which is not specifically limited here.

[0008] In this way, when the first anti-shake data is not within the recognizable data range, the first anti-shake data is modified to the second anti-shake data within the recognizable data range. The electronic device can identify the second anti-shake data, thereby performing subsequent anti-shake processing, image output and display, etc. This method can reduce the occurrence of freezes and freezes in the electronic device display caused by the anti-shake data not being within the recognizable data range.

[0009] In one possible implementation, the first anti-shake data includes: gyroscope data, which is used to indicate the angular velocity of the electronic device in various directions; the second anti-shake data includes: a first preset value; when the first anti-shake data is not within the data range that can be recognized by the first chip, the second anti-shake data is transmitted to the first chip through the second chip, including: when the data in at least one direction in the gyroscope data is greater than a first threshold, the first preset value is transmitted to the first chip through the second chip.

[0010] In this way, the inability of electronic equipment to perform anti-shake processing, image output and other processes due to the inability to recognize gyroscope data can be reduced, and the display freeze phenomenon can be reduced.

[0011] In one possible implementation, when data in at least one direction of the gyroscope data is greater than a first threshold, a first preset value is transmitted to the first chip through the second chip, including: when data in at least one direction of the gyroscope data is greater than the first threshold, the second chip adds 1 to the number of first counts; the first count is a count of the case where data in at least one direction of the gyroscope data is greater than the first threshold; when the first count is greater than the second threshold, the first preset value is transmitted to the first chip through the second chip.

[0012] In this way, the accidental impact of gyroscope data errors can be reduced.

[0013] In a possible implementation, the first anti-shake data further includes: a first timestamp, which is a timestamp when the sensor reports the gyroscope data; and the second anti-shake data further includes: a second timestamp, which is later than the first timestamp.

[0014] In this way, the disorder of timestamp sorting can be reduced, which causes electronic devices to be unable to perform anti-shake processing, image output and display, and reduces display freezes.

[0015] In some embodiments, the second timestamp is a timestamp when the second chip transmits the gyroscope data to the first chip.

[0016] In some other embodiments, the second timestamp may also be the timestamp of the second chip encapsulating the gyroscope data. It is understandable that the second chip may encapsulate the gyroscope data to facilitate data transmission or the first chip's identification and reading of the data. The second chip may modify the first timestamp to the second timestamp when encapsulating the gyroscope data. The embodiments of the present application do not specifically limit the second timestamp.

[0017] In a possible implementation, the first anti-shake data also includes: a third timestamp, which is the timestamp when the sensor reports the acceleration data; the acceleration data is used to indicate the acceleration of the electronic device in various directions; the second anti-shake data also includes: a fourth timestamp, which is later than the third timestamp.

[0018] In this way, the disorder of timestamp sorting can be reduced, which causes electronic devices to be unable to perform anti-shake processing, image output and display, and reduces display freezes.

[0019] In some embodiments, the fourth timestamp is a timestamp when the second chip transmits acceleration data to the first chip.

[0020] In some other embodiments, the fourth timestamp may also be the timestamp of the acceleration data packaged by the second chip. It is understandable that the second chip may package the acceleration data to facilitate data transmission or identification and reading of the data by the first chip. The second chip may modify the first timestamp to the fourth timestamp when packaging the acceleration data. The fourth timestamp is not specifically limited in the embodiments of the present application.

[0021] In a possible implementation, the first anti-shake data includes: acceleration data, and the second anti-shake data includes: a second preset value; when the first anti-shake data is not within the data range that can be recognized by the first chip, the second anti-shake data is transmitted to the first chip through the second chip, including: when the acceleration in each direction in the acceleration data is less than a third threshold, the second preset value is transmitted to the first chip through the second chip.

[0022] In this way, the inability to recognize acceleration data can reduce the inability of electronic devices to perform anti-shake processing, image output and other processes, and reduce display freezes.

[0023] In one possible implementation, when the acceleration in each direction in the acceleration data is less than a third threshold, a second preset value is transmitted to the first chip through the second chip, including: when the acceleration in each direction in the acceleration data is less than the third threshold, the second chip adds 1 to the number of second counts; the second count is a count of the case where the acceleration in each direction in the acceleration data is less than the third threshold; when the second count is greater than a fourth threshold, the second preset value is transmitted to the first chip through the second chip.

[0024] In this way, the accidental impact of acceleration data errors can be reduced.

[0025] In a possible implementation, the first preset value is (0, 0, 0); the second preset value is (0, 9.8, 0).

[0026] In a possible implementation manner, the method further includes: the first chip performs anti-shake processing on the image captured by the camera by using the second anti-shake data to obtain a processed image.

[0027] In this way, the electronic device can perform anti-shake processing on the image captured by the camera through the second anti-shake data.

[0028] In a possible implementation, before obtaining the first anti-shake data reported by the sensor through the second chip, the method further includes: in response to an operation of entering a video recording mode, the first chip transmits a message for instructing video recording to the second chip.

[0029] In this way, the image stabilization process can be applied in the video recording mode.

[0030] In a second aspect, an embodiment of the present application provides a data processing method, which is applied to a second chip, and the method includes: the second chip obtains first anti-shake data reported by a sensor; when the first anti-shake data is not within a data range that can be recognized by the first chip, the second chip transmits second anti-shake data to the first chip, and the second anti-shake data is within the data range that can be recognized by the first chip.

[0031] In one possible implementation, the first anti-shake data includes: gyroscope data, which is used to indicate the angular velocity of the electronic device in various directions; the second anti-shake data includes: a first preset value; when the first anti-shake data is not within the data range that the first chip can recognize, the second chip transmits the second anti-shake data to the first chip, including: when the data in at least one direction in the gyroscope data is greater than a first threshold, the second chip transmits the first preset value to the first chip.

[0032] In one possible implementation, when data in at least one direction of the gyroscope data is greater than a first threshold, a first preset value is transmitted to the first chip, including: when data in at least one direction of the gyroscope data is greater than the first threshold, the second chip adds 1 to the number of first counts; the first count is a count of the case where data in at least one direction of the gyroscope data is greater than the first threshold; when the first count is greater than the second threshold, the second chip transmits the first preset value to the first chip.

[0033] In a possible implementation, the first anti-shake data further includes: a first timestamp, which is a timestamp when the sensor reports the gyroscope data; and the second anti-shake data further includes: a second timestamp, which is later than the first timestamp.

[0034] In a possible implementation, the first anti-shake data also includes: a third timestamp, which is the timestamp when the sensor reports the acceleration data; the acceleration data is used to indicate the acceleration of the electronic device in various directions; the second anti-shake data also includes: a fourth timestamp, which is later than the third timestamp.

[0035] In a possible implementation, the first anti-shake data includes: acceleration data, and the second anti-shake data includes: a second preset value; when the first anti-shake data is not within the data range that the first chip can recognize, the second chip transmits the second anti-shake data to the first chip, including: when the acceleration in each direction in the acceleration data is less than a third threshold, the second chip transmits the second preset value to the first chip.

[0036] In one possible implementation, when the acceleration in each direction in the acceleration data is less than a third threshold, the second chip transmits a second preset value to the first chip, including: when the acceleration in each direction in the acceleration data is less than the third threshold, the second chip adds 1 to the number of second counts; the second count is a count of the case where the acceleration in each direction in the acceleration data is less than the third threshold; when the second count is greater than a fourth threshold, the second chip transmits the second preset value to the first chip.

[0037] In a possible implementation, the first preset value is (0, 0, 0); the second preset value is (0, 9.8, 0).

[0038] In a possible implementation, before the second chip obtains the first anti-shake data reported by the sensor, the method further includes: the second chip receives a message for instructing video recording, where the message is generated by the first chip in response to an operation of entering a video recording mode.

[0039] In a third aspect, an embodiment of the present application provides a data processing device, which may be an electronic device, or a chip or chip system in an electronic device. The data processing device may include a communication unit and a processing unit. When the data processing device is an electronic device, the communication unit is used to perform the step of data transmission, and the processing unit is used to implement the step of responding to the first anti-shake data reported by the sensor, so that the electronic device implements the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect. A data processing method described in the method. When the data processing device is an electronic device, the processing unit may be a processor. The communication unit is used to perform the step of data transmission. The data processing device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the electronic device implements a data processing method described in the first aspect or any possible implementation of the first aspect. When the data processing device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit, so that the electronic device implements a data processing method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit within the chip (eg, a register, a cache, etc.), or a storage unit within the electronic device that is located outside the chip (eg, a read-only memory, a random access memory, etc.).

[0040] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0041] In a sixth aspect, the present application provides a chip, comprising a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the second aspect or any possible implementation of the second aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run on a computer, the computer executes the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0044] In a sixth aspect, the present application provides a chip system, the chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect. The communication interface in the chip can be an input / output interface, a pin, or a circuit, etc.

[0045] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0046] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of a data processing method provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a processing flow for determining abnormal gyroscope data provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of a processing flow for abnormal acceleration data judgment provided in an embodiment of the present application;

[0053] Figure 7 A flowchart of a data processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0055] 1. Other terms

[0056] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0057] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0058] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0059] 3. Electronic equipment

[0060] The electronic device of the embodiment of the present application may include a handheld device with a shooting function, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (public landmobile networks). The embodiments of the present application do not limit this.

[0061] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0062] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0063] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0064] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0065] For ease of understanding, the following Figure 1 The application scenarios involved in the embodiments of the present application are described.

[0066] For example, Figure 1 This is a schematic diagram of an interface of an electronic device provided in an embodiment of the present application. Figure 1 In the illustrated embodiment, an electronic device is taken as a mobile phone as an example for illustration, and this example does not constitute a limitation on the embodiments of the present application.

[0067] When the electronic device detects that the user has opened the camera application, the Figure 1 The interface shown in a in the figure may include: a camera control 101, a thumbnail 102, a preview screen 103, a control for switching cameras, and a shooting mode selection item. The shooting mode selection items include but are not limited to: portrait, photo, video 104, professional or more. The preview screen 103 displays the real-time picture collected by the camera.

[0068] In response to the user's triggering operation on the video 104, the electronic device displays the following Figure 1 The interface shown in b in the figure may include: a preview screen 105, a recording control 106, a thumbnail, etc. Other contents displayed in the interface are similar to Figure 1 The interface shown in a is similar and will not be described here.

[0069] In a possible design, when the electronic device is using the video recording mode for previewing, the preview screen corresponding to the video recording mode freezes, and the content of the preview screen 105 remains unchanged.

[0070] It is generally understood that the reason why the video recording of an electronic device freezes is that the software system of the electronic device fails and cannot operate normally. Therefore, when the electronic device cannot operate normally, the means adopted may be to restart the electronic device or reinstall the software system of the electronic device.

[0071] However, after the electronic device is restarted, or the software system of the electronic device is reinstalled, when the electronic device is previewing in the video recording mode, the preview screen may freeze.

[0072] Based on this, the reasons why the electronic device cannot use the video recording mode for preview are further analyzed in the embodiments of the present application. It is found that the electronic device cannot recognize the anti-shake data reported by the sensor in the electronic device, and then the electronic device cannot perform anti-shake processing on the image captured by the camera according to the anti-shake data and display the processed image, causing the preview screen to freeze.

[0073] For example, by performing data analysis on electronic devices that are stuck, it can be found that a sensor failure in the electronic device may cause the timestamp corresponding to the sensor data to be incorrect. The electronic device fails to verify the timestamp of the sensor data in the recording mode, resulting in the electronic device being unable to perform anti-shake processing on the image, and thus being unable to display the image, causing the preview screen to freeze.

[0074] In view of this, the embodiments of the present application provide a data processing method and related devices, and the electronic device can modify the first anti-shake data to the second anti-shake data within the recognizable data range when the first anti-shake data is not within the recognizable data range. The electronic device can identify the second anti-shake data, so as to perform subsequent anti-shake processing, image output and display, etc. This method can reduce the situation where the electronic device display freezes due to the anti-shake data not being within the recognizable data range, and improve the user experience.

[0075] In order to better understand the embodiments of the present application, the structure of the electronic device of the embodiments of the present application is introduced below. Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0076] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, an indicator 192, a camera 193, and a display screen 194, etc.

[0077] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc. This is not specifically limited in the embodiments of the present application.

[0078] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0079] The gyro sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. Exemplarily, the gyro sensor 180B can detect the angle of the electronic device shaking, and the electronic device can crop the image captured by the camera according to the angle, and compensate for the camera shaking through the cropped content to achieve anti-shake.

[0080] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0081] The processor 110 may include one or more processing units. Among them, different processing units may be independent devices or integrated into one or more processors. A memory may also be provided in the processor 110 for storing instructions and data. For example, the processor 110 is used to implement the steps performed in the data processing method provided in the embodiment of the present application, and to store instructions and data related to the data processing method.

[0082] The charging management module 140 is used to receive charging input from a charger, which may be a wireless charger or a wired charger. The power management module 141 is used to connect the charging management module 140 to the processor 110 .

[0083] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0084] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. For example, the GPU is used to perform the graphics rendering process in the image processing method.

[0085] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1. For example, the display screen 194 is used to display a shooting interface in a camera application, and to display a preview image or a recording image in response to a user triggering a video recording mode.

[0086] The electronic device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0087] The camera 193 is used to capture static images or videos. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1. For example, in response to a user's camera operation, the camera 193 may be used to obtain a raw image sequence.

[0088] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area. For example, the internal memory 121 can be used to store executable program codes in the image processing method.

[0089] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0090] The touch sensor can be disposed on the display screen 194, and the touch sensor and the display screen 194 form a touch screen, or "touch screen". For example, the touch sensor is used to receive a user's trigger operation for a photo control, and receive a user's trigger operation for a video recording mode.

[0091] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc., which will not be elaborated here.

[0092] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0093] For example, Figure 3 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application. The layered architecture divides the system into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces.

[0094] Taking an electronic device including two processors: an application processor (application process, AP) and a coprocessor (coprocessor, CP) as an example, the software architecture of the electronic device includes: an application processor side and a coprocessor side.

[0095] The application processor side (AP side) may include, from top to bottom: application (APP), application framework layer (FWK), hardware abstraction layer (HAL), driver layer and hardware layer.

[0096] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include: camera application, third-party camera application, etc. Among them, the third-party camera application may include: The camera application and the third-party camera application can both control the electronic device to take pictures and display the taken images.

[0097] The application framework layer provides an application programming interface (API) and a programming framework for the application of the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and a programming framework for camera applications.

[0098] The hardware abstraction layer is an interface layer between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. In the embodiment of the present application, the hardware abstraction layer may include camera hardware abstraction, non-camera sensor (NCS) service, and electronic image stabilization (EIS) processing module.

[0099] The camera hardware abstraction may provide virtual hardware of camera device 1, camera device 2, or more camera devices to control corresponding cameras to be turned on or off.

[0100] The NCS service is used to cache the sensor data on the CP side and transmit the sensor data to the EIS processing module. For example, the NCS service can obtain the sensor data from the CP side through a message interface. The message interface can be a Qualcomm message interface (QMI), etc., which is not specifically limited here.

[0101] The EIS processing module is used to perform anti-shake processing on images collected by the electronic device using sensor data. In some embodiments, the EIS processing module can determine the jitter amplitude of the electronic device through sensor data, and dynamically adjust the exposure (ISO), shutter and imaging algorithm of the electronic device to correct blur and other methods to reduce the image quality to compensate for the jitter of the electronic device and achieve anti-shake processing. Exemplarily, the EIS processing module can determine the jitter of the electronic device based on the gyroscope data, and determine the gravity direction of the electronic device based on the acceleration data, thereby determining the center picture of the image and the picture of the edge part. The EIS processing module can crop the picture of the edge part, and reversely compensate the center picture of the image based on the picture of the edge part and the jitter of the electronic device according to the gyroscope data.

[0102] The driver layer is a layer between hardware and software. The driver layer includes drivers for various hardware. The driver layer may include camera device drivers, display drivers, etc. The camera device driver is used to drive the camera to capture images.

[0103] The hardware layer includes: multiple camera devices, such as camera 1 and camera 2. The camera device can be called a camera, which is not limited here.

[0104] The coprocessor side (CP side) may include: a sensor processing layer (sensor hub) and a hardware layer. The sensor processing layer may connect and process sensor data from a variety of sensors. In an embodiment of the present application, the sensor processing layer may connect and process sensor data from an acceleration sensor and sensor data from a gyroscope sensor.

[0105] In the embodiment of the present application, the sensor processing layer includes: sensor driver. The hardware layer includes: acceleration sensor and gyroscope sensor.

[0106] The processing module is used to manage multiple sensors.

[0107] The sensor driver is used to drive the sensor to collect sensor data, determine whether the sensor data is abnormal, etc. When the data transmitted by the sensor is abnormal, the sensor driver can modify the sensor data and transmit the modified data to the AP side. When the data transmitted by the sensor is normal, the sensor driver transmits the sensor data to the AP side.

[0108] In the embodiment of the present application, the sensor driver can drive the acceleration sensor to collect the acceleration of the electronic device in various directions, obtain acceleration data, and determine whether the acceleration data is abnormal; when the acceleration data is abnormal, modify the acceleration data, and transmit the modified acceleration data to the AP side. The sensor driver can drive the gyroscope sensor to collect the angular velocity of the electronic device in various directions, obtain gyroscope data, and determine whether the gyroscope data is abnormal; when the gyroscope data is abnormal, modify the gyroscope data, and transmit the modified gyroscope data to the AP side.

[0109] In some embodiments, acceleration data is used to indicate the acceleration of the electronic device in various directions, and the acceleration data may include acceleration of three axes (e.g., acceleration of the x-axis, acceleration of the y-axis, and acceleration of the z-axis). Gyroscope data is used to indicate the angular velocity of the electronic device in various directions. Gyroscope data includes angular velocity of three axes (e.g., angular velocity of the x-axis, angular velocity of the y-axis, and angular velocity of the z-axis).

[0110] The following is a detailed description of the data processing method in the embodiment of the present application in combination with the above system structure:

[0111] In response to the user's operation of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera device by calling the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction. The camera hardware abstraction sends the instruction to the camera device driver of the kernel layer and the sensor driver on the CP side.

[0112] The camera device driver can start the corresponding camera and collect images through the camera. A camera device in the camera hardware abstraction corresponds to a camera in the hardware layer. The camera transmits the collected image to the EIS processing module in the camera hardware abstraction through the camera device driver.

[0113] Taking the case where the camera is turned on and in video recording mode as an example, the sensor driver obtains acceleration data of the electronic device and gyroscope data of the electronic device from the acceleration sensor and the gyroscope sensor respectively.

[0114] In the embodiment of the present application, when the acceleration data and gyroscope data of the electronic device are normal, the sensor driver transmits the acceleration data of the electronic device and the gyroscope data of the electronic device to the NCS service in the camera hardware abstraction on the AP side. When the acceleration data or gyroscope data of the electronic device is abnormal, the sensor driver modifies the acceleration data or gyroscope data, and transmits the modified acceleration data or modified gyroscope data to the NCS service in the camera hardware abstraction on the AP side.

[0115] After receiving the image, the EIS processing module performs anti-shake processing on the image according to the data driven and transmitted by the sensor.

[0116] It should be understood that the software architecture provided in the embodiments of the present application is merely an example and does not constitute a limitation on the embodiments of the present application.

[0117] Combine the following Figures 4 to 7 The method provided in the embodiment of the present application is described. For example, Figure 4 A flow chart of a data processing method provided in an embodiment of the present application. Taking the sensor data including acceleration data and gyroscope data as an example, Figure 4 As shown, the method includes:

[0118] S401. In response to an operation of entering a recording mode, the application calls an information interface in an NCS service to transmit a message indicating the recording mode to a sensor driver.

[0119] The operation of entering the video recording mode may be an operation in which the user clicks on the video recording mode, an operation in which the user clicks on a video call, or an operation in which the user clicks on a video recording. The embodiment of the present application does not specifically limit the operation of entering the video recording mode.

[0120] The information interface is an interface for multi-processor inter-process communication. In the embodiment of the present application, the information interface can realize data transmission between the application processor and the coprocessor. Exemplarily, the information interface can be a QMI interface, which is not specifically limited here.

[0121] After receiving the message of the recording mode, the sensor driver can implement the following S402 to S407 processing of sensor data through the application programming interface (API). Exemplarily, the API may be sensor_sensor_instance_api, which is not specifically limited here. In some embodiments, the sensor driver can implement the processing of the sensor (sensor) instance through the API interface. For example, sensor instance initialization, clearing sensor data parameters (for example, acceleration data, gyroscope data), setting sensor parameters, and notification event (notify_event) processing can be implemented. No specific limitation is made here.

[0122] S402: In response to the message for indicating the video recording mode, the sensor driver on the CP side sets a notification event, where the notification event is used to instruct to obtain acceleration data and gyroscope data.

[0123] In the embodiment of the present application, the notification event may be an interruption event or any type of event, which is not specifically limited herein. The interruption event may be understood as an event generated in the form of an interruption.

[0124] Setting a notification event includes: setting a trigger condition of the notification event. Exemplarily, taking hardware triggering as an example, setting a notification event may include setting the hardware corresponding to the notification event, setting a time interval for triggering the notification event, and the like.

[0125] Taking the hardware corresponding to the notification event as a timer, and the notification event as an interrupt event as an example, the sensor driver can set the time of the timer, etc. The sensor driver can detect the signal transmitted by the timer. When the timer ends, the sensor driver generates an interrupt event when it detects that the signal is interrupted. Exemplarily, taking the signal as a voltage signal as an example, the interruption of the signal may refer to the conversion of the voltage signal transmitted by the timer (for example, a falling edge is detected, or a rising edge is detected).

[0126] In some embodiments, the AP side may trigger the notification event function (eg, sensor_inst_notify_event) on the CP side through the above-mentioned API interface (application programming interface, API) to set the notification event.

[0127] It is understandable that in response to the API call, the sensor driver can also set the bus power on or off through the event function (for example, controlling the power-on of the accelerometer and the gyroscope sensor), set the timer stream for operating some events (for example, controlling the timer to perform timing so that the sensor reports sensor data after a certain time interval), and set the event stream for processing asynchronous COM port data streams, as well as the event stream for processing interrupts uploaded by the sensor (for example, generating an interrupt event when the timing ends), etc.

[0128] The following embodiment takes the notification event as an interrupt event as an example to illustrate the process of acquiring sensor data.

[0129] S403: The sensor driver performs interrupt detection on the signal transmitted by the hardware, and generates an interrupt event when an interrupt is detected in the signal.

[0130] Exemplarily, the sensor driver can detect the signal transmitted by the hardware through an interrupt signal stream processing function (eg, sensor_process_interrupt_data_stream). Taking a voltage signal as an example, an interrupt event is generated when a voltage conversion is detected.

[0131] S404: After generating an interrupt event, the sensor driver creates a buffer A corresponding to the acceleration data and a buffer B for the gyroscope data.

[0132] Exemplarily, the sensor driver may create a cache for storing acceleration data and a cache for storing gyroscope data by processing an interrupt event function (eg, sensor_handle_interrupt_event).

[0133] S405: The sensor driver reads acceleration data from the acceleration sensor and reads gyroscope data from the gyroscope sensor.

[0134] It is understandable that the sensor driver can trigger the reading of acceleration data and the reading of gyroscope data through one interrupt event, or can trigger the reading of acceleration data and the reading of gyroscope data respectively through two interrupt events. The embodiment of the present application does not specifically limit the order of reading acceleration data and reading gyroscope data.

[0135] The sensor driver reading acceleration data from the acceleration sensor can also be understood as the acceleration sensor reporting acceleration data to the sensor driver; the sensor driver reading gyroscope data from the gyroscope sensor can also be understood as the gyroscope sensor reporting gyroscope data to the sensor driver.

[0136] It can be understood that the acceleration sensor can transmit the acceleration data to the buffer A driven by the sensor for data storage, and the gyroscope sensor can transmit the gyroscope data to the buffer B driven by the sensor for data storage.

[0137] Exemplarily, the sensor driver may read acceleration data from cache A and gyroscope data from cache B through a data cache function (eg, sensor_process_fifo_data_buffer).

[0138] The following describes the process of determining data anomalies of acceleration data and gyroscope data by the sensor driver.

[0139] S406: The sensor driver processes the gyroscope data.

[0140] Exemplarily, the sensor driver implements the processing of the gyroscope data through a gyroscope data processing function (eg, sensor_gyro_sample or sensor_handle_gyro_sample).

[0141] In the embodiment of the present application, S406 may include: S4061-S4063.

[0142] S4061. Analyze the gyroscope data for abnormality.

[0143] Exemplarily, the gyroscope data processing function includes: a gyroscope abnormality judgment function (eg, check_sensor_gyro_data_unusual). The abnormality judgment of the gyroscope data is implemented by the gyroscope abnormality judgment function.

[0144] The process of abnormal judgment of gyroscope data can refer to the following Figure 5 The corresponding instructions are not repeated here.

[0145] S4062: When the gyroscope data is abnormal, modify the gyroscope data to a preset value A and a timestamp A.

[0146] In the embodiment of the present application, timestamp A is later than the timestamp corresponding to the gyroscope data.

[0147] In some embodiments, timestamp A is the timestamp of transmitting gyroscope data from the CP side to the AP side.

[0148] In some other embodiments, timestamp A may also be the timestamp of the gyroscope data encapsulated by the CP side. It is understandable that the CP side may encapsulate the gyroscope data to facilitate data transmission or identification and reading of data by the AP side. The sensor driver may modify the current timestamp when encapsulating the gyroscope data. The embodiments of the present application do not specifically limit timestamp A.

[0149] The timestamp corresponding to the gyroscope data can be added by the gyroscope sensor (for example, added when collecting gyroscope data) or added by the sensor driver (for example, the timestamp added by the sensor driver when acquiring gyroscope data, the timestamp added when storing gyroscope data). There is no specific limitation on the timestamp corresponding to the gyroscope data in the embodiment of the present application.

[0150] In some embodiments, timestamp A is recorded as a timestamp of the current time. Timestamp A is represented by sns_get_system_time.

[0151] The preset value A may be an empirical value or may be obtained by statistically analyzing normal data according to an algorithm. The embodiment of the present application does not limit the method for determining the preset value A. For example, the preset value A may be (0, 0, 0) or (1, 1, 1), which is not specifically limited here.

[0152] In this way, when the gyroscope data is abnormal, the gyroscope data and timestamp are modified so that subsequent electronic devices can perform image processing based on the modified gyroscope data and output images normally. In addition, modifying the timestamp can reduce the disorder in the device timing, reduce the mismatch between the timestamp of the gyroscope data and the timestamp obtained by the subsequent EIS processing module, so that the timestamp of the gyroscope data is less than the timestamp matched by the EIS processing module, and reduce the abnormal phenomenon of the camera recording screen being stuck. S4063. When the gyroscope data is normal, the gyroscope data and the timestamp corresponding to the gyroscope data are not modified.

[0153] S407: The sensor driver processes the acceleration data.

[0154] Exemplarily, the sensor driver implements the processing of gyroscope data through an acceleration data processing function (eg, sensor_acc_sample or sensor_handle_acc_sample).

[0155] In this embodiment of the application, S407 may include: S4071-S4075.

[0156] S4071. Detect abnormality in acceleration data.

[0157] Exemplarily, the acceleration data processing function includes: an acceleration abnormality judgment function (eg, check_sensor_acc_data_unusual). The acceleration abnormality judgment function is used to realize abnormality judgment of acceleration data.

[0158] The process of abnormal judgment of acceleration data can refer to the following Figure 6 The corresponding instructions are not repeated here.

[0159] S4072: When the acceleration data is abnormal, modify the acceleration data to a preset value B.

[0160] The preset value B may be an empirical value or may be obtained by statistically analyzing normal data according to an algorithm. The embodiment of the present application does not limit the method for determining the preset value B. For example, the preset value B may be (0, 9.8, 0) or (0, 10, 0), which is not specifically limited here.

[0161] S4073: When the acceleration data is normal, the acceleration data is not modified.

[0162] S4074: When the acceleration data is normal, the gyroscope data is judged to be abnormal.

[0163] In some embodiments, the following Figure 5 The gyroscope data can be judged by the process shown in the figure, or by querying the identifier (for example, the following Figure 5 The identification A in the sensor driver realizes abnormal judgment of the gyroscope data (the sensor driver adds an identification when the gyroscope data is abnormal), which is not specifically limited here. S4075, when the acceleration data is normal and the gyroscope data is abnormal, the sensor driver modifies the timestamp of the acceleration data to timestamp B. Timestamp B is later than the timestamp corresponding to the gyroscope data.

[0164] In some embodiments, timestamp B is the timestamp of acceleration data transmitted from the CP side to the AP side.

[0165] In some other embodiments, timestamp B may also be the timestamp of the acceleration data encapsulated by the CP side. It is understandable that the CP side may encapsulate the acceleration data to facilitate data transmission or identification and reading of the data by the AP side. The sensor driver may modify the acceleration data to the current timestamp when encapsulating the acceleration data. The embodiments of the present application do not specifically limit the timestamp B.

[0166] The timestamp corresponding to the acceleration data may be added by the acceleration sensor (for example, added when the acceleration data is collected) or by the sensor driver (for example, the timestamp added by the sensor driver when acquiring the acceleration data or the timestamp added when storing the acceleration data). There is no specific limitation on the timestamp corresponding to the acceleration data in the embodiments of the present application.

[0167] In some embodiments, timestamp B is recorded as a timestamp of the current time. Timestamp B is represented by sns_get_system_time.

[0168] Timestamp B may be the same as or different from timestamp A, and is not specifically limited here.

[0169] S4076: When the acceleration data and the gyroscope data are normal, the sensor driver does not modify the timestamp of the acceleration data.

[0170] S408: The sensor driver transmits the processed sensor data to the NCS service on the AP side. Adaptively, the NCS service stores the processed sensor data.

[0171] Exemplarily, the sensor driver may transmit the processed sensor data to the NCS service through a sending function (eg, pb_send_sensor_stream function).

[0172] In the embodiment of the present application, when the acceleration data and the gyroscope data are normal, the processed sensor data includes: acceleration data, a timestamp corresponding to the acceleration data, gyroscope data, and a timestamp corresponding to the gyroscope data.

[0173] When the acceleration data is normal and the gyroscope data is abnormal, the processed sensor data includes: acceleration data, timestamp B, preset value A, and timestamp A. In this way, it can reduce the verification failure caused by the mismatch between the timestamp of the AP side image and the timestamp corresponding to the gyroscope data, or reduce the verification failure caused by the mismatch between the timestamp of the AP side image and the timestamp corresponding to the acceleration data, and then the electronic device cannot perform anti-shake processing and output images. When the acceleration data and preset value A are transmitted at the same time, timestamp A

[0174] When the acceleration data is abnormal and the gyroscope data is normal, the processed sensor data includes: a preset value B, a timestamp corresponding to the acceleration data, the gyroscope data, and a timestamp corresponding to the gyroscope data.

[0175] When the acceleration data and the gyroscope data are abnormal, the processed sensor data includes: a preset value B, a timestamp corresponding to the acceleration data, a preset value A, and a timestamp A.

[0176] In some embodiments, the sensor data and the processed sensor data further include: the name of the gyroscope sensor, an event request regarding the gyroscope sensor; the name of the acceleration sensor, an event request regarding the acceleration sensor, and the like.

[0177] It is understandable that the sensor driver may send the processed acceleration data and the processed gyroscope data simultaneously, or may send the processed acceleration data and the processed gyroscope data separately, which is not specifically limited here.

[0178] In the embodiment of the present application, the sensor driver may transmit the processed sensor data to the NCS service on the AP side at a certain time interval. The sensor driver may also transmit the processed sensor data to the NCS service on the AP side when the NCS service is called through the information interface. No specific limitation is made here.

[0179] S409 , the EIS processing module performs anti-shake processing on the image according to the sensor data processed in the NCS service to obtain a processed image.

[0180] The EIS processing module can determine the jitter of the electronic device based on the gyroscope data, determine the gravity direction of the electronic device based on the acceleration data, and determine the center of the image and the edge of the image based on the jitter of the electronic device and the gravity direction. The EIS processing module can cut off the edge of the image, and perform reverse compensation on the center of the image based on the edge of the image and the jitter of the electronic device according to the gyroscope data.

[0181] In some embodiments, the EIS processing module can align the timestamp of the image with the timestamp corresponding to the gyroscope data. The timestamp of the image is the time when the camera captures the image. The camera driver will add the start of frame (SOF) time, and then combine the rolling shutter time and exposure time to obtain the time range covered by the image exposure.

[0182] S410 : The EIS processing module transmits the processed image to the application for display.

[0183] In some embodiments, the NCS service may store the sensor data in a cache pool. The EIS processing module may obtain the sensor data from the cache pool, and when the timestamp corresponding to the sensor data in the cache of the NCS service is greater than the timestamp matched by the EIS processing module, the EIS processing module may perform anti-shake processing on the image.

[0184] In this way, the CP side can modify the first anti-shake data to the pre-set data when the sensor data is abnormal. The electronic device can identify the pre-set data, so as to perform subsequent anti-shake processing, image output and display, etc. This method can reduce the situation where the electronic device display freezes due to abnormal sensor data.

[0185] For example, Figure 5 A schematic diagram of a processing flow for determining abnormal gyroscope data provided by an embodiment of the present application. Figure 5 As shown, the method includes:

[0186] S501: The sensor driver determines whether at least one of the three-axis data in the gyroscope data is abnormal. This can also be understood as the sensor driver determining whether the three-axis data are all normal.

[0187] In the embodiment of the present application, the electronic device is pre-set with a threshold value A. When the data of one axis in the three-axis data of the gyroscope data is greater than the threshold value A, the axis data is abnormal. For example, taking the data of the X axis as 80 and the threshold value A as 60 as an example, the X axis data is abnormal.

[0188] It can be understood that the situation equal to the threshold value A may correspond to abnormal data or normal data, which is not specifically limited here.

[0189] Threshold A may be an empirical value, or may be obtained by statistically analyzing abnormal data using an algorithm. The embodiment of the present application does not limit the method for determining threshold A.

[0190] When the data of at least one axis in the three-axis data is abnormal (for example, the data of at least one axis is greater than threshold A), the electronic device executes S502 and S503; when the data of the three axes are all normal (for example, the data of the three axes are all less than threshold A), the electronic device does not modify the gyroscope data.

[0191] S502: The sensor driver adds 1 to the count of the number of abnormal times of the gyroscope data, and determines whether the number of abnormal times reaches a threshold value B.

[0192] The threshold B can be 50, 100 or any value, which is not specifically limited here.

[0193] When the number of abnormalities does not reach the threshold, the sensor driver does not modify the gyroscope data and the timestamp corresponding to the gyroscope data.

[0194] In this way, by judging the number of abnormalities, the abnormal data in accidental situations can be reduced, and the modification of the gyroscope data can be reduced. When the gyroscope data is continuously abnormal, the gyroscope data is modified.

[0195] In some embodiments, the sensor driver may not count the number of abnormal times, and when the gyroscope data is abnormal, the gyroscope data is modified to the preset value A and the timestamp A.

[0196] S503: When the number of abnormalities reaches a threshold value B, the sensor driver modifies the gyroscope data to a preset value A and a timestamp A. The timestamp A is the current time of the system.

[0197] In some embodiments, the sensor driver adds an identifier A, and when the identifier A is recognized, the gyroscope data is modified to a preset value A and a timestamp A. The identifier A is used to indicate that the gyroscope data is abnormal, or that the event corresponding to the gyroscope data is an abnormal event.

[0198] Exemplarily, when the number of abnormalities reaches threshold B, the sensor driver outputs false, the sensor driver adds identification A to record the gyroscope abnormal event, and when the gyroscope abnormal event with identification A is identified, the gyroscope data is modified to preset value A and timestamp A.

[0199] Exemplarily, timestamp A is sns_get_system_time.

[0200] In some embodiments, after the number of abnormalities reaches a threshold value B, the sensor driver modifies the gyroscope data received from the gyroscope sensor to a preset value A within a period of time, and modifies the timestamp corresponding to the gyroscope data. In this way, the judgment of the gyroscope data within a period of time can be reduced, and the workload of the judgment can be reduced.

[0201] Above Figure 5 The embodiment shown in the figure illustrates the abnormality judgment process of the gyroscope data. Figure 6 The abnormal flow of acceleration data is explained.

[0202] For example, Figure 6 A schematic diagram of a processing flow for abnormal acceleration data judgment provided in an embodiment of the present application. Figure 6 As shown, the method includes:

[0203] S601: The sensor driver determines whether the three-axis data in the acceleration data are all less than a threshold C.

[0204] In the embodiment of the present application, the electronic device is pre-set with a threshold C. When the three-axis data in the acceleration data are all less than the threshold C, the acceleration data is abnormal. When at least one of the three-axis data in the acceleration data is greater than the threshold C, the acceleration data is normal.

[0205] It can be understood that the situation where the value is equal to the threshold value C may correspond to abnormal data or normal data, which is not specifically limited here.

[0206] The threshold C may be an empirical value or may be obtained by counting abnormal data according to an algorithm. The embodiment of the present application does not limit the method for determining the threshold C. For example, the preset value C may be 60 meters per square second, or any value, which is not specifically limited here.

[0207] When the three-axis data are all smaller than the threshold C, the electronic device executes S602 and S603; when at least one of the three-axis data is larger than the threshold C, the electronic device does not modify the acceleration data.

[0208] S602: The sensor driver adds 1 to the count of the number of abnormal times of acceleration data, and determines whether the number of abnormal times reaches a threshold value D.

[0209] The threshold D can be 50, 100 or any value, and is not specifically limited here.

[0210] S603: When the number of abnormalities reaches a threshold value D, the sensor drives the acceleration data to be modified to a preset value B.

[0211] In some embodiments, when the number of abnormal times reaches a threshold value D, the timestamp corresponding to the acceleration is also modified to timestamp B.

[0212] When the number of abnormalities reaches a threshold value D, the sensor driver may add a flag B. By identifying the flag B, the gyroscope data is modified to a preset value B and a timestamp A. The flag B is used to indicate that the acceleration data is abnormal, or that the event corresponding to the acceleration data is an abnormal event.

[0213] Exemplarily, when the number of abnormalities reaches a threshold value D, the sensor driver outputs false, the sensor driver adds a flag B to record the acceleration abnormality event, and modifies the acceleration data to a preset value B when the flag B is identified.

[0214] When the number of abnormalities does not reach the threshold value D, the sensor driver does not modify the acceleration data.

[0215] In this way, by judging the number of abnormalities, the abnormal data in accidental situations can be reduced, and the modification of acceleration data can be reduced. When the acceleration data is continuously abnormal, the acceleration data is modified.

[0216] In some embodiments, when the acceleration data is normal, the sensor driver also determines whether the gyroscope data is normal. When the gyroscope data is abnormal, the timestamp corresponding to the acceleration is modified to timestamp B. The determination of abnormal gyroscope data can refer to the above corresponding description, which will not be repeated here.

[0217] In some embodiments, after the number of abnormalities reaches a threshold value D, the sensor driver modifies the acceleration data received from the acceleration sensor to a preset value B within a period of time, and modifies the timestamp corresponding to the acceleration data. In this way, the judgment of the acceleration data within a period of time can be reduced, and the workload of the judgment can be reduced.

[0218] The data method of the embodiment of the present application can be applied to the anti-shake processing in the video recording mode, and can also be applied to the anti-shake processing in other modes, for example, the anti-shake processing in the camera photo mode, which is not specifically limited here.

[0219] For example, Figure 7 A flowchart of a data processing method provided in an embodiment of the present application. Taking an electronic device including a first chip, a second chip and a sensor as an example, Figure 7 As shown, the method includes:

[0220] S701. Obtain first anti-shake data reported by a sensor through a second chip.

[0221] In the embodiment of the present application, the first chip is located on the AP side, which may correspond to the application processor mentioned above, that is, the second chip is located on the CP side, which may correspond to the coprocessor mentioned above.

[0222] The first anti-shake data may be data collected by an anti-shake related sensor. For example, gyroscope data collected by a gyroscope sensor, acceleration data collected by an acceleration sensor, Hall data collected by a Hall sensor, etc. The acceleration data and gyroscope data may refer to the above corresponding descriptions, which will not be repeated here. Hall data is used to indicate the position of the lens in the camera.

[0223] S702: When the first anti-shake data is not within a data range recognizable by the first chip, transmit second anti-shake data to the first chip through the second chip, where the second anti-shake data is within a data range recognizable by the first chip.

[0224] The data range can be the range of anti-shake data in the shooting scene. Normal shaking of electronic devices when shooting

[0225] In the embodiment of the present application, when the first anti-shake data is not within the data range that can be recognized by the first chip due to one or more reasons, for example, when the sensor fails, when the sensor reports a transmission error, etc., no specific limitation is made here.

[0226] It can be understood that when the first anti-shake data is within the data range that can be recognized by the first chip, the first anti-shake data is transmitted to the first chip through the second chip.

[0227] In this way, when the first anti-shake data is not within the recognizable data range, the first anti-shake data is modified to the second anti-shake data within the recognizable data range. The electronic device can identify the second anti-shake data, thereby performing subsequent anti-shake processing, image output and display, etc. This method can reduce the situation where the electronic device display freezes due to the anti-shake data not being within the recognizable data range.

[0228] In one possible implementation, the first anti-shake data includes: gyroscope data, which is used to indicate the angular velocity of the electronic device in various directions; the second anti-shake data includes: a first preset value; when the first anti-shake data is not within the data range that can be recognized by the first chip, the second anti-shake data is transmitted to the first chip through the second chip, including: when the data in at least one direction in the gyroscope data is greater than a first threshold, the first preset value is transmitted to the first chip through the second chip.

[0229] It can be understood that, when the data in each direction of the gyroscope data is less than or equal to the first threshold, the first anti-shake data is transmitted to the first chip through the second chip.

[0230] In the above embodiments, the situation where the value is equal to the first threshold corresponds to normal data, and the first anti-shake data is transmitted; in other embodiments, the situation where the value is equal to the first threshold may correspond to abnormal data. For example, when the data in at least one direction of the gyroscope data is greater than or equal to the first threshold, the first preset value is transmitted to the first chip through the second chip; no specific limitation is made here.

[0231] The first threshold value may correspond to the threshold value A mentioned above. The first preset value may correspond to the preset value A mentioned above.

[0232] In this way, the first threshold can be used to confirm whether the gyroscope data is within the data range that can be recognized by the first chip, thereby reducing the inability of the electronic device to perform anti-shake processing, image output, etc. due to the inability to recognize the gyroscope data, and reducing the display freeze phenomenon.

[0233] In the above embodiment, the same threshold is used to determine whether the angular velocity in each direction is abnormal, thereby determining whether the gyroscope data is abnormal. The embodiment of the present application can also determine whether the gyroscope data is abnormal in any manner. Exemplarily, a threshold is set for each direction, and the angular velocity in each direction is determined by the corresponding threshold, thereby determining whether the gyroscope data is abnormal. The thresholds corresponding to each direction may be the same or different. The embodiment of the present application does not limit the specific implementation of the determination of gyroscope data abnormality.

[0234] In a possible implementation, when data in at least one direction of the gyroscope data is greater than a first threshold, transmitting a first preset value to the first chip through the second chip, including: when data in at least one direction of the gyroscope data is greater than the first threshold, the second chip adds 1 to the number of first counts; the first count is a count of the case where data in at least one direction of the gyroscope data is greater than the first threshold; when the first count is greater than the second threshold, transmitting the first preset value to the first chip through the second chip. It can be understood that when the first count is less than or equal to the second threshold, the first anti-shake data is transmitted to the first chip through the second chip.

[0235] The second threshold value may correspond to the threshold value B mentioned above. In the above embodiments, the case where the second threshold value is equal to the second threshold value corresponds to normal data, and the first anti-shake data is transmitted; in other embodiments, the case where the second threshold value is equal to the second threshold value may correspond to abnormal data, for example, when the first count is greater than or equal to the second threshold value, the first preset value is transmitted to the first chip through the second chip; no specific limitation is made here.

[0236] In this way, the abnormal situations are counted until the second threshold is reached, so that the accidental influence of gyroscope data errors can be reduced.

[0237] In a possible implementation, the first anti-shake data further includes: a first timestamp, which is a timestamp when the sensor reports the gyroscope data; and the second anti-shake data further includes: a second timestamp, which is a timestamp of the current time.

[0238] In this way, the disorder of timestamp sorting can be reduced, which causes electronic devices to be unable to perform anti-shake processing, image output and display, and reduces display freezes.

[0239] In a possible implementation, the first anti-shake data includes: acceleration data, and the second anti-shake data includes: a second preset value; when the first anti-shake data is not within the data range that the first chip can recognize, the second anti-shake data is transmitted to the first chip through the second chip, including: when the acceleration in each direction of the acceleration data is less than a third threshold, the second preset value is transmitted to the first chip through the second chip. It can be understood that when the acceleration in any direction of the acceleration data is greater than or equal to the third threshold, the first anti-shake data is transmitted to the first chip through the second chip.

[0240] In the above embodiments, the situation where the value is equal to the third threshold corresponds to normal data, and the first anti-shake data is transmitted; in other embodiments, the situation where the value is equal to the third threshold may correspond to abnormal data. For example, when the acceleration in each direction in the acceleration data is less than or equal to the third threshold, the second preset value is transmitted to the first chip through the second chip; no specific limitation is made here.

[0241] The third threshold may correspond to threshold C mentioned above.

[0242] In this way, the third threshold can be used to confirm whether the acceleration data is within the data range that can be recognized by the first chip, which can reduce the inability of the electronic device to perform anti-shake processing, image output and other processes due to the inability to recognize the acceleration data, and reduce the display freeze phenomenon.

[0243] In a possible implementation, when the acceleration in each direction in the acceleration data is less than the third threshold, the second preset value is transmitted to the first chip through the second chip, including: when the acceleration in each direction in the acceleration data is less than the third threshold, the second chip adds 1 to the number of second counts; the second count is the count of the acceleration in each direction in the acceleration data is less than the third threshold; when the second count is greater than the fourth threshold, the second preset value is transmitted to the first chip through the second chip. It can be understood that when the second count is less than or equal to the fourth threshold, the first anti-shake data is transmitted to the first chip through the second chip.

[0244] In the above embodiments, the situation equal to the fourth threshold corresponds to normal data, and the first anti-shake data is transmitted; in other embodiments, the situation equal to the fourth threshold may correspond to abnormal data, for example, when the second count is greater than or equal to the fourth threshold, the second preset value is transmitted to the first chip through the second chip; no specific limitation is made here.

[0245] The fourth threshold may correspond to the threshold D mentioned above.

[0246] In this way, the abnormal situations are counted until the fourth threshold is reached, so that the accidental influence of acceleration data errors can be reduced.

[0247] In a possible implementation, the first anti-shake data also includes: a third timestamp, which is the timestamp of the acceleration data reported by the sensor; the acceleration data is used to indicate the acceleration of the electronic device in various directions; the second anti-shake data also includes: a second timestamp.

[0248] In this way, the disorder of timestamp sorting can be reduced, which causes electronic devices to be unable to perform anti-shake processing, image output and display, and reduces display freezes.

[0249] In a possible implementation, the first preset value is (0, 0, 0); the second preset value is (0, 9.8, 0).

[0250] In this way, the impact on the anti-shake processing is small. The first preset value and the second preset value may also be other values, which are not specifically limited here.

[0251] In a possible implementation manner, the method further includes: the first chip performs anti-shake processing on the image captured by the camera by using the second anti-shake data to obtain a processed image.

[0252] In this way, the electronic device can perform anti-shake processing on the image captured by the camera through the second anti-shake data.

[0253] In a possible implementation, before obtaining the first anti-shake data reported by the sensor through the second chip, the method further includes: in response to an operation of entering a video recording mode, the first chip transmits a message for instructing video recording to the second chip.

[0254] In this way, the image stabilization process can be applied in the video recording mode.

[0255] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0256] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0257] The data processing method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above method.

[0258] The embodiment of the present application provides a data processing device, which may include a communication unit and a processing unit. The communication unit is used to perform the step of data transmission, and the processing unit is used to implement the step of responding to the first anti-shake data reported by the sensor, so that the electronic device implements the above data processing method.

[0259] The data processing method provided in the embodiment of the present application can be applied to an electronic device with a shooting function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be repeated here.

[0260] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0261] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0262] The embodiment of the present application provides a chip system, which includes at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the technical solution in the above embodiment. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0263] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0264] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0265] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0266] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0267] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: Applied to an electronic device, the electronic device includes a first chip, a second chip and a sensor, and the method includes: Obtaining first anti-shake data reported by the sensor through the second chip; wherein the first anti-shake data includes: gyroscope data and / or acceleration data; In a case where the first anti-shake data is not within a data range recognizable by the first chip, second anti-shake data is transmitted to the first chip through the second chip, and the second anti-shake data is within a data range recognizable by the first chip; wherein the first anti-shake data further includes: a timestamp of the first anti-shake data; the timestamp of the first anti-shake data is a timestamp of the first anti-shake data reported by the sensor; and the timestamp included in the second anti-shake data is later than the timestamp of the first anti-shake data; The first chip performs anti-shake processing on the image captured by the camera according to the second anti-shake data to obtain a processed image.

2. The method according to claim 1, characterized in that The gyroscope data is used to indicate the angular velocity of the electronic device in various directions; When the first anti-shake data includes gyroscope data, the second anti-shake data includes: a first preset value; The transmitting, by the second chip, second anti-shake data to the first chip when the first anti-shake data is not within the data range recognizable by the first chip comprises: When data of at least one direction in the gyroscope data is greater than a first threshold, the first preset value is transmitted to the first chip through the second chip.

3. The method according to claim 2, characterized in that When data of at least one direction in the gyroscope data is greater than a first threshold, transmitting the first preset value to the first chip through the second chip includes: When the data in at least one direction of the gyroscope data is greater than the first threshold, the second chip adds 1 to the number of first counts; the first count is a count of the number of times when the data in at least one direction of the gyroscope data is greater than the first threshold; When the first count is greater than a second threshold, the first preset value is transmitted to the first chip through the second chip.

4. The method according to claim 2 or 3, characterized in that: The timestamp of the first anti-shake data includes: a first timestamp, which is the timestamp when the sensor reports the gyroscope data; the timestamps included in the second anti-shake data include: a second timestamp, which is later than the first timestamp.

5. The method according to claim 2 or 3, characterized in that: The timestamp of the first anti-shake data includes: a third timestamp, which is the timestamp of the acceleration data reported by the sensor; the acceleration data is used to indicate the acceleration of the electronic device in various directions; the timestamps included in the second anti-shake data include: a fourth timestamp, which is later than the third timestamp.

6. The method according to any one of claims 1 to 3, characterized in that: When the first anti-shake data includes acceleration data, the second anti-shake data includes: a second preset value; The transmitting, by the second chip, second anti-shake data to the first chip when the first anti-shake data is not within the data range recognizable by the first chip comprises: When the accelerations in each direction in the acceleration data are all less than a third threshold, the second preset value is transmitted to the first chip through the second chip.

7. The method according to claim 6, characterized in that When the accelerations in each direction in the acceleration data are all less than a third threshold, transmitting the second preset value to the first chip through the second chip includes: When the acceleration in each direction in the acceleration data is less than the third threshold, the second chip adds 1 to the number of second counts; the second count is the count of the case where the acceleration in each direction in the acceleration data is less than the third threshold; When the second count is greater than a fourth threshold, the second preset value is transmitted to the first chip through the second chip.

8. The method according to any one of claims 1 to 3 and 7, characterized in that: The first preset value is (0, 0, 0); the second preset value is (0, 9.8, 0).

9. The method according to any one of claims 1 to 3 and 7, characterized in that: Before obtaining the first anti-shake data reported by the sensor through the second chip, the method further includes: In response to an operation of entering a video recording mode, the first chip transmits a message for instructing video recording to the second chip.

10. A data processing method, characterized in that: Applied to the second chip, the method includes: The second chip obtains first anti-shake data reported by the sensor; wherein the first anti-shake data includes: gyroscope data and / or acceleration data; When the first anti-shake data is not within the data range recognizable by the first chip, the second chip transmits the second anti-shake data to the first chip, so that the first chip performs anti-shake processing on the image captured by the camera through the second anti-shake data to obtain a processed image, and the second anti-shake data is within the data range recognizable by the first chip; wherein the first anti-shake data also includes: a timestamp of the first anti-shake data; the timestamp of the first anti-shake data is the timestamp of the first anti-shake data reported by the sensor; the timestamp included in the second anti-shake data is later than the timestamp of the first anti-shake data.

11. The method according to claim 10, characterized in that The gyroscope data is used to indicate the angular velocity of the electronic device in various directions; when the first anti-shake data includes the gyroscope data, the second anti-shake data includes: a first preset value; When the first anti-shake data is not within the data range recognizable by the first chip, the second chip transmits the second anti-shake data to the first chip, including: When data of at least one direction in the gyroscope data is greater than a first threshold, the second chip transmits the first preset value to the first chip.

12. The method according to claim 11, characterized in that When data of at least one direction in the gyroscope data is greater than a first threshold, transmitting the first preset value to the first chip includes: When the data in at least one direction of the gyroscope data is greater than the first threshold, the second chip adds 1 to the number of first counts; the first count is a count of the number of times when the data in at least one direction of the gyroscope data is greater than the first threshold; When the first count is greater than a second threshold, the second chip transmits the first preset value to the first chip.

13. The method according to claim 11 or 12, characterized in that: The timestamp of the first anti-shake data includes: a first timestamp, which is the timestamp when the sensor reports the gyroscope data; the timestamps included in the second anti-shake data include: a second timestamp, which is later than the first timestamp.

14. The method according to claim 11 or 12, characterized in that: The timestamp of the first anti-shake data includes: a third timestamp, the third timestamp is the timestamp of the acceleration data reported by the sensor; the acceleration data is used to indicate the acceleration of the electronic device in various directions; The timestamp included in the second anti-shake data includes: a fourth timestamp, and the fourth timestamp is later than the third timestamp.

15. The method according to any one of claims 10 to 12, characterized in that: When the first anti-shake data includes acceleration data, the second anti-shake data includes: a second preset value; When the first anti-shake data is not within the data range recognizable by the first chip, the second chip transmits the second anti-shake data to the first chip, including: When the accelerations in each direction in the acceleration data are all less than a third threshold, the second chip transmits the second preset value to the first chip.

16. The method according to claim 15, characterized in that When the acceleration in each direction in the acceleration data is less than the third threshold, the second chip transmits the second preset value to the first chip, including: When the acceleration in each direction in the acceleration data is less than the third threshold, the second chip adds 1 to the number of second counts; the second count is the count of the case where the acceleration in each direction in the acceleration data is less than the third threshold; When the second count is greater than a fourth threshold, the second chip transmits the second preset value to the first chip.

17. The method according to any one of claims 10-12 and 16, characterized in that: The first preset value is (0, 0, 0); the second preset value is (0, 9.8, 0).

18. The method according to any one of claims 10-12 and 16, characterized in that: Before the second chip obtains the first anti-shake data reported by the sensor, the method further includes: The second chip receives a message for instructing recording, where the message is generated by the first chip in response to an operation of entering a recording mode.

19. A chip, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 18.

20. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 18.

21. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 18 is implemented.

22. A chip system, characterized in that: It comprises at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is used to run a computer program or instruction to execute the method according to any one of claims 1 to 18.

23. A computer program product, characterized in that The method comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 18.

Images