Integrated controls and display system

By integrating sensor units and security modules through an integrated control device, sensor data is managed and encrypted for verification, thus solving the problem of insufficient sensor data security and achieving low-power, high-security sensor management.

CN107808104BActive Publication Date: 2024-11-05CHIPONE TECHNOLOGY (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201711239634.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2017-11-30
Publication Date
2024-11-05
Estimated Expiration
2037-11-30

AI Technical Summary

Technical Problem

In existing technologies, sensor data security is insufficient, especially since the electronic device continues to work and record data even when it is in sleep mode. This results in a heavy workload for the main control chip and huge power consumption. At the same time, sensitive data is not adequately protected, posing security risks.

Method used

The integrated control device integrates multiple sensor units and security modules. It manages sensor data acquisition and transmission through a processor and performs encryption and security verification in the security module to achieve hardware-level data protection.

Benefits of technology

This reduces the workload of the main control unit, decreases power consumption, and improves the security of sensor data, preventing malicious applications from impersonating users and enhancing system security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN107808104B_ABST
    Figure CN107808104B_ABST
Patent Text Reader

Abstract

The present application discloses an integrated control device and a display system. The integrated control device includes: a plurality of sensor units, respectively connected to a plurality of sensors, for collecting sensing data from the plurality of sensors; a first processor, for managing the plurality of sensor units to collect sensing data from the plurality of sensors, and outputting the collected sensing data according to preset rules or according to received instructions; a security module, for performing security verification and / or encrypting at least part of the sensing data under the control of the first processor; a communication interface, connected to the first processor, for communicating with a main control unit of an electronic device. By integrating a plurality of sensor units and a security module in the integrated control device, the security is improved while playing the role of sensor relay.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and more specifically, to an integrated control device and a display system. Background Art

[0002] With the advent of the big data era, the security of massive data has become a key issue that must be effectively solved in "human-computer interaction". In order to meet the growing needs of users, more and more sensors are used in electronic devices, such as gravity sensors, gyroscopes, ambient light sensors, compasses, pressure sensors, touch sensors, fingerprint sensors, microphones, cameras, etc. The control of these sensors has corresponding algorithms and control logic. Some applications even need to work and record relevant data when the electronic device is dormant, such as recording the user's walking trajectory / speed. If the main control chip of the electronic device participates in the calculation and control throughout the process, the workload of the main control chip is heavy and the power consumption is huge. In order to solve this problem, an additional chip is often used as a relay for the sensor. For example, a low-power MCU can be added between the main control chip of the electronic device and many sensors. This MCU connects various sensors to play a relay role. It can be called a sensor relay (Sensor Hub), which is responsible for scheduling and processing data from each sensor to reduce the burden on the main control chip.

[0003] However, with the development of security verification technology, more and more sensor data are used to perform security verification related operations, such as the currently commonly used fingerprint data, face images, iris images, voiceprint data, touch data when entering passwords or verification codes, etc. However, these sensor data are not fully protected as sensitive data. Usually, SensorHub directly provides the obtained sensor data to the main control chip of the electronic device. The main control chip adopts the Trusted Execution Environment (TEE) security protection mechanism. Operations such as biometric verification and PIN code verification are all performed in the TEE environment. However, TEE is a combination of software and hardware solutions, and it still cannot fully protect information from the hardware level. If the application APP manages to obtain sensitive data, it will be extremely detrimental to the protection of user privacy. Some malicious application APPs can even use sensitive data to impersonate the user's identity, which creates a huge security risk.

[0004] Therefore, further improvements in sensor data security are expected. Summary of the invention

[0005] In view of the above problems, an object of the present invention is to provide an integrated control device and a display system, wherein a variety of sensor units and security modules are integrated into the integrated control device, which not only plays a role of sensor relay but also improves security.

[0006] According to a first aspect of the present invention, there is provided an integrated control device, comprising: a plurality of sensor units, respectively connected to a plurality of sensors, for collecting sensing data from the plurality of sensors; a first processor, for managing the plurality of sensor units to collect sensing data from the plurality of sensors, and outputting the collected sensing data according to preset rules or according to received instructions; a security module, for performing security verification and / or encrypting at least a portion of the sensing data under the control of the first processor; and a communication interface, connected to the first processor, for communicating with a main control unit of the electronic device.

[0007] Preferably, the security module includes: a second processor, used to perform security verification based on at least part of the sensing data and / or encrypt at least part of the sensing data under the control of the first processor; a storage unit, used to store at least one of the sensing data, encrypted data, a verification template for security verification, and a verification result.

[0008] Preferably, the first processor is further configured to determine whether a condition for waking up the main control unit is met based on the collected sensing data, and if so, wake up the main control unit in a dormant state.

[0009] Preferably, the first processor is further used to control the security module to perform security verification according to the collected sensing data when receiving the security verification instruction, and determine whether the condition for waking up the main control unit is met according to the verification result.

[0010] Preferably, the integrated control device further comprises: a display unit, configured to provide a display driving signal to the display device; and the first processor is further configured to control the display unit to provide a display driving signal to the display device.

[0011] Preferably, at least one of the multiple sensor units includes: a first interface for communicating with a corresponding sensor; a logic module for providing a driving signal to the corresponding sensor and receiving a sensing signal from the corresponding sensor, and amplifying and performing digital-to-analog conversion on the received sensing signal to generate sensing data; and a second interface for communicating with the first processor.

[0012] Preferably, the multiple sensors include at least two of a gravity sensor, a gyroscope, an ambient light sensor, a compass, an air pressure sensor, a temperature sensor, a humidity sensor, a pressure sensor, a touch sensor, a fingerprint sensor, a palm print sensor, an audio sensor, and an image sensor.

[0013] Preferably, the integrated control device is a single chip.

[0014] According to a second aspect of the present invention, there is provided a display system, comprising: a display device for displaying an image according to display data; a plurality of sensors for generating sensing signals; and the above-mentioned integrated control device.

[0015] Preferably, the display device includes a liquid crystal display or an AMOLED display.

[0016] According to an embodiment of the present invention, the processor in the integrated control device can assist the main control unit of the electronic device to manage the data collection and transmission of various sensors. The integrated control device is also provided with a security module, which can perform data processing such as encryption, security verification, and secure storage based on the sensing data. By performing security operations such as encryption and security verification in the security module of the integrated control device, sensitive data will not be provided to the open environment of the main control unit 300 without protection, and hardware-level protection can be performed on the sensing data involved in security, which improves the system security compared to the traditional protection method combining software and hardware.

[0017] The processor of the embodiment of the present invention can use the security module to execute the judgment of the wake-up condition. For example, it can judge that the wake-up condition is met when the fingerprint verification is passed, so as to wake up the main control unit. While ensuring the low power consumption of the system, it also improves the security and improves the flexibility of the wake-up operation.

[0018] The use of an integrated control device can reduce the structural size of the existing display module, reduce the number of electronic components, reduce design complexity, and improve the yield rate.

[0019] In a preferred embodiment, each module of the integrated control device is integrated into a single chip. Furthermore, a flash memory is also integrated into the chip of the integrated control device. Since the characteristic data is locally stored inside the chip, security can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0021] Figure 1 A schematic block diagram of an electronic device with an integrated control device according to a first embodiment of the present invention is shown.

[0022] Figure 2 A schematic block diagram of a security module according to an embodiment of the present invention is shown.

[0023] Figure 3 A schematic block diagram of an electronic device with an integrated control device according to another second embodiment of the present invention is shown.

[0024] Figure 4A schematic block diagram of a sensor unit according to an embodiment of the present invention is shown.

[0025] Figure 5 A schematic block diagram of an integrated control device according to a third embodiment of the present invention is shown.

[0026] Figure 6 A schematic block diagram of an integrated control device according to a fourth embodiment of the present invention is shown.

[0027] Figure 7 A timing diagram showing display and touch control in a display system according to an embodiment of the present invention using a time division multiplexing method is shown. DETAILED DESCRIPTION

[0028] The present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are represented by similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale. In addition, some well-known parts may not be shown.

[0029] Many specific details of the present invention are described below, such as device structure, materials, dimensions, processing technology and techniques, so as to more clearly understand the present invention. However, as those skilled in the art will appreciate, the present invention may be implemented without following these specific details.

[0030] In this application, the term "local" means that the chip of the integrated control device is set at the end of the display screen, or is located on the same printed circuit board as the integrated control device. For example, "local verification" means that the verification program is executed by the processor inside the chip of the integrated control device, and "local storage" means that the non-volatile memory for storing feature data, encryption program and verification program is located inside the chip of the integrated control device, or is located on the same printed circuit board as the integrated control device.

[0031] The present invention may be embodied in various forms, some examples of which will be described below.

[0032] The integrated control device according to the embodiment of the present invention can be installed as a sensor relay in an electronic device having multiple sensors, and the processor in the integrated control device can assist the main control unit of the electronic device to manage the data collection and transmission of the multiple sensors. The integrated control device is also provided with a security module, which can perform data processing such as encryption, security verification, and secure storage based on the sensing data.

[0033] Figure 1 FIG. 2 shows a schematic block diagram of an electronic device with an integrated control device according to a first embodiment of the present invention. Figure 1 As shown, examples of the electronic device include but are not limited to mobile phones, tablet computers, laptop computers, VR devices, AR devices, watches, cars, bicycles, etc.

[0034] The electronic device is provided with a variety of sensors 100, for example, Figure 1 The example includes a first sensor 101 and a second sensor 102, and the first sensor 101 and the second sensor 102 can be different types of sensors, for example, two of the following: gravity sensor, gyroscope, ambient light sensor, compass, air pressure sensor, temperature sensor, humidity sensor, pressure sensor, touch sensor, fingerprint sensor, palm print sensor, audio sensor, image sensor, etc. However, it should be clear to those skilled in the art that the number and type of the sensor 100 are not limited thereto, and the required type and number of sensors can be selected as needed.

[0035] In addition, the electronic device further includes an integrated control device 200 and a main control unit 300 .

[0036] The main control unit 300 is used to implement the functions of the operating system, which can be implemented by an application processor (AP) of the electronic device. The integrated control device 200 can be installed in an electronic device as a sensor relay to assist the main control unit 300 in managing the data collection and transmission of the various sensors 100 and realizing secure data processing.

[0037] The integrated control device 200 includes a plurality of sensor units, a security module 202 , a processor 203 , a storage unit 204 and a communication interface 205 .

[0038] exist Figure 1 In the example, the integrated control device includes a first sensor unit 2011 and a second sensor unit 2012 (hereinafter collectively referred to as the sensor unit 201). The first sensor unit 2011 is connected to the first sensor 101 and is used to collect sensing data from the first sensor 101. The second sensor unit 2011 is connected to the second sensor 102 and is used to collect sensing data from the second sensor 102.

[0039] The security module 202 is used to perform security-related operations, such as data encryption, security verification, secure storage, etc. Figure 2 As shown, the security module 202 may include a processor 2021 and a memory 2022. The processor 2021 is used to perform security verification according to at least part of the sensing data and / or encrypt at least part of the sensing data under the control of the processor 203, and may also store at least one of the sensing data, the encrypted data, the verification template for security verification, and the verification result in the memory 2022 as needed. The memory 2022 may also store other security-related contents, such as encryption programs, security verification programs, and the like.

[0040] The processor 203 can be implemented by an MCU, and is used to manage multiple sensor units 201, and output the collected sensing data according to preset rules or according to received instructions. For example, when the multiple sensors include a touch sensor, a fingerprint sensor, a heartbeat detection sensor, a pulse detection sensor, a camera, a microphone, etc., a scheduling rule can be pre-set, for example, it can be set to send the data of the heartbeat sensor and the pulse sensor to the main control unit regularly, and send the data from the touch sensor and the fingerprint sensor to the main control unit 300 according to the instruction from the main control unit 300, and send the data of the camera and the microphone to the main control unit 300 in real time, and so on.

[0041] The processor 203 can also determine whether the conditions for waking up the main control unit 300 are met based on the collected sensing data and / or the instructions from the main control unit 300. For example, for touch wake-up, the processor 203 can periodically scan the touch sensor, and determine whether the preset position is touched or the touch time reaches a preset threshold based on the touch sensing data from the touch sensor according to a preset rule. If so, it is considered that the user's touch operation has reached the wake-up condition, otherwise it is considered that the touch has not reached the wake-up condition. Of course, multiple wake-up methods can also be set as needed, for example, shaking wake-up can be set. When the sensing data from the acceleration sensor and / or the gravity sensor shows that the shaking operation performed by the user on the electronic device has reached the wake-up condition, the processor 203 sends a wake-up signal to the main control unit 300 to wake it up. As another example, fingerprint verification wake-up, sound wake-up, temperature wake-up, pressure wake-up, etc. can also be set. The corresponding wake-up program and related data can be stored in the storage unit 204. After the main control unit 300 is awakened, the integrated control device 200 may continue to manage the sensors 100 . In some embodiments, one or more of the sensors 100 may be switched to the main control unit 300 for management.

[0042] The processor 200 can also control the security module 202 to perform security-related operations. For example, when the application needs to perform a security verification operation, such as when unlocking the device, paying, and other scenarios requiring security verification, the main control unit 300 can send a security verification request to the processor 203 through the communication interface 205, and the processor 203 can control the security module 202 to perform security verification locally. The processor 2021 of the security verification module 202 compares the collected sensing data with the verification template stored locally in the memory 2022. If it matches, it is determined that the verification is passed, otherwise it is determined that the verification fails. The security module 202 can provide the verification result to the processor 203 so that it can be sent to the main control unit 300 via the communication interface 205. Preferably, the security module 202 can also encrypt the verification result before outputting it to further improve security. As another example, when sensitive data related to security such as verification and identification needs to be obtained from the sensor 100, such as in the biometric template registration stage, the main control unit 300 can request the integrated control device 200 to perform encrypted transmission. In this case, the processor 203 of the integrated control device 200 can control the security module 202 to encrypt the above sensitive data and provide the encrypted sensitive data (such as the encrypted biometric template) to the main control unit 300. The security module 202 can also store the encrypted sensitive data in the local memory 2022 to further improve security.

[0043] The storage unit 204 may include a non-volatile memory for storing sensing data, a wake-up program, and the like.

[0044] The communication interface 205 can support multiple communication protocols and digital I / O, such as I2C protocol and SPI protocol, and provide multiple digital I / O pins. The processor 203 can communicate with the master blank unit 300 via the communication interface 231 .

[0045] According to an embodiment of the present invention, the processor in the integrated control device can assist the main control unit of the electronic device to manage the data collection and transmission of the multiple sensors. The integrated control device is also provided with a security module, which can perform data processing such as encryption, security verification, and secure storage based on the sensing data. By performing security operations such as encryption and security verification in the security module of the integrated control device, sensitive data will not be provided to the open environment of the main control unit 300 without protection, and hardware-level protection can be performed on the sensing data involved in security, which improves the system security compared to the traditional protection method combining software and hardware.

[0046] The processor of the embodiment of the present invention can use the security module to execute the judgment of the wake-up condition. For example, it can judge that the wake-up condition is met when the fingerprint verification is passed, so as to wake up the main control unit. While ensuring the low power consumption of the system, it also improves the security and improves the flexibility of the wake-up operation.

[0047] Figure 3 FIG. 2 is a schematic block diagram of an electronic device with an integrated control device according to a second embodiment of the present invention. Figure 2 Similar to the embodiment of the present invention, the electronic device includes a plurality of sensors 100, an integrated control device 210 and a main control unit 300.

[0048] The difference is at least that Figure 3 The electronic device is an electronic device with a display device 110, such as a mobile phone, a tablet computer, an AR device, a VR device, a smart watch, and the like. With the development of technology, the display screen configured on the electronic device is not only used to display images and texts, but also further developed into an important way of human-computer interaction. Touch sensors, fingerprint sensors, audio sensors (such as microphones), image sensors (such as cameras), etc. can be integrated on the display screen to form a display system. Users can directly input text, select icons, gesture control, sound, face recognition and other operations on the display screen. As the entrance for information collection and the exit for display content, the display system plays an irreplaceable and important role in the security of data interaction. The interactive data collected on the display screen includes not only text information such as keyboard input, but may also include privacy information such as fingerprints and facial features. These sensitive data will be transmitted from the driver chip of the display screen to the processor on the motherboard, and processed by the operating system to implement functions such as security verification. In Figure 3 In the embodiment of the present invention, the display device 110 and the multiple sensors 100 are integrated together to form a display system. The display device is, for example, any one selected from a liquid crystal display, an LED display, an AMOLED display, a quantum dot display, an electronic paper, and a MicroLED display. The sensor 100 is, for example, at least two selected from a touch device, a fingerprint sensor, an optical sensor, and an acoustic sensor. Figure 3 In the embodiment of the present invention, a touch-controlled liquid crystal display screen is taken as an example for description, wherein the display device is a liquid crystal display device, and the sensor 100 includes a touch sensor 120, a fingerprint sensor 130, an audio sensor 140 and an image sensor 150.

[0049] The integrated control device 210 includes a processor 211 , a storage unit 204 , a display unit 21 , a plurality of sensor units 22 , and a communication interface 205 .

[0050] like Figure 3As shown, the multiple sensor units 22 include a touch unit 221, a fingerprint unit 222, an audio unit 223 and an image unit 224, which are respectively connected to the touch sensor 120, the fingerprint sensor 130, the audio sensor 140 and the image sensor 150, and are used to collect sensing data from the corresponding sensors under the control of the processor 211 and provide it to the processor 211.

[0051] The main control unit 300 of the electronic device is used to implement the functions of the operating system, which can be implemented by the application processor (AP) of the electronic device.

[0052] The security module 202 is used to perform security-related operations, such as data encryption, security verification, secure storage, etc. Figure 2 As shown, the security module 202 may include a processor 2021 and a memory 2022. The processor 2021 is used to perform security verification and / or encrypt at least part of the sensing data according to at least part of the sensing data under the control of the processor 203. The memory 2022 may store at least one of the sensing data, encrypted data, a verification template and verification result for security verification, an encryption program, a security verification program, and the like. In this embodiment, the verification template of the memory 2022 may come from the main control unit 300, or may be obtained by local collection and data processing under the control of the processor 211. In the latter case, not only the user's sensitive data but also the verification template are generated locally. The operating system only obtains the verification result or encrypted data from the integrated control chip 210, but cannot obtain both the sensitive data and the verification template, which is conducive to improving security.

[0053] The processor 211 controls the display unit 21 and the multiple sensor units 22, manages the multiple sensor units 22 according to preset rules and / or received instructions, and wakes up the main control unit 300 in a dormant state when conditions are met. The processor 200 can also control the security module 202 to perform security-related operations, such as encrypting the sensing data according to instructions from the main control unit and / or performing security verification based on the sensing data. For example, when the device is unlocked, when paying, and in other scenarios requiring security verification, the main control unit 300 can send a security verification request to the processor 203 through the communication interface 205, and the processor 203 can control the security module 202 to perform security verification locally. The processor 2021 of the security verification module 202 compares the collected sensing data with the verification template stored locally in the memory 2022. If it matches, it is determined that the verification is passed, otherwise it is determined that the verification fails. The security module 202 can provide the verification result to the processor 203 so that it can be sent to the main control unit 300 via the communication interface 205. Preferably, the security module 202 can also encrypt the verification result before outputting it to further improve security. As another example, when it is necessary to obtain sensitive data related to security such as verification and identification from the sensor 100, such as in the biometric template registration stage, the main control unit 300 can request the integrated control device 200 to perform encrypted transmission. In this case, the processor 203 of the integrated control device 200 can control the security module 202 to encrypt the above sensitive data and provide the encrypted sensitive data (such as the encrypted biometric template) to the main control unit 300. The security module 202 can also store the encrypted sensitive data in the local memory 2022 to further improve security.

[0054] In some embodiments, a security verification wake-up method may be used, in which case the processor 211 may control the security module 202 to perform security verification based on the collected sensing data when the main control unit 300 is in a sleep state, and wake up the main control unit 300 when the security verification passes.

[0055] The processor 211 can package the sensing data, encrypted data, security verification results, etc. and provide them to the main control unit 300. For example, a start bit and a type identifier can be added before the data content, and an end bit and a check bit can be added after the data content. The main control unit 300 can identify which type the data content belongs to according to the type identifier, and can also identify which sensor the sensing data comes from.

[0056] The storage unit 204 may be a non-volatile memory, such as a flash memory, which may store sensing data, a wake-up program, and the like.

[0057] The communication interface 205 can support multiple communication protocols and digital I / O, such as I2C protocol and SPI protocol, and provide multiple digital I / O pins. The processor 211 communicates with the main control unit 300 via the communication interface 231 .

[0058] Reference below Figure 4 To describe the structure of the plurality of sensor units 22, at least one of the plurality of sensor units 22 may have Figure 4 The structure of Figure 4 As shown, the sensor unit may include a first interface 1, a second interface 2, a third interface 3 and a logic module 4. The first interface 1 is used to communicate with the corresponding sensor, the second interface 2 is used to communicate with the processor 211, and the third interface 3 is optional and is used to communicate with the main control unit 300. For example, Figure 3 The fingerprint unit 222 may include a first interface 1 for communicating with the fingerprint sensor 130, a second interface 2 for communicating with the processor 211, and a fingerprint logic module 4. The fingerprint logic module 4 may apply a driving signal to the fingerprint sensor 130 via the first interface 1 and obtain a sensing signal from the fingerprint sensor 130, amplify and convert the received sensing signal into digital-to-analog signals to generate sensing data, and provide the sensing data to the processor 211 via the second interface 2. Optionally, the fingerprint unit 222 may also include a third interface 3 for communicating with the main control unit 300, and the processor 211 may control the fingerprint unit 222 to provide the sensing data to the main control unit 300 via the third interface 3. In some embodiments, when the sensor unit 22 has both the second interface 2 and the third interface 3, two working modes may be set. In the first working mode, the sensing data collected by the sensor unit 22 is provided to the processor 211, and the main control unit 300 obtains the sensing data or the processed data thereof with the assistance of the processor 211; in the second working mode, the main control unit 300 directly obtains the sensing data from the control sensor unit 22 via the third interface 3, thereby enhancing flexibility and improving processing efficiency.

[0059] In addition, in some embodiments, the sensor unit 22 may also adopt other structures as needed. For example, for an image sensor 150 such as a camera, it can provide image data that can be processed by the processor 211 and the main control unit 300. The image unit 224 may include an image data interface for connecting to the processor 211 to achieve communication between the image sensor 150 and the processor 211. In some embodiments, the image unit 244 may include two image data interfaces, one of which is used to connect to the processor 211 and the other is used to connect to the main control unit 300. In this case, similar to the above method, the main control unit 300 can obtain image data with the assistance of the processor 211 or directly obtain data from the image sensor 150.

[0060] The integrated control device of the embodiment of the present invention integrates a display unit, multiple sensor units for multiple sensors, and a security module 202, and can be installed as a single chip in an electronic device with a display device, and has the functions of display driving, sensor relay management, and sensor data protection. On the one hand, while ensuring low power consumption of the system, hardware-level data protection can be achieved, thereby improving security; on the other hand, the system design is simplified and the workload of the main control unit is also reduced.

[0061] Figure 5 A schematic block diagram of an integrated control device according to a third embodiment of the present invention is shown. The integrated control device is suitable for electronic equipment with a liquid crystal display screen.

[0062] As shown in the figure, the integrated control chip 220 includes a processor 211 , a security module 202 , a communication interface 205 , a storage unit 204 , a display unit 21 and a sensor unit 22 .

[0063] The sensor unit 22 includes but is not limited to the above-mentioned touch unit 221, fingerprint unit 222, audio unit 223 and image unit 224, which are respectively connected to the touch sensor 120, fingerprint sensor 130, audio sensor 140 and image sensor 150, and are used to collect sensing data from corresponding sensors under the control of the processor 211 and provide it to the processor 211. At least one of the multiple sensor units 22 may have the above-mentioned reference Figure 4 In some embodiments, the integrated control chip 220 obtains the identification of each sensor through the hardware interface of the sensor unit 22. For example, it can be considered that the sensing data from the touch unit 221 is provided by the touch sensor 120, and the sensing data from the fingerprint unit 222 is provided by the fingerprint sensor 130. In an alternative embodiment, each sensor 120 to 150 can transmit the identification and the sensing signal to the integrated control chip 220, so that the integrated control chip 220 can identify which sensor the sensing data comes from.

[0064] The security module 202 is used to perform security-related operations, such as data encryption, security verification, secure storage, etc. Figure 2As shown, the security module 202 may include a processor 2021 and a memory 2022. The processor 2021 is used to perform security verification and / or encrypt at least part of the sensing data under the control of the processor 203. The memory 2022 can store at least one of the sensing data, encrypted data, a verification template and verification result for security verification, an encryption program, a security verification program, and the like. In this embodiment, the memory 2022 verification template may come from the main control unit 300, or it may be obtained by local collection and data processing under the control of the processor 211. In the latter case, not only the user's sensitive data but also the verification template are generated locally. The operating system only obtains the verification result or encrypted data from the integrated control chip 220, but cannot obtain both the sensitive data and the verification template, which is conducive to improving security.

[0065] The processor 211 is a RISC CPU of von Neumann or Harvard architecture, including but not limited to ARM, MIPS, OPENRISC, etc., preferably ARM; it can also be DSP, etc. The processor 211 can manage the multiple sensor units 22 to collect and transmit sensing data according to the received instructions. The processor 211 can also be optimized for touch detection or for other types of sensors. The touch input can be processed locally to determine whether it is necessary to wake up the operating system to process the user request. The processor 200 can also control the security module 202 to perform security-related operations, such as encrypting the sensing data and / or performing security verification based on the sensing data according to the instructions from the main control unit. In some embodiments, a security verification wake-up method can be adopted. In this case, the processor 211 can control the security module 202 to perform security verification based on the collected sensing data when the main control unit 300 is in a dormant state, and wake up the main control unit 300 when the security verification passes. The processor 211 can package the sensing data, encrypted data, security verification results, etc. and provide them to the main control unit 300. For example, a start bit and a type identifier may be added before the data content, and an end bit and a check bit may be added after the data content. The main control unit 300 may identify which type the data content belongs to based on the type identifier, and may also identify which sensor the sensing data comes from.

[0066] The communication interface 205 can support multiple communication protocols and digital I / O, such as I2C protocol and SPI protocol, and provide multiple digital I / O pins. The user interface 205 can communicate with the main processor on the mainboard.

[0067] The storage unit 204 may include a data RAM 241, a program RAM 242, a boot ROM 243, and a flash memory 244. In the flash memory 244, sensing data, a wake-up program, a scheduling rule, etc. may be stored. For example, during power-on of the integrated control chip 220, the boot program in the boot ROM 243 may detect the flash memory 244, and load the wake-up program and / or the scheduling program from the flash memory 244 to perform wake-up and / or sensor scheduling. The data generated by the processor 211 during operation may be stored in the data RAM 241.

[0068] The display unit 21 includes a display controller 212, a graphics engine 213, a timing controller 214, a display and graphics interface 215, a gate driver module 216, a source driver module 217, a common voltage driver module 218, a backlight control module 219 and a gamma reference module 251. The display controller 212 is used to generate graphic data according to the input display data. The graphics engine 213 is used to control the memory window, cursor, pointer and sprite graphics, so as to provide high-performance optimized graphics for touch control. The display and graphics interface 215 provides a variety of industrial standard display interfaces for receiving display data, such as DSI TCVR, DBI I / F, DPI I / F. The backlight control module 219 is used to control the backlight of the liquid crystal screen to achieve low power consumption management, so that it can be combined with the existing backlight energy saving technology. The gate driver module 216, the source driver module 217, and the common voltage driver module 218 are used to generate gate voltage, grayscale voltage and common voltage respectively. The timing controller 214 is used to control the output timing of the gate electrode and the grayscale voltage, so that in one image frame period, the gate voltages G1 to Gm are provided in a scanning manner, and when the thin film transistors of the corresponding row are turned on, the grayscale voltages S1 to Sn corresponding to the grayscale are provided to the plurality of thin film transistors T of each column, so that the voltage is applied to the pixel capacitor to change the orientation of the liquid crystal molecules to achieve the transmittance corresponding to the grayscale. The gamma reference module 251 is used to store the gamma correction curve, and to provide a correction signal to the source driving module 217 for correcting the grayscale voltage to meet the nonlinear requirements of the human eye for brightness changes.

[0069] The integrated control device of the embodiment of the present invention internally integrates a display unit, a touch unit, sensor units such as a fingerprint unit, an audio unit, an image unit, etc. for sensors that can be set on a display screen, and a security module, and can be installed as a single chip in an electronic device with touch and display functions, and has the functions of display driving, touch driving, on-screen sensor relay management, and sensor data protection. While ensuring low power consumption of the system, hardware-level data protection can be achieved, which improves security. The integrated design also simplifies the system design and reduces the workload of the main control unit. However, the present invention is not limited to this. The integrated control chip 220 can form multiple chips and be installed together on a circuit board at the display screen end.

[0070] In the integrated control device of this embodiment, a flash memory is used to store sensing data, a wake-up program, etc. However, the present invention is not limited thereto. In an alternative embodiment, the integrated control chip can use any type of non-volatile memory, such as any one selected from flash memory, SRAM, DRAM, EEPROM, and EPROM.

[0071] In this embodiment, the flash memory is integrated into the integrated control chip to improve security. In an alternative embodiment, the flash memory may be located outside the integrated control chip and connected to the integrated control chip via a bus to reduce system cost.

[0072] Figure 6 A schematic block diagram of an integrated control device according to a fourth embodiment of the present invention is shown. The integrated control device is suitable for electronic devices with an AMOLED display screen.

[0073] As shown in the figure, the integrated control chip 230 includes a processor 211 , a security module 202 , a communication interface 205 , a storage unit 204 , a display unit 21 and a sensor unit 22 .

[0074] The sensor unit 22 includes but is not limited to the above-mentioned touch unit 221, fingerprint unit 222, audio unit 223 and image unit 224, which are respectively connected to the touch sensor 120, fingerprint sensor 130, audio sensor 140 and image sensor 150, and are used to collect sensing data from corresponding sensors under the control of the processor 211 and provide it to the processor 211. At least one of the multiple sensor units 22 may have the above-mentioned reference Figure 4In some embodiments, the integrated control chip 230 obtains the identification of each sensor through the hardware interface of the sensor unit 22. For example, it can be considered that the sensing data from the touch unit 221 is provided by the touch sensor 120, and the sensing data from the fingerprint unit 222 is provided by the fingerprint sensor 130. In an alternative embodiment, each sensor 120 to 150 can transmit the identification and the sensing signal to the integrated control chip 230, so that the integrated control chip 230 can identify which sensor the sensing data comes from.

[0075] The security module 202 is used to perform security-related operations, such as data encryption, security verification, secure storage, etc. Figure 2 As shown, the security module 202 may include a processor 2021 and a memory 2022. The processor 2021 is used to perform security verification and / or encrypt at least part of the sensing data according to at least part of the sensing data under the control of the processor 203. The memory 2022 may store at least one of the sensing data, encrypted data, a verification template and verification result for security verification, an encryption program, a security verification program, and the like. In this embodiment, the verification template of the memory 2022 may come from the main control unit 300, or may be obtained by local collection and data processing under the control of the processor 211. In the latter case, not only the user's sensitive data but also the verification template are generated locally. The operating system only obtains the verification result or encrypted data from the integrated control chip 230, but cannot obtain both the sensitive data and the verification template, which is conducive to improving security.

[0076] The processor 211 is a RISC CPU of von Neumann or Harvard architecture, including but not limited to ARM, MIPS, OPENRISC, etc., preferably ARM; it can also be DSP, etc. The processor 211 can manage the multiple sensor units 22 to collect and transmit sensing data according to the received instructions. The processor 211 can also be optimized for touch detection or for other types of sensors. The touch input can be processed locally to determine whether it is necessary to wake up the operating system to process the user request. The processor 200 can also control the security module 202 to perform security-related operations, such as encrypting the sensing data and / or performing security verification based on the sensing data according to the instructions from the main control unit. In some embodiments, a security verification wake-up method can be adopted. In this case, the processor 211 can control the security module 202 to perform security verification based on the collected sensing data when the main control unit 300 is in a dormant state, and wake up the main control unit 300 when the security verification passes. The processor 211 can package the sensing data, encrypted data, security verification results, etc. and provide them to the main control unit 300. For example, a start bit and a type identifier may be added before the data content, and an end bit and a check bit may be added after the data content. The main control unit 300 may identify which type the data content belongs to based on the type identifier, and may also identify which sensor the sensing data comes from.

[0077] The communication interface 205 can support multiple communication protocols and digital I / O, such as I2C protocol and SPI protocol, and provide multiple digital I / O pins. The user interface 205 can communicate with the main processor on the mainboard.

[0078] The storage unit 204 may include a data RAM 241, a program RAM 242, a boot ROM 243, and a flash memory 244. In the flash memory 244, sensing data, a wake-up program, a scheduling rule, etc. may be stored. For example, during power-on of the integrated control chip 230, the boot program in the boot ROM 243 may detect the flash memory 244, and load the wake-up program and / or the scheduling program from the flash memory 244 to perform wake-up and / or sensor scheduling. The data generated by the processor 211 during operation may be stored in the data RAM 241.

[0079] The display unit 21 includes a display controller 212, a graphics engine 213, a timing controller 214, a display and graphics interface 215, a row driver module 226, a column driver module 227 and a gamma reference module 251. The display controller 212 is used to generate graphic data according to the input display data. The graphics engine 213 is used to control the memory window, cursor, pointer and sprite graphics, so as to provide high-performance optimized graphics for touch control. The display and graphics interface 215 provides a variety of industrial standard display interfaces for receiving display data, such as DSI TCVR, DBI I / F, DPI I / F. The row driver module 226 and the column driver module 227 are used to generate gate voltage and grayscale voltage respectively. The timing controller 214 is used to control the output timing of the gate electrode and the grayscale voltage, so that in one image frame period, the gate voltages G1 to Gm are provided in a scanning manner, and when the thin film transistors of the corresponding row are turned on, the grayscale voltages S1 to Sn corresponding to the grayscale are provided to the plurality of thin film transistors T of each column, so that the light emitting diode is applied with a current corresponding to the grayscale voltage to emit light, so as to achieve the light emitting brightness corresponding to the grayscale. The gamma reference module 251 is used to store the gamma correction curve, and to provide a correction signal to the column driving module 227 for correcting the grayscale voltage to meet the nonlinear requirements of the human eye for brightness changes.

[0080] The integrated control device of the embodiment of the present invention internally integrates a display unit, a touch unit, sensor units such as a fingerprint unit, an audio unit, an image unit, etc. for sensors that can be set on a display screen, and a security module, and can be installed as a single chip in an electronic device with touch and display functions, and has the functions of display driving, touch driving, on-screen sensor relay management, and sensor data protection. While ensuring low power consumption of the system, hardware-level data protection can be achieved, which improves security. The integrated design also simplifies the system design and reduces the workload of the main control unit. However, the present invention is not limited to this. The integrated control chip 230 can form multiple chips and be installed together on a circuit board at the display screen end.

[0081] In the integrated control device of this embodiment, a flash memory is used to store sensing data, a wake-up program, etc. However, the present invention is not limited thereto. In an alternative embodiment, the integrated control chip 230 may use any type of non-volatile memory, such as any one selected from flash memory, SRAM, DRAM, EEPROM, and EPROM.

[0082] In this embodiment, the flash memory is integrated into the integrated control chip 230 to improve security. In an alternative embodiment, the flash memory may be located outside the integrated control chip 230 and connected to the integrated control chip via a bus to reduce system cost.

[0083] Figure 7A timing diagram showing display and touch control in a display system according to an embodiment of the present invention using a time division multiplexing method is shown.

[0084] In practical applications, the display system can integrate multiple types of sensors. The processor in the integrated control chip 210 has three functions: display driving, sensor driving, and security verification, thereby providing a hardware-level security verification mechanism. Various sensor units are in a periodic polling state at a lower operating frequency.

[0085] For touch detection, when the sensing unit senses that an object has touched it, it switches to a working state and collects touch data. At this time, the display data and the touch data are multiplexed in a time-sharing manner.

[0086] For contact biometrics, such as fingerprint recognition, contact biometrics requires touch, so the biometric data collected in this way is similar to touch data and is time-division multiplexed with display data.

[0087] For contactless biometrics, presets can be made. For acquisitions that require manual supervision, serial data frames can be used to switch the display data to the current acquisition data. After the currently displayed acquisition data is terminated, it can be switched to the original display screen. If the contactless biometrics acquisition process does not require manual supervision (unsupervised), it can be processed in the background using time-sharing multiplexing with the display data. For example, when using an image sensor to acquire images, it is necessary to manually check whether the real-time acquisition image meets the requirements. In this case, the object currently being captured can be displayed. At this time, the display data is the current acquisition data of the image sensor.

[0088] The following is an example where touch control and display need to be performed synchronously.

[0089] During the blanking period of the image frame switching process, the display unit has little effect on the noise of the touch unit. Therefore, when the device is actually working, the display and touch control adopt the time-division multiplexing principle, and the display data processing can be separated from the touch data processing in time to reduce mutual interference. In the image frame scanning, some time slots are divided as touch frames.

[0090] like Figure 7 As shown, during one image frame, there are multiple display time periods TP and multiple touch time periods TP. In different time periods, display data and touch data are processed alternately. That is, display and touch are time-shared. Because the human eye has a certain recognition time window for picture changes, there are certain requirements for the frame rate and the time ratio of the two time periods. This driving method can effectively reduce the impact of the noise electrical signal of the liquid crystal display array on the touch working layer, save the shielding stack, and reduce the thickness of the touch display screen.

[0091] The above time-division multiplexing function can be operated by a software program, or can be switched in conjunction with a MUX multiplexing selection unit.

[0092] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0093] The embodiments of the present invention are described above. These embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, according to the above description, many modifications and changes can be made, including but not limited to changes in the local structure of the circuit and replacement of the type or model of components. This specification selects and describes these embodiments in detail in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An integrated control device, characterized in that: include: A plurality of sensor units, respectively connected to the plurality of sensors, for collecting sensing data from the plurality of sensors; A first processor, configured to manage the plurality of sensor units to collect sensing data from the plurality of sensors, and output the collected sensing data according to a preset rule or according to a received instruction; a security module, configured to perform security verification according to at least part of the sensing data and / or encrypt at least part of the sensing data under the control of the first processor; A communication interface, connected to the first processor, and used to communicate with a main control unit of the electronic device; Wherein, the security module includes: a second processor, the second processor being configured to perform security verification according to at least part of the sensing data and / or encrypt at least part of the sensing data under the control of the first processor; A storage unit, used to store at least one of the sensing data, the encrypted data, a verification template for security verification, and a verification result; The first processor is further used to determine whether a condition for waking up the main control unit is met according to the collected sensing data, and if so, wake up the main control unit in a dormant state; The first processor is also used to control the security module to perform security verification according to the collected sensing data when receiving the security verification instruction, and determine whether the conditions for waking up the main control unit are met according to the verification result.

2. The integrated control device according to claim 1, characterized in that: Also includes: A display unit, used for providing a display driving signal to the display device; The first processor is further configured to control the display unit to provide a display driving signal to the display device.

3. The integrated control device according to claim 1, characterized in that: At least one sensor unit among the plurality of sensor units comprises: A first interface, used to communicate with a corresponding sensor; a logic module, configured to provide a driving signal to a corresponding sensor and receive a sensing signal from the corresponding sensor, and amplify and perform digital-to-analog conversion on the received sensing signal to generate sensing data; and The second interface is used to communicate with the first processor.

4. The integrated control device according to claim 1, characterized in that: The multiple sensors include at least two of a gravity sensor, a gyroscope, an ambient light sensor, a compass, an air pressure sensor, a temperature sensor, a humidity sensor, a pressure sensor, a touch sensor, a fingerprint sensor, a palm print sensor, an audio sensor, and an image sensor.

5. The integrated control device according to claim 1, characterized in that: The integrated control device is a single chip.

6. A display system, characterized in that: include: A display device for displaying an image according to display data; A plurality of sensors for generating sensing signals; An integrated control device according to any one of claims 1 to 5.

7. The display system according to claim 6, characterized in that: The display device includes a liquid crystal display or an AMOLED display.