Two-dimensional code identification method and device

Abstract

The embodiment of the invention provides a two-dimensional code recognition method and device, and the method comprises the steps: transmitting an event notification message of a wrist turning event to an AO module if a sensing module monitors the wrist turning event; and the AO module sends a starting instruction to the camera driving module according to the event notification message. And the camera driving module responds to the starting instruction and drives a front-facing camera of the terminal equipment to acquire an image. The AO module obtains the collected image and identifies the two-dimensional code information in the image. According to the scheme of the embodiment, the identification of the two-dimensional code can be triggered when the user turns the wrist, so that the identification efficiency of the two-dimensional code can be effectively improved. And the sensing module monitors the wrist flipping event, so that frequent registration and unregistration of the code recognition APP can be effectively avoided, the function stability can be improved, and the power consumption can be reduced. And the AO module is responsible for identifying the two-dimensional code information in the image, so that the complicated interaction process among the modules can be saved, and the purposes of improving the function stability and reducing the power consumption can be achieved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a two-dimensional code recognition method and device. Background Art

[0002] With the development of terminal technology, terminal devices have become important tools in people's lives, and terminal devices can provide users with multiple functions.

[0003] The various functions provided by current terminal devices include a QR code scanning function. In the prior art, the terminal devices usually respond to a series of operations by the user to scan the QR code, thereby completing life and work purposes.

[0004] However, in the process of the terminal device scanning the QR code, the user needs to perform a series of operations on the terminal device to trigger the terminal device to scan the QR code. The triggering process takes a certain amount of time, resulting in low QR code recognition efficiency. Summary of the invention

[0005] The embodiments of the present application provide a two-dimensional code recognition method and device, which are applied in the field of terminal technology.

[0006] In a first aspect, an embodiment of the present application provides a two-dimensional code recognition method. The method includes:

[0007] If the perception module detects a wrist-turning event, it sends an event notification message of the wrist-turning event to the AO module;

[0008] The AO module sends a start instruction to the camera driving module according to the event notification message;

[0009] The camera driving module drives the front camera of the terminal device to capture images in response to the start instruction;

[0010] The AO module acquires the collected image and identifies the two-dimensional code information in the image.

[0011] In this implementation, the perception module monitors the wrist flip event, which can effectively avoid frequent registration and deregistration of the code recognition APP, thereby improving functional stability and reducing power consumption. The AO module is responsible for recognizing the QR code information in the image, which can save the complicated interaction process between the modules and also achieve the purpose of improving functional stability and reducing power consumption.

[0012] In some implementations, the AO module includes a main control module, a timing module, a camera control module, and a CA module.

[0013] In some implementations, the sending the event notification message of the wrist turning event to the AO module includes:

[0014] An event notification message of the wrist turning event is sent to the main control module.

[0015] In some implementations, the AO module sends a start instruction to the camera driver module according to the event notification message, including:

[0016] The camera control module sends a start instruction to the camera driving module according to the event notification message.

[0017] In some implementations, the terminal device further includes a TA module running in a TEE environment;

[0018] After sending the event notification message of the wrist turning event to the AO module, the method further includes:

[0019] The main control module pulls up the CA module;

[0020] The CA module pulls up the TA module;

[0021] The CA module generates a key and sends the key to the TA module.

[0022] In some implementations, after the camera driving module drives the front camera of the terminal device to capture an image in response to the start instruction, the method further includes:

[0023] The camera driving module sends the image address of the image to the CA module;

[0024] The CA module sends the image address and the preset image size to the TA module;

[0025] The TA module acquires the image according to the image address and the image size;

[0026] The TA module encrypts the image according to the key to obtain an encrypted image, and stores the encrypted image in a shared memory of the TA module and the CA module.

[0027] In this implementation, the TA module does not need to know what business function is currently being executed. It only requires the AO service to pass the image address and image size. The encryption function in the TA module will automatically encrypt the data under the corresponding address and then return it to the CA module in the AO service, thereby reducing the data dependence on the AO service to a certain extent and improving the generalization and generalization capabilities of the encryption and decryption functions.

[0028] In some implementations, the AO module acquires the collected image, including:

[0029] The CA module obtains the encrypted image from the shared memory;

[0030] The CA module decrypts the encrypted image using the key to obtain the decrypted image;

[0031] The CA module sends the decrypted image to the main control module.

[0032] In some implementations, after sending the event notification message of the wrist-turning event to the AO module, the method further includes:

[0033] The main control module sends a timing request to the timing module;

[0034] The timing module counts the wrist turning event.

[0035] In some implementations, the terminal device also includes a system service module and a code recognition application;

[0036] After the AO module recognizes the two-dimensional code information in the image, the method further includes:

[0037] If the main control module successfully identifies the two-dimensional code information in the image, the main control module sends the two-dimensional code information to the perception module;

[0038] The perception module sends the two-dimensional code information to the system service module;

[0039] The system service module sends the two-dimensional code information to the code recognition application.

[0040] In some implementations, after the AO module identifies the QR code information in the image, the method further includes:

[0041] If the main control module fails to identify the two-dimensional code information in the image, the main control module obtains the timing duration for the wrist turning event in the timing module;

[0042] If the timing duration is greater than or equal to the preset duration, the main control module ends the code recognition process of the wrist turning event;

[0043] If the timing duration is less than the preset duration, the operation of identifying the two-dimensional code information in the image is repeated until the code is successfully identified or the timeout occurs.

[0044] In a second aspect, an embodiment of the present application provides a two-dimensional code recognition device. The device includes:

[0045] A sending unit, configured to send an event notification message of the wrist-turning event to the AO module if the perception module monitors the wrist-turning event;

[0046] The sending unit is also used for the AO module to send a start instruction to the camera driving module according to the event notification message;

[0047] A processing unit, configured for the camera driving module to drive the front camera of the terminal device to capture an image in response to the start instruction;

[0048] The processing unit is also used by the AO module to acquire the collected image and identify the QR code information in the image.

[0049] In some implementations, the AO module includes a main control module, a timing module, a camera control module, and a CA module.

[0050] In some implementations, the sending unit is specifically configured to:

[0051] An event notification message of the wrist turning event is sent to the main control module.

[0052] In some implementations, the sending unit is specifically configured to:

[0053] The camera control module sends a start instruction to the camera driving module according to the event notification message.

[0054] In some implementations, the terminal device further includes a TA module running in a TEE environment;

[0055] The processing unit is further used for:

[0056] After sending the event notification message of the wrist-turning event to the AO module, the main control module pulls up the CA module;

[0057] The CA module pulls up the TA module;

[0058] The CA module generates a key and sends the key to the TA module.

[0059] In some implementations, the processing unit is further configured to:

[0060] After the camera driving module drives the front camera of the terminal device to capture an image in response to the start instruction, the camera driving module sends an image address of the image to the CA module;

[0061] The CA module sends the image address and the preset image size to the TA module;

[0062] The TA module acquires the image according to the image address and the image size;

[0063] The TA module encrypts the image according to the key to obtain an encrypted image, and stores the encrypted image in a shared memory of the TA module and the CA module.

[0064] In some implementations, the processing unit is specifically configured to:

[0065] The CA module obtains the encrypted image from the shared memory;

[0066] The CA module decrypts the encrypted image using the key to obtain the decrypted image;

[0067] The CA module sends the decrypted image to the main control module.

[0068] In some implementations, the processing unit is further configured to:

[0069] After sending the event notification message of the wrist-turning event to the AO module, the main control module sends a timing request to the timing module;

[0070] The timing module counts the wrist turning event.

[0071] In some implementations, the sending unit is further configured to:

[0072] After the AO module recognizes the two-dimensional code information in the image, if the main control module successfully recognizes the two-dimensional code information in the image, the main control module sends the two-dimensional code information to the perception module;

[0073] The perception module sends the two-dimensional code information to the system service module;

[0074] The system service module sends the QR code information to the code recognition APP.

[0075] In some implementations, the processing unit is further configured to:

[0076] After the AO module recognizes the two-dimensional code information in the image, if the main control module fails to recognize the two-dimensional code information in the image, the main control module obtains the timing duration for the wrist turning event in the timing module;

[0077] If the timing duration is greater than or equal to the preset duration, the main control module ends the code recognition process of the wrist turning event;

[0078] If the timing duration is less than the preset duration, the operation of identifying the two-dimensional code information in the image is repeated until the code is successfully identified or the timeout occurs.

[0079] In a third aspect, an embodiment of the present application provides a terminal device, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc.

[0080] The terminal device comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method of the first aspect.

[0081] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0082] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method of the first aspect.

[0083] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method described in the first aspect.

[0084] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 1 ;

[0086] Figure 2 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 2 ;

[0087] Figure 3 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0088] Figure 4 A software structure block diagram of a terminal device according to an embodiment of the present application;

[0089] Figure 5 A schematic diagram of the software architecture of the two-dimensional code recognition method provided in the embodiment of the present application;

[0090] Figure 6 An interactive flow chart of a two-dimensional code recognition method provided in an embodiment of the present application;

[0091] Figure 7 Interaction process of the two-dimensional code recognition method provided in the embodiment of the present application Figure 2 ;

[0092] Figure 8 A schematic diagram of a process flow for determining a processing strategy for a wrist flip event provided in an embodiment of the present application;

[0093] Fig. 9 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 1 ;

[0094] Fig.10 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 2 ;

[0095] Fig.11 A schematic diagram of the structure of a two-dimensional code recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0096] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0097] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0098] It should be noted that the "at..." in the embodiment of the present application can be the instant when a certain situation occurs, or a period of time after a certain situation occurs, and the embodiment of the present application does not specifically limit this. In addition, the display interface provided in the embodiment of the present application is only an example, and the display interface can also include more or less content.

[0099] With the development of terminal technology, terminal devices have become an important tool in people's lives. Terminal devices provide users with a variety of functions. For example, terminal devices can respond to a series of operations of users, scan QR codes, and thus complete life and work purposes.

[0100] For example, when a user uses a terminal device to scan a QR code, the user usually triggers the terminal device's application or applet first, and then triggers the terminal device's QR code recognition function based on the application or applet's entry. Then, the terminal device can call the camera to obtain an image and recognize the QR code based on the obtained image.

[0101] It can be seen that in the process of terminal equipment recognizing QR codes, users are required to perform a series of operations on the terminal equipment to achieve the purpose of code recognition. Both triggering code recognition and the actual code recognition process take a certain amount of time, resulting in low QR code recognition efficiency.

[0102] In view of this, an embodiment of the present application provides a QR code recognition method, wherein the method can monitor wrist-turning events through the posture change information and screen direction of the terminal device, and then directly call the front camera of the terminal device to capture images and recognize QR codes based on the monitored wrist-turning events, thereby avoiding a series of tedious operations for triggering code recognition, saving time for recognizing QR codes, improving QR code recognition efficiency, and enhancing user experience.

[0103] Figure 1 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 1 .like Figure 1 As shown in (a) of FIG. 1 , the user holds the terminal device 100, and the terminal device 100 is in a bright screen unlocked state. Figure 1 As shown in (b), the user flips the terminal device 100, that is, the user has a wrist-turning action; then based on the QR code recognition method of the embodiment of the present application, the terminal device 100 calls the front camera of the terminal device to obtain the image on the desktop and recognizes the QR code information in the image. Based on the recognized QR code information, the terminal device 100 calls the interface of the application corresponding to the QR code information.

[0104] Figure 2 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 2 .like Figure 2 As shown, the user holds a terminal device 100; based on the QR code recognition method of the embodiment of the present application, the terminal device 100 calls the rear camera of the terminal device to obtain an image on the wall and recognizes the QR code information in the image. Based on the recognized QR code information, the terminal device 100 calls the interface of the application corresponding to the QR code information.

[0105] The terminal device 100 may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0106] The terminal device 100 has a function of displaying an interface and a function of acquiring an image.

[0107] In order to better understand the embodiment of the present application, the structure of the terminal device in the embodiment of the present application is introduced below:

[0108] Figure 3 A schematic diagram of the structure of a terminal device 100 provided in an embodiment of the present application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0109] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0110] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0111] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0112] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0113] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0114] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0115] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0116] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0117] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0118] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0119] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0120] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0121] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0122] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0123] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0124] Gyro sensor 180B is used to collect angle change data. Gyro sensor 180B is used to collect angle data.

[0125] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in all directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device and applied to applications such as horizontal and vertical screen switching and pedometers.

[0126] The acceleration change data is collected based on the acceleration sensor 180E. The terminal device determines the posture change information of the terminal device based on the acceleration change data collected by the acceleration sensor 180E.

[0127] The terminal device determines the screen direction based on the angle data collected by the gyro sensor 180B.

[0128] The terminal device determines the posture change information of the terminal device based on the angle change data collected by the gyroscope sensor 180B. Alternatively, the terminal device determines the posture change information of the terminal device based on the acceleration change data collected by the acceleration sensor 180E. Alternatively, the terminal device determines the posture change information of the terminal device based on the angle change data collected by the gyroscope sensor 180B and the acceleration change data collected by the acceleration sensor 180E.

[0129] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and pocket mode automatically unlocks and locks the screen.

[0130] The proximity light sensor 180G collects proximity light data. The terminal device determines proximity light information based on the proximity light data collected by the light sensor 180G.

[0131] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touch.

[0132] The ambient light sensor 180L collects ambient light data. The terminal device determines ambient light information based on the ambient light data collected by the ambient light sensor 180L.

[0133] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0134] Figure 4 This is a software structure block diagram of the terminal device 100 according to an embodiment of the present application.

[0135] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0136] The application layer can include a series of application packages.

[0137] like Figure 4 As shown, the application package may include applications such as camera, calendar, phone, map, call, music, settings, mailbox, video, social, etc.

[0138] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0139] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a resource manager, a view system, a notification manager, and the like.

[0140] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, touch the screen, drag the screen, capture the screen, etc.

[0141] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0142] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0143] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0144] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0145] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0146] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0147] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0148] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0149] The following is an illustrative description of the software and hardware workflow of the terminal device 100 in conjunction with a scenario where an application is started or an interface switch occurs in an application.

[0150] The following is a detailed description of the operation process of the two-dimensional code recognition provided by the embodiment of the present application and the display process of the interface during the two-dimensional code recognition process in conjunction with the accompanying drawings. It should be noted that the "at..." in the embodiment of the present application can be the instant when a certain situation occurs, or it can be a period of time after a certain situation occurs, and the embodiment of the present application does not make specific limitations on this.

[0151] First combine Figure 5 Introduce the software architecture of the terminal device in this application. Figure 5 A schematic diagram of the software architecture of the two-dimensional code recognition method provided in the embodiment of the present application. Each embodiment provided in the present application can be adapted to Figure 5 The software architecture provided.

[0152] like Figure 5As shown, the terminal device includes REE (Rich Execution Environment) and TEE (Trusted Execution Environment), and a secure buffer and a shared buffer are deployed between the REE environment and the TEE environment, wherein the secure buffer is a buffer that can be accessed by both the REE environment and the TEE environment, thereby enabling data interaction between the REE environment and the TEE environment. The shared buffer is specifically a buffer shared by the CA (Client Application) module in the REE environment and the TA (Trusted Application) module in the TEE environment, which is used for data interaction between the two modules.

[0153] Among them, the application layer, framework layer, HAL (Hardware Abstraction Layer, hardware abstraction) layer, kernel layer and hardware layer are deployed in the REE environment.

[0154] Furthermore, a code recognition APP (Application) is deployed in the application layer, wherein the code recognition APP can be used to process part of the code scanning process and flow in.

[0155] System services, camera services, and perception platforms (also called perception modules) are deployed in the framework layer. The perception platform further includes subscription management, code recognition fences, and image data modules. Among them, the camera service is a unit in the framework layer used to handle camera scheduling, the system service is used to transmit data between the code recognition APP and the perception platform, the subscription management in the perception platform is used to detect screen unlocking, monitor wrist flip events, and monitor the management of code events. The code recognition fence in the perception service is used to specifically detect screen unlocking and monitor wrist flip events. The image data module in the perception platform is responsible for image transmission between the AO service and the code recognition APP.

[0156] The camera HAL and AO (Always On) services are deployed in the HAL layer, where the AO service can be understood as a service module resident in the system, used to implement some operations that need to be implemented frequently. The camera control module, CA module, main control module and timing module are further deployed in the AO service. Among them, the camera HAL is a unit in the HAL layer used to handle camera scheduling. The camera control module in the AO service is used to issue instructions to start the camera, the CA module is used to collaborate with the TA module to realize image encryption and decryption processing, the main control module is used to implement the code recognition process for wrist flip events, and the timing module is used to timely respond to wrist flip events.

[0157] A camera driver is deployed in the kernel layer, wherein the camera driver is a unit in the framework layer for processing camera scheduling.

[0158] The hardware layer deploys cameras and IFE (Image Front End). The camera is used to collect images, and the IFE is responsible for receiving the raw data from the camera and processing it, such as noise reduction and color correction. After that, the IFE can pass the processed image data to subsequent units, such as Figure 5 In a possible implementation, the IFE output may be a YUV image (full channel image), where YUV is a color coding mode, in which Y represents luminance, i.e., grayscale value, and UV represents chrominance and chroma, respectively.

[0159] A trusted camera service is also deployed in the TEE environment. This service can obtain images from the secure cache and pass them to the TA module, which is responsible for image encryption.

[0160] based on Figure 5 The introduced software architecture can determine that in the technical solution of the present application, the perception module detects the screen unlocking and monitors the wrist turning event, and the AO module is responsible for the related code recognition processing of the wrist turning event.

[0161] In another possible implementation, the code recognition APP may detect screen unlocking and monitor wrist turning events, and the code recognition APP may also be responsible for the subsequent processing of part of the code recognition process for the wrist turning event.

[0162] However, in this case, the code recognition APP needs to remain resident, so there may be a problem of high power consumption. Also, if the code recognition APP monitors the wrist flip event, the code recognition APP needs to inform the perception middle station. The subsequent process is that the perception middle station interacts with the AO service, the AO service pulls up the camera and processes the image encryption and decryption, and then the AO service processes the decrypted image and passes it to the code recognition APP through the perception middle station, and the code recognition APP then performs code recognition based on the image.

[0163] The above process has two problems:

[0164] On the one hand, every time the code recognition APP sends data to the perception center, it needs to first send a registration command to the perception center to establish a connection. This means that every time the code recognition APP monitors a wrist flip event, it needs to perform a registration operation, and then a corresponding deregistration operation is required. The trigger frequency of wrist flip events is relatively high during the daily use of terminal devices, which will cause the code recognition APP and the perception center to frequently register and deregister. Frequent registration and deregistration will increase the probability of errors and cause power consumption problems. Therefore, the current implementation method will have a certain impact on the functional stability and power consumption of wrist flip code recognition.

[0165] On the other hand, if the code recognition processing for the wrist turning event is performed by the code recognition APP, the code recognition APP needs to first inform the perception middle station that it has monitored the wrist turning event, and then the perception middle station will instruct the AO service to pull up the camera to collect images. After obtaining the image, the AO service also needs to pass the image to the code recognition APP through the perception middle station, which makes the interaction between the various modules more complicated and cumbersome.

[0166] In summary, in the technical solution of the present application, the terminal device can monitor the wrist flip event to call the front camera of the terminal device to capture images and identify the QR code, thereby improving the recognition efficiency of the QR code. However, it can be understood that because the triggering frequency of the wrist flip event itself is relatively high, it is very important to ensure the stability of the wrist flip code recognition function and the stability of the power consumption of the terminal device. The frequent registration and deregistration operations of the code recognition APP, as well as the complicated interactions between the various modules, will affect the stability of the function and the power consumption. Therefore, the QR code recognition method provided in this application solves the problem currently introduced.

[0167] In the above Figure 5 Based on the software architecture introduced above, the following Figure 6 The specific technical solution of the two-dimensional code recognition method provided by this application is described. Figure 6 An interactive flow chart of the QR code recognition method provided in an embodiment of the present application.

[0168] like Figure 6 As shown, the method includes:

[0169] S601. The perception module monitors a wrist-turning event.

[0170] In this embodiment, the perception module can detect screen-on unlocking and monitor wrist-turning events. Exemplarily, the perception module can monitor wrist-turning events when the terminal device is in the screen-on unlocking state.

[0171] The reason why the code recognition APP needs to register and unregister frequently is that every time the code recognition APP monitors a wrist flip event, it needs to inform the perception module, so it needs to establish a link through a registration request. In this embodiment, the perception module is directly responsible for monitoring the wrist flip event, which can effectively avoid the frequent registration and unregistration of the code recognition APP, thereby effectively improving the functional stability and power consumption stability.

[0172] In one possible implementation, the perception module may adopt the following method when monitoring a wrist-turning event: if it is determined that the value represented by the posture change information of the terminal device is greater than or equal to a first preset threshold, and the screen direction is a direction other than the first direction, then it is determined that a wrist-turning event is monitored, wherein the first direction is the positive direction of the z-axis of the world coordinate system.

[0173] In one example, an accelerometer and a gyroscope sensor are provided in the terminal device, and the acceleration sensor collects acceleration change data within a preset time period, and the gyroscope sensor collects angle change data within a preset time period; wherein the preset time period is a time period before the current moment. Then, the posture change information of the terminal device can be determined in the following manner: obtaining the acceleration change data collected by the accelerometer in the terminal device within the preset time period, and / or obtaining the angle change data collected by the gyroscope sensor in the terminal device within the preset time period; determining the posture change information according to the acceleration change data and / or the angle change data. wherein the posture change information represents the posture change angle of the terminal device, or the posture change information represents the rotation change angle of the terminal device.

[0174] In one example, when determining the screen direction of a terminal device, the following method can be used: obtain the angle data collected by the gyroscope sensor in the terminal device at the current moment; determine the screen direction of the terminal device based on the angle data. For example, the gyroscope sensor collects xyz axis data (that is, angle data) at the current moment; the terminal device obtains the xyz axis data collected by the gyroscope sensor, and then the terminal device determines the screen direction of the terminal device based on the xyz axis data, whether it is not facing the positive direction of the z axis of the world coordinate system, that is, whether the screen direction of the terminal device is not facing upward.

[0175] The “upward” direction of the screen of the terminal device means that the screen of the terminal device faces the “positive direction of the z-axis of the world coordinate system”; and the first direction refers to the positive direction of the z-axis of the world coordinate system.

[0176] The “downward” direction of the screen of the terminal device means that the screen of the terminal device faces the “negative direction of the z-axis of the world coordinate system”; and the second direction refers to the negative direction of the z-axis of the world coordinate system.

[0177] Based on the above principle, the perception module determines that the terminal device has flipped over according to the value represented by the posture change information obtained, which is greater than or equal to the first preset threshold; it can be known that the user holding the terminal device flips the wrist to flip the screen of the terminal device. In addition, the perception module determines that the screen direction is a direction other than the first direction, and then determines that the screen direction of the terminal device is not facing upward. Then, the perception module determines that a wrist flip event is monitored.

[0178] S602: The perception module sends an event notification message of the wrist turning event to the AO module.

[0179] When the perception module confirms that it has detected a wrist-turning event, it can send an event notification message of the wrist-turning event to the AO module, where the event notification message is used to inform the AO module that a wrist-turning event has been detected. Figure 5 AO service introduced in .

[0180] Based on the above introduction, it can be determined that the AO module includes a main control module. In a possible implementation, the perception module may, for example, send an event notification message to the main control module in the AO module.

[0181] S603. The AO module sends a start instruction to the camera driving module according to the event notification message.

[0182] After receiving the event notification message, the AO module can confirm that a wrist-flipping event has been monitored. In this embodiment, when processing the wrist-flipping event, it is necessary to call the front camera of the terminal device to acquire an image, and then perform QR code recognition based on the acquired image.

[0183] Therefore, the AO module can send a start instruction to the camera driver module according to the event notification message, wherein the start instruction is used to instruct to start the front camera of the terminal device to acquire an image.

[0184] The camera driver module described in this embodiment can be understood as the above Figure 5 Therefore, in a possible implementation, the AO module in this implementation does not send a startup instruction directly to the camera driver module, but sends a startup instruction to the camera driver module via some intermediate modules.

[0185] Exemplarily, the camera control module in the AO module is responsible for sending a startup instruction, wherein the camera control module first sends a startup instruction to the camera service in the framework layer, and then the camera service in the framework layer sends a startup instruction to the cameraHAL in the HAL layer, and the cameraHAL in the HAL layer sends a startup instruction to the camera driver in the kernel layer, thereby sending the startup instruction for starting the front camera to the camera driver module.

[0186] S604: The camera driving module drives the front camera of the terminal device to capture images in response to the start instruction.

[0187] After receiving the start command, the camera driver module can drive the front camera of the terminal device to capture images.

[0188] S605. The AO module obtains the collected image and identifies the QR code information in the image.

[0189] After the front camera acquires the image, the AO module can obtain the acquired image and perform code recognition processing on the image to identify the two-dimensional code information in the image. That is to say, in this embodiment, the code recognition processing is completed by the AO module.

[0190] In a possible implementation, after the AO module performs code recognition processing on the captured image to obtain the QR code information, the QR code information can be provided to the code recognition APP, and the code recognition APP is responsible for pulling up the service interface corresponding to the QR code information.

[0191] Therefore, in the present embodiment, the AO module is directly responsible for starting the camera, and after acquiring the image, the AO module is directly responsible for the image recognition processing, thereby effectively reducing the interaction between modules and avoiding the problem of reduced functional stability and high system power consumption caused by complicated interaction processes.

[0192] The QR code recognition method provided in the embodiment of the present application includes: if the perception module monitors the wrist turning event, an event notification message of the wrist turning event is sent to the AO module. The AO module sends a start instruction to the camera driver module according to the event notification message. The camera driver module responds to the start instruction and drives the front camera of the terminal device to capture an image. The AO module obtains the captured image and recognizes the QR code information in the image. The scheme of this embodiment can trigger the recognition of the QR code when the user turns his wrist, so the recognition efficiency of the QR code can be effectively improved. And the perception module monitors the wrist turning event, which can effectively avoid the frequent registration and deregistration of the code recognition APP, thereby improving the functional stability and reducing power consumption. And the AO module is responsible for recognizing the QR code information in the image, which can save the complicated interaction process between the modules, and can also achieve the purpose of improving functional stability and reducing power consumption.

[0193] Based on the above introduction, the following Figure 7 The complete interaction between the various modules in the two-dimensional code recognition method in this application is further introduced in detail. Figure 7 Interaction process of the two-dimensional code recognition method provided in the embodiment of the present application Figure 2 .

[0194] like Figure 7 As shown, the method includes:

[0195] 1. The system service module sends a registration request to the perception module.

[0196] Among them, the registration request is used to request the perception module to register the code scanning fence.

[0197] In a possible implementation, when the terminal device is turned on, the system service module sends a registration request to the perception module to request the perception module to register the code scanning fence, where the code scanning fence can be understood as a processing module for processing code scanning related matters, such as detecting the device screen light status, unlock status, monitoring wrist turn events, etc. Then the subsequent perception module can realize operations such as detecting screen light unlocking, monitoring wrist turn events, etc.

[0198] Therefore, in the technical solution of this embodiment, it is only necessary to register once when the device is turned on, and then the perception module can perform relevant detection and monitoring, thereby avoiding the problems of functional instability and high power consumption caused by frequent registration and deregistration of the code recognition APP.

[0199] 2. The perception module responds to the registration request and registers the scanned fence.

[0200] 3. When the device screen is on and unlocked, the sensing module monitors the wrist flip event.

[0201] Among them, the related operations in step 3 can be performed by the code scanning fence introduced above.

[0202] 4. The perception module sends a notification message of the wrist-turning event to the main control module in the AO service module.

[0203] When the perception module detects a wrist-turning event, it needs to inform the AO module, and then the AO module can perform the code recognition process for the detected wrist-turning event. Therefore, the perception module can send a notification message to the main control module in the AO service module, and the notification message is used to inform the main control module that a wrist-turning event has been detected.

[0204] 5. The perception module sends an instruction to start the APP to the system service module to start the code recognition APP.

[0205] 6. The system service module sends a startup instruction to the code recognition APP to start the code recognition APP.

[0206] Furthermore, after monitoring the wrist-turning event, the perception module can also launch the code recognition APP at the application layer, so as to subsequently pass the QR code recognition result to the code recognition APP.

[0207] For example, the perception module can call back the code recognition service in the system service module by scanning the code fence, thereby starting the code recognition APP. Figure 7 As shown in the figure, the perception module first sends a command to the system service module to start the code recognition APP, which can be understood as the perception module notifying the system service module that the code recognition APP needs to be launched by calling back the code recognition service in the system service module. Then the system service module sends a startup command to the code recognition APP to start the code recognition APP.

[0208] Figure 7 The situation shown in the figure is that the system service module acts as an intermediate transmission module between the code recognition APP and the perception module to realize the startup of the code recognition APP. Alternatively, the perception module can also directly send a startup instruction to the code recognition APP to launch the code recognition APP, but because the system service module has a higher priority, the efficiency of starting the code recognition APP can be improved to a certain extent by intermediate transmission through the system service module.

[0209] It should also be noted that the access direction of the perception module to the code recognition APP can be directly accessed through the interface, that is, access can be achieved without registration. However, the access direction of the code recognition APP to the perception module must be achieved by establishing a link through a registration request.

[0210] 7. The main control module in the AO service module detects whether there is a wrist flip event code recognition process being executed.

[0211] After receiving the notification message of the wrist flip event, the main control module first needs to detect whether there is a wrist flip event code recognition process currently being executed, and then determine how to handle the newly monitored wrist flip event. The code recognition process in this embodiment includes: calling the front camera of the terminal device to obtain an image and identifying the QR code information in the image.

[0212] If there is a wrist-flip event while the code recognition process is being executed, the wrist-flip event indicated by the newly received notification message can be recorded as the next wrist-flip event (also called the first wrist-flip event), and the wrist-flip event while the code recognition process is being executed is the current wrist-flip event (also called the second wrist-flip event). The processing strategy for the next wrist-flip event can then be determined based on the execution result of the current wrist-flip event.

[0213] 8. The main control module in the AO service module can pull up the CA module in the AO service module.

[0214] If there is no wrist-turning event in the code recognition process, the newly monitored wrist-turning event can be directly processed.

[0215] For example, when the AO module acquires the collected image, it also acquires the image based on a certain module transmission link. Therefore, in order to ensure the security of image transmission, during the transmission process, the image encryption and decryption processing can be implemented through the CA module in the AO module and the TA module in the TEE environment.

[0216] In order to save power consumption, the CA module and the TA module are usually set not to be resident, so the main control module can pull up the CA module to perform subsequent image encryption and decryption operations.

[0217] 9. The CA module pulls up the TA module in the TEE environment.

[0218] After the CA module is pulled up, the CA module can continue to pull up the TA module in the TEE environment to perform subsequent image encryption and decryption operations.

[0219] 10. The CA module generates keys.

[0220] 11. The CA module sends the key to the TA module in the TEE environment.

[0221] In order to facilitate efficient and effective encryption and decryption processing in the current code recognition process, the CA module can generate a key when the code recognition process just starts, and then send the generated key to the TA module to synchronize the key between the CA module and the TA module. Thereafter, in the code recognition process of this wrist flip event, the encryption and decryption processing of each frame of the collected image can use the currently synchronized key.

[0222] Therefore, when the code recognition process just starts to be executed, the key synchronization is performed, which can effectively save system resources on the basis of ensuring the successful execution of image encryption and decryption.

[0223] The key generated by the CA module may be a random number, or may be any other form of information as long as it can be used for encryption.

[0224] 12. The main control module sends a timing request to the timing module.

[0225] 13. The timing module is used to time the wrist turning event.

[0226] The timing request is used to request the timing module to time the wrist flip event.

[0227] If the result of the judgment in step 7 is that there is no wrist-turning event currently executing the code recognition process, then there is only the newly monitored wrist-turning event, so it is only necessary to time the newly monitored wrist-turning event. For example, the execution time of the code recognition process of the wrist-turning event can be timed (the code recognition process starts after step 7), and the timing module can also record the time when the wrist-turning event is monitored.

[0228] Alternatively, if the judgment result of the above step 7 is that there is a wrist-flip event that is currently executing the code recognition process, then there is a current wrist-flip event and a next wrist-flip event. In this case, the main control module can also send a timing request to the timing module, and the timing request includes the time when the next wrist-flip event is monitored. The timing module responds to the timing request and records the time when the next wrist-flip event is monitored.

[0229] Exemplarily, the moment when the wrist-flip event is detected can be carried in the notification message of the wrist-flip event, or the moment when the main control module receives the notification message can also be confirmed as the moment when the wrist-flip event is detected. The main control module then carries the moment when the wrist-flip event is detected in the timing request to inform the timing module.

[0230] 14. The main control module sends a notification message to the camera control module to start the camera.

[0231] 15. The camera control module sends a command to start the camera to the camera driver module.

[0232] In the code recognition process of the wrist flip event, it is necessary to call the front camera of the terminal device to capture images. Therefore, after confirming the start of the code recognition process, the main control module can send a notification message to the camera control module to start the camera. After receiving the notification message, the camera control module sends a command to the camera driver module to start the camera.

[0233] 16. The camera driver module drives the front camera of the terminal device to capture images.

[0234] The camera driving module responds to the start-up instruction to drive the front camera of the terminal device to capture images. The specific implementation can refer to the introduction of the above embodiment and will not be repeated here.

[0235] In one implementation, the timing relationship of the three operations introduced in this embodiment, namely, pulling up the CA and TA synchronization keys (steps 8 to 11), timing (steps 12 and 13), and starting the camera (steps 14 to 16), is not limited to the order described above. The timing relationship of these three operations can be arbitrarily changed according to actual needs, and even these three operations can be executed in parallel. Therefore, this embodiment does not limit the specific execution order of these three operations.

[0236] 17. The camera driver module sends the image address to the CA module.

[0237] For example, after the front camera captures an image, the image is stored in a corresponding location, such as in the shared buffer described above. The camera driver module can then send an image address to the CA module, and the image address is used to indicate the storage location of the captured image.

[0238] In one possible implementation, after the camera in the hardware layer captures an image and stores it in a secure buffer, the camera in the hardware layer can first inform the camera driver in the kernel layer of the image address, and then pass the image address to the camera control module in the AO module via the link of the camera driver in the kernel layer, cameraHAL in the HAL layer, and camera service in the framework layer, and then the camera driver module sends the image address to the CA module.

[0239] 18. The CA module sends the image address and image size to the TA module.

[0240] In order for the TA module to acquire the image, the CA module can send the image address to the TA module. The CA module also sends the image size to the TA module, where the image size can be preset. During the wrist flip code recognition process, the image size captured by the front camera is the preset size.

[0241] The image size is sent to the TA module so that the TA module reserves sufficient storage space and writes the image acquired later into the storage space of the TA module to avoid the problem of being unable to obtain image data after the data in the security cache area is overwritten.

[0242] 19. The TA module obtains the image based on the image address and image size.

[0243] After receiving the image address and image size, the TA module can obtain the image captured by the front camera according to the image address and image size. After obtaining, the image can be stored in the storage space of the TA module, for example.

[0244] 20. The TA module encrypts the image according to the key and stores the encrypted image in the shared cache area of ​​the CA and TA.

[0245] Because the CA module and the TA module have synchronized the key in advance, the TA module can encrypt the image according to the key and store the encrypted image in the shared buffer area.

[0246] Exemplarily, the TA module can directly encrypt according to the key generated by the CA module. Alternatively, the TA module can also generate a further advanced key according to the key generated by the CA module and the TA's uuid (Universally Unique Identifier), and then use the advanced key to encrypt the image.

[0247] 21. The TA module sends an indication message to the CA module.

[0248] The indication message is used to inform the CA module that the encrypted image has been stored in the shared cache area.

[0249] 22. The CA module obtains the encrypted image from the shared buffer and decrypts it using the key to obtain the decrypted image.

[0250] After receiving the indication message, the CA module can obtain the encrypted image from the shared buffer area and decrypt it using the key to obtain the decrypted image.

[0251] Exemplarily, the CA module can directly decrypt according to the key. Alternatively, the CA module can also generate a further advanced key according to the key and the uuid of the TA, and then use the advanced key to decrypt the image.

[0252] The specific encryption and decryption method can be determined according to actual needs, as long as the TA module and the CA module use the same method.

[0253] 23. The CA module sends the decrypted image to the main control module.

[0254] 24. The main control module identifies whether there is a QR code in the image.

[0255] In this application, the code recognition process of the wrist flip event may include a first stage and a second stage. In the first stage, the terminal device may call the front camera to acquire an image and identify whether there is a QR code in the acquired image. The second stage will be entered only when it is determined that there is a QR code in the acquired image. In the second stage, the QR code in the image may be decoded to obtain the information of the QR code, which can also be understood as a code value.

[0256] Therefore, the main control module can first perform the first stage of processing based on the decrypted image to identify whether there is a QR code in the image. It should be noted here that the current step is not a simple determination of whether there is a QR code in a frame of image. In the first stage of the code recognition process, the front camera will continue to capture images. The current situation is to determine whether there is a QR code based on multiple frames of images captured in the first stage.

[0257] 25. If there is no QR code in the image, the main control module terminates the code recognition process of the wrist flip event.

[0258] 26. The main control module controls the code recognition process for the next wrist flip event.

[0259] If there is no QR code in the image, it means that no QR code is recognized in the first stage of the code recognition process, so the code recognition process of the wrist flip event can be directly terminated. If there is a next wrist flip event, the code recognition process of the next wrist flip event will be entered, that is, the operation starting from step 8 above will be executed again.

[0260] Alternatively, if there is no next wrist-turning event, the code recognition process of the current wrist-turning event is terminated, and the wrist-turning event can be continuously monitored.

[0261] 27. The main control module recognizes the code value of the QR code.

[0262] If a QR code is identified in the image in the first stage, the process proceeds to the second stage, which is to identify the code value of the QR code.

[0263] 28. If the code value recognition of the QR code fails, the main control module can obtain the timing information of the current wrist flip event from the timing module.

[0264] In the current step, the timing information of the current wrist-flipping event may be the execution duration of the current wrist-flipping event.

[0265] 29. The main control module determines whether the code recognition process of the current wrist flip event has timed out based on the timing information of the current wrist flip event.

[0266] If the QR code parsing for a certain frame image is unsuccessful in the second stage, that is, the code value recognition fails, then you can first determine whether the code recognition process of the current wrist flip event has timed out based on the timing information of the current wrist flip event. If it has not timed out, you can continue to acquire the image and parse the QR code. If it has timed out, you can directly determine that the execution result of the second stage is an execution failure, and then you can determine that the code recognition process of the current wrist flip event has naturally ended, and enter the code recognition process of the next wrist flip event.

[0267] 30. If the timeout is exceeded, then when there is a next wrist flip event, the main control module enters the code recognition process for the next wrist flip event.

[0268] When the code recognition process of the current wrist-flipping event times out, it means that the code recognition process of the current wrist-flipping event is finished, and thus the code recognition process of the next wrist-flipping event can be entered. Specifically, for the next wrist-flipping event, the operations starting from step 8 above will be re-executed.

[0269] Alternatively, when the code recognition process of the current wrist-turning event times out, if there is no next wrist-turning event, the code recognition process of the current wrist-turning event is terminated, and then the wrist-turning event is continuously monitored.

[0270] 31. If the code value of the QR code is successfully recognized, the main control module can obtain the timing information of the current wrist flip event and the timing information of the next wrist flip event from the timing module.

[0271] 32. The main control module determines whether the time interval between the current wrist flip event and the next wrist flip event is less than a preset interval.

[0272] If the QR code parsing for a certain frame image is successful in the second stage, that is, the code value recognition is successful, then the execution result of the second stage can be directly determined as successful. Further, it can be determined whether the time interval between the current wrist flip event and the next wrist flip event is less than or equal to a preset threshold. The time interval in this embodiment refers to the time interval between two wrist flip events monitored.

[0273] Therefore, the main control module can obtain the timing information of the current wrist-flipping event and the timing information of the next wrist-flipping event from the timing module. In the current step, the timing information of the current wrist-flipping event can be the moment when the current wrist-flipping event is monitored, and the timing information of the next wrist-flipping event can be the moment when the next wrist-flipping event is monitored, which are all recorded in the timing module.

[0274] 33. If it is less than the preset interval, when there is a next wrist flip event, the main control module discards the next wrist flip event.

[0275] If the time interval between the current wrist-flip event and the next wrist-flip event is less than the preset interval, the main control module discards the next wrist-flip event.

[0276] Because the time interval between the first wrist-turning event (next wrist-turning event) and the second wrist-turning event (current wrist-turning event) is very short, generally, after the code is successfully recognized for a wrist-turning event (the second wrist-turning event in this embodiment), the user turns back the phone to check the result and triggers another wrist-turning event (the first wrist-turning event in this embodiment). Therefore, when the time interval between the two wrist-turning events is relatively small, the first wrist-turning event will be ignored in this embodiment. In this case, the second wrist-turning event itself has been successfully recognized, and ignoring the first wrist-turning event will not affect the recognition success rate, and unnecessary recognition processing can be avoided, saving system resources.

[0277] Alternatively, if there is no next wrist-turning event, after the code value of the QR code is successfully recognized, the wrist-turning event can be continuously monitored.

[0278] 34. If it is greater than or equal to the preset interval, when there is the next wrist flip event, the main control module enters the code recognition process of the next wrist flip event.

[0279] If the time interval is greater than or equal to the preset interval, when there is the next wrist flip event, the main control module determines that the code recognition process of the current wrist flip event is naturally ended, and enters the code recognition process of the next wrist flip event, that is, the operation starting from step 8 above will be executed again.

[0280] If the time interval between the first wrist-flipping event and the second wrist-flipping event is relatively long, it is possible that after the user successfully identifies a wrist-flipping event (the second wrist-flipping event in this embodiment), another wrist-flipping event (the first wrist-flipping event in this embodiment) that requires identification is triggered again. Therefore, when the time interval between the two wrist-flipping events is relatively large, in this embodiment, the identification process of the second wrist-flipping event is determined to be completed and the identification process of the first wrist-flipping event is executed to ensure that the wrist-flipping events that require identification are processed as comprehensively as possible.

[0281] Alternatively, if there is no next wrist-turning event, after the code value of the QR code is successfully recognized, the wrist-turning event can be continuously monitored.

[0282] 35. The main control module sends the QR code recognition result to the perception module.

[0283] After the QR code recognition is successful, the main control module can send the QR code recognition result to the system service module, where the QR code recognition result can be understood as the QR code information introduced above.

[0284] 36. The perception module sends the QR code recognition result to the system service module.

[0285] 37. The system service module sends the QR code recognition result to the code recognition APP.

[0286] After that, the perception module sends the QR code recognition result to the system service module, and the system service module sends the QR code recognition result to the code recognition APP. The code recognition APP can pull up the service interface corresponding to the QR code information based on the QR code recognition result. After that, the code recognition channel between the code recognition APP and the AO service will automatically release resources, for example, the CA module and TA module will be restored to their original state, etc.

[0287] Alternatively, if the code recognition APP does not receive the QR code recognition result after a certain period of time, the code recognition channel will also be released.

[0288] In this embodiment, the perception module is responsible for detecting the screen unlocking and monitoring the wrist turning event, which can effectively avoid the frequent registration and deregistration of the code recognition APP, thereby improving the functional stability and reducing power consumption. And the various modules in the AO module interact, so as to schedule the front camera to collect images and identify the QR code information in the image, so as to save the complicated interaction process between the modules and minimize the unnecessary startup and shutdown of application functions. The purpose of improving functional stability and reducing power consumption can also be achieved. And in this embodiment, when the image is encrypted and decrypted, the TA module does not need to know what business function is currently executed. It only needs the AO service (module) to pass the image address and image size. The encryption function in the TA module will automatically encrypt the data under the corresponding address and then return it to the CA module in the AO service, thereby reducing the data dependence on the AO service to a certain extent, integrating the encryption function under the TEE as highly as possible, reducing the information interaction with the business end, and effectively improving the general ability and generalization ability of the encryption and decryption function, which can adapt to more functions with encryption scenarios in the future.

[0289] Based on the above introduction, another possible implementation method of the CA module and the TA module to implement encryption and decryption processing of the image is described below.

[0290] In another possible implementation, the image data collected by the camera of some terminal devices is dual-stream, for example, a TOF (Time of flight) type image and an RGB (red, green, and blue) type image may be output.

[0291] For this type of terminal device with dual-stream image data, during the wrist-flipping code recognition process, the algorithm design needs to wait until both types of images are collected before the TA module can perform encryption processing. Therefore, in this case, the encryption function in the TA module has a strong binding relationship with the business function of wrist-flipping code recognition.

[0292] Correspondingly, in step 18 described above, the data sent by the CA module to the TA module may include: service function identifier, image address, image size, and image type, wherein the image sizes of different types of images may be different.

[0293] Then, the TA module first determines that the encryption processing of the wrist-flipping code recognition service is to be processed according to the service function identifier sent by the CA module, and then determines to ensure that both types of images are collected. The TA module then obtains the corresponding image according to the image address and image size, and verifies whether both types of images have been collected according to the image type. After determining that the collection is completed, adaptive image processing can be performed for different types of images, such as denoising, and then the image is encrypted. The implementation method of the subsequent encryption processing is similar to that described above, and will not be repeated here.

[0294] It can be understood that this implementation method can ensure that the terminal device for collecting dual-stream image data can also effectively adapt to the QR code recognition method provided by this application.

[0295] Based on the above introduction, it can be determined that if the first wrist-turning event is monitored, but the code recognition process of the second wrist-turning event is in progress, it is necessary to determine the processing strategy for the first wrist-turning event according to the code recognition process of the second wrist-turning event. Figure 8 The implementation method of determining the processing strategy of the first wrist flip event is further introduced. Figure 8 A schematic flow chart of a processing strategy for determining a wrist flip event provided in an embodiment of the present application.

[0296] like Figure 8 As shown, the method includes:

[0297] First, the first stage of processing is performed on the wrist flip event (that is, the second wrist flip event) to identify whether there is a QR code in the image obtained in the first stage of the wrist flip event. Then, according to the judgment result of the first stage, there may be the following three situations:

[0298] Case 1: Among the consecutive t frames, s frames are identified as having codes.

[0299] Wherein, t and s are integers greater than or equal to 1, and t is greater than s.

[0300] In this case, it can be confirmed that the code is recognized in the first stage, which means that the execution result of the first stage is successful, and then enter the second stage processing process of the wrist flip event, and then determine the subsequent processing strategy based on the execution result of the second stage.

[0301] For example, take t=3, s=2 as an example (at least 2 of the 3 consecutive frames have codes), assuming that 6 frames of images have been acquired up to the current moment, and the judgment results of the 5th and 6th frames are that there are QR codes in the images, it can be understood that there are 2 frames with codes in the 4th to 6th consecutive 3 frames, so it can be determined that the execution result of the first stage is successful. After determining that the execution result of the first stage is successful, the second stage can be entered without continuing to collect images for the first stage, thereby effectively improving the processing efficiency of the code recognition process.

[0302] Case 2: m frames of images are continuous and uncoded, where m is an integer greater than or equal to 1.

[0303] In this case, it can be confirmed that the QR code is not recognized in the first stage, which means that the execution result of the first stage is execution failure. Correspondingly, it means that the QR code is not recognized during the processing of the second wrist flip event. Therefore, the code recognition process of the current wrist flip event (that is, the second wrist flip event) can be directly terminated, and then the code recognition process of the next wrist flip event (that is, the first wrist flip event) can be entered to improve the success rate of wrist flip code recognition.

[0304] For example, taking m=10 as an example (10 consecutive frames without code), assuming that 12 frames of images have been acquired up to the current moment, and the judgment results of the 1st to 12th frames of images are all that there is no QR code in the image, then it can be determined that there are 10 consecutive frames of images with the judgment results that there is no QR code in the image, so it can be determined that the execution result of the first stage is execution failure. After determining that the execution result of the first stage is execution failure, the current code recognition process can be terminated without entering the second stage, so the processing efficiency of the code recognition process can be effectively improved.

[0305] Case 3: intermittent coding but does not meet Case 1 and Case 2, and the number of frames of the image acquired in the first stage is greater than or equal to the preset number of frames or the duration of the first stage is greater than or equal to the preset duration.

[0306] In this case, it can be confirmed that enough images have been acquired in the first stage or the process lasted for a sufficient period of time, but no QR code was recognized, which means that the execution result of the first stage is execution failure. Therefore, the code recognition process of the current wrist flip event (that is, the second wrist flip event) is directly terminated, and then the code recognition process of the next wrist flip event (that is, the first wrist flip event) is entered.

[0307] Alternatively, if the number of frames of the acquired image is less than the preset number of frames and the duration of the first stage is less than the preset duration, it means that the number of images currently collected and the execution time of the first stage are not sufficient to make a final judgment. Therefore, the front camera of the terminal device can continue to be called to acquire images, and determine whether the QR code is recognized for the acquired images, and then determine which of the above-mentioned situations is met based on the judgment results of each image acquired in the first stage.

[0308] Then the subsequent process of situation 1 is explained. Situation 1 corresponds to the successful execution of the first stage, and then the second stage is executed for the wrist flip event. In the second stage, the code value of the QR code is identified and decoded.

[0309] If the QR code parsing for a certain frame image is unsuccessful in the second stage, it can be determined whether the second stage has timed out. If not, the image can continue to be acquired and the QR code parsing can be performed. If it has timed out, the execution result of the second stage can be directly determined to be an execution failure. Then, it can be determined that the code recognition process of the current wrist flip event has naturally ended and the code recognition process of the next wrist flip event can be entered.

[0310] If the QR code parsing of a certain frame image is successful in the second stage, the execution result of the second stage can be directly determined to be successful. Further, it can be determined whether the time interval between the current wrist flip event and the next wrist flip event is less than or equal to the preset interval. If it is less than or equal to the preset interval, the next wrist flip event is ignored. If it is greater than the preset interval, it is determined that the code recognition process of the current wrist flip event is naturally ended, and the code recognition process of the next wrist flip event is entered.

[0311] In this embodiment, it can be determined based on the recognition results of the first stage and the recognition results of the second stage whether to ignore the first wrist-flip event, terminate the current recognition process of the second wrist-flip event and execute the recognition process of the first wrist-flip event, or confirm that the current recognition process of the second wrist-flip event ends naturally and execute the recognition process of the first wrist-flip event. This can effectively avoid the problem of low wrist-flip recognition success rate caused by directly ignoring the first wrist-flip event when the second wrist-flip event is being executed, and improve the success rate of wrist-flip recognition as much as possible.

[0312] Based on the above embodiments, Fig. 9 and Fig.10 Taking a specific application scenario as an example, the changes in the interface are explained exemplarily. Fig. 9 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 1 , Fig.10 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 2 .

[0313] First, let’s introduce Fig. 9 ,like Fig. 9 As shown in (a), the terminal device is in the desktop state, that is, the terminal device does not run an application in the front end; at this time, the screen direction of the terminal device is upward (upward refers to the positive direction of the z-axis of the world coordinate system). Fig. 9 As shown in (b) in the figure, the user holding the terminal device turns his wrist, and the user turns the screen of the terminal device downward (the negative direction of the z-axis of the world coordinate system). Furthermore, according to the method provided in this embodiment, if it is determined that the terminal device is in a bright screen unlocked state, the terminal device is in a desktop state, the proximity light is not blocked (not a necessary condition), the screen direction is not facing upward, there is a wrist turn, the terminal device is within a preset geographic fence (not a necessary condition), and the value represented by the ambient light information is greater than or equal to a second preset threshold value (not a necessary condition), then the front camera is turned on to capture images and recognize the QR code. Then, as Fig. 9 As shown in (c), the terminal device will not change Fig. 9 The interface (a) in the figure still displays the desktop of the terminal device, but the terminal device uses the front camera of the terminal device to obtain the image in the background and recognizes the QR code information in the image in the background. Then, after the terminal device recognizes the QR code information, Fig. 9 As shown in (d) in the figure, the terminal device directly calls up the interface corresponding to the QR code information, for example, the transfer interface of application A is displayed.

[0314] Again Fig.10 To introduce, such as Fig.10 As shown in (a), the terminal device is in the desktop state, that is, the terminal device does not run an application in the front end; at this time, the screen direction of the terminal device is upward (upward refers to the positive direction of the z-axis of the world coordinate system). Fig.10 As shown in (b), the user holding the terminal device turns his wrist, and the user turns the screen of the terminal device downward (the negative direction of the z-axis of the world coordinate system). Furthermore, according to the method provided in this embodiment, if it is determined that the terminal device is in a bright screen unlocked state, the terminal device is in a desktop state, the proximity light is not blocked (not a necessary condition), the screen direction is not facing upward, there is a wrist turn, the terminal device is within a preset geographic fence (not a necessary condition), and the value of the ambient light information is greater than or equal to a second preset threshold (not a necessary condition), then the front camera is turned on to capture images and recognize the QR code. Fig.10 As shown in (c) of the figure, the terminal device calls the front camera of the terminal device to obtain an image and recognize the QR code information in the image. The terminal device can display a code scanning interface. Then, after the terminal device recognizes the QR code information, Fig.10As shown in (d) in the figure, the terminal device directly calls up the interface corresponding to the QR code information, for example, the transfer interface of application A is displayed.

[0315] Fig. 9 and Fig.10 The situations described above are all triggered when the terminal device is in desktop state. However, based on the above description, it can be determined that the terminal device being in desktop state is optional, that is, the wrist flip code recognition can also be triggered when the terminal device is not in desktop state. For example, Fig. 9 and Fig.10 In the example, if a video playback application is running in the foreground, the figure (a) corresponds to the video playback interface, which can also trigger subsequent related operations. Its implementation method is similar to that described above and will not be repeated here.

[0316] Therefore, in the technical solution of the present application, it is possible to directly determine to call the front camera to collect images and identify the QR code information according to the operating status and posture information of the terminal device. It is no longer necessary for the user to perform a series of touch operations on the terminal device to trigger the terminal device to scan the QR code. This saves the time of identifying the QR code and improves the efficiency of QR code recognition.

[0317] The two-dimensional code recognition method of the embodiment of the present application has been described above, and the device for executing the two-dimensional code recognition method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the two-dimensional code recognition method.

[0318] Fig.11 This is a schematic diagram of the structure of the two-dimensional code recognition device provided in the embodiment of the present application. Fig.11 As shown, the device 110 includes: a sending unit 1101, a processing unit 1102;

[0319] The sending unit 1101 is configured to send an event notification message of the wrist-turning event to the AO module if the perception module monitors the wrist-turning event;

[0320] The sending unit 1101 is also used for the AO module to send a start instruction to the camera driving module according to the event notification message;

[0321] Processing unit 1102, configured for the camera driving module to drive the front camera of the terminal device to capture an image in response to the start instruction;

[0322] The processing unit 1102 is also used by the AO module to obtain the collected image and identify the two-dimensional code information in the image.

[0323] In some implementations, the AO module includes a main control module, a timing module, a camera control module, and a CA module.

[0324] In some implementations, the sending unit 1101 is specifically configured to:

[0325] An event notification message of the wrist turning event is sent to the main control module.

[0326] In some implementations, the sending unit 1101 is specifically configured to:

[0327] The camera control module sends a start instruction to the camera driving module according to the event notification message.

[0328] In some implementations, the terminal device further includes a TA module running in a TEE environment;

[0329] The processing unit 1102 is further configured to:

[0330] After sending the event notification message of the wrist-turning event to the AO module, the main control module pulls up the CA module;

[0331] The CA module pulls up the TA module;

[0332] The CA module generates a key and sends the key to the TA module.

[0333] In some implementations, the processing unit 1102 is further configured to:

[0334] After the camera driving module drives the front camera of the terminal device to capture an image in response to the start instruction, the camera driving module sends an image address of the image to the CA module;

[0335] The CA module sends the image address and the preset image size to the TA module;

[0336] The TA module acquires the image according to the image address and the image size;

[0337] The TA module encrypts the image according to the key to obtain an encrypted image, and stores the encrypted image in a shared memory of the TA module and the CA module.

[0338] In some implementations, the processing unit 1102 is specifically used for: the CA module obtains the encrypted image from the shared memory; the CA module uses the key to decrypt the encrypted image to obtain the decrypted image; and the CA module sends the decrypted image to the main control module.

[0339] In some implementations, the processing unit 1102 is further used for: after sending the event notification message of the wrist-flipping event to the AO module, the main control module sends a timing request to the timing module; and the timing module performs timing for the wrist-flipping event.

[0340] In some implementations, the sending unit 1101 is also used for: after the AO module identifies the QR code information in the image, if the main control module successfully identifies the QR code information in the image, the main control module sends the QR code information to the perception module; the perception module sends the QR code information to the system service module; the system service module sends the QR code information to the code recognition APP.

[0341] In some implementations, the processing unit 1102 is further configured to:

[0342] After the AO module identifies the QR code information in the image, if the main control module fails to identify the QR code information in the image, the main control module obtains the timing duration for the wrist-flipping event in the timing module; if the timing duration is greater than or equal to the preset duration, the main control module ends the code recognition process of the wrist-flipping event; if the timing duration is less than the preset duration, the operation of identifying the QR code information in the image is repeated until the code recognition is successful or times out.

[0343] The two-dimensional code recognition method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal device can refer to the above related descriptions, which will not be repeated here.

[0344] An embodiment of the present application provides a terminal device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0345] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0346] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0347] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0348] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is run, the computer executes the above method. The embodiment of the present application is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0349] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A two-dimensional code recognition method, characterized in that: Applied to a terminal device, the terminal device includes a perception module, an AO module and a camera driving module; the method includes: If the perception module detects a wrist-turning event, it sends an event notification message of the wrist-turning event to the AO module; The AO module sends a start instruction to the camera driving module according to the event notification message; The camera driving module drives the front camera of the terminal device to capture images in response to the start instruction; The AO module acquires the collected image and identifies the two-dimensional code information in the image.

2. The method according to claim 1, characterized in that The AO module includes a main control module, a timing module, a camera control module and a client application CA module.

3. The method according to claim 2, characterized in that The sending the event notification message of the wrist turning event to the AO module includes: An event notification message of the wrist turning event is sent to the main control module.

4. The method according to claim 2, characterized in that: The AO module sends a start instruction to the camera driving module according to the event notification message, including: The camera control module sends a start instruction to the camera driving module according to the event notification message.

5. The method according to any one of claims 2 to 4, characterized in that: The terminal device also includes a TA module running in a trusted execution environment TEE environment; After sending the event notification message of the wrist turning event to the AO module, the method further includes: The main control module pulls up the CA module; The CA module pulls up the TA module; The CA module generates a key and sends the key to the TA module.

6. The method according to claim 5, characterized in that After the camera driving module drives the front camera of the terminal device to capture an image in response to the start instruction, the method further includes: The camera driving module sends the image address of the image to the CA module; The CA module sends the image address and the preset image size to the TA module; The TA module obtains the image according to the image address and the image size; The TA module encrypts the image according to the key to obtain an encrypted image, and stores the encrypted image in a shared memory of the TA module and the CA module.

7. The method according to claim 6, characterized in that The AO module acquires the collected image, including: The CA module obtains the encrypted image from the shared memory; The CA module decrypts the encrypted image using the key to obtain the decrypted image; The CA module sends the decrypted image to the main control module.

8. The method according to any one of claims 2 to 7, characterized in that: After sending the event notification message of the wrist turning event to the AO module, the method further includes: The main control module sends a timing request to the timing module; The timing module counts the wrist turning event.

9. The method according to claim 8, characterized in that The terminal device also includes a system service module and a code recognition application; After the AO module recognizes the two-dimensional code information in the image, the method further includes: If the main control module successfully identifies the two-dimensional code information in the image, the main control module sends the two-dimensional code information to the perception module; The perception module sends the two-dimensional code information to the system service module; The system service module sends the two-dimensional code information to the code recognition application.

10. The method according to claim 8, characterized in that After the AO module recognizes the two-dimensional code information in the image, the method further includes: If the main control module fails to identify the two-dimensional code information in the image, the main control module obtains the timing duration for the wrist turning event in the timing module; If the timing duration is greater than or equal to the preset duration, the main control module ends the code recognition process of the wrist turning event; If the timing duration is less than the preset duration, the operation of identifying the two-dimensional code information in the image is repeated until the code is successfully identified or the timeout occurs.

11. A two-dimensional code recognition device, characterized in that: include: A sending unit, configured to send an event notification message of the wrist-turning event to the AO module if the perception module monitors the wrist-turning event; The sending unit is also used for the AO module to send a start instruction to the camera driving module according to the event notification message; A processing unit, configured for the camera driving module to drive the front camera of the terminal device to capture an image in response to the start instruction; The processing unit is also used by the AO module to acquire the collected image and identify the QR code information in the image.

12. A terminal device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method according to any one of claims 1 to 10.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 10.

Images