Two-dimensional code identification method and device

Abstract

The embodiment of the invention provides a two-dimensional code recognition method and device, and the method comprises the steps: determining whether a code recognition process of a second wrist turning event is carried out or not if a terminal device monitors a first wrist turning event; and if the code identification process of the second wrist turning event is in progress, determining an execution result of the code identification process of the second wrist turning event. And according to the execution result, determining a processing strategy for the first wrist turning event, the processing strategy comprising at least one of the following: ignoring the first wrist turning event, terminating the code identification process of the second wrist turning event and executing the code identification process for the first wrist turning event, and determining that the code identification process of the second wrist turning event is ended and executing the code identification process for the first wrist turning event. Wherein the code recognition process comprises the steps of calling a front camera of the terminal equipment to obtain an image, and recognizing two-dimensional code information in the image. According to the technical scheme, the identification efficiency of the two-dimensional code can be improved through a processing mode of identifying the code by turning the wrist, and the success rate of identifying the code by turning the wrist can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a two-dimensional code recognition method and device. Background Art

[0002] With the development of terminal technology, terminal devices have become important tools in people's lives, and terminal devices can provide users with multiple functions.

[0003] The various functions provided by current terminal devices include a QR code scanning function. In the prior art, the terminal devices usually respond to a series of operations by the user to scan the QR code, thereby completing life and work purposes.

[0004] However, in the process of the terminal device scanning the QR code, the user needs to perform a series of operations on the terminal device to trigger the terminal device to scan the QR code. The triggering process takes a certain amount of time, resulting in low QR code recognition efficiency. Summary of the invention

[0005] The embodiments of the present application provide a two-dimensional code recognition method and device, which are applied in the field of terminal technology.

[0006] In a first aspect, an embodiment of the present application provides a two-dimensional code recognition method. The method includes:

[0007] If the terminal device monitors the first wrist-turning event, a code recognition process for determining whether there is a second wrist-turning event is in progress;

[0008] If there is a code recognition process for the second wrist-turning event in progress, determining the execution result of the code recognition process for the second wrist-turning event;

[0009] Determine, according to the execution result, a processing strategy for the first wrist-flip event, the processing strategy comprising at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is terminated and executing the code recognition process for the first wrist-flip event;

[0010] The code recognition process includes: calling the front camera of the terminal device to obtain an image, and identifying the QR code information in the image.

[0011] In this implementation, when a first wrist-flipping event is monitored, it can be first determined whether a second wrist-flipping event is executing a code recognition process. If so, a processing strategy for the first wrist-flipping event is determined based on the execution result of the code recognition process of the second wrist-flipping event, wherein the processing strategy can indicate to ignore the first wrist-flipping event or to process the first wrist-flipping event, thereby effectively improving the success rate of wrist-flipping code recognition, and when processing the first wrist-flipping event, the recognition efficiency of the QR code can be improved by the wrist-flipping code recognition processing method.

[0012] In some implementations, the code recognition process includes a first stage and a second stage, in which an image is acquired and whether a two-dimensional code exists in the image is identified, and in which the two-dimensional code in the image is decoded;

[0013] The execution result of the code recognition process for determining the second wrist turning event includes:

[0014] If the code recognition process of the second wrist-flipping event is in the first stage, the execution result of the first stage is determined according to the judgment results corresponding to the acquired images, and the judgment results are used to indicate whether there is a QR code in the image;

[0015] If the code recognition process of the second wrist flip event is in the second stage, or if the execution result of the first stage is successful, determining the execution result of the second stage according to the decoding result of at least one frame of image acquired in the second stage, wherein the decoding result is used to indicate whether the decoding of the two-dimensional code in the image is successful;

[0016] The execution result of the code recognition process of the second wrist-turning event includes the execution result of the first stage and / or the execution result of the second stage.

[0017] In some implementations, determining the execution result of the first stage according to the identification results corresponding to the acquired images includes:

[0018] If the discrimination results corresponding to the acquired images meet the first condition, the execution result of the first stage is determined to be execution failure, and the first condition is: the discrimination result of the existence of m consecutive frames of images is that there is no QR code in the image, and m is an integer greater than or equal to 1;

[0019] If the discrimination results corresponding to the acquired images meet the second condition, the execution result of the first stage is determined to be successful, and the second condition is: the discrimination result of at least s frames of images in the continuous t frames of images is that the two-dimensional code exists in the image, and t and s are integers greater than or equal to 1;

[0020] If the identification results corresponding to the acquired images do not satisfy the first condition and the second condition, the execution result of the first stage is determined according to the number of frames of the acquired images and the duration of the first stage.

[0021] In this implementation, multiple conditions are preset, and then it is determined which condition the discrimination result of the acquired image specifically satisfies, thereby determining the execution result of the first stage. Specifically, in the judgment process, if it is determined that no QR code is recognized in multiple consecutive frames of images, the execution result of the first stage can be directly determined as execution failure, so as to end the processing of the first stage as soon as possible and improve the efficiency of the code recognition processing. And if it is determined that the QR code is stably recognized in multiple consecutive frames of images, the execution result of the first stage can be directly determined as execution success, and the processing of the first stage can also be ended as soon as possible to enter the next stage.

[0022] In some implementations, determining the execution result of the first stage according to the number of frames of the acquired image and the duration of the first stage includes:

[0023] If the number of frames of the acquired image is greater than or equal to the preset number of frames or the duration of the first stage is greater than or equal to the preset duration, determining that the execution result of the first stage is execution failure;

[0024] If the number of frames of the acquired image is less than the preset number of frames and the duration of the first stage is less than the preset duration, continue to call the front camera of the terminal device to acquire the image and determine the image recognition result, and repeat the operation of determining the execution result of the first stage according to the recognition results corresponding to each of the acquired images until the execution result of the first stage is determined to be a success or a failure.

[0025] In this implementation, when it is determined that there are images containing QR codes according to the discrimination results, but the discrimination results of these images do not meet the two conditions described above, the judgment is continued according to the number of frames of the acquired images and the duration of the first stage. When the number of frames is large enough or the duration of the first stage is long enough, the execution result of the first stage is directly determined to be execution failure, so as to avoid the first stage being unable to end for a long time, resulting in the continuous process of code recognition processing being too long. Or when the number of frames is insufficient and the duration of the first stage is not long enough, the image acquisition and discrimination can be continued to ensure that the execution result is determined based on a sufficient amount of data, thereby effectively improving the success rate of the determined execution result.

[0026] In some implementations, determining the execution result of the second stage according to the decoding result of at least one frame of image acquired in the second stage includes:

[0027] In the second stage, at least one frame of image is acquired through the front camera;

[0028] If the decoding result corresponding to any frame of the at least one frame of the image is a decoding success, determining that the execution result of the second stage is an execution success;

[0029] If the decoding results of the at least one frame of image are all decoding failures, it is determined that the execution result of the second stage is execution failure.

[0030] In some implementations, determining a processing strategy for the first wrist-flip event according to the execution result includes:

[0031] If the execution result of the first stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as: terminating the code recognition process of the second wrist-flipping event and executing the code recognition process for the first wrist-flipping event;

[0032] If the execution result of the first stage of the code recognition process of the second wrist-flipping event is successful, a processing strategy for the first wrist-flipping event is determined according to the execution result of the second stage of the code recognition process of the second wrist-flipping event.

[0033] In some implementations, determining a processing strategy for the first wrist-flipping event according to the execution result of the second stage of the code recognition process for the second wrist-flipping event includes:

[0034] If the execution result of the second stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as follows: determining that the code recognition process of the second wrist-flipping event ends and executing the code recognition process for the first wrist-flipping event;

[0035] If the execution result of the second stage of the code recognition process of the second wrist-flipping event is successful, obtaining the time interval between the first wrist-flipping event and the second wrist-flipping event;

[0036] If the time interval is less than or equal to the preset interval, determining that the processing strategy for the first wrist-flipping event is: ignoring the first wrist-flipping event;

[0037] If the time interval is greater than the preset interval, the processing strategy for the first wrist-flipping event is determined as: determining that the code recognition process for the second wrist-flipping event is finished and executing the code recognition process for the first wrist-flipping event.

[0038] In this implementation, a determination is made based on the recognition results of the first stage and the recognition results of the second stage whether to ignore the first wrist-flip event, terminate the current recognition process of the second wrist-flip event and instead execute the recognition process of the first wrist-flip event, or confirm that the current recognition process of the second wrist-flip event ends naturally and then execute the recognition process of the first wrist-flip event. This can effectively avoid the problem of a low wrist-flip recognition success rate caused by directly ignoring the first wrist-flip event when the second wrist-flip event is being executed, and can maximize the success rate of wrist-flip recognition.

[0039] In some implementations, the code recognition process for determining whether there is a second wrist flip event is in progress, including:

[0040] Access process execution flag;

[0041] If the process execution flag is the first value, it is determined that the code recognition process of the second wrist flip event is being executed; or,

[0042] If the process execution flag is the second value, it is determined that there is no code recognition process of the second wrist turning event being executed.

[0043] In this implementation, it is possible to quickly and efficiently determine whether there is a second wrist-turning event while the code recognition process is being executed.

[0044] In some implementations, monitoring the first wrist-flip event includes:

[0045] If it is determined that the value represented by the posture change information is greater than or equal to a first preset threshold and the screen direction is a direction other than the first direction, it is determined that a first wrist flip event is monitored, wherein the first direction is the positive direction of the z-axis of the world coordinate system.

[0046] In a second aspect, an embodiment of the present application provides a two-dimensional code recognition device. The device includes:

[0047] a determination unit, configured to determine whether a code recognition process for a second wrist-turning event is in progress if the terminal device monitors the first wrist-turning event;

[0048] The determining unit is further configured to determine the execution result of the code recognition process for the second wrist-flipping event if the code recognition process for the second wrist-flipping event is in progress;

[0049] The determining unit is further used to determine a processing strategy for the first wrist-flip event according to the execution result, wherein the processing strategy includes at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is terminated and executing the code recognition process for the first wrist-flip event;

[0050] The code recognition process includes: calling the front camera of the terminal device to obtain an image, and identifying the QR code information in the image.

[0051] In some implementations, the code recognition process includes a first stage and a second stage, in which an image is acquired and whether a two-dimensional code exists in the image is identified, and in which the two-dimensional code in the image is decoded;

[0052] The determining unit is specifically used for:

[0053] If the code recognition process of the second wrist-flipping event is in the first stage, the execution result of the first stage is determined according to the judgment results corresponding to the acquired images, and the judgment results are used to indicate whether there is a QR code in the image;

[0054] If the code recognition process of the second wrist flip event is in the second stage, or if the execution result of the first stage is successful, determining the execution result of the second stage according to the decoding result of at least one frame of image acquired in the second stage, wherein the decoding result is used to indicate whether the decoding of the two-dimensional code in the image is successful;

[0055] The execution result of the code recognition process of the second wrist-turning event includes the execution result of the first stage and / or the execution result of the second stage.

[0056] In some implementations, the determining unit is specifically configured to:

[0057] If the discrimination results corresponding to the acquired images meet the first condition, the execution result of the first stage is determined to be execution failure, and the first condition is: the discrimination result of the existence of m consecutive frames of images is that there is no QR code in the image, and m is an integer greater than or equal to 1;

[0058] If the discrimination results corresponding to the acquired images meet the second condition, the execution result of the first stage is determined to be successful, and the second condition is: the discrimination result of at least s frames of images in the continuous t frames of images is that the two-dimensional code exists in the image, and t and s are integers greater than or equal to 1;

[0059] If the identification results corresponding to the acquired images do not satisfy the first condition and the second condition, the execution result of the first stage is determined according to the number of frames of the acquired images and the duration of the first stage.

[0060] In some implementations, the determining unit is specifically configured to:

[0061] If the number of frames of the acquired image is greater than or equal to the preset number of frames or the duration of the first stage is greater than or equal to the preset duration, determining that the execution result of the first stage is execution failure;

[0062] If the number of frames of the acquired image is less than the preset number of frames and the duration of the first stage is less than the preset duration, continue to call the front camera of the terminal device to acquire the image and determine the image recognition result, and repeat the operation of determining the execution result of the first stage according to the recognition results corresponding to each of the acquired images until the execution result of the first stage is determined to be a success or a failure.

[0063] In some implementations, the determining unit is specifically configured to:

[0064] In the second stage, at least one frame of image is acquired through the front camera;

[0065] If the decoding result corresponding to any frame of the at least one frame of the image is a decoding success, determining that the execution result of the second stage is an execution success;

[0066] If the decoding results of the at least one frame of image are all decoding failures, it is determined that the execution result of the second stage is execution failure.

[0067] In some implementations, the determining unit is specifically configured to:

[0068] If the execution result of the first stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as: terminating the code recognition process of the second wrist-flipping event and executing the code recognition process for the first wrist-flipping event;

[0069] If the execution result of the first stage of the code recognition process of the second wrist-flipping event is successful, a processing strategy for the first wrist-flipping event is determined according to the execution result of the second stage of the code recognition process of the second wrist-flipping event.

[0070] In some implementations, the determining unit is specifically configured to:

[0071] If the execution result of the second stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as follows: determining that the code recognition process of the second wrist-flipping event ends and executing the code recognition process for the first wrist-flipping event;

[0072] If the execution result of the second stage of the code recognition process of the second wrist-flipping event is successful, obtaining the time interval between the first wrist-flipping event and the second wrist-flipping event;

[0073] If the time interval is less than or equal to the preset interval, determining that the processing strategy for the first wrist-flipping event is: ignoring the first wrist-flipping event;

[0074] If the time interval is greater than the preset interval, the processing strategy for the first wrist-flipping event is determined as: determining that the code recognition process for the second wrist-flipping event is finished and executing the code recognition process for the first wrist-flipping event.

[0075] In some implementations, the determining unit is specifically configured to:

[0076] Access process execution flag;

[0077] If the process execution flag is the first value, it is determined that the code recognition process of the second wrist flip event is being executed; or,

[0078] If the process execution flag is the second value, it is determined that there is no code recognition process of the second wrist turning event being executed.

[0079] In some implementations, the determining unit is specifically configured to:

[0080] If it is determined that the value represented by the posture change information is greater than or equal to a first preset threshold and the screen direction is a direction other than the first direction, it is determined that a first wrist flip event is monitored, wherein the first direction is the positive direction of the z-axis of the world coordinate system.

[0081] In a third aspect, an embodiment of the present application provides a terminal device, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc.

[0082] The terminal device comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method of the first aspect.

[0083] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0084] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method of the first aspect.

[0085] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method described in the first aspect.

[0086] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 1 ;

[0088] Figure 2 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 2 ;

[0089] Figure 3 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0090] Figure 4 A software structure diagram of a terminal device provided in an embodiment of the present application;

[0091] Figure 5 A flowchart of a two-dimensional code recognition method provided in an embodiment of the present application;

[0092] Figure 6 A schematic diagram of a process for determining the execution result of the first stage provided in an embodiment of the present application;

[0093] Figure 7 The image discrimination result provided in the embodiment of the present application is shown in FIG. Figure 1 ;

[0094] Figure 8 The image discrimination result provided in the embodiment of the present application is shown in FIG. Figure 2 ;

[0095] Fig. 9 The image discrimination result provided in the embodiment of the present application is shown in FIG. Figure 3 ;

[0096] Fig.10 A schematic diagram of a flow chart for determining the execution result of the second stage provided in an embodiment of the present application;

[0097] Fig.11 The decoding result of the image provided in the embodiment of the present application is shown as follows Figure 1 ;

[0098] Fig.12 The decoding result of the image provided in the embodiment of the present application is shown as follows Figure 2 ;

[0099] Fig.13 A schematic diagram of a decoding process provided in an embodiment of the present application;

[0100] Fig.14 A schematic diagram of an implementation of determining a processing strategy according to an execution result provided in an embodiment of the present application;

[0101] Fig.15 A logical schematic diagram of a processing strategy for determining a first wrist-flip event provided in an embodiment of the present application;

[0102] Fig.16 An interactive flow chart of a two-dimensional code recognition method provided in an embodiment of the present application;

[0103] Fig.17 A schematic diagram of the software architecture of the two-dimensional code recognition method provided in the embodiment of the present application;

[0104] Fig.18 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 1 ;

[0105] Fig.19 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 2 ;

[0106] Fig. 20 A schematic diagram of the structure of a two-dimensional code recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0107] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0108] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0109] It should be noted that the "at..." in the embodiment of the present application can be the instant when a certain situation occurs, or a period of time after a certain situation occurs, and the embodiment of the present application does not specifically limit this. In addition, the display interface provided in the embodiment of the present application is only an example, and the display interface can also include more or less content.

[0110] With the development of terminal technology, terminal devices have become an important tool in people's lives. Terminal devices provide users with a variety of functions. For example, terminal devices can respond to a series of operations of users, scan QR codes, and thus complete life and work purposes.

[0111] For example, when a user uses a terminal device to scan a QR code, the user usually triggers the terminal device's application or applet first, and then triggers the terminal device's QR code recognition function based on the application or applet's entry. Then, the terminal device can call the camera to obtain an image and recognize the QR code based on the obtained image.

[0112] It can be seen that in the process of terminal equipment recognizing QR codes, users are required to perform a series of operations on the terminal equipment to achieve the purpose of code recognition. Both triggering code recognition and the actual code recognition process take a certain amount of time, resulting in low QR code recognition efficiency.

[0113] In view of this, an embodiment of the present application provides a QR code recognition method, wherein the method can monitor wrist-turning events through the posture change information and screen direction of the terminal device, and then directly call the front camera of the terminal device to capture images and recognize QR codes based on the monitored wrist-turning events, thereby avoiding a series of tedious operations for triggering code recognition, saving time for recognizing QR codes, improving QR code recognition efficiency, and enhancing user experience.

[0114] Figure 1 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 1 .like Figure 1 As shown in (a) of FIG. 1 , the user holds the terminal device 100, and the terminal device 100 is in a bright screen unlocked state. Figure 1 As shown in (b), the user flips the terminal device 100, that is, the user has a wrist-turning action; then based on the QR code recognition method of the embodiment of the present application, the terminal device 100 calls the front camera of the terminal device to obtain the image on the desktop and recognizes the QR code information in the image. Based on the recognized QR code information, the terminal device 100 calls the interface of the application corresponding to the QR code information.

[0115] Figure 2 Schematic diagram of the scenario applicable to the two-dimensional code recognition method of the embodiment of the present application Figure 2 .like Figure 2 As shown, the user holds a terminal device 100; based on the QR code recognition method of the embodiment of the present application, the terminal device 100 calls the rear camera of the terminal device to obtain an image on the wall and recognizes the QR code information in the image. Based on the recognized QR code information, the terminal device 100 calls the interface of the application corresponding to the QR code information.

[0116] The terminal device 100 may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0117] The terminal device 100 has a function of displaying an interface and a function of acquiring an image.

[0118] In order to better understand the embodiment of the present application, the structure of the terminal device in the embodiment of the present application is introduced below:

[0119] Figure 3A schematic diagram of the structure of a terminal device 100 provided in an embodiment of the present application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0120] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0121] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0122] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0123] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0124] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0125] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0126] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0127] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0128] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0129] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0130] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0131] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0132] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0133] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0134] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0135] Gyro sensor 180B is used to collect angle change data. Gyro sensor 180B is used to collect angle data.

[0136] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in all directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device and applied to applications such as horizontal and vertical screen switching and pedometers.

[0137] The acceleration change data is collected based on the acceleration sensor 180E. The terminal device determines the posture change information of the terminal device based on the acceleration change data collected by the acceleration sensor 180E.

[0138] The terminal device determines the screen direction based on the angle data collected by the gyro sensor 180B.

[0139] The terminal device determines the posture change information of the terminal device based on the angle change data collected by the gyroscope sensor 180B. Alternatively, the terminal device determines the posture change information of the terminal device based on the acceleration change data collected by the acceleration sensor 180E. Alternatively, the terminal device determines the posture change information of the terminal device based on the angle change data collected by the gyroscope sensor 180B and the acceleration change data collected by the acceleration sensor 180E.

[0140] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and pocket mode automatically unlocks and locks the screen.

[0141] The proximity light sensor 180G collects proximity light data. The terminal device determines proximity light information based on the proximity light data collected by the light sensor 180G.

[0142] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touch.

[0143] The ambient light sensor 180L collects ambient light data. The terminal device determines ambient light information based on the ambient light data collected by the ambient light sensor 180L.

[0144] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0145] Figure 4 A software structure block diagram of the terminal device 100 provided in an embodiment of the present application.

[0146] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0147] The application layer can include a series of application packages.

[0148] like Figure 4 As shown, the application package may include applications such as camera, calendar, phone, map, call, music, settings, mailbox, video, social, etc.

[0149] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0150] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a resource manager, a view system, a notification manager, and the like.

[0151] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, touch the screen, drag the screen, capture the screen, etc.

[0152] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0153] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0154] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0155] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0156] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0157] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0158] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0159] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0160] The following is an illustrative description of the software and hardware workflow of the terminal device 100 in conjunction with a scenario where an application is started or an interface switch occurs in an application.

[0161] The following is a detailed description of the operation process of the two-dimensional code recognition provided by the embodiment of the present application and the display process of the interface during the two-dimensional code recognition process in conjunction with the accompanying drawings. It should be noted that the "at..." in the embodiment of the present application can be the instant when a certain situation occurs, or it can be a period of time after a certain situation occurs, and the embodiment of the present application does not make specific limitations on this.

[0162] Based on the above introduction, it can be determined that in the technical solution of this application, the terminal device can monitor the wrist flip event to call the front camera of the terminal device to capture images and recognize the QR code, thereby improving the recognition efficiency of the QR code. However, it can be understood that in actual use scenarios, users may frequently flip their wrists. When there are two or more wrist flip events, the processing strategy for the multiple wrist flip events monitored before and after has a great impact on the success rate of wrist flip scanning. Therefore, this application also introduces the processing method when multiple wrist flip events are monitored.

[0163] Let's first combine Figure 5 The two-dimensional code recognition method provided in the embodiment of the present application is introduced. Figure 5 A flowchart of a QR code recognition method provided in an embodiment of the present application.

[0164] like Figure 5 As shown, the method includes:

[0165] S501: If the terminal device monitors a first wrist-turning event, a code recognition process for determining whether a second wrist-turning event is in progress.

[0166] In this embodiment, the terminal device can continuously monitor the wrist-flipping event, because the user may frequently perform the wrist-flipping operation, so the terminal device may frequently monitor the wrist-flipping event. For performance considerations, it is not possible to directly perform code recognition processing every time a wrist-flipping event is monitored. In a possible implementation, the terminal device can first determine whether there is a code recognition process for a historical wrist-flipping event in progress each time a new wrist-flipping event is monitored, and then decide how to process the newly monitored wrist-flipping event.

[0167] For example, an access process execution flag may be set. If the process execution flag is a first value, it is determined that a code recognition process with a second wrist-flipping event is being executed; or, if the process execution flag is a second value, it is determined that a code recognition process with no second wrist-flipping event is being executed.

[0168] For example, the first value may be 1, and the second value may be 0. When the code recognition process of a wrist flip event starts to be executed each time, the process execution flag is set to 1.

[0169] In this embodiment, the newly monitored wrist-flip event is referred to as the first wrist-flip event, and the wrist-flip event monitored before the first wrist-flip event and for which the current terminal device is performing a code recognition process is referred to as the second wrist-flip event.

[0170] That is to say, when the terminal device monitors the first wrist flip event, it will first determine whether there is a code recognition process for the second wrist flip event in progress. The code recognition process in this embodiment includes calling the front camera of the terminal device to obtain an image and identifying the QR code information in the image.

[0171] In one possible implementation, when the terminal device monitors a wrist-turning event, the following method may be adopted: if it is determined that the value represented by the posture change information of the terminal device is greater than or equal to a first preset threshold value, and the screen direction is a direction other than the first direction, then it is determined that a wrist-turning event is monitored, wherein the first direction is the positive direction of the z-axis of the world coordinate system.

[0172] In one example, an accelerometer and a gyroscope sensor are provided in the terminal device, and the acceleration sensor collects acceleration change data within a preset time period, and the gyroscope sensor collects angle change data within a preset time period; wherein the preset time period is a time period before the current moment. Then, the posture change information of the terminal device can be determined in the following manner: obtaining the acceleration change data collected by the accelerometer in the terminal device within the preset time period, and / or obtaining the angle change data collected by the gyroscope sensor in the terminal device within the preset time period; determining the posture change information according to the acceleration change data and / or the angle change data. wherein the posture change information represents the posture change angle of the terminal device, or the posture change information represents the rotation change angle of the terminal device.

[0173] In one example, when determining the screen direction of a terminal device, the following method can be used: obtain the angle data collected by the gyroscope sensor in the terminal device at the current moment; determine the screen direction of the terminal device based on the angle data. For example, the gyroscope sensor collects xyz axis data (that is, angle data) at the current moment; the terminal device obtains the xyz axis data collected by the gyroscope sensor, and then the terminal device determines the screen direction of the terminal device based on the xyz axis data, whether it is not facing the positive direction of the z axis of the world coordinate system, that is, whether the screen direction of the terminal device is not facing upward.

[0174] The “upward” direction of the screen of the terminal device means that the screen of the terminal device faces the “positive direction of the z-axis of the world coordinate system”; and the first direction refers to the positive direction of the z-axis of the world coordinate system.

[0175] The “downward” direction of the screen of the terminal device means that the screen of the terminal device faces the “negative direction of the z-axis of the world coordinate system”; and the second direction refers to the negative direction of the z-axis of the world coordinate system.

[0176] Based on the above principle, the terminal device determines that the terminal device has flipped over according to the value represented by the obtained posture change information, which is greater than or equal to the first preset threshold value; it can be known that the user holding the terminal device flips the wrist to flip the screen of the terminal device. In addition, the terminal device determines that the screen direction is a direction other than the first direction, and then determines that the screen direction of the terminal device is not facing upward. Then, the terminal device determines that a wrist flip event is monitored.

[0177] Therefore, in the present embodiment, when the terminal device determines that the terminal device is flipped over and the screen direction of the terminal device is not facing upward, it can determine that a wrist-turning event has been monitored. If the wrist-turning event is to be processed, the front camera of the terminal device will be called to capture an image and identify the QR code in the image. This avoids the drawback that multiple human-computer interactions between the user and the terminal device are required to trigger the terminal device to identify the QR code, reduces the number of user operations, speeds up the efficiency of identifying the QR code, and improves the user experience.

[0178] S502: If the code recognition process of the second wrist-turning event is in progress, determine the execution result of the code recognition process of the second wrist-turning event.

[0179] In a possible implementation, if there is a code recognition process for the second wrist-flipping event in progress, the execution result of the code recognition process for the second wrist-flipping event can be determined, where the execution result can be execution success or execution failure.

[0180] Alternatively, if there is no code recognition process in progress for the second wrist-flipping event, that is, there is no wrist-flipping event currently undergoing a code recognition process, the code recognition process can be directly executed on the first wrist-flipping event to call the front camera of the terminal device to capture images and recognize the QR code, thereby improving the recognition efficiency of the QR code.

[0181] S503. Determine a processing strategy for the first wrist-flip event according to the execution result, the processing strategy comprising at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is terminated and executing the code recognition process for the first wrist-flip event.

[0182] After determining the execution result of the code recognition process of the second wrist-flipping event, how to process the first wrist-flipping event can be determined according to the execution result of the code recognition process of the second wrist-flipping event, that is, determining the processing strategy of the first wrist-flipping event.

[0183] In this embodiment, the processing strategy for the first wrist-flip event may include at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is completed and executing the code recognition process for the first wrist-flip event.

[0184] That is to say, in this embodiment, how to process the first wrist-flip event is determined according to the execution result of the code recognition process of the second wrist-flip event, specifically, whether to directly ignore the first wrist-flip event or process the first wrist-flip event.

[0185] For performance reasons, when the first wrist-flipping event is monitored, the first wrist-flipping event will be processed only when there is no second wrist-flipping event that is executing the code recognition process. Otherwise, the first event will be directly ignored, which will greatly reduce the success rate of wrist-flipping code recognition. This is because if the first wrist-flipping code recognition in the wrist-flipping code recognition process is unsuccessful, the user may frequently perform wrist-flipping operations in an attempt to check the screen display content and trigger wrist-flipping code recognition again. However, the wrist-flipping algorithm usually cannot distinguish whether the user flips the wrist outward to trigger code recognition or flips it back inward to check the screen display content to view the scan result. This will lead to but not limited to the following situation: the first time the user triggers the wrist-flipping operation, the terminal device executes the code recognition process for the first wrist-flipping operation, but the code recognition processing of the first wrist-flipping operation is unsuccessful, and the user flips the terminal device back to view the scan result, which triggers the second wrist-flipping operation (the wrist-flipping operation to flip back to view the result), and the code recognition process of the second wrist-flipping operation begins to execute. After checking the scanning result and finding that the scanning is not successful, the user will then perform another wrist flip operation to try to trigger another wrist flip scanning. This will trigger the third wrist flip operation (flipping the wrist outward to trigger code recognition). However, when the third wrist flip operation is triggered, the code recognition process of the second wrist flip operation is still being executed. Therefore, the third wrist flip operation will be ignored and will not take effect. However, in fact, the code recognition process should be executed for the third wrist flip operation. Therefore, the current implementation method greatly reduces the success rate of wrist flip code recognition.

[0186] That is to say, when the first wrist-flip event is monitored, the first wrist-flip event will be processed only when there is no second wrist-flip event in the code recognition process. Otherwise, the processing strategy of directly ignoring the first event will reduce the success rate of wrist-flip code recognition. In the technical solution of the present application, the processing strategy for the first wrist-flip event is determined according to the execution result of the second wrist-flip event in the code recognition process, wherein the processing strategy includes ignoring the first wrist-flip event and processing the first wrist-flip event to decide whether to choose the first wrist-flip event, thereby effectively improving the success rate of wrist-flip code recognition.

[0187] The two-dimensional code recognition method provided by the embodiment of the present application includes: if the terminal device monitors the first wrist-flipping event, it is determined whether there is a code recognition process for the second wrist-flipping event in progress. If there is a code recognition process for the second wrist-flipping event in progress, the execution result of the code recognition process for the second wrist-flipping event is determined. According to the execution result, the processing strategy for the first wrist-flipping event is determined, and the processing strategy includes at least one of the following: ignoring the first wrist-flipping event, terminating the code recognition process for the second wrist-flipping event and executing the code recognition process for the first wrist-flipping event, determining that the code recognition process for the second wrist-flipping event is terminated and executing the code recognition process for the first wrist-flipping event. Among them, the code recognition process includes: calling the front camera of the terminal device to obtain an image and identifying the two-dimensional code information in the image. When the first wrist-flipping event is monitored, it can be first determined whether there is a code recognition process for the second wrist-flipping event. If not, the code recognition process can be directly executed for the first wrist-flipping event to call the front camera to obtain an image and identify the two-dimensional code information in the image, thereby improving the recognition efficiency of the two-dimensional code. If it exists, the processing strategy for the first wrist-flipping event is determined according to the execution result of the code recognition process of the second wrist-flipping event, wherein the processing strategy can indicate to ignore the first wrist-flipping event or to process the first wrist-flipping event, thereby effectively improving the success rate of wrist-flipping code recognition, and when processing the first wrist-flipping event, the above-mentioned technical effect of improving the recognition efficiency of the QR code can also be achieved. Therefore, in the technical solution of the present application, on the one hand, the recognition efficiency of the QR code can be improved by the processing method of wrist-flipping code recognition, and on the other hand, the success rate of wrist-flipping code recognition can be effectively improved.

[0188] Based on the above introduction, the specific implementation of determining the execution result of the second wrist flip event code recognition process is further introduced in detail below.

[0189] In the present application, the code recognition process of the wrist flip event may include a first stage and a second stage. In the first stage, the terminal device may call the front camera to acquire an image and identify whether there is a QR code in the acquired image. The second stage will be entered only when it is determined that there is a QR code in the acquired image. In the second stage, the QR code in the image may be decoded to obtain the code value of the QR code.

[0190] Therefore, when determining the execution result of the code recognition process of the second wrist flip event, it can be divided into the following two situations:

[0191] If the code recognition process of the second wrist flip event is in the first stage, the execution result of the first stage is determined according to the discrimination results corresponding to the acquired images, and the discrimination results are used to indicate whether there is a QR code in the image;

[0192] If the code recognition process of the second wrist flip event is in the second stage, or if the execution result of the first stage is successful, then according to the judgment result that at least one frame of the image contains a QR code, the execution result of the second stage is determined;

[0193] The execution result of the code recognition process of the second wrist-flipping event includes the execution result of the first stage and / or the execution result of the second stage.

[0194] Specifically, when the first wrist-flipping event is monitored, it will be determined whether there is a recognition process for the second wrist-flipping event in progress. If so, the execution result of the recognition process for the second wrist-flipping event will be determined. However, at this time (the moment of starting to determine the execution result), the recognition process for the second wrist-flipping event may be in the first stage or the second stage.

[0195] If the code recognition process of the second wrist flip event is in the first stage at this time, the execution result of the first stage can be determined according to the discrimination results corresponding to the multiple frames of images acquired in the first stage up to the current moment, where the execution result can be a success or failure. In one implementation, in the first stage of the code recognition process, discrimination processing is performed in sequence for each frame of image collected by calling the front camera to obtain the discrimination result corresponding to each frame of image, and the discrimination result is used to indicate whether there is a QR code in the image.

[0196] If the code recognition process of the second wrist-flipping event is in the second stage at this time, or the code recognition process of the second wrist-flipping event is in the first stage when the execution result is determined, but as time goes by, the first stage ends, and the execution result of the first stage is successful execution (which means that there is an image in which the judgment result is that there is a QR code in the image), then the execution result of the second stage can be determined based on at least one frame of the image in which the judgment result is that there is a QR code in the image.

[0197] The following further describes in detail the processing flow of determining the execution result of the first stage and the execution result of the second stage in conjunction with specific embodiments and drawings.

[0198] First combine Figures 6 to 9 The process of determining the execution result of the first stage is introduced. Figure 6 A schematic diagram of a process for determining the execution result of the first stage provided in an embodiment of the present application, Figure 7 The image discrimination result provided in the embodiment of the present application is shown in FIG. Figure 1 , Figure 8 The image discrimination result provided in the embodiment of the present application is shown in FIG. Figure 2 , Fig. 9 The image discrimination result provided in the embodiment of the present application is shown in FIG. Figure 3 .

[0199] In this embodiment, the execution result of the first stage can be determined according to the discrimination results corresponding to the acquired images. Figure 6 As shown, in this embodiment, some conditions can be preset, and then the preset condition that is specifically satisfied by the current situation is determined based on the discrimination results corresponding to the acquired images, and then the corresponding subsequent steps are executed.

[0200] In a possible implementation, if the discrimination results corresponding to the acquired images meet the first condition: the discrimination result of m consecutive frames of images is that there is no QR code in the image, and m is an integer greater than or equal to 1, then the execution result of the first stage is determined to be execution failure, wherein the execution result of the first stage is execution failure, which means that the QR code is not recognized in the first stage. m is a preset value, and the specific implementation can be set according to actual needs.

[0201] For example, taking m=10 (10 consecutive frames without code), refer to Figure 7 Explain the current first condition, such as Figure 7 As shown, assuming that 12 frames of images have been acquired up to the current moment, Figure 7 In the example, a check mark indicates that the image is identified as having a QR code, and a wrong mark indicates that the image is identified as not having a QR code. Figure 7 As shown in the figure, the judgment results of the first to twelfth frames are that there is no QR code in the image, so it can be determined that there are 10 consecutive frames of images with the judgment results that there is no QR code in the image, so it can be determined that the execution result of the first stage is execution failure. After determining that the execution result of the first stage is execution failure, the current code recognition process can be ended without entering the second stage, so the processing efficiency of the code recognition process can be effectively improved.

[0202] In another possible implementation, if the discrimination results corresponding to the acquired images meet the second condition: the discrimination results of at least s frames of the continuous t frames are that the two-dimensional code exists in the image, and t and s are integers greater than or equal to 1 (t is greater than s), then the execution result of the first stage is determined to be successful, wherein the execution result of the first stage is successful, which means that the two-dimensional code is recognized in the first stage. t and s are also preset values, and the specific implementation can be set according to actual needs.

[0203] For example, taking t=3, s=2 as an example (at least 2 of the 3 consecutive frames have codes), refer to Figure 8 Explain the current second condition, such as Figure 8As shown in the figure, assuming that 6 frames of images have been acquired up to the current moment, and the judgment results of the 5th and 6th frames are that there is a QR code in the image, it can be understood that there are 2 frames with codes in the 3 consecutive frames 4 to 6, so it can be determined that the execution result of the first stage is successful. After determining that the execution result of the first stage is successful, the second stage can be entered without continuing to collect images for the first stage, thereby effectively improving the processing efficiency of the code recognition process.

[0204] In another possible implementation, if the discrimination results corresponding to the acquired images do not satisfy the first and second conditions described above, the execution result of the first stage can be determined based on the number of frames of the acquired images and the duration of the first stage.

[0205] Continuing with m=10, t=3, s=2 as an example, refer to Fig. 9 Describe the current situation, such as Fig. 9 As shown, assuming that 12 frames of images have been acquired up to the current moment, and the judgment results of the 1st, 5th, 8th and 11th frames are that there is a QR code in the image, it can be understood that in this case, the judgment results of some image frames are that there is a QR code in the image, but neither the first condition (the judgment results of 10 consecutive frames of images are that there is no QR code) nor the second condition (the judgment results of at least 2 frames out of 3 consecutive frames are that there is a QR code). In this case, the execution result of the first stage can be determined based on the number of frames of the acquired images and the duration of the first stage.

[0206] Specifically, it may be determined whether the number of frames of the acquired image is greater than or equal to a preset number of frames, and whether the duration of the first stage is greater than or equal to a preset duration.

[0207] If the number of frames of the acquired image is greater than or equal to the preset number of frames, or the duration of the first stage is greater than or equal to the preset duration, it means that a sufficient number of images have been collected or the first stage has been executed for a sufficient period of time, but no conclusion has been reached that the QR code has been recognized. Therefore, it can be determined that the execution result of the first stage is a failure.

[0208] If the number of frames of the acquired image is less than the preset number of frames and the duration of the first stage is less than the preset duration, it means that the number of images currently collected and the execution time for the first stage are not sufficient to make a final judgment. Therefore, the front camera of the terminal device can continue to be called to acquire images, and the judgment results are determined for the acquired images. Then the newly acquired images are used as the acquired images. Therefore, the judgment results of the acquired images can be further updated according to the judgment results of the newly determined images, and the above-mentioned operations of determining the execution results of the first stage according to the judgment results corresponding to the acquired images are repeated until the execution result of the first stage is determined to be a success or a failure.

[0209] In this embodiment, by presetting a plurality of preset conditions, it is determined that the discrimination result of the acquired image satisfies which condition specifically, and then the execution result of the first stage is determined. Specifically, in the judgment process, if the first condition is met, it can be determined that no two-dimensional code is recognized in the continuous multiple frames of images, so the execution result of the first stage can be directly determined as execution failure, so as to end the processing of the first stage as soon as possible and improve the efficiency of the code recognition processing. And if the second condition is met, it can be determined that the two-dimensional code is stably recognized in the continuous multiple frames of images, and the execution result of the first stage can be directly determined as execution success this time, and the processing of the first stage can be ended as soon as possible to enter the next stage, so the efficiency of the code recognition processing can also be improved. And, when it is determined according to the discrimination result that there are images containing two-dimensional codes, but the discrimination results of these images do not meet the first and second conditions described above, the judgment is continued according to the number of frames of the acquired images and the duration of the first stage. When the number of frames is large enough or the duration of the first stage is long enough, the execution result of the first stage is directly determined as execution failure, so as to avoid the first stage being unable to end for a long time, resulting in the continuous process of the code recognition processing being too long. Or when the number of frames is insufficient and the duration of the first stage is not long enough, image acquisition and identification can continue to be performed to ensure that the execution result is determined based on a sufficient amount of data, thereby effectively improving the success rate of the determined execution result.

[0210] The above describes the implementation method of determining the execution results of the first stage. Fig.10 and Fig.11 The implementation method for determining the execution result of the second stage is described. Fig.10 A flowchart of determining the execution result of the second stage provided in an embodiment of the present application, Fig.11 The decoding result of the image provided in the embodiment of the present application is shown as follows Figure 1 , Fig.12 The decoding result of the image provided in the embodiment of the present application is shown as follows Figure 2 .

[0211] It is understandable that the second stage of the code recognition process may include the following two situations:

[0212] Scenario 1: The code recognition process is in the second stage when the execution result is determined;

[0213] Scenario 2: When the execution result is determined, the code recognition process is in the first stage, but as time goes by, the first stage ends, and the execution result of the first stage is successful, after which the code recognition process enters the second stage.

[0214] Regardless of the above situation, when the code recognition process is executed to the second stage, it means that the QR code has been confirmed to be recognized in the first stage (otherwise the code recognition process will not enter the second stage). Therefore, in the second stage, the front camera can continue to be called to capture images, and at least one frame of images captured in the second stage can be decoded and processed. After that, the execution result of the second stage is determined based on the decoding result of at least one frame of images obtained in the second stage. The decoding result is used to indicate whether the decoding of the QR code in the image is successful.

[0215] like Fig.10 As shown, in the second stage, at least one frame of image can be acquired through the front camera, and then decoding processing is performed on each acquired frame of image to obtain a decoding result of the image to determine whether the two-dimensional code in the image is successfully decoded.

[0216] Afterwards, if the decoding result corresponding to any frame image in at least one frame image is decoding success, it can be determined that the execution result of the second stage is execution success;

[0217] Alternatively, if the decoding results of the at least one frame of image are all decoding failures, it can be determined that the execution result of the second stage is execution failure.

[0218] In one possible implementation, Fig.10 As shown, each time a frame of image is captured, the captured frame of image is decoded and then the decoding result of the image is obtained. If the decoding result of the image is a successful decoding, the execution result of the second stage is directly determined to be a successful execution, and the corresponding second stage is ended, and the code recognition process of the wrist flip event is also ended accordingly.

[0219] However, if the decoding result of the image is a decoding failure, you can continue to call the front camera to capture images and repeat the decoding operation until the decoding is successful or the second stage times out. If the image is still not decoded successfully when the second stage times out, the execution result of the second stage is determined to be an execution failure.

[0220] The duration for judging whether the second stage has timed out may be determined based on the total duration of the code recognition process and the duration of the first stage, or may be a separate duration requirement set for the second stage, which is not limited in this embodiment.

[0221] For example, you can refer to Fig.11 To understand, such as Fig.11 As shown, assuming that the decoding results of the first and second image frames obtained in the second stage are both decoding failures, and the second stage has not timed out, the third image frame can be obtained, and then the decoding process is performed on the third image frame to confirm that the decoding is successful. Then, the execution result of the second stage can be determined to be successful, and the code recognition process of the current wrist flip event is completed.

[0222] And also refer to Fig.12 To understand, such as Fig.12 As shown, assuming that the decoding results of the 1st to 10th image frames obtained in the second stage are all decoding failures, and after the 10th image frame is decoded, it is confirmed that the second stage has timed out, then it can be determined that the execution result of the second stage is execution failure, and then the code recognition process of the current wrist flip event is also terminated.

[0223] In this embodiment, by calling the front camera to acquire images in the second stage, and then performing decoding processing on the images, as long as the decoding result of one frame of the image is a decoding success, the execution result of the second stage can be confirmed as an execution success, and the second stage can be confirmed to be over, so that the code recognition process can be ended as soon as possible when the code recognition is completed. In addition, only when the decoding results of each frame of the image acquired in the second stage are all decoding failures, the execution result of the second stage is confirmed as an execution failure, so that the accuracy of the execution result of the second stage that is finally confirmed can be effectively improved.

[0224] This time the combination Fig.13 The possible implementation methods of the second stage decoding process are introduced. Fig.13 A flowchart of the decoding process provided in an embodiment of the present application.

[0225] like Fig.13 As shown, when decoding an image, the code area is first detected to confirm the specific area where the QR code is located in the entire image. Then, the adjustment coefficient can be determined according to the proportion of the detected code area in the entire image, and the focal length of the front camera can be adjusted through the ZOOM algorithm based on the adjustment coefficient. The purpose of adjusting the focal length is to make the area where the QR code is located account for a larger proportion in the image captured by the front camera, so as to improve the success rate of QR code parsing.

[0226] In one implementation, because some jitter may occur when the user is using a handheld terminal device to recognize the code, the above-mentioned steps can be performed multiple times in this embodiment: determining the adjustment coefficient based on the proportion of the detected code area in the entire image, and adjusting the focal length of the front camera through the ZOOM algorithm based on the adjustment coefficient, so as to ensure stable acquisition of images in which the area where the QR code is located accounts for a relatively large proportion, thereby improving the success rate of the decoding process.

[0227] After the focal length of the front camera is adjusted, in the subsequent images captured by the front camera, the area where the QR code is located accounts for a relatively large proportion of the entire image, thereby effectively improving the success rate of subsequent decoding processing.

[0228] However, it is understandable that adjusting the focal length of the camera through the ZOOM algorithm requires a certain amount of time for internal and underlying processing, which means that the current image frame and the subsequent image frames may still be based on the camera acquisition before the focal length adjustment.

[0229] In order to improve the decoding success rate when the area where the QR code is located accounts for a small proportion of the entire image, in this embodiment, the number of pixels on the shortest side of the QR code area in the image can be obtained, and then the number of pixels on the shortest side can be compared with a preset threshold.

[0230] If the number of pixels on the shortest side of the QR code area is less than the preset threshold, it means that the area where the QR code is located accounts for a relatively small proportion of the image. Therefore, the super-resolution algorithm can be used to process the QR code area in the image. On the one hand, the super-resolution algorithm can make the image clearer, and on the other hand, it can also enlarge the QR code area, thereby improving the image quality of the QR code area and further improving the success rate of QR code parsing.

[0231] Alternatively, if the number of pixels of the shortest side of the QR code area is greater than or equal to the preset threshold, it can be shown that the area where the QR code is located in the image accounts for a relatively large proportion, so the super-resolution algorithm can be omitted and subsequent processing can be performed directly, which can also ensure the image quality of the QR code area. At the same time, when the area where the QR code is located accounts for a relatively large proportion, there is no need to perform super-resolution algorithm processing, which can achieve the purpose of saving power consumption to a certain extent.

[0232] At the same time, it is understandable that Fig.13 The figure shows the complete recognition process for one frame of image. Then, after the focus adjustment of the subsequent front camera is completed, the same process is performed. Fig.13In the process illustrated in FIG, when it comes to the step of determining whether the number of pixels on the shortest side of the two-dimensional code area is less than the preset threshold, it is determined that the number of pixels on the shortest side of the two-dimensional code area is not less than the preset threshold, so there is no need to perform super-resolution processing.

[0233] Regardless of whether the super-resolution algorithm introduced above is used, or the image is taken with the front camera after adjusting the focal length, a good quality image of the QR code area can be obtained. After that, it can be further determined whether the QR code area meets the correction conditions, where the correction conditions can be, for example, that the ratio of the longest side to the shortest side of the QR code area is greater than a preset ratio.

[0234] If the correction conditions are met, the code correction can be performed on the QR code area, specifically, a perspective transformation is performed. The effect achieved is to correct the tilted QR code area so that the QR code area appears as a rectangle as much as possible. Then, the corrected QR code area is decoded, which can effectively improve the decoding success rate.

[0235] Alternatively, if the correction condition is not met, it means that the perspective relationship of the two-dimensional code area is normal. At this time, the decoding process can be directly performed on the corrected two-dimensional code area to effectively save unnecessary processing and save system resources.

[0236] The above embodiment introduces a possible implementation method for determining the execution result of the first stage and the execution result of the second stage. After determining the execution result of the first stage and the execution result of the second stage, the processing strategy for the first wrist flip event can be determined according to the execution results of these two parts. Fig.14 Introduce the implementation method of determining the processing strategy. Fig.14 A schematic diagram of an implementation of determining a processing strategy based on an execution result provided in an embodiment of the present application.

[0237] like Fig.14 As shown, when determining the processing strategy, the following situations may exist:

[0238] Case 1: If the execution result of the first stage of the code recognition process of the second wrist-flipping event is execution failure, the processing strategy for the first wrist-flipping event is determined as: terminating the code recognition process of the second wrist-flipping event and executing the code recognition process for the first wrist-flipping event.

[0239] Reference Fig.14 The first judgment branch in the process, when the first stage fails, indicates that the QR code is not recognized in the processing of the second wrist flip event, then the code recognition process of the second wrist flip event can be terminated, and then the code recognition process of the first wrist flip event can be started to improve the success rate of wrist flip code recognition.

[0240] However, if the execution result of the first stage of the code recognition process of the second wrist-flipping event is successful, it is necessary to continue to determine the processing strategy for the first wrist-flipping event according to the subsequent execution result of the second stage of the code recognition process of the second wrist-flipping event.

[0241] Reference Fig.14 In the second judgment branch, when determining the processing strategy for the first wrist-flipping event according to the execution result of the second stage of the code recognition process of the second wrist-flipping event, the following situation exists:

[0242] Case 2: If the execution result of the second stage of the code recognition process of the second wrist-flipping event is execution failure, the processing strategy for the first wrist-flipping event is determined as: determining that the code recognition process of the second wrist-flipping event is ended and executing the code recognition process for the first wrist-flipping event.

[0243] When the execution result of the second stage of the code recognition process of the second wrist-flipping event is execution failure, it can be determined that there is no successful decoding in the second stage, and the second stage of the code recognition process of the second wrist-flipping event is automatically ended. It should be noted that the situation here is not the termination of the code recognition process introduced above, but it is determined that the code recognition process of the second wrist-flipping event has naturally ended. Then, the code recognition process can be executed for the first wrist-flipping event accordingly to improve the success rate of wrist-flipping code recognition.

[0244] Alternatively, if the execution result of the second stage of the code recognition process of the second wrist-flipping event is successful, the time interval between the first wrist-flipping event and the second wrist-flipping event is obtained, and then the specific processing strategy is determined by judging the size of the time interval and the preset interval. The time interval in this embodiment refers to the time interval between monitoring two wrist-flipping events.

[0245] Reference Fig.14 The second judgment branch in the above example can include the following situations:

[0246] Case 3: if the time interval is less than or equal to the preset interval, the processing strategy for the first wrist-flipping event is determined as: ignoring the first wrist-flipping event.

[0247] Because the time interval between the first wrist-turning event and the second wrist-turning event is very short, generally, after the code is successfully recognized for one wrist-turning event (the second wrist-turning event in this embodiment), the user turns back the phone to check the result and triggers another wrist-turning event (the first wrist-turning event in this embodiment). Therefore, when the time interval between the two wrist-turning events is relatively small, the first wrist-turning event will be ignored in this embodiment. In this case, the second wrist-turning event itself has been successfully recognized, and ignoring the first wrist-turning event will not affect the recognition success rate, and unnecessary recognition processing can be avoided, saving system resources.

[0248] Case 4: if the time interval is greater than the preset interval, the processing strategy for the first wrist-turning event is determined as: determining that the code recognition process for the second wrist-turning event is completed and executing the code recognition process for the first wrist-turning event.

[0249] If the time interval between the first wrist-flipping event and the second wrist-flipping event is relatively long, it is possible that the user has re-triggered a wrist-flipping event (the first wrist-flipping event in this embodiment) that requires code recognition after successfully recognizing a wrist-flipping event (the second wrist-flipping event in this embodiment). Therefore, when the time interval between the two wrist-flipping events is relatively large, in this embodiment, the code recognition process of the second wrist-flipping event will be determined to be completed and the code recognition process will be executed for the first wrist-flipping event to ensure that the wrist-flipping events that require code recognition are processed as comprehensively as possible.

[0250] In this embodiment, it can be determined based on the recognition results of the first stage and the recognition results of the second stage whether to ignore the first wrist-flip event, terminate the current recognition process of the second wrist-flip event and execute the recognition process of the first wrist-flip event, or confirm that the current recognition process of the second wrist-flip event ends naturally and execute the recognition process of the first wrist-flip event. This can effectively avoid the problem of low wrist-flip recognition success rate caused by directly ignoring the first wrist-flip event when the second wrist-flip event is being executed, and improve the success rate of wrist-flip recognition as much as possible.

[0251] Based on the above introduction, the following can be combined Fig.15 The overall logic of the processing strategy for determining the first wrist flip event in this application is described. Fig.15 A logical diagram of a processing strategy for determining a first wrist flip event provided in an embodiment of the present application.

[0252] like Fig.15 As shown, the method includes:

[0253] First, the first stage of processing is performed on the wrist flip event (that is, the second wrist flip event) to identify whether there is a QR code in the image obtained in the first stage of the wrist flip event. Then, according to the judgment result of the first stage, there may be the following three situations:

[0254] Case 1: Among the consecutive t frames, s frames are identified as having codes.

[0255] In this case, it can be confirmed that the code is recognized in the first stage, which corresponds to the execution result of the first stage described above being successful, and then entering the second stage processing process of the wrist flip event, and then determining the subsequent processing strategy based on the execution result of the second stage.

[0256] Case 2: m frames of images are continuous and without code.

[0257] In this case, it is directly confirmed that the QR code is not recognized in the first stage, which corresponds to the execution result of the first stage described above being execution failure. Therefore, the code recognition process of the current wrist flip event (that is, the second wrist flip event) can be directly terminated, and then the code recognition process of the next wrist flip event (that is, the first wrist flip event) can be entered.

[0258] Case 3: intermittent coding but does not meet Case 1 and Case 2, and the number of frames of the image acquired in the first stage is greater than or equal to the preset number of frames or the duration of the first stage is greater than or equal to the preset duration.

[0259] In this case, it can be confirmed that enough images have been acquired in the first stage or the process lasted for a sufficient period of time, but no QR code was recognized, which means that the execution result of the first stage described above is execution failure. Therefore, the code recognition process of the current wrist flip event (that is, the second wrist flip event) is directly terminated, and then the code recognition process of the next wrist flip event (that is, the first wrist flip event) is entered.

[0260] Then the subsequent process of situation 1 is explained. Situation 1 corresponds to the successful execution of the first stage, and then the second stage is executed for the wrist flip event. In the second stage, the code value of the QR code is identified and decoded.

[0261] If the QR code parsing for a certain frame image is unsuccessful in the second stage, it can be determined whether the second stage has timed out. If not, the image can continue to be acquired and the QR code parsing can be performed. If it has timed out, the execution result of the second stage can be directly determined to be an execution failure. Then, it can be determined that the code recognition process of the current wrist flip event has naturally ended and the code recognition process of the next wrist flip event can be entered.

[0262] If the QR code parsing of a certain frame image is successful in the second stage, the execution result of the second stage can be directly determined to be successful. Further, it can be determined whether the time interval between the current wrist flip event and the next wrist flip event is less than or equal to the preset interval. If it is less than or equal to the preset interval, the next wrist flip event is ignored. If it is greater than the preset interval, it is determined that the code recognition process of the current wrist flip event is naturally ended, and the code recognition process of the next wrist flip event is entered.

[0263] In view of the contents of the above embodiments, the following describes the software unit of the terminal device from the perspective of Fig.16 The specific interaction mode between the various software modules in the terminal device when implementing the technical solution of this application is introduced. Fig.16 An interactive flow chart of the QR code recognition method provided in an embodiment of the present application.

[0264] like Fig.16 As shown, the method includes:

[0265] 1. The system service module sends a registration request to the perception module.

[0266] Among them, the registration request is used to request the perception module to register the code scanning fence.

[0267] In a possible implementation, when the terminal device is turned on, the system service module sends a registration request to the perception module to request the perception module to register the code scanning fence, where the code scanning fence can be understood as a processing module for processing code scanning related matters, such as detecting the device screen light status, unlock status, monitoring wrist turn events, etc. Then the subsequent perception module can realize operations such as detecting screen light unlocking, monitoring wrist turn events, etc.

[0268] Therefore, in the technical solution of this embodiment, it is only necessary to register once when the device is turned on, and then the perception module can perform relevant detection and monitoring, thereby avoiding the problems of functional instability and high power consumption caused by frequent registration and deregistration of the code recognition APP (Application).

[0269] 2. The perception module responds to the registration request and registers the scanned fence.

[0270] 3. When the device screen is on and unlocked, the sensing module monitors the wrist flip event.

[0271] Among them, the related operations in step 3 can be performed by the code scanning fence introduced above.

[0272] 4. The perception module sends a notification message of the wrist-turning event to the main control module in the AO service module.

[0273] When the perception module detects a wrist-flipping event, it needs to inform the AO module, and then the AO module can execute the code recognition process for the detected wrist-flipping event. Therefore, the perception module can send a notification message to the main control module in the AO service module, and the notification message is used to inform the main control module that a wrist-flipping event has been detected. The AO service can be understood as a service module resident in the system, which is used to implement some operations that need to be implemented frequently. In this embodiment, the AO module, AO service module, and AO service are all the same concept.

[0274] 5. The perception module sends an instruction to start the APP to the system service module to start the code recognition APP.

[0275] 6. The system service module sends a startup instruction to the code recognition APP to start the code recognition APP.

[0276] Furthermore, after monitoring the wrist-turning event, the perception module can also launch the code recognition APP at the application layer, so as to subsequently pass the QR code recognition result to the code recognition APP.

[0277] For example, the perception module can call back the code recognition service in the system service module by scanning the code fence, thereby starting the code recognition APP. Figure 7 As shown in the figure, the perception module first sends a command to the system service module to start the code recognition APP, which can be understood as the perception module notifying the system service module that the code recognition APP needs to be launched by calling back the code recognition service in the system service module. Then the system service module sends a startup command to the code recognition APP to start the code recognition APP.

[0278] Figure 7 The situation shown in the figure is that the system service module acts as an intermediate transmission module between the code recognition APP and the perception module to realize the startup of the code recognition APP. Alternatively, the perception module can also directly send a startup instruction to the code recognition APP to launch the code recognition APP, but because the system service module has a higher priority, the efficiency of starting the code recognition APP can be improved to a certain extent by intermediate transmission through the system service module.

[0279] It should also be noted that the access direction of the perception module to the code recognition APP can be directly accessed through the interface, that is, access can be achieved without registration. However, the access direction of the code recognition APP to the perception module must be achieved by establishing a link through a registration request.

[0280] 7. The main control module in the AO service module detects whether there is a wrist flip event code recognition process being executed.

[0281] After receiving the notification message of the wrist flip event, the main control module first needs to detect whether there is a wrist flip event code recognition process currently being executed, and then determine how to handle the newly monitored wrist flip event. The code recognition process in this embodiment includes: calling the front camera of the terminal device to obtain an image and identifying the QR code information in the image.

[0282] If there is a wrist-flip event while the code recognition process is being executed, the wrist-flip event indicated by the newly received notification message can be recorded as the next wrist-flip event (also called the first wrist-flip event), and the wrist-flip event while the code recognition process is being executed is the current wrist-flip event (also called the second wrist-flip event). The processing strategy for the next wrist-flip event can then be determined based on the execution result of the current wrist-flip event.

[0283] 8. The main control module in the AO service module can pull up the CA module in the AO service module.

[0284] If there is no wrist-turning event in the code recognition process, the newly monitored wrist-turning event can be directly processed.

[0285] For example, when the AO module acquires the collected image, it also acquires the image based on a certain module transmission link. Therefore, in order to ensure the security of image transmission, during the transmission process, the image encryption and decryption processing can be implemented through the CA module in the AO module and the TA module in the TEE environment.

[0286] In order to save power consumption, the CA module and the TA module are usually set not to be resident, so the main control module can pull up the CA module to perform subsequent image encryption and decryption operations.

[0287] 9. The CA module pulls up the TA module in the TEE environment.

[0288] After the CA module is pulled up, the CA module can continue to pull up the TA module in the TEE environment to perform subsequent image encryption and decryption operations.

[0289] 10. The CA module generates keys.

[0290] 11. The CA module sends the key to the TA module in the TEE environment.

[0291] In order to facilitate efficient and effective encryption and decryption processing in the current code recognition process, the CA module can generate a key when the code recognition process just starts, and then send the generated key to the TA module to synchronize the key between the CA module and the TA module. Thereafter, in the code recognition process of this wrist flip event, the encryption and decryption processing of each frame of the collected image can use the currently synchronized key.

[0292] Therefore, when the code recognition process just starts to be executed, the key synchronization is performed, which can effectively save system resources on the basis of ensuring the successful execution of image encryption and decryption.

[0293] The key generated by the CA module may be a random number, or may be any other form of information as long as it can be used for encryption.

[0294] 12. The main control module sends a timing request to the timing module.

[0295] 13. The timing module is used to time the wrist turning event.

[0296] The timing request is used to request the timing module to time the wrist flip event.

[0297] If the result of the judgment in step 7 is that there is no wrist-turning event currently executing the code recognition process, then there is only the newly monitored wrist-turning event, so it is only necessary to time the newly monitored wrist-turning event. For example, the execution time of the code recognition process of the wrist-turning event can be timed (the code recognition process starts after step 7), and the timing module can also record the time when the wrist-turning event is monitored.

[0298] Alternatively, if the judgment result of the above step 7 is that there is a wrist-flip event that is currently executing the code recognition process, then there is a current wrist-flip event and a next wrist-flip event. In this case, the main control module can also send a timing request to the timing module, and the timing request includes the time when the next wrist-flip event is monitored. The timing module responds to the timing request and records the time when the next wrist-flip event is monitored.

[0299] Exemplarily, the moment when the wrist-flip event is detected can be carried in the notification message of the wrist-flip event, or the moment when the main control module receives the notification message can also be confirmed as the moment when the wrist-flip event is detected. The main control module then carries the moment when the wrist-flip event is detected in the timing request to inform the timing module.

[0300] 14. The main control module sends a notification message to the camera control module to start the camera.

[0301] 15. The camera control module sends a command to start the camera to the camera driver module.

[0302] In the code recognition process of the wrist flip event, it is necessary to call the front camera of the terminal device to capture images. Therefore, after confirming the start of the code recognition process, the main control module can send a notification message to the camera control module to start the camera. After receiving the notification message, the camera control module sends a command to the camera driver module to start the camera.

[0303] 16. The camera driver module drives the front camera of the terminal device to capture images.

[0304] The camera driving module responds to the start-up instruction to drive the front camera of the terminal device to capture images. The specific implementation can refer to the introduction of the above embodiment and will not be repeated here.

[0305] In one implementation, the timing relationship of the three operations introduced in this embodiment, namely, pulling up the CA and TA synchronization keys (steps 8 to 11), timing (steps 12 and 13), and starting the camera (steps 14 to 16), is not limited to the order described above. The timing relationship of these three operations can be arbitrarily changed according to actual needs, and even these three operations can be executed in parallel. Therefore, this embodiment does not limit the specific execution order of these three operations.

[0306] 17. The camera driver module sends the image address to the CA module.

[0307] For example, after the front camera captures an image, the image is stored in a corresponding location, such as in the shared buffer described above. The camera driver module can then send an image address to the CA module, and the image address is used to indicate the storage location of the captured image.

[0308] In a possible implementation, after the camera in the hardware layer captures an image and stores it in a secure buffer, the camera in the hardware layer can first inform the camera driver in the kernel layer of the image address, and then pass the image address to the camera control module in the AO module through the link of the camera driver in the kernel layer, cameraHAL in the HAL layer, and camera service in the framework layer, and then the camera driver module sends the image address to the CA module.

[0309] 18. The CA module sends the image address and image size to the TA module.

[0310] In order for the TA module to acquire the image, the CA module can send the image address to the TA module. The CA module also sends the image size to the TA module, where the image size can be preset. During the wrist flip code recognition process, the image size captured by the front camera is the preset size.

[0311] The image size is sent to the TA module so that the TA module reserves sufficient storage space and writes the image acquired later into the storage space of the TA module to avoid the problem of being unable to obtain image data after the data in the security cache area is overwritten.

[0312] 19. The TA module obtains the image based on the image address and image size.

[0313] After receiving the image address and image size, the TA module can obtain the image captured by the front camera according to the image address and image size. After obtaining, the image can be stored in the storage space of the TA module, for example.

[0314] 20. The TA module encrypts the image according to the key and stores the encrypted image in the shared cache area of ​​the CA and TA.

[0315] Because the CA module and the TA module have synchronized the key in advance, the TA module can encrypt the image according to the key and store the encrypted image in the shared buffer area.

[0316] Exemplarily, the TA module can directly encrypt according to the key generated by the CA module. Alternatively, the TA module can also generate a further advanced key according to the key generated by the CA module and the TA's uuid (Universally Unique Identifier), and then use the advanced key to encrypt the image.

[0317] 21. The TA module sends an indication message to the CA module.

[0318] The indication message is used to inform the CA module that the encrypted image has been stored in the shared cache area.

[0319] 22. The CA module obtains the encrypted image from the shared buffer and decrypts it using the key to obtain the decrypted image.

[0320] After receiving the indication message, the CA module can obtain the encrypted image from the shared buffer area and decrypt it using the key to obtain the decrypted image.

[0321] Exemplarily, the CA module can directly decrypt according to the key. Alternatively, the CA module can also generate a further advanced key according to the key and the uuid of the TA, and then use the advanced key to decrypt the image.

[0322] The specific encryption and decryption method can be determined according to actual needs, as long as the TA module and the CA module use the same method.

[0323] 23. The CA module sends the decrypted image to the main control module.

[0324] 24. The main control module identifies whether there is a QR code in the image.

[0325] In this application, the code recognition process of the wrist flip event may include a first stage and a second stage. In the first stage, the terminal device may call the front camera to acquire an image and identify whether there is a QR code in the acquired image. The second stage will be entered only when it is determined that there is a QR code in the acquired image. In the second stage, the QR code in the image may be decoded to obtain the information of the QR code, which can also be understood as a code value.

[0326] Therefore, the main control module can first perform the first stage of processing based on the decrypted image to identify whether there is a QR code in the image. It should be noted here that the current step is not a simple determination of whether there is a QR code in a frame of image. In the first stage of the code recognition process, the front camera will continue to capture images. The current situation is to determine whether there is a QR code based on multiple frames of images captured in the first stage.

[0327] 25. If there is no QR code in the image, the main control module terminates the code recognition process of the wrist flip event.

[0328] 26. The main control module controls the code recognition process for the next wrist flip event.

[0329] If there is no QR code in the image, it means that no QR code is recognized in the first stage of the code recognition process, so the code recognition process of the wrist flip event can be directly terminated. If there is a next wrist flip event, the code recognition process of the next wrist flip event will be entered, that is, the operation starting from step 8 above will be executed again.

[0330] Alternatively, if there is no next wrist-turning event, the code recognition process of the current wrist-turning event is terminated, and the wrist-turning event can be continuously monitored.

[0331] 27. The main control module recognizes the code value of the QR code.

[0332] If a QR code is identified in the image in the first stage, the process proceeds to the second stage, which is to identify the code value of the QR code.

[0333] 28. If the code value recognition of the QR code fails, the main control module can obtain the timing information of the current wrist flip event from the timing module.

[0334] In the current step, the timing information of the current wrist-flipping event may be the execution duration of the current wrist-flipping event.

[0335] 29. The main control module determines whether the code recognition process of the current wrist flip event has timed out based on the timing information of the current wrist flip event.

[0336] If the QR code parsing for a certain frame image is unsuccessful in the second stage, that is, the code value recognition fails, then you can first determine whether the code recognition process of the current wrist flip event has timed out based on the timing information of the current wrist flip event. If it has not timed out, you can continue to acquire the image and parse the QR code. If it has timed out, you can directly determine that the execution result of the second stage is an execution failure, and then you can determine that the code recognition process of the current wrist flip event has naturally ended, and enter the code recognition process of the next wrist flip event.

[0337] 30. If the timeout period is exceeded, then when there is a next wrist flip event, the main control module enters the code recognition process for the next wrist flip event.

[0338] When the code recognition process of the current wrist-flipping event times out, it means that the code recognition process of the current wrist-flipping event is finished, and thus the code recognition process of the next wrist-flipping event can be entered. Specifically, for the next wrist-flipping event, the operations starting from step 8 above will be re-executed.

[0339] Alternatively, when the code recognition process of the current wrist-turning event times out, if there is no next wrist-turning event, the code recognition process of the current wrist-turning event is terminated, and then the wrist-turning event is continuously monitored.

[0340] 31. If the code value of the QR code is successfully recognized, the main control module can obtain the timing information of the current wrist flip event and the timing information of the next wrist flip event from the timing module.

[0341] 32. The main control module determines whether the time interval between the current wrist flip event and the next wrist flip event is less than a preset interval.

[0342] If the QR code parsing for a certain frame image is successful in the second stage, that is, the code value recognition is successful, then the execution result of the second stage can be directly determined as successful. Further, it can be determined whether the time interval between the current wrist flip event and the next wrist flip event is less than or equal to a preset threshold. The time interval in this embodiment refers to the time interval between two wrist flip events monitored.

[0343] Therefore, the main control module can obtain the timing information of the current wrist-flipping event and the timing information of the next wrist-flipping event from the timing module. In the current step, the timing information of the current wrist-flipping event can be the moment when the current wrist-flipping event is monitored, and the timing information of the next wrist-flipping event can be the moment when the next wrist-flipping event is monitored, which are all recorded in the timing module.

[0344] 33. If it is less than the preset interval, when there is a next wrist flip event, the main control module discards the next wrist flip event.

[0345] If the time interval between the current wrist-flip event and the next wrist-flip event is less than the preset interval, the main control module discards the next wrist-flip event.

[0346] Because the time interval between the first wrist-turning event (next wrist-turning event) and the second wrist-turning event (current wrist-turning event) is very short, generally, after the code is successfully recognized for a wrist-turning event (the second wrist-turning event in this embodiment), the user turns back the phone to check the result and triggers another wrist-turning event (the first wrist-turning event in this embodiment). Therefore, when the time interval between the two wrist-turning events is relatively small, the first wrist-turning event will be ignored in this embodiment. In this case, the second wrist-turning event itself has been successfully recognized, and ignoring the first wrist-turning event will not affect the recognition success rate, and unnecessary recognition processing can be avoided, saving system resources.

[0347] Alternatively, if there is no next wrist-turning event, after the code value of the QR code is successfully recognized, the wrist-turning event can be continuously monitored.

[0348] 34. If it is greater than or equal to the preset interval, when there is the next wrist flip event, the main control module enters the code recognition process of the next wrist flip event.

[0349] If the time interval is greater than or equal to the preset interval, when there is the next wrist flip event, the main control module determines that the code recognition process of the current wrist flip event is naturally ended, and enters the code recognition process of the next wrist flip event, that is, the operation starting from step 8 above will be executed again.

[0350] If the time interval between the first wrist-flipping event and the second wrist-flipping event is relatively long, it is possible that after the user successfully identifies a wrist-flipping event (the second wrist-flipping event in this embodiment), another wrist-flipping event (the first wrist-flipping event in this embodiment) that requires identification is triggered again. Therefore, when the time interval between the two wrist-flipping events is relatively large, in this embodiment, the identification process of the second wrist-flipping event is determined to be completed and the identification process of the first wrist-flipping event is executed to ensure that the wrist-flipping events that require identification are processed as comprehensively as possible.

[0351] Alternatively, if there is no next wrist-turning event, after the code value of the QR code is successfully recognized, the wrist-turning event can be continuously monitored.

[0352] 35. The main control module sends the QR code recognition result to the perception module.

[0353] After the QR code recognition is successful, the main control module can send the QR code recognition result to the system service module, where the QR code recognition result can be understood as the QR code information introduced above.

[0354] 36. The perception module sends the QR code recognition result to the system service module.

[0355] 37. The system service module sends the QR code recognition result to the code recognition APP.

[0356] After that, the perception module sends the QR code recognition result to the system service module, and the system service module sends the QR code recognition result to the code recognition APP. The code recognition APP can pull up the service interface corresponding to the QR code information based on the QR code recognition result. After that, the code recognition channel between the code recognition APP and the AO service will automatically release resources, for example, the CA module and TA module will be restored to their original state, etc.

[0357] Alternatively, if the code recognition APP does not receive the QR code recognition result after a certain period of time, the code recognition channel will also be released.

[0358] In this embodiment, the perception module is responsible for detecting the screen unlocking and monitoring the wrist turning event, which can effectively avoid the frequent registration and deregistration of the code recognition APP, thereby improving the functional stability and reducing power consumption. And the various modules in the AO module interact, so as to schedule the front camera to collect images and identify the QR code information in the image, so as to save the complicated interaction process between the modules and minimize the unnecessary startup and shutdown of application functions. The purpose of improving functional stability and reducing power consumption can also be achieved. And in this embodiment, when the image is encrypted and decrypted, the TA module does not need to know what business function is currently executed. It only needs the AO service (module) to pass the image address and image size. The encryption function in the TA module will automatically encrypt the data under the corresponding address and then return it to the CA module in the AO service, thereby reducing the data dependence on the AO service to a certain extent, integrating the encryption function under the TEE as highly as possible, reducing the information interaction with the business end, and effectively improving the general ability and generalization ability of the encryption and decryption function, which can adapt to more functions with encryption scenarios in the future.

[0359] In the above Fig.16 Based on this, the following is combined with Fig.17 The specific software architecture of the terminal device is further explained. Fig.17 A schematic diagram of the software architecture of the two-dimensional code recognition method provided in the embodiment of the present application. Each embodiment provided in the present application can be adapted to Fig.17 The software architecture provided.

[0360] like Fig.17 As shown, the terminal device includes REE (Rich Execution Environment) and TEE (Trusted Execution Environment), and a secure buffer and a shared buffer are deployed between the REE environment and the TEE environment, wherein the secure buffer is a buffer that can be accessed by both the REE environment and the TEE environment, thereby enabling data interaction between the REE environment and the TEE environment. The shared buffer is specifically a buffer shared by the CA (Client Application) module in the REE environment and the TA (Trusted Application) module in the TEE environment, which is used for data interaction between the two modules.

[0361] Among them, the application layer, framework layer, HAL (Hardware Abstraction Layer, hardware abstraction) layer, kernel layer and hardware layer are deployed in the REE environment.

[0362] Furthermore, a code recognition APP is deployed in the application layer, wherein the code recognition APP can be used to process part of the code scanning process and flow in.

[0363] System services, camera services, and perception platforms (also called perception modules) are deployed in the framework layer. The perception platform further includes subscription management, code recognition fences, and image data modules. Among them, the camera service is a unit in the framework layer used to handle camera scheduling, the system service is used to transmit data between the code recognition APP and the perception platform, the subscription management in the perception platform is used to detect screen unlocking, monitor wrist flip events, and monitor the management of code events. The code recognition fence in the perception service is used to specifically detect screen unlocking and monitor wrist flip events. The image data module in the perception platform is responsible for image transmission between the AO service and the code recognition APP.

[0364] The camera HAL and AO service are deployed in the HAL layer, and the camera control module, CA module, main control module and timing module are further deployed in the AO service. Among them, the camera HAL is the unit in the HAL layer used to handle camera scheduling. The camera control module in the AO service is used to issue instructions to start the camera, the CA module is used to collaborate with the TA module to realize image encryption and decryption processing, the main control module is used to implement the code recognition process for wrist flip events, and the timing module is used to timely respond to wrist flip events.

[0365] A camera driver is deployed in the kernel layer, wherein the camera driver is a unit in the framework layer for processing camera scheduling.

[0366] The hardware layer deploys cameras and IFE (Image Front End). The camera is used to collect images, and the IFE is responsible for receiving the raw data from the camera and processing it, such as noise reduction and color correction. After that, the IFE can pass the processed image data to subsequent units, such as Fig.15 In a possible implementation, the IFE output may be a YUV image (full channel image), where YUV is a color coding mode, in which Y represents luminance, i.e., grayscale value, and UV represents chrominance and chroma, respectively.

[0367] A trusted camera service is also deployed in the TEE environment. This service can obtain images from the secure cache and pass them to the TA module, which is responsible for image encryption.

[0368] Based on the above embodiments, Fig.18and Fig.19 Taking a specific application scenario as an example, the changes in the interface are explained exemplarily. Fig.18 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 1 , Fig.19 A schematic diagram of an interface of a two-dimensional code recognition method provided in an embodiment of the present application Figure 2 .

[0369] First, let’s introduce Fig.18 ,like Fig.18 As shown in (a), the terminal device is in the desktop state, that is, the terminal device does not run an application in the front end; at this time, the screen direction of the terminal device is upward (upward refers to the positive direction of the z-axis of the world coordinate system). Fig.18 As shown in (b) in the figure, the user holding the terminal device turns his wrist, and the user turns the screen of the terminal device downward (the negative direction of the z-axis of the world coordinate system). Furthermore, according to the method provided in this embodiment, if it is determined that the terminal device is in a bright screen unlocked state, the terminal device is in a desktop state, the proximity light is not blocked (not a necessary condition), the screen direction is not facing upward, there is a wrist turn, the terminal device is within a preset geographic fence (not a necessary condition), and the value represented by the ambient light information is greater than or equal to a second preset threshold value (not a necessary condition), then the front camera is turned on to capture images and recognize the QR code. Then, as Fig.18 As shown in (c), the terminal device will not change Fig.18 The interface (a) in the figure still displays the desktop of the terminal device, but the terminal device uses the front camera of the terminal device to obtain the image in the background and recognizes the QR code information in the image in the background. Then, after the terminal device recognizes the QR code information, Fig.18 As shown in (d) in the figure, the terminal device directly calls up the interface corresponding to the QR code information, for example, the transfer interface of application A is displayed.

[0370] Again Fig.19 To introduce, such as Fig.19 As shown in (a), the terminal device is in the desktop state, that is, the terminal device does not run an application in the front end; at this time, the screen direction of the terminal device is upward (upward refers to the positive direction of the z-axis of the world coordinate system). Fig.19As shown in (b), the user holding the terminal device turns his wrist, and the user turns the screen of the terminal device downward (the negative direction of the z-axis of the world coordinate system). Furthermore, according to the method provided in this embodiment, if it is determined that the terminal device is in a bright screen unlocked state, the terminal device is in a desktop state, the proximity light is not blocked (not a necessary condition), the screen direction is not facing upward, there is a wrist turn, the terminal device is within a preset geographic fence (not a necessary condition), and the value of the ambient light information is greater than or equal to a second preset threshold (not a necessary condition), then the front camera is turned on to capture images and recognize the QR code. Fig.19 As shown in (c) of the figure, the terminal device calls the front camera of the terminal device to obtain an image and recognize the QR code information in the image. The terminal device can display a code scanning interface. Then, after the terminal device recognizes the QR code information, Fig.19 As shown in (d) in the figure, the terminal device directly calls up the interface corresponding to the QR code information, for example, the transfer interface of application A is displayed.

[0371] Fig.18 and Fig.19 The situations described above are all triggered when the terminal device is in desktop state. However, based on the above description, it can be determined that the terminal device being in desktop state is optional, that is, the wrist flip code recognition can also be triggered when the terminal device is not in desktop state. For example, Fig.18 and Fig.19 In the example, if a video playback application is running in the foreground, the figure (a) corresponds to the video playback interface, which can also trigger subsequent related operations. Its implementation method is similar to that described above and will not be repeated here.

[0372] Therefore, in the technical solution of the present application, it is possible to directly determine to call the front camera to collect images and identify the QR code information according to the operating status and posture information of the terminal device. It is no longer necessary for the user to perform a series of touch operations on the terminal device to trigger the terminal device to scan the QR code. This saves the time of identifying the QR code and improves the efficiency of QR code recognition.

[0373] The two-dimensional code recognition method of the embodiment of the present application has been described above, and the device for executing the two-dimensional code recognition method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the two-dimensional code recognition method.

[0374] Fig. 20 This is a schematic diagram of the structure of the two-dimensional code recognition device provided in the embodiment of the present application. Fig. 20 As shown, the device 200 includes: a determining unit 2001.

[0375] The determination unit 2001 is used to determine whether a code recognition process of a second wrist-turning event is in progress if the terminal device monitors the first wrist-turning event;

[0376] The determining unit 2001 is further configured to determine the execution result of the code recognition process of the second wrist-turning event if the code recognition process of the second wrist-turning event is in progress;

[0377] The determining unit 2001 is further used to determine a processing strategy for the first wrist-flip event according to the execution result, wherein the processing strategy includes at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is terminated and executing the code recognition process for the first wrist-flip event;

[0378] The code recognition process includes: calling the front camera of the terminal device to obtain an image, and identifying the QR code information in the image.

[0379] In some implementations, the code recognition process includes a first stage and a second stage, in which an image is acquired and whether a two-dimensional code exists in the image is identified, and in which the two-dimensional code in the image is decoded;

[0380] The determining unit 2001 is specifically used for:

[0381] If the code recognition process of the second wrist-flipping event is in the first stage, the execution result of the first stage is determined according to the judgment results corresponding to the acquired images, and the judgment results are used to indicate whether there is a QR code in the image;

[0382] If the code recognition process of the second wrist flip event is in the second stage, or if the execution result of the first stage is successful, determining the execution result of the second stage according to the decoding result of at least one frame of image acquired in the second stage, wherein the decoding result is used to indicate whether the decoding of the two-dimensional code in the image is successful;

[0383] The execution result of the code recognition process of the second wrist-turning event includes the execution result of the first stage and / or the execution result of the second stage.

[0384] In some implementations, the determining unit 2001 is specifically configured to:

[0385] If the discrimination results corresponding to the acquired images meet the first condition, the execution result of the first stage is determined to be execution failure, and the first condition is: the discrimination result of the existence of m consecutive frames of images is that there is no QR code in the image, and m is an integer greater than or equal to 1;

[0386] If the discrimination results corresponding to the acquired images meet the second condition, the execution result of the first stage is determined to be successful, and the second condition is: the discrimination result of at least s frames of images in the continuous t frames of images is that the two-dimensional code exists in the image, and t and s are integers greater than or equal to 1;

[0387] If the identification results corresponding to the acquired images do not satisfy the first condition and the second condition, the execution result of the first stage is determined according to the number of frames of the acquired images and the duration of the first stage.

[0388] In some implementations, the determining unit 2001 is specifically configured to:

[0389] If the number of frames of the acquired image is greater than or equal to the preset number of frames or the duration of the first stage is greater than or equal to the preset duration, determining that the execution result of the first stage is execution failure;

[0390] If the number of frames of the acquired image is less than the preset number of frames and the duration of the first stage is less than the preset duration, continue to call the front camera of the terminal device to acquire the image and determine the image recognition result, and repeat the operation of determining the execution result of the first stage according to the recognition results corresponding to each of the acquired images until the execution result of the first stage is determined to be a success or a failure.

[0391] In some implementations, the determining unit 2001 is specifically configured to:

[0392] In the second stage, at least one frame of image is acquired through the front camera;

[0393] If the decoding result corresponding to any frame of the at least one frame of the image is a decoding success, determining that the execution result of the second stage is an execution success;

[0394] If the decoding results of the at least one frame of image are all decoding failures, it is determined that the execution result of the second stage is execution failure.

[0395] In some implementations, the determining unit 2001 is specifically configured to:

[0396] If the execution result of the first stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as: terminating the code recognition process of the second wrist-flipping event and executing the code recognition process for the first wrist-flipping event;

[0397] If the execution result of the first stage of the code recognition process of the second wrist-flipping event is successful, a processing strategy for the first wrist-flipping event is determined according to the execution result of the second stage of the code recognition process of the second wrist-flipping event.

[0398] In some implementations, the determining unit 2001 is specifically configured to:

[0399] If the execution result of the second stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as follows: determining that the code recognition process of the second wrist-flipping event ends and executing the code recognition process for the first wrist-flipping event;

[0400] If the execution result of the second stage of the code recognition process of the second wrist-flipping event is successful, obtaining the time interval between the first wrist-flipping event and the second wrist-flipping event;

[0401] If the time interval is less than or equal to the preset interval, determining that the processing strategy for the first wrist-flipping event is: ignoring the first wrist-flipping event;

[0402] If the time interval is greater than the preset interval, the processing strategy for the first wrist-flipping event is determined as: determining that the code recognition process for the second wrist-flipping event is finished and executing the code recognition process for the first wrist-flipping event.

[0403] In some implementations, the determining unit 2001 is specifically configured to:

[0404] Access process execution flag;

[0405] If the process execution flag is the first value, it is determined that the code recognition process of the second wrist flip event is being executed; or,

[0406] If the process execution flag is the second value, it is determined that there is no code recognition process of the second wrist turning event being executed.

[0407] In some implementations, the determining unit 2001 is specifically configured to:

[0408] If it is determined that the value represented by the posture change information is greater than or equal to a first preset threshold and the screen direction is a direction other than the first direction, it is determined that a first wrist flip event is monitored, wherein the first direction is the positive direction of the z-axis of the world coordinate system.

[0409] The two-dimensional code recognition method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal device can refer to the above related descriptions, which will not be repeated here.

[0410] An embodiment of the present application provides a terminal device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0411] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0412] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0413] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0414] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0415] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0416] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the protection scope of the present invention.

Claims

1. A two-dimensional code recognition method, characterized in that: include: If the terminal device monitors the first wrist-turning event, a code recognition process for determining whether there is a second wrist-turning event is in progress; If there is a code recognition process for the second wrist-turning event in progress, determining the execution result of the code recognition process for the second wrist-turning event; Determine, according to the execution result, a processing strategy for the first wrist-flip event, the processing strategy comprising at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is terminated and executing the code recognition process for the first wrist-flip event; The code recognition process includes: calling the front camera of the terminal device to obtain an image, and identifying the QR code information in the image.

2. The method according to claim 1, characterized in that The code recognition process includes a first stage and a second stage, in which an image is acquired and a QR code is identified in the image, and in the second stage, the QR code in the image is decoded; The execution result of the code recognition process for determining the second wrist turning event includes: If the code recognition process of the second wrist-flipping event is in the first stage, the execution result of the first stage is determined according to the judgment results corresponding to the acquired images, and the judgment results are used to indicate whether there is a QR code in the image; If the code recognition process of the second wrist flip event is in the second stage, or if the execution result of the first stage is successful, determining the execution result of the second stage according to the decoding result of at least one frame of image acquired in the second stage, wherein the decoding result is used to indicate whether the decoding of the two-dimensional code in the image is successful; The execution result of the code recognition process of the second wrist-turning event includes the execution result of the first stage and / or the execution result of the second stage.

3. The method according to claim 2, characterized in that Determining the execution result of the first stage according to the discrimination results corresponding to the acquired images includes: If the discrimination results corresponding to the acquired images meet the first condition, the execution result of the first stage is determined to be execution failure, and the first condition is: the discrimination result of the existence of m consecutive frames of images is that there is no QR code in the image, and m is an integer greater than or equal to 1; If the discrimination results corresponding to the acquired images meet the second condition, the execution result of the first stage is determined to be successful, and the second condition is: the discrimination result of at least s frames of images in the continuous t frames of images is that the two-dimensional code exists in the image, and t and s are integers greater than or equal to 1; If the identification results corresponding to the acquired images do not satisfy the first condition and the second condition, the execution result of the first stage is determined according to the number of frames of the acquired images and the duration of the first stage.

4. The method according to claim 3, characterized in that The step of determining the execution result of the first stage according to the number of frames of the acquired image and the duration of the first stage includes: If the number of frames of the acquired image is greater than or equal to the preset number of frames or the duration of the first stage is greater than or equal to the preset duration, determining that the execution result of the first stage is execution failure; If the number of frames of the acquired image is less than the preset number of frames and the duration of the first stage is less than the preset duration, continue to call the front camera of the terminal device to acquire the image and determine the image recognition result, and repeat the operation of determining the execution result of the first stage according to the recognition results corresponding to each of the acquired images until the execution result of the first stage is determined to be a success or a failure.

5. The method according to claim 2, characterized in that: The step of determining the execution result of the second stage according to the decoding result of at least one frame of image acquired in the second stage includes: In the second stage, at least one frame of image is acquired through the front camera; If the decoding result corresponding to any frame of the at least one frame of the image is a decoding success, determining that the execution result of the second stage is an execution success; If the decoding results of the at least one frame of image are all decoding failures, it is determined that the execution result of the second stage is execution failure.

6. The method according to any one of claims 1 to 5, characterized in that: The step of determining a processing strategy for the first wrist-flip event according to the execution result includes: If the execution result of the first stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as: terminating the code recognition process of the second wrist-flipping event and executing the code recognition process for the first wrist-flipping event; If the execution result of the first stage of the code recognition process of the second wrist-flipping event is successful, a processing strategy for the first wrist-flipping event is determined according to the execution result of the second stage of the code recognition process of the second wrist-flipping event.

7. The method according to claim 6, characterized in that The step of determining a processing strategy for the first wrist-flipping event according to the execution result of the second stage of the code recognition process for the second wrist-flipping event includes: If the execution result of the second stage of the code recognition process of the second wrist-flipping event is execution failure, determining a processing strategy for the first wrist-flipping event as follows: determining that the code recognition process of the second wrist-flipping event ends and executing the code recognition process for the first wrist-flipping event; If the execution result of the second stage of the code recognition process of the second wrist-flipping event is successful, obtaining the time interval between the first wrist-flipping event and the second wrist-flipping event; If the time interval is less than or equal to the preset interval, determining that the processing strategy for the first wrist-flipping event is: ignoring the first wrist-flipping event; If the time interval is greater than the preset interval, the processing strategy for the first wrist-flipping event is determined as: determining that the code recognition process for the second wrist-flipping event is finished and executing the code recognition process for the first wrist-flipping event.

8. The method according to any one of claims 1 to 7, characterized in that: The code recognition process for determining whether a second wrist flip event occurs is in progress, including: Access process execution flag; If the process execution flag is the first value, it is determined that the code recognition process of the second wrist flip event is being executed; or, If the process execution flag is the second value, it is determined that there is no code recognition process of the second wrist turning event being executed.

9. The method according to any one of claims 1 to 8, characterized in that: The monitoring of the first wrist-turning event includes: If it is determined that the value represented by the posture change information is greater than or equal to the first preset threshold and the screen direction is a direction other than the first direction, it is determined that a first wrist flip event is monitored, wherein the first direction is the positive direction of the z-axis of the world coordinate system.

10. A two-dimensional code recognition device, characterized in that: include: a determination unit, configured to determine whether a code recognition process for a second wrist-turning event is in progress if the terminal device monitors the first wrist-turning event; The determining unit is further configured to determine the execution result of the code recognition process for the second wrist-flipping event if the code recognition process for the second wrist-flipping event is in progress; The determining unit is further used to determine a processing strategy for the first wrist-flip event according to the execution result, wherein the processing strategy includes at least one of the following: ignoring the first wrist-flip event, terminating the code recognition process of the second wrist-flip event and executing the code recognition process for the first wrist-flip event, determining that the code recognition process of the second wrist-flip event is terminated and executing the code recognition process for the first wrist-flip event; The code recognition process includes: calling the front camera of the terminal device to obtain an image, and identifying the QR code information in the image.

11. A terminal device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that The method comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 9.

Images