Code scanning method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a code scanning method, an electronic device and a storage medium. In the method, the electronic device acquires a first callback image frame in response to an opening operation of an application code scanning function, performs distortion correction on a first two-dimensional code image in the image frame to obtain a second two-dimensional code image, regenerates a second callback image frame based on the second two-dimensional code image, so that the position of the second two-dimensional code image in the second callback image frame is consistent with the position of the first two-dimensional code image in the first callback image frame, sends the second callback image frame to the application for identification, and displays a positioning mark on a preview image when the application identifies a two-dimensional code based on the second callback image frame, the position of the positioning mark being determined based on the position of the second two-dimensional code image in the second callback image frame. In this way, the problem of unsuccessful or even failed code scanning for a long time due to poor user scanning angle can be solved, and the problem of improper display position of the two-dimensional code positioning mark on the preview image can be avoided.

Description

[0001] The present application is a divisional application, the original application is named a code scanning method, an electronic device and a storage medium, the original application has an application number 202310207187.1, an original application date is February 27, 2023, and the entire contents of the original application are incorporated by reference into the present application. TECHNICAL FIELD

[0002] The present application relates to the technical field of intelligent terminals, and in particular to a code scanning method, an electronic device and a storage medium. BACKGROUND

[0003] With the development of science and technology, barcodes are ubiquitous in our lives. Barcodes can help us quickly pay, add friends, follow public accounts, and so on. When a user uses a terminal to scan a barcode, due to factors such as the position of the barcode or the scanning angle, the terminal may not be able to recognize the barcode. At this time, the user needs to adjust the scanning angle and keep trying until the terminal can successfully scan the barcode, which brings a poor user experience to the user. SUMMARY

[0004] To solve the above technical problems, the present application provides a code scanning method, an electronic device and a storage medium. In the method, the electronic device performs distortion correction processing on a two-dimensional code in a callback image frame, and regenerates the callback image frame based on the two-dimensional code image after distortion correction, so that the scanning application performs scanning recognition based on the regenerated callback image frame. The position of the two-dimensional code in the regenerated callback image frame is consistent with the position of the two-dimensional code in the original callback image frame. In this way, the code scanning method provided by the present application not only solves the problem of long-time unsuccessful scanning or even failure caused by the poor scanning angle of the user, but also avoids the problem of improper display position of the two-dimensional code positioning mark on the preview image.

[0005] In a first aspect, an embodiment of the present application provides a code scanning method. The method comprises: in response to an opening operation of a target application to a code scanning function, an electronic device acquires a first callback image frame and a first preview image frame; the electronic device extracts a first two-dimensional code image from the first callback image frame; the electronic device performs distortion correction processing on the first two-dimensional code image to obtain a second two-dimensional code image; the electronic device creates a blank image frame, determines the position of the first two-dimensional code image in the first callback image frame, and adds the second two-dimensional code image to the blank image frame to obtain a second callback image frame, so that the position of the second two-dimensional code image in the second callback image frame is consistent with the position of the first two-dimensional code image in the first callback image frame; wherein the blank image frame has the same image size as the first callback image frame; the electronic device sends the second callback image frame to the target application for code scanning recognition; the electronic device displays the first preview image; when the target application recognizes a two-dimensional code based on the second callback image frame, the electronic device displays a two-dimensional code positioning mark on the first preview image; wherein the position of the two-dimensional code positioning mark is determined based on the position of the second two-dimensional code image in the second callback image frame.

[0006] Wherein, the target application refers to a third-party application with a code scanning function, such as WeChat application, Alipay application, etc.

[0007] In the embodiments of the present application, the callback image frame is used for the code scanning application to perform code scanning recognition operation, and the preview image frame is used for the code scanning application to perform image preview operation. Wherein, the image acquisition time of the first callback image frame is close to the image acquisition time of the first preview image frame, and since the callback image frame and the preview image frame correspond to the same image acquisition scene, the picture information in the first callback image frame is almost completely the same as the picture information of the first preview image frame, and the difference is very small. Therefore, the position of the two-dimensional code in the first callback image frame is close to the position of the two-dimensional code in the first preview image frame, such as the position of the two-dimensional code center point, which can also be called that the two-dimensional code region in the first callback image frame and the two-dimensional code region in the first preview image frame overlap greatly, even completely.

[0008] When the target application recognizes a two-dimensional code based on the second callback image frame, the electronic device determines the position of the two-dimensional code in the second callback image frame, such as the coordinates of the two-dimensional code center point in the second callback image frame. Further, the electronic device displays a two-dimensional code positioning mark on the first preview image according to the coordinates, so that the coordinates of the center point of the two-dimensional code positioning mark in the first preview image are the same as the coordinates of the center point of the two-dimensional code in the second callback image frame.

[0009] When the angle between the plane where the two-dimensional code to be scanned and the image capturing plane of the mobile phone is too large (e.g., 45 degrees to 75 degrees), the two-dimensional code in the callback image frame will have an image distortion problem. The electronic device performs distortion correction processing on the two-dimensional code image in the first callback image frame, and regenerates a second callback image frame based on the two-dimensional code image after distortion correction, for the scanning application to recognize. In this way, the positioning and recognition success rate of the scanning application for the two-dimensional code can be improved, and the problem of long-time unsuccessful scanning or even failure due to poor user scanning angle can be solved. In addition, since the shooting scenes corresponding to the first preview image frame and the first callback image frame are consistent, and the interval of the image capturing time is small, the picture information of the first preview image frame and the picture information of the first callback frame are less different, and the two-dimensional code region in the first callback image frame and the two-dimensional code region in the first preview image frame are mostly overlapped, or even completely overlapped. Since the position of the two-dimensional code in the regenerated second callback image frame is consistent with the position of the two-dimensional code with distortion problem in the first callback image frame, the two-dimensional code positioning mark determined by the scanning application based on the regenerated second callback image frame will be displayed in the two-dimensional code region in the first preview image, and the problem of incorrect display position of the two-dimensional code positioning mark in the first preview image can be avoided.

[0010] According to the first aspect, the method further includes: the electronic device acquires a third callback image frame, the third callback image frame being a callback image frame after the first callback image frame; the electronic device sends the third callback image frame to the target application for scanning recognition; the electronic device acquires a fourth callback image frame, the fourth callback image frame being a callback image frame after the third callback image frame; the electronic device extracts a third two-dimensional code image from the fourth callback image frame, performs distortion correction processing on the third two-dimensional code image to obtain a fourth two-dimensional code image; the electronic device creates a blank image frame, determines the position of the third two-dimensional code image in the fourth callback image frame, and adds the fourth two-dimensional code image to the blank image frame to obtain a fifth callback image frame, so that the position of the fourth two-dimensional code image in the fifth callback image frame is consistent with the position of the third two-dimensional code image in the fourth callback image frame; the electronic device sends the fifth callback image frame to the target application for scanning recognition.

[0011] For example, the third callback image frame is the next callback image frame after the first callback image frame, and the fourth callback image frame is the next callback image frame after the third callback image frame.

[0012] In the scan code application sequentially received each callback image frame, the callback image frame after distortion correction processing and the callback image frame without distortion correction processing appear alternately, and the scan code application sequentially performs scan code identification based on each callback image frame according to the receiving order of the callback image frame. In this way, the problem that the scan code application cannot identify the callback image frame after distortion correction processing and the two-dimensional code is not successfully identified for a long time can be avoided. Once the scan code application can identify the callback image frame without distortion correction processing, the two-dimensional code can still be successfully identified, so as to ensure the success rate of the scan code method.

[0013] According to the first aspect or any one of the implementations of the above first aspect, the method further includes: the electronic device acquires a third callback image frame, the third callback image frame being a callback image frame after the first callback image frame; and the electronic device sends the third callback image frame to the target application for scan code identification. The electronic device displays the preview image; and when the target application identifies the two-dimensional code based on the third callback image frame, the electronic device displays a two-dimensional code positioning mark on the corresponding preview image; and the position of the two-dimensional code positioning mark is consistent with the position of the two-dimensional code in the third callback image frame.

[0014] Similarly, when the target application identifies the two-dimensional code based on the fourth callback image frame, a two-dimensional code positioning mark is displayed on the corresponding preview image; and the position of the two-dimensional code positioning mark is consistent with the position of the fourth two-dimensional code image in the fourth callback image frame.

[0015] The scan code application can complete the positioning and identification operation of the two-dimensional code based on the callback image frame N1 after distortion correction processing, and the position of the two-dimensional code positioning mark is determined based on the position of the two-dimensional code in the callback image frame N1. The scan code application can complete the positioning and identification operation of the two-dimensional code based on the callback image frame N2 without distortion correction processing, and the position of the two-dimensional code positioning mark is determined based on the position of the two-dimensional code in the callback image frame N2. After the scan code application completes the positioning and identification operation of the two-dimensional code based on a certain callback image frame, the positioning and identification operation of the two-dimensional code based on the next callback image frame is no longer performed.

[0016] According to the first aspect or any one of the implementations of the above first aspect, the position of the second two-dimensional code image in the second callback image frame is consistent with the position of the first two-dimensional code image in the first callback image frame, including: the coordinates of the center point of the second two-dimensional code image in the second callback image frame are consistent with the coordinates of the center point of the first two-dimensional code image in the first callback image frame.

[0017] In this way, the problem that the position of the two-dimensional code positioning mark displayed in the preview image is incorrect can be avoided.

[0018] According to a first aspect, or any possible implementation mode of the above first aspect, the electronic device performs the distortion correction processing on the first two-dimensional code image in the first callback image frame, including: when the first two-dimensional code image in the first callback image frame does not satisfy the zooming condition, the electronic device performs the distortion correction processing on the first two-dimensional code image in the first callback image frame.

[0019] According to the first aspect, or any possible implementation mode of the above first aspect, when the first two-dimensional code image in the first callback image frame satisfies the zooming condition, the electronic device performs zooming processing on the first callback image frame to enlarge the two-dimensional code image, to obtain a sixth callback image frame; and the electronic device sends the sixth callback image frame to the target application for code scanning recognition.

[0020] In this way, before the electronic device returns the callback image frame of the code scanning application to the code scanning application, when the two-dimensional code image in the callback image frame is small, the electronic device performs zooming processing on the callback image frame to enlarge the two-dimensional code image, so that the callback image frame after zooming processing is more convenient for the code scanning application to recognize, thereby improving the code scanning success rate and code scanning efficiency of the code scanning application on the callback image frame, and improving the user experience.

[0021] According to the first aspect, or any possible implementation mode of the above first aspect, the method further includes: the electronic device displays a second preview image; when the target application recognizes the two-dimensional code based on the sixth callback image frame, the electronic device displays a two-dimensional code positioning mark on the second preview image; and a position of the two-dimensional code positioning mark is consistent with a position of the two-dimensional code image in the sixth callback image frame.

[0022] In this way, when the electronic device is in a state that the two-dimensional code to be scanned occupies a small proportion of the image in the callback image, the electronic device performs zooming processing on the callback image frame to enlarge the two-dimensional code image, for the code scanning application to recognize. At this time, if the code scanning application recognizes the two-dimensional code based on the callback image frame after zooming, the position of the two-dimensional code positioning mark is determined based on the position of the two-dimensional code in the callback image frame, and the two-dimensional code positioning mark is displayed on the preview image according to the position.

[0023] According to the first aspect, or any possible implementation mode of the above first aspect, the distortion correction processing on the first two-dimensional code image includes: when a maximum rotation angle of the first two-dimensional code image is greater than a first threshold, performing the distortion correction processing on the first two-dimensional code image in the first callback image frame according to a perspective transformation. The maximum rotation angle is the maximum value of a first included angle, a second included angle, a third included angle and a fourth included angle, the first included angle is an included angle between a left boundary of the first two-dimensional code image and a vertical direction, the second included angle is an included angle between an upper boundary of the first two-dimensional code image and a horizontal direction, the third included angle is an included angle between a right boundary of the first two-dimensional code image and the vertical direction, and the fourth included angle is an included angle between a lower boundary of the first two-dimensional code image and the horizontal direction.

[0024] For the related explanations of the first angle, the second angle, the third angle, and the fourth angle, refer to the following Figure 7c and related explanations, which are not repeated here.

[0025] According to the first aspect, or any one of the implementations of the first aspect, the zooming condition comprises that the image proportion of the two-dimensional code image in the callback image frame is less than the second threshold value, and the target zooming ratio corresponding to the two-dimensional code is greater than the third threshold value; wherein the target zooming ratio is used to indicate the maximum zooming ratio value allowed when zooming processing is performed on the callback image frame to enlarge the two-dimensional code image.

[0026] For example, the electronic device can calculate the image proportion r0 of the two-dimensional code image in the callback image frame according to the contour of the two-dimensional code image. Wherein r0=S0 / S, S0 is the pixel area of the two-dimensional code image, and S is the pixel area of the callback image frame.

[0027] Wherein, the target zooming ratio is the maximum allowed zooming ratio r1 in the following. The electronic device can calculate the maximum allowed zooming ratio r1 corresponding to the two-dimensional code in the callback image frame according to the two-dimensional code image size information and the two-dimensional code image position information in the callback image frame. For detailed description of the maximum allowed zooming ratio r1, refer to the following Figure 7b and related explanations, which are not repeated here.

[0028] According to the first aspect, or any one of the implementations of the first aspect, the zooming condition comprises that the image proportion of the two-dimensional code image in the callback image frame is less than the second threshold value.

[0029] According to the first aspect, or any one of the implementations of the first aspect, the zooming processing performed by the electronic device on the first callback image frame comprises: the electronic device performs zooming processing on the first callback image frame according to the target zooming ratio.

[0030] In this way, the two-dimensional code image in the callback image frame is enlarged as much as possible under the premise of ensuring the integrity of the two-dimensional code image, so that a certain boundary of the two-dimensional code image is infinitely close to a certain boundary of the callback image frame, or even coincides with this boundary, which is more conducive to positioning and identifying the two-dimensional code by the code scanning application.

[0031] According to a first aspect, or any possible implementation mode of the first aspect, the electronic device performs the distortion correction processing on the first QR code image in the first callback image frame, including: determining, by the electronic device, four vertex coordinates of the first QR code image; determining, by the electronic device, a side length of a second QR code image according to the four vertex coordinates of the first QR code image; determining, by the electronic device, a coordinate of one vertex of the second QR code image, and calculating coordinates of other three vertices of the second QR code image based on the side length of the second QR code image; calculating, by the electronic device, a perspective transformation matrix according to the four vertex coordinates of the first QR code image and the four vertex coordinates of the second QR code image; and performing, by the electronic device, perspective transformation processing on the first QR code image according to the perspective transformation matrix to obtain the second QR code image.

[0032] The coordinate of the first vertex of the second QR code image determined by the electronic device can also be a coordinate of an upper-left vertex. For example, the coordinate of the upper-left vertex can be set in a left upper corner region of the image, so that the four vertices of the second QR code image can all fall within the image region.

[0033] According to the first aspect, or any possible implementation mode of the first aspect, the electronic device determines the side length of the first QR code image according to the four vertex coordinates of the first QR code image, including: calculating, by the electronic device, four side lengths of the first QR code image according to the four vertex coordinates of the first QR code image; and taking, by the electronic device, a maximum side length of the four side lengths of the first QR code image as the side length of the second QR code image.

[0034] In this way, the electronic device takes the maximum side length of the four side lengths of the first QR code image as the side length of the second QR code image, and the image size of the second QR code image obtained after the distortion correction processing on the first QR code image is larger than that when one of the other three side lengths is taken as the side length of the second QR code image, which is beneficial to the recognition of the code scanning application.

[0035] According to a second aspect, the embodiments of the present application provide an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the code scanning method of the first aspect and any one of the first aspect.

[0036] The second aspect and any possible implementation mode of the second aspect correspond to the first aspect and any possible implementation mode of the first aspect respectively. The technical effects corresponding to the second aspect and any possible implementation mode of the second aspect can refer to the technical effects corresponding to the first aspect and any possible implementation mode of the first aspect described above, which will not be described here.

[0037] In a third aspect, an embodiment of the present application provides a computer readable storage medium. The computer readable storage medium includes a computer program, when the computer program is run on an electronic device, causes the electronic device to execute the code scanning method in the first aspect and any one of the implementation manners of the first aspect.

[0038] The third aspect and any one of the implementation manners of the third aspect correspond to the first aspect and any one of the implementation manners of the first aspect respectively. For details, refer to the technical effects of the first aspect and any one of the implementation manners of the first aspect, which will not be repeated here.

[0039] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, when the computer program is run, causes a computer to execute the code scanning method in the first aspect or any one of the implementation manners of the first aspect.

[0040] The fourth aspect and any one of the implementation manners of the fourth aspect correspond to the first aspect and any one of the implementation manners of the first aspect respectively. For details, refer to the technical effects of the first aspect and any one of the implementation manners of the first aspect, which will not be repeated here.

[0041] In a fifth aspect, the present application provides a chip, including processing circuit, transceiver pin. Wherein, the transceiver pin and the processing circuit communicate with each other through internal connection path, the processing circuit executes the code scanning method in the first aspect or any one of the implementation manners of the first aspect, to control the receiving pin to receive the signal, to control the sending pin to send the signal.

[0042] The fifth aspect and any one of the implementation manners of the fifth aspect correspond to the first aspect and any one of the implementation manners of the first aspect respectively. For details, refer to the technical effects of the first aspect and any one of the implementation manners of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 An application scenario is exemplarily shown;

[0044] Figure 2 A large angle code scanning scenario is exemplarily shown;

[0045] Figure 3 A hardware structure schematic diagram of an electronic device is exemplarily shown;

[0046] Figure 4 A software structure schematic diagram of an electronic device is exemplarily shown;

[0047] Figure 5 A flowchart of a scanning code method provided by the embodiment shown as an example is provided.

[0048] Figure 6 A flowchart of processing of a callback image frame is provided as an example.

[0049] Figure 7a An example diagram of a two-dimensional code detection result is provided as an example.

[0050] Figure 7b A diagram of a position of a two-dimensional code image in a callback image frame is provided as an example.

[0051] Figure 7c A diagram of a rotation angle of a two-dimensional code image is provided as an example.

[0052] Figure 7d An example diagram of perspective transformation of a two-dimensional code image is provided as an example.

[0053] Figure 7e Example diagrams of a callback image frame before and after processing are provided as an example.

[0054] Figure 7f A diagram of a scanning code positioning scene is provided as an example.

[0055] Figure 8 A flowchart of a scanning code method provided by the embodiment shown as an example is provided. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0057] The term “and / or” in the present document is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of existence of A alone, existence of A and B at the same time, and existence of B alone.

[0058] The terms “first” and “second” and the like in the specification and claims of the embodiments of the present application are used to distinguish different objects, and are not used to describe a specific order of the objects. For example, the first target object and the second target object are used to distinguish different target objects, and are not used to describe a specific order of the target objects.

[0059] In the embodiments of the present application, the word "exemplary" or "for example" is used to mean serving as an example, instance, or illustration. Any embodiment or design described in the embodiments of the present application as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the word "exemplary" or "for example" is used to present concepts in a particular manner.

[0060] In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise specified. For example, a plurality of processing units means two or more processing units; a plurality of systems means two or more systems.

[0061] With the development of science and technology, barcodes are ubiquitous in our life. Barcodes can help us to pay, add friends, follow public accounts, etc. Exemplarily, barcodes include, but are not limited to, barcodes, two-dimensional codes, etc.

[0062] At present, more and more application programs (APPs) on electronic devices add the function of using a camera, so that the APPs can use the camera to realize the function of scanning a code. Taking the scanning code APP as WeChat and the barcode as a two-dimensional code as an example, Figure 1 An application scenario is exemplarily shown. As shown in (1) of FIG. 1, a user clicks an icon 101 of a WeChat application, a mobile phone detects the user operation, opens the WeChat application, and displays an interface of the WeChat application. The user performs a related operation in the WeChat application, so that the WeChat application displays an interface including a "scan" function, as shown in (2) of FIG. 1. Figure 1 Figure 1 Figure 1 Figure 1 Figure 1 Figure 1 In the process, the WeChat application locates and identifies the two-dimensional code in the image. After the WeChat application completes the operation of locating and identifying the two-dimensional code, the mobile phone displays a code scanning positioning interface (as shown in (4) of FIG. 1) and immediately jumps to a code scanning result interface (not shown in the figure) corresponding to the content information of the two-dimensional code. The code scanning positioning interface is used to indicate the positioning result of the two-dimensional code in the image, and the code scanning result interface is used to indicate the content identification result of the two-dimensional code. Exemplarily, the code scanning result interface can be an operation interface (such as a payment interface, a friend adding interface, etc.) or an information display interface (a public account interface, etc.) corresponding to the content of the two-dimensional code. In the case of the WeChat application, the code scanning result interface can be an operation interface or an information display interface corresponding to the content of the two-dimensional code.​​​​​Figure 1 As shown in (4), the code scanning and positioning interface displays the currently displayed preview image frame 104 of the WeChat application and the code scanning and positioning mark (or QR code positioning mark) 1041. The code scanning and positioning mark 1041 is used to indicate the position information of the QR code recognized by the WeChat application in the scanned image, so that the user can understand the positioning of the QR code in the image by the WeChat application. For example, in the WeChat application, the code scanning and positioning mark 1041 is a green dot.

[0063] If the WeChat application is unable to locate and recognize the QR code, the scan line 1031 will continue to move from top to bottom, constantly showing the user that it is scanning. In one case, if the QR code is poorly positioned or the scanning angle is too large, the WeChat application may be unable to locate and recognize the QR code, and the scan line 1031 will continue to move from top to bottom. Figure 2 An example application scenario is shown. Figure 2 As shown, when a user holds a mobile phone 100_1 and uses the "Scan" function in the WeChat application, if the angle between the plane containing the QR code 200 and the image acquisition plane of the mobile phone is too large, the QR code 200 will be distorted in the scanned image of the WeChat application, causing the WeChat application to be unable to complete the positioning and recognition of the QR code. The scanning line 1031 will continue to move from top to bottom, always showing the user that it is scanning. At this time, the user can only continuously adjust the image acquisition direction of the mobile phone 100_1 to make the image acquisition plane of the mobile phone as parallel as possible to the plane containing the QR code 200, thereby reducing the degree of distortion of the QR code 200 in the scanned image and enabling the WeChat application to complete the positioning and recognition of the QR code. Among them, the image acquisition plane of the mobile phone can be understood as the plane containing the lens of the mobile phone camera, or a plane parallel to the plane containing the lens.

[0064] Therefore, when the angle between the plane of the barcode to be scanned and the image acquisition plane of the electronic device is too large (such as 45 degrees to 75 degrees), how to improve the positioning and recognition success rate of the barcode scanning application is an urgent problem to be solved.

[0065] like Figure 3 The figure shows a schematic diagram of the hardware structure of the electronic device 100. Optionally, the electronic device 100 can be called a terminal or a terminal device. The electronic device 100 can be a mobile phone, a tablet computer, a wearable device or other device with a camera. The present application does not limit the type of the electronic device 100. It should be noted that the schematic diagram of the structure of the electronic device 100 can be applied to Figures 1-2 Mobile phone in.

[0066] It should be understood that Figure 3The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have a different configuration of components. Figure 3 The various components shown in the drawings may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0067] like Figure 3 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an acceleration sensor, a temperature sensor, a motion sensor, an air pressure sensor, a magnetic sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0068] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0069] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0070] The processor 110 can also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0071] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0072] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 can include 1 or N display screens 194, N being a positive integer greater than 1.

[0073] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor, etc.

[0074] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, and the optical signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0075] The camera 193 is used to capture still images or videos. For example, in a code scanning scenario, the camera 193 can capture an image frame including a barcode. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, etc. format image signal. In some embodiments, the electronic device 100 can include 1 or N cameras 193, N being a positive integer greater than 1.

[0076] The internal memory 121 can be used to store computer executable program codes including instructions. The processor 110 performs various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121, for example, to enable the electronic device 100 to implement the code scanning method in the embodiments of the present application.

[0077] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0078] Figure 4 is a software structure block diagram of the electronic device 100 in the embodiments of the present application.

[0079] The layered architecture of the electronic device 100 divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the HAL (Hardware Abstraction Layer), and the kernel layer (also known as the driver layer).

[0080] It can be understood that, Figure 4 For example, only as an example, between the application framework layer and the HAL, an Android runtime and a system library layer, etc. can also be included. Among them, the Android Runtime includes the core library and the virtual machine, which is responsible for the scheduling and management of the Android system. The system library can include a plurality of functional modules. For example: surface manager, media library, three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL) and the like.

[0081] The application layer can include a series of application packages. As Figure 4 indicated, the application package can include a camera, a gallery, a code scanning application (a third-party application), a scene recognition module, etc. The application package can also include calling, calendar, map, navigation, music, video, short message, WLAN, Bluetooth, and the like.

[0082] In the embodiments of the present application, the scene recognition module can be used to determine the value of the flag bit FLAG corresponding to the current scene. The value of the flag bit FLAG is used to indicate whether the callback stream needs to be created. For example, if the scene recognition module determines that FLAG=1 corresponding to the current scene, the callback stream needs to be created for the current scene; if the scene recognition module determines that FLAG=0 corresponding to the current scene, the callback stream does not need to be created for the current scene.

[0083] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0084] As shown in Figure 4 , the application framework layer can include a window manager, a content provider, a resource manager, a view system, a notification manager, a camera service, and the like.

[0085] The camera service can be used to manage the camera, including starting and stopping the camera, creating a preview stream (previewStream) and a callback stream (callbackStream), obtaining a preview image frame and a callback image frame, reporting the preview stream to the upper-layer application, and reporting the callback stream to the upper-layer application through a callback function. The preview stream is composed of multiple preview image frames, and the preview image frame is generated based on the preview image data collected by the camera sensor. The callback stream is composed of multiple callback image frames, and the callback image frame is generated based on the callback image data collected by the camera sensor.

[0086] In the embodiments of the present application, the camera service can determine whether the callback stream needs to be created according to the value of the flag bit FLAG determined by the scene recognition module.

[0087] The window manager is used to manage the window program. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen.

[0088] The content provider is used to store and obtain data, and make the data accessible to the application. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, and the like.

[0089] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

[0090] The view system includes visual controls, such as controls that display text, controls that display pictures, and the like. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface that includes a short message notification icon can include a view that displays text and a view that displays a picture.

[0091] The notification manager enables an application to display notification information in a status bar, which can be used to convey a message of an informing type that can automatically disappear after a short stay without user interaction. The notification information can be used to inform, for example, of a download completion, a message reminder, and the like. The notification information can also be a notification that appears in the form of a chart or a scroll bar text in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification that appears in the form of a dialog window on the screen. The notification information can also be, for example, a text information prompt in the status bar, a prompt sound, a vibration of the electronic device, a flashing of an indicator light, and the like.

[0092] The HAL is used to abstract the underlying hardware to provide a unified service for the upper layer. The HAL can encapsulate the driver in the kernel layer and provide an interface for the application framework layer to call, and shield the implementation details of the low-level hardware. As shown in Figure 4 The HAL can include a camera HAL (Camera HAL) and the like.

[0093] The Camera HAL is a camera core software framework, and the Camera HAL includes a sensor node (Sensor node), an image processing module, a barcode detection module, a barcode distortion correction module, a barcode zoom module, and the like.

[0094] The Sensor node can be a control node for a camera sensor, and the Sensor node can control the camera sensor to expose a picture through a camera driver.

[0095] The image processing module can be used for, but is not limited to, image signal front-end processing, spatial alignment transformation processing, image signal back-end processing, image format conversion processing (such as format conversion between a GRB image and a YUV image), barcode distortion detection processing, barcode size detection processing, and image new creation operations, and the like.

[0096] The barcode detection module can be used to detect a barcode in an image and determine a barcode image region. Optionally, the barcode detection module is a two-dimensional code detection module, which is used to detect a two-dimensional code in an image and determine a two-dimensional code image region.

[0097] The barcode distortion correction module can be configured to perform distortion correction on the barcode image satisfying the distortion condition. In an example, the barcode distortion correction module can be configured to perform distortion correction on the barcode image satisfying the distortion condition based on the principle of perspective transformation. In an example, the barcode distortion correction module can be a two-dimensional code distortion correction module configured to perform distortion correction on the two-dimensional code image satisfying the distortion condition based on the principle of perspective transformation.

[0098] The barcode zoom module can be configured to perform zoom operation on the image satisfying the zoom condition to enlarge the image proportion of the barcode image in the whole frame image. In an example, the barcode zoom module can be a two-dimensional code zoom module configured to perform zoom operation on the image satisfying the zoom condition to enlarge the image proportion of the two-dimensional code image in the whole frame image.

[0099] It should be noted that the barcode detection module, the barcode distortion correction module and the barcode zoom module are divided based on functions, and each module is configured to implement a certain function. The division of the barcode detection module, the barcode distortion correction module and the barcode zoom module is not limited in the present application. The barcode detection module, the barcode distortion correction module and the barcode zoom module can be separately arranged or integrated together, for example, integrated in the image processing module.

[0100] The kernel layer is a layer between hardware and software. The kernel layer at least includes a camera driver and a display driver. The hardware at least includes a processor, a camera, a display screen, a sensor, etc.

[0101] It should be understood that, Figure 4 The layers in the software structure and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer layers than shown, and each layer can include more or fewer components, which are not limited in the present application.

[0102] It should be understood that the electronic device includes hardware and / or software modules corresponding to each function in order to implement the code scanning method in the embodiments of the present application. The algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered beyond the scope of the present application.

[0103] Reference is made to FIG. 1, which shows a software structure of an electronic device 100 according to an embodiment of the present application. Figure 1 and Figure 2In this example, the WeChat app's "Scan" function is implemented based on the electronic device's camera. When implementing this function, the WeChat app uses the preview image frames captured by the camera app for interface preview and the callback image frames captured by the camera app for code scanning and recognition.

[0104] The following example uses the WeChat app as the third-party application and the "Scan" function as the example. Figure 5 , the code scanning method provided in the embodiment of the present application is explained. Among them, the third-party application can also be other applications with a code scanning function, such as the Alipay application.

[0105] refer to Figure 1 In (1), the user clicks the WeChat application icon 101, and the WeChat application starts. After the WeChat application starts, an activity thread corresponding to the WeChat application is generated. Figure 1 In (2), the user clicks the "Scan" function option 102. In response to the user operation, the WeChat SDK (Software Development Kit) in the mobile phone starts the camera SDK, such as Figure 5 As shown. After the "Scan" function is started, the active thread corresponding to the WeChat application sends the interface scene parameters to the scene recognition module. Among them, the interface scene parameters are used to describe the scene and state of the current interface of the mobile phone. The interface scene parameters may include but are not limited to the application package name (PackegeName), the activity name (ActivityName), or the interface scene parameters include but are not limited to the application package name, the activity name and the fragment name (FragmentName) (or page name), etc. Furthermore, the scene recognition module can determine the value of the scene flag FLAG based on the interface scene parameters. Among them, the value of the scene flag FLAG is used to indicate whether the current scene needs to create a callback flow. For example, when FLAG=1, it means that the current scene needs to create a callback flow; when FLAG=0, it means that the current scene does not need to create a callback flow.

[0106] It should be noted that the preview stream is composed of multiple preview image frames, which are generated based on the preview image data collected by the camera sensor. The preview image data is the image data collected by the camera sensor based on the preview image frame request; the callback stream is composed of multiple callback image frames, which are generated based on the callback image data collected by the camera sensor. The callback image data is the image data collected by the camera sensor based on the callback image frame request. Among them, the preview stream is used for third-party applications to display preview images, while the callback stream is used for third-party applications to implement specific functions. In this application, the callback stream is specifically used for the WeChat application to implement the code scanning and recognition function.

[0107] For example, the scene recognition module can match the received interface scene parameters with the interface scene parameters corresponding to the preset scene. The value of the flag bit FLAG corresponding to the preset scene is 1. If the match is successful, the scene recognition module can determine that the flag bit FLAG corresponding to the current scene is 1; if the match is unsuccessful, the scene recognition module can determine that the flag bit FLAG corresponding to the current scene is 0. After the scene recognition module determines the FLAG value corresponding to the current scene, it sends the FLAG value to the camera SDK.

[0108] Since the WeChat application performs code scanning and recognition operations based on the callback image frame when implementing the "Scan" function, the value of the scene flag FLAG corresponding to the WeChat application's "Scan" function is "1", and the scene recognition module sends the indication information "FLAG=1" to the camera SDK.

[0109] Continue to refer to Figure 5 After receiving the start instruction sent by the WeChat SDK, the camera SDK sends a preview instruction to the camera service. The camera service generates a preview image frame request based on the preview instruction and sends the preview image frame request to the camera HAL.

[0110] In this embodiment of the present application, the scene recognition module sends the indication information "FLAG=1" to the camera SDK. After receiving the indication information "FLAG=1", the camera SDK sends the indication information "FLAG=1" to the camera service. The camera service generates a callback image frame request based on the indication information "FLAG=1" and sends the callback image frame request to the camera HAL.

[0111] After the camera HAL receives the preview image frame request, the Sensor node in the camera HAL sends the preview image frame request to the camera driver, which controls the camera sensor to perform exposure and image output operations to obtain preview image data.

[0112] After the camera HAL receives the callback image frame request, the Sensor node in the camera HAL sends the callback image frame request to the camera driver, which controls the camera sensor to perform exposure and image output operations to obtain the callback image data.

[0113] Due to the rotation buffer mechanism of the camera system in the Android architecture between the Framework layer and the HAL. After the camera service receives the preview indication, the camera service first continuously sends multiple (for example, 8) preview image frame requests to the camera HAL, and sends a preview image frame request to the camera HAL after receiving a preview image frame feedback from the camera HAL each time until the preview ends. In the camera HAL, the Sensor node controls the camera sensor to perform exposure and image generation operations based on the multiple preview image frame requests in turn. Similarly, after the camera service receives the indication information "FLAG = 1", the camera service first continuously sends multiple (for example, 8) callback image frame requests to the camera HAL, and sends a callback image frame request to the camera HAL after receiving a callback image frame feedback from the camera HAL each time until the code scanning identification ends. In the camera HAL, the Sensor node controls the camera sensor to perform exposure and image generation operations based on the multiple callback image frame requests in turn.

[0114] In this way, the camera service realizes the creation of the preview stream and the creation of the callback stream. Among them, the preview image frames in the preview stream are continuously acquired and updated, that is, the preview picture of the WeChat application is generated according to the preview image frames acquired in turn; the callback image frames in the callback stream are also continuously acquired and updated, that is, the WeChat application realizes the code scanning identification function according to the callback image frames acquired in turn.

[0115] It should be pointed out that the order of creating the preview stream and creating the callback stream by the camera service is not limited by the embodiments of the present application. For example, the camera service first creates the preview stream, and then creates the callback stream. For another example, the camera service creates the preview stream and the callback stream in parallel.

[0116] If the camera service receives the preview indication and the indication information "FLAG = 0", the camera service generates a preview image frame request according to the preview indication, and sends the generated preview image frame request to the camera HAL. Since the indication information "FLAG = 0" indicates that the callback stream does not need to be created in the current scene, the camera service will not generate a callback image frame request in this case.

[0117] Continuing to refer to Figure 5 After the camera sensor exposes to generate preview image data, the preview image data is sent to the image processing module through the camera driver. The image processing module performs related processing on the preview image data, including but not limited to front-end processing, spatial alignment transformation, back-end processing, etc., to obtain the corresponding preview image frame. Further, the preview image frame generated by the image processing module is sent to the UI module of the WeChat SDK through the camera service for display.

[0118] After the camera sensor exposure generates the callback image data, the callback image data is sent to the image processing module through the camera driver. The image processing module performs relevant processing on the callback image data, including but not limited to front-end processing, spatial alignment transformation, back-end processing, etc., to obtain a corresponding callback image frame.

[0119] In the embodiments of the present application, unlike the preview image frame, the callback image frame is not directly sent to the WeChat SDK, but needs to be further processed in the camera HAL.

[0120] Referring to Figure 5 In an optional implementation, before the callback image frame is fed back to the camera service, the image processing module also performs distortion correction processing thereon. The image processing module calls the two-dimensional code detection module to extract a two-dimensional code image contour in the callback image frame, and cuts out a two-dimensional code image in the callback image frame according to the two-dimensional code image contour. The image processing module determines whether the two-dimensional code image satisfies distortion correction conditions, and if so, calls the two-dimensional code distortion correction module to perform distortion correction processing on the two-dimensional code image, and regenerates a callback image frame according to the two-dimensional code image after the distortion correction processing, as a processed callback image frame. After obtaining the processed callback image frame, the image processing module sends the processed callback image frame to the camera service.

[0121] Further, the camera service returns the processed callback image frame to the recognition interface in the WeChat SDK through a callback function set by the WeChat SDK. The WeChat SDK can determine the timing of setting the callback function (for example, after focus processing) according to its own business logic, and call the callback function interface in the camera SDK interface at the timing to set the callback function. After the WeChat SDK receives the callback image frame through the recognition interface, the WeChat SDK sends the callback image frame to the recognition module, and performs positioning and recognition processing on the two-dimensional code in the callback image frame through the recognition module. After the recognition module of the WeChat SDK obtains the processed callback image frame through the recognition interface, the recognition module can perform positioning and recognition processing on the two-dimensional code in the callback image frame. The positioning of the two-dimensional code by the recognition module can refer to determining the position information of the two-dimensional code in the image, and the recognition of the two-dimensional code can refer to determining the content information contained in the two-dimensional code, such as link information, text information, etc.

[0122] When the angle between the plane where the two-dimensional code to be scanned and the image acquisition plane of the mobile phone is too large (e.g., 45 degrees to 75 degrees), the two-dimensional code in the callback image frame will have distortion problems. Since the image processing module performs distortion correction processing on the two-dimensional code image in the callback image frame before feeding it back to the camera service, and sends the processed callback frame image to the recognition module in the WeChat SDK for recognition, the two-dimensional code image recognized by the recognition module in the WeChat SDK does not have distortion, which facilitates the two-dimensional code positioning and recognition operation of the recognition module in the WeChat SDK, thereby improving the positioning and recognition success rate of the two-dimensional code by the recognition module in the WeChat SDK, and accelerating the positioning and recognition efficiency of the two-dimensional code by the recognition module in the WeChat SDK.

[0123] After the recognition module completes the positioning and recognition processing of the two-dimensional code in the callback image frame, the two-dimensional code positioning result is determined and sent to the UI module. The two-dimensional code positioning result is used to indicate the position information of the two-dimensional code in the callback image frame, for example, the coordinate information of the center point of the two-dimensional code in the callback image frame. Then, the UI module can determine the display position of the scan positioning mark 1041 in the preview image according to the two-dimensional code positioning result, for example, taking the coordinate indicated by the two-dimensional code positioning result as the coordinate of the center point of the scan positioning mark 1041 in the preview image, and displaying the scan positioning mark on the currently displayed preview image frame. At this time, refer to Figure 1 for details.

[0124] It should be noted that the above explanation is based on the example of including one two-dimensional code in the callback frame image. When multiple two-dimensional codes are included in the callback image frame, the image processing module can call the two-dimensional code detection module to extract the contour of each two-dimensional code image in the callback image frame, and extract the two-dimensional code image according to each two-dimensional code image contour in the callback image frame, and perform distortion correction processing on each two-dimensional code image, so that each two-dimensional code in the processed callback image frame does not have distortion.

[0125] After the recognition module in the WeChat SDK performs two-dimensional code positioning processing on the processed callback image frame, if only one two-dimensional code is positioned, the recognition module sends the positioning result corresponding to the two-dimensional code to the UI module. At this time, the UI module can determine the display position of the scan positioning mark 1041 in the preview image according to the two-dimensional code positioning result, and display the scan positioning mark 1041 on the currently displayed preview image frame. At this time, refer to Figure 1As shown in FIG. 4, only one scan code positioning mark is displayed. In this case, the recognition module performs recognition processing on the positioned two-dimensional code to obtain two-dimensional code information content, and the WeChat application interface is immediately switched to a scan code result interface corresponding to the two-dimensional code content information, such as an operation interface (such as a payment interface, a friend adding interface, etc.) or an information display interface (a public number interface, etc.) corresponding to the two-dimensional code content information.

[0126] In the WeChat SDK, after the recognition module performs two-dimensional code positioning processing on the processed callback image frame, if multiple two-dimensional codes are positioned, the recognition module sends each positioning result corresponding to each two-dimensional code to the UI module. At this time, the UI module can determine the display position of each scan code positioning mark 1041 in the preview image according to each two-dimensional code positioning result, and display each scan code positioning mark on the currently displayed preview image frame. In this case, in response to a selection operation (such as a click operation, etc.) of a user on a certain scan code positioning mark, the recognition module performs recognition processing on the two-dimensional code selected by the user (that is, the two-dimensional code corresponding to the scan code positioning mark selected by the user) to obtain two-dimensional code information content, and the WeChat application interface is immediately switched to a scan code result interface corresponding to the two-dimensional code indicated by the scan code positioning mark. The recognition module can also perform recognition on each positioned two-dimensional code before the user selects the scan code positioning mark, so that after the user selects a certain scan code positioning mark, the WeChat application interface can directly display a scan code result interface corresponding to the two-dimensional code indicated by the scan code positioning mark, which is not limited in the present embodiment.

[0127] If the image processing module determines that the cropped two-dimensional code image does not satisfy the distortion correction condition, the callback image frame is not processed, and the callback image frame is directly sent to the camera service, so that the camera service returns the callback image frame to the recognition interface in the WeChat SDK through the callback function of the WeChat SDK, and then the recognition module of the WeChat SDK can perform two-dimensional code positioning and recognition processing on the callback image frame.

[0128] Reference Figure 5In another alternative embodiment, the image processing module calls the two-dimensional code detection module to extract a two-dimensional code image contour in the callback image frame, and to crop a two-dimensional code image in the callback image frame according to the two-dimensional code image contour. The image processing module first determines whether the two-dimensional code image in the callback image frame satisfies the zoom condition. If the two-dimensional code image in the callback image frame satisfies the zoom condition, the image processing module calls the two-dimensional code zoom module to perform zoom processing on the callback image frame to enlarge the two-dimensional code image. Since the image size of the callback image frame does not change before and after the zoom processing, the image proportion of the two-dimensional code image in the callback image frame increases after the zoom processing. At this time, the image processing module takes the callback image frame after the zoom processing as the processed callback image frame, and sends the processed callback image frame to the camera service. If the two-dimensional code image in the callback image frame does not satisfy the zoom condition, the image processing module further determines whether the cropped two-dimensional code image satisfies the distortion correction condition. If the cropped two-dimensional code image satisfies the distortion correction condition, the image processing module calls the two-dimensional code distortion correction module to perform distortion correction processing on the two-dimensional code image, and regenerates a callback image frame according to the two-dimensional code image after the distortion correction processing as the processed callback image frame, and sends the processed callback image frame to the camera service. If the cropped two-dimensional code image does not satisfy the distortion correction condition, the image processing module does not process the callback image frame, and directly sends the callback image frame to the camera service. Further, the camera service can return the received callback image frame to the recognition interface in the WeChat SDK through the callback function of the WeChat SDK, and the recognition module of the WeChat SDK performs positioning and recognition processing of the two-dimensional code on the received callback image frame. The processing flow of the recognition module of the WeChat SDK after receiving the callback image frame can be referred to the foregoing, and will not be described here.

[0129] When the angle between the plane where the two-dimensional code to be scanned and the image acquisition plane of the mobile phone is too large (such as 45 degrees to 75 degrees), the two-dimensional code in the callback image frame can have the problem of too small image, or the problem of image distortion. Since the image processing module performs zoom processing or distortion correction processing on the two-dimensional code image in the callback image frame before feeding back the callback image frame to the camera service, and sends the processed callback image frame to the recognition module in the WeChat SDK through the camera service for recognition, the two-dimensional code recognized by the recognition module in the WeChat SDK has a proper size in the image and does not have distortion phenomenon, which facilitates the recognition module in the WeChat SDK to perform positioning and recognition operation of the two-dimensional code, thereby improving the positioning and recognition success rate of the two-dimensional code by the recognition module in the WeChat SDK, and accelerating the positioning and recognition efficiency of the two-dimensional code by the recognition module in the WeChat SDK.

[0130] For example, as shown in FIG. 8, a callback image frame is taken as an example to describe the processing flow of the image processing module. Figure 6 As shown in FIG. 8, a callback image frame is taken as an example to describe the processing flow of the image processing module. Figure 6The processing procedure of the image processing module before sending the image to the camera service is described in detail. Among them, Figure 6 The flow of the processing shown is taken as an example to explain the processing flow, which can also be completed by the image processing module alone, such as the operations performed by the two-dimensional code detection module, the two-dimensional code zoom module, and the two-dimensional code distortion correction module can be completed by the image processing module. Figure 6 The processing flow shown is taken as an example to explain the processing flow, which can also be completed by the image processing module alone, such as the operations performed by the two-dimensional code detection module, the two-dimensional code zoom module, and the two-dimensional code distortion correction module can be completed by the image processing module.

[0131] As shown in Figure 6 As shown in

[0132] S301, the image processing module obtains the callback image frame A.

[0133] Assuming that the callback image frame A refers to the nth callback image frame, the callback image frame A corresponds to the nth callback image frame request. The camera sensor captures the callback image data based on the nth callback image frame request, and sends the callback image data to the image processing module through the camera driver. The image processing module performs related processing on the callback image data, such as image signal front-end processing, spatial alignment transformation processing, image signal back-end processing, etc., to obtain the callback image frame A. Among them, the image format of the callback image frame A is YUV.

[0134] S302, the image processing module converts the image format of the callback image frame A from YUV to RGB to obtain the callback image frame B.

[0135] S303, the image processing module calls the two-dimensional code detection module to extract the two-dimensional code image contour in the callback image frame B.

[0136] As shown in Figure 7a The image processing module inputs the callback image frame B 401 into the two-dimensional code detection module, and the two-dimensional code detection module extracts the two-dimensional code image contour 402 in the callback image frame B 401.

[0137] Exemplarily, the two-dimensional code detection module extracts the vertex coordinates of the two-dimensional code image contour 402 in the callback image frame B 401.

[0138] In an example, the two-dimensional code detection module is a two-dimensional code AI (Artificial Intelligence) detection module generated by pre-training. The callback image frame B 401 is input into the two-dimensional code AI detection module, and the two-dimensional code AI detection module labels the two-dimensional code image contour 402 on the callback image frame B 401.

[0139] In training the AI detection module for the two-dimensional code, a standard AI detection model (such as a machine learning model) can be trained according to multiple groups of image samples to obtain the AI detection module for the two-dimensional code. Each group of image samples includes an image containing a two-dimensional code and a two-dimensional code image contour corresponding thereto.

[0140] In yet another example, the two-dimensional code detection module includes an image mean filtering submodule, an image graying submodule, an image edge detection submodule, an image binarization submodule, an image dilation and erosion submodule, and an image contour extraction submodule.

[0141] The image mean filtering submodule is configured to perform mean filtering (also referred to as linear filtering) on the image to achieve image denoising. The image mean filtering submodule uses a neighborhood averaging method, the basic principle of which is to replace each pixel value in the original image with a mean value. For a current pixel point (x, y) to be processed, a template (composed of a number of neighboring pixels) is selected, the mean value of all pixels in the template is calculated, and the mean value is assigned to the current pixel point (x, y) as the pixel value of the processed image at the point.

[0142] The image graying submodule is configured to convert the image from an RGB image to a grayscale image.

[0143] The image edge detection submodule is configured to perform edge detection on the grayscale image to identify pixel points with obvious brightness changes in the grayscale image. The image edge detection submodule can be implemented based on a lookup method or a zero-crossing method. The lookup method detects boundaries by finding the maximum and minimum values in the first derivative of the image, which is usually to locate the boundary in the direction with the maximum gradient. The zero-crossing method finds the boundary by finding the zero-crossing of the second derivative of the image, which is usually the zero-crossing of the Laplacian or the zero-crossing of a nonlinear difference representation.

[0144] The image binarization submodule is configured to perform binarization processing on the grayscale image, and set the grayscale value of the pixel point on the grayscale image to 0 or 255 to convert the grayscale image into a binary image. For example, the image binarization submodule performs binarization processing on the grayscale image according to a certain threshold, sets the grayscale value of the pixel point with a grayscale value greater than the threshold to 255, and sets the grayscale value of the pixel point with a grayscale value less than or equal to the threshold to 0.

[0145] The image dilation and erosion submodule is configured to perform dilation and / or erosion processing on the binary image. The erosion and dilation of the image are two basic morphological operations, mainly used to find the maximum region and the minimum region in the binary image.

[0146] The image contour extraction submodule is configured to extract the external contour feature of the image in the binary image. The image contour extraction submodule can employ a common contour extraction algorithm, such as threshold segmentation, extraction of high-frequency information of Fourier transform, etc.

[0147] In the present example, the callback image frame B is sequentially processed by the image mean filtering submodule, the image graying submodule, the image edge detection submodule, the image binarization submodule, the image dilation and erosion submodule, and the image contour extraction submodule, to obtain the contour of the two-dimensional code in the callback image frame B.

[0148] In S304, the image processing module determines whether the two-dimensional code image in the callback image frame B satisfies the zoom condition. If not, S305 is performed; if yes, S309 is performed.

[0149] In an exemplary embodiment, the zoom condition includes that the image proportion of the two-dimensional code image in the callback image frame is less than a preset proportion threshold, and the satisfaction of the zoom condition indicates that the callback image frame needs to be zoomed to enlarge the image proportion of the two-dimensional code image in the callback image frame.

[0150] In an alternative embodiment, the image processing module calculates the image proportion r0 of the two-dimensional code image in the callback image frame according to the contour of the two-dimensional code image. Wherein, r0=S0 / S, S0 is the pixel area of the two-dimensional code image, and S is the pixel area of the callback image frame.

[0151] The image processing module further calculates the maximum allowed zoom ratio r1 corresponding to the two-dimensional code in the callback image frame according to the two-dimensional code image size information and the two-dimensional code image position information in the callback image frame. The maximum allowed zoom ratio r1 is used to indicate the maximum zoom ratio value allowed when zooming the callback image frame to enlarge the two-dimensional code image. When zooming the callback image frame, if the zoom ratio is greater than r1, the two-dimensional code image will exceed the boundary of the callback image frame, i.e., the two-dimensional code image is no longer complete in the zoomed callback image frame.

[0152] In the present embodiment, zooming the callback image frame to enlarge the two-dimensional code image means that the size of the callback image frame does not change before and after zooming, and the field of view angle changes (i.e., the field of view angle of the image becomes smaller). Since the size of the callback image frame does not change before and after zooming, it can be understood that the boundary position of the callback image frame does not change.

[0153] As Figure 7bAs shown, the width w and the height h of the QR code image are determined, and the distance w1 of the center point of the QR code image from the left boundary of the callback image frame, the distance h1 of the center point of the QR code image from the top boundary of the callback image frame, the distance w2 of the center point of the QR code image from the right boundary of the callback image frame, and the distance h2 of the center point of the QR code image from the bottom boundary of the callback image frame are calculated. The maximum allowed zoom ratio r1 = min{w1 / 0.5w, h1 / 0.5h, w2 / 0.5w, h2 / 0.5h}.

[0154] Optionally, the zoom condition further comprises a limitation on the maximum allowed zoom ratio r1, for example, the maximum allowed zoom ratio r1 corresponding to the QR code in the callback image frame is greater than a threshold value B. In this embodiment, the zoom condition can be that the image proportion r0 of the QR code image in the callback image frame is less than a threshold value A, and the maximum allowed zoom ratio r1 corresponding to the QR code is greater than a threshold value B.

[0155] If the image proportion r0 of the QR code image in the callback image frame B is not less than the threshold value A, or the maximum allowed zoom ratio r1 corresponding to the QR code in the callback image frame is not greater than the threshold value B, the QR code image in the callback image frame B does not satisfy the zoom condition.

[0156] If the image proportion r0 of the QR code image in the callback image frame B is less than the threshold value A, and the maximum allowed zoom ratio r1 corresponding to the QR code in the callback image frame is greater than the threshold value B, the QR code image in the callback image frame B satisfies the zoom condition.

[0157] S305, the image processing module determines whether the QR code image satisfies the distortion correction condition, if yes, S306 is executed, if not, S311 is executed.

[0158] When the QR code image in the callback image frame B does not satisfy the zoom condition, the image processing module continues to determine whether the QR code image satisfies the distortion correction condition.

[0159] In an optional embodiment, the image processing module determines whether the QR code image satisfies the distortion correction condition according to the maximum rotation angle M of the QR code image, and the distortion correction condition can be that the maximum rotation angle M is greater than a threshold value C. An exemplary threshold value C is 10°.

[0160] The maximum rotation angle M of the QR code image is the maximum angle value of the horizontal rotation angle and the vertical rotation angle of the QR code image. For example, the horizontal rotation angle of the QR code image is the angle between the horizontal direction of the QR code image and the horizontal direction of the callback image frame, and the vertical rotation angle of the QR code image is the angle between the vertical direction of the QR code image and the vertical direction of the callback image frame. Figure 7cAs shown, angle a1 is the included angle between the left boundary of the two-dimensional code image and the vertical direction, angle a2 is the included angle between the upper boundary of the two-dimensional code image and the horizontal direction, angle a3 is the included angle between the right boundary of the two-dimensional code image and the vertical direction, and angle a4 is the included angle between the lower boundary of the two-dimensional code image and the horizontal direction. Angle a1 and angle a3 are vertical rotation angles, and angle a2 and angle a4 are horizontal rotation angles. The maximum rotation angle M = max{angle a1, angle a2, angle a3, angle a4}.

[0161] In S306, the image processing module calls the two-dimensional code distortion correction module to perform distortion correction processing on the two-dimensional code image.

[0162] When the two-dimensional code image satisfies the distortion correction condition, the image processing module calls the two-dimensional code distortion correction module to perform distortion correction processing on the two-dimensional code image. The two-dimensional code distortion correction module performs perspective transformation processing on the two-dimensional code image to obtain a distortion-corrected two-dimensional code image.

[0163] Perspective transformation (Perspective Transformation) refers to using the condition that the perspective center, image point, and target point are collinear, rotating the projection surface (perspective surface) by a certain angle around the trace line (perspective axis) according to the perspective rotation law, destroying the original projection light beam, and still maintaining the projection geometric figure on the projection surface unchanged. In short, it is to project a plane onto a specified plane through a projection matrix, and the general transformation formula is as follows:

[0164]

[0165] where the perspective transformation matrix

[0166] represents image linear transformation, T2 = [a 13 a 23 ] T is used to generate image perspective transformation, and T3 = [a 31 a 32 ] represents image translation.

[0167] where u and v are original image coordinates, the parameter w is equal to 1, x and y are picture coordinates obtained after perspective transformation,

[0168] The above perspective transformation matrix is divided into four parts, the T1 matrix is mainly used for image scaling and rotation operations, the T3 matrix is used for image translation operations, the T2 matrix is used to generate perspective transformation, a 33 is equal to 1. Since the perspective transformation matrix has a total of 8 parameters, 4 coordinate pairs are required to solve it. The expressions for x and y after perspective transformation are:

[0169]

[0170]

[0171] Thus, given the four pairs of pixel coordinates corresponding to the perspective transformation, the perspective transformation matrix Transform can be obtained, and then the original image can be processed according to the perspective transformation matrix, so that the image after perspective transformation can be obtained.

[0172] In the embodiment of the present application, the image processing module extracts four vertex coordinates of the two-dimensional code image, and sorts the four vertices in a clockwise order, such as the left upper vertex P1(u1, v1), the right upper vertex P2(u2, v2), the right lower vertex P3(u3, v3), and the left lower vertex P4(u4, v4).

[0173] The image processing module determines the edge length L of the two-dimensional code image after distortion correction according to the pixel coordinates of the vertices P1, P2, P3, and P4. In the embodiment, the image processing module takes the length of the longest side of the two-dimensional code image as the edge length L of the two-dimensional code image after distortion correction. Wherein, the edge length L of the two-dimensional code image after distortion correction = max{P1P2, P2P3, P3P4, P4P1}

[0174] Suppose that the image processing module sets the coordinates of the left upper vertex P1' of the two-dimensional code image after distortion correction as (u0, v0), then the image processing module can calculate the right upper vertex P2'(u0+L, v0), the right lower vertex P3'(u0+L, v0+L), and the left lower vertex P4'(u0, v0+L) according to the edge length L of the two-dimensional code image after distortion correction. Wherein, the left upper vertex P1'(u0, v0) can be set in the left upper corner region of the image, for example, u0 is greater than 20, and v0 is greater than 20.

[0175] In this way, the image processing module obtains four pairs of pixel coordinates, which are the pixel coordinates of P1 and P1', the pixel coordinates of P2 and P2', the pixel coordinates of P3 and P3', and the pixel coordinates of P4 and P4'. Further, the image processing module calculates the perspective transformation matrix Transform based on the pixel coordinates of the vertices P1, P2, P3, and P4 of the source two-dimensional code image (i.e. the two-dimensional code image to be distortion corrected), and the pixel coordinates of the vertices P1', P2', P3', and P4' of the target two-dimensional code image (i.e. the two-dimensional code image after distortion correction).

[0176] After the perspective transformation matrix Transform is calculated, the image processing module can perform perspective transformation processing on the source two-dimensional code image according to the perspective transformation matrix Transform, so that the target two-dimensional code image, i.e. the two-dimensional code image after distortion correction, can be obtained. For reference, the perspective transformation processing can be performed according to the perspective transformation matrix Transform. Figure 7dAn example of perspective transformation processing is shown.

[0177] S307 : The image processing module regenerates the callback image frame C according to the distortion-corrected QR code image, and converts the image format of the callback image frame C from RGB to YUV to obtain the callback image frame D.

[0178] The image processing module creates a new blank image frame, and adds the two-dimensional code image after the distortion correction processing to the blank image frame to generate a callback image frame C.

[0179] The image size of callback image frame C is the same as that of callback image frame B. That is, the image size of the blank image frame is the same as that of callback image frame B. The position of the QR code image after distortion correction in callback image frame C is consistent with the position of the QR code image before distortion correction in callback image frame B (or position alignment).

[0180] Optionally, the coordinates of the center point of the QR code image after distortion correction in the callback image frame C are the same as the coordinates of the center point of the QR code image before distortion correction in the callback image frame B. The image processing module determines the coordinates of the center point of the QR code image before distortion correction in the callback image frame B, and based on the coordinates, adds the QR code image after distortion correction to the blank image frame to generate the callback image frame C.

[0181] Figure 7e Examples of callback image frame C and callback image frame B are exemplarily shown. Figure 7e (1) shows an example of a callback image frame B, in which the two-dimensional code image meets the distortion correction condition and needs to be subjected to distortion correction processing. Figure 7e (2) shows the Figure 7e The callback image frame B shown in (1) corresponds to the callback image frame C. Figure 7e As shown in (2), the image size of the callback image frame C is the same as that of the callback image frame B. The callback image frame C is obtained by adding the distortion-corrected QR code image to a blank image frame with the same image size as the callback image frame B. The coordinates of the center point of the distortion-corrected QR code image in the callback image frame C are the same as the coordinates of the center point of the distortion-corrected QR code image in the callback image frame B.

[0182] If the image processing module randomly adds the two-dimensional code image after the distortion correction process to the blank image frame, for example, the image processing module adds the two-dimensional code image after the distortion correction process to the upper left corner of the blank image frame to generate the callback image frame C, then combined with Figure 5As shown in the flow, the recognition module in the WeChat SDK can perform two-dimensional code positioning and recognition based on the callback image frame D (obtained by converting the image format of the callback image frame C, see S308). When the recognition module performs two-dimensional code positioning based on the callback image frame D, the two-dimensional code positioning result (i.e., the display position of the scan code positioning mark 1041 in the image) is related to the position of the two-dimensional code image after the distortion correction in the blank image frame. If the image processing module adds the two-dimensional code image after the distortion correction to the top left corner of the blank image frame, the display position of the scan code positioning mark 1041 in the image is also the top left corner of the image. Since the shooting scene corresponding to the preview image frame and the callback image frame A is consistent, and the image capture time is also relatively close, the picture information in the callback image frame A (or the callback image frame B) is almost completely the same as the picture information of the preview image frame, and the difference is very small. Therefore, the two-dimensional code position in the callback image frame A (or the callback image frame B) is also close to the two-dimensional code position in the preview image frame, such as the two-dimensional code center position, which can also be referred to as the two-dimensional code region in the callback image frame A (or the callback image frame B) and the two-dimensional code region in the preview image frame are mostly overlapped, or even completely overlapped. If the image processing module randomly adds the two-dimensional code image after the distortion correction to the blank image frame to generate the callback image frame C, the two-dimensional code position in the callback image frame C will be inconsistent with the two-dimensional code position in the callback image frame B, i.e., inconsistent with the two-dimensional code position in the preview image frame, and further, the scan code positioning mark 1041 determined based on the two-dimensional code position in the callback image frame C will be misaligned with the two-dimensional code image in the preview image frame. That is, the scan code positioning mark 1041 displayed on the scan code positioning interface of the WeChat application will appear in the non-two-dimensional code region, which can be referred to as FIG. 1. Figure 7f

[0183] In the embodiments of the present application, the position of the two-dimensional code image after the distortion correction is consistent with the position of the two-dimensional code image before the distortion correction, such as the two-dimensional code image center points are aligned, and the position of the two-dimensional code image before the distortion correction is consistent with the two-dimensional code position in the corresponding preview image frame. Therefore, when the recognition module of the WeChat SDK performs two-dimensional code positioning and recognition based on the callback image frame D, it can ensure that the display position of the scan code positioning mark 1041 is consistent with the two-dimensional code position in the preview image frame displayed by the WeChat application, so that the scan code positioning mark 1041 is displayed in the two-dimensional code region in the preview image, and the problem of incorrect display position of the scan code positioning mark 1041 in the preview image frame is avoided.

[0184] S308, the image processing module converts the image format of the callback image frame C from RGB to YUV to obtain the callback image frame D, and sends the callback image frame D to the WeChat SDK through the camera service.

[0185] ​In this case, the callback image frame D instead of the callback image frame A is sent to the WeChat SDK, and since the two-dimensional code image in the callback image frame D has been corrected by distortion, the positioning and recognition success rate of the two-dimensional code by the recognition module of the WeChat SDK is improved, and the positioning and recognition efficiency of the two-dimensional code by the recognition module of the WeChat SDK is also accelerated, thereby improving the user experience.

[0186] S309, the image processing module calls the two-dimensional code zoom module to perform zoom processing on the callback image frame B to obtain a callback image frame E.

[0187] When the two-dimensional code image in the callback image frame B meets the zoom condition, the image processing module calls the two-dimensional code zoom module to perform zoom processing on the callback image frame B to enlarge the image proportion of the two-dimensional code image in the callback image frame B, and obtain a callback image frame E.

[0188] For example, the two-dimensional code zoom module can determine a suitable zoom ratio according to the maximum allowed zoom ratio r1 corresponding to the two-dimensional code in the callback image frame, the zoom ratio is smaller than the maximum allowed zoom ratio r1, and perform zoom processing on the callback image frame B according to the zoom ratio to enlarge the image proportion of the two-dimensional code image in the callback image frame B, so that the image proportion of the two-dimensional code image in the callback image frame B is more suitable for scanning requirements, thereby facilitating the positioning and recognition of the two-dimensional code by the recognition module of the WeChat SDK.

[0189] For example, the two-dimensional code zoom module can perform zoom processing on the callback image frame B according to the maximum allowed zoom ratio r1 corresponding to the two-dimensional code in the callback image frame to enlarge the image proportion of the two-dimensional code image in the callback image frame B. In this way, the two-dimensional code image in the callback image frame is enlarged as much as possible under the premise of ensuring the integrity of the two-dimensional code image, so that a certain boundary of the two-dimensional code image is infinitely close to or even coincides with a certain boundary of the callback image frame, thereby facilitating the positioning and recognition of the two-dimensional code by the recognition module of the WeChat SDK.

[0190] S310, the image processing module converts the image format of the callback image frame E from RGB to YUV to obtain a callback image frame F, and sends the callback image frame F to the WeChat SDK through the camera service.

[0191] In this case, the callback image frame F instead of the callback image frame A is sent to the WeChat SDK, and since the two-dimensional code image in the callback image frame F has been enlarged, the positioning and recognition efficiency of the two-dimensional code by the recognition module of the WeChat SDK is accelerated, thereby improving the user experience.

[0192] In an optional embodiment, the image processing module can also call the two-dimensional code zoom module to perform zoom processing on the preview image frame, and display the zoom-processed preview image frame.

[0193] In the embodiment, the image processing module judges whether the two-dimensional code image in the preview image frame satisfies the zoom condition after generating the preview image frame in RGB format. For example, the zoom condition is that the image proportion r0 of the two-dimensional code image in the callback image frame is less than a threshold A. For another example, the zoom condition is that the image proportion r0 of the two-dimensional code image in the callback image frame is less than a threshold A, and the maximum allowed zoom ratio r1 corresponding to the two-dimensional code is greater than a threshold B.

[0194] If the two-dimensional code image in the preview image frame satisfies the zoom condition, the image processing module calls the two-dimensional code zoom module to perform zoom processing on the preview image frame, and sends the preview image frame after zoom processing for display.

[0195] When the callback image frame is zoomed to enlarge the two-dimensional code image, if the preview image frame is not zoomed to enlarge the two-dimensional code image, the display position of the scan code positioning mark 1041 can also be improper, so that the scan code positioning mark 1041 is displayed on the non-two-dimensional code region of the preview image frame.

[0196] In this way, when the callback image frame is zoomed to enlarge the two-dimensional code image, the preview image frame is also zoomed to enlarge the two-dimensional code image, so that when the recognition module of the WeChat SDK locates and recognizes the two-dimensional code based on the callback image frame F after zoom processing, the display position of the scan code positioning mark can be ensured to be consistent with the position of the two-dimensional code in the preview image frame displayed by the WeChat application, and the problem of incorrect display position of the scan code positioning mark in the preview image frame can be avoided.

[0197] S311, the image processing module sends the callback image frame A to the WeChat SDK through the camera service.

[0198] If the two-dimensional code image in the callback image frame B neither satisfies the zoom condition nor satisfies the distortion correction condition, the image processing module directly sends the callback image frame A to the WeChat SDK through the camera service, so that the recognition module of the WeChat SDK locates and recognizes the two-dimensional code based on the callback image frame A.

[0199] In this case, if the image processing module caches the callback image frame A, the image processing module can directly send the callback image frame A in the cache to the WeChat SDK through the camera service. If the image processing module does not cache the callback image frame A, the image processing module converts the image format of the callback image frame B from RGB to YUV to obtain the callback image frame A again, and then sends the callback image frame A to the WeChat SDK through the camera service.

[0200] In the embodiment, continue to refer to Figure 5The callback image frame received by the WeChat SDK is either a zoomed callback image frame, or a callback image frame after distortion correction of the two-dimensional code image, or a callback image frame without zooming and distortion correction processing. Therefore, the callback image frame received by the WeChat SDK is more convenient for the recognition module to locate and recognize the two-dimensional code, thereby improving the success rate of the recognition module of the WeChat SDK in locating and recognizing the two-dimensional code, and accelerating the positioning and recognition efficiency of the recognition module of the WeChat SDK.

[0201] In the code scanning method provided in the embodiments of the present application, the electronic device performs zooming processing or distortion correction processing on the callback image frame before returning the callback image frame to the code scanning application, thereby improving the code scanning success rate of the code scanning application on the callback image frame, solving the problem of long-time unsuccessful code scanning or even code scanning failure due to poor user scanning angle, and improving the user experience.

[0202] On the basis of the foregoing embodiments, Figure 8 Another optional embodiment is exemplarily shown. Taking the third-party application as the WeChat application and the code scanning function as the "Scan" function as an example, as shown in Figure 8 The user opens the "Scan" function of the WeChat application, and the scene recognition module determines that the flag corresponding to the current scene is FLAG = 1. The camera service generates a preview image frame request based on the preview instruction sent by the camera application and sends it to the camera HAL, and generates a callback image frame request based on the instruction information "FLAG = 1" sent by the camera application and sends it to the camera HAL. The Sensor node in the camera HAL controls the camera sensor to expose an image through the camera driver, and obtains preview image data corresponding to the preview image frame request and callback image data corresponding to the callback image frame request. The preview image data can be processed by the image processing module to obtain a preview image frame, and the preview image frame is sent to the WeChat application for display.

[0203] In the embodiment, after the image processing module processes the received callback image data to obtain the callback image frame n, the image processing module further performs distortion correction processing or zooming processing (for example, according to Figure 6The callback image frame n is processed by the processing flow shown in the figure), and the processed callback image frame n is sent to the WeChat SDK through the camera service for QR code positioning and recognition. Regarding the next frame of the callback image frame n, that is, the callback image frame n+1, after the image processing module processes the received callback image data to obtain the callback image frame n+1, the callback image frame n+1 is directly sent to the WeChat SDK through the camera service for QR code positioning and recognition. For the next frame of the callback image frame n+1, that is, the callback image frame n+2, after the image processing module processes the received callback image data to obtain the callback image frame n+2, the image processing module continues to perform distortion correction processing or zoom processing on it (such as according to Figure 6 The processed callback image frame n+2 is sent to the WeChat SDK via the camera service for QR code positioning and recognition. The same process can be applied to other callback image frames.

[0204] Among them, regarding the callback image frame 1, the image processing module can directly send it to the WeChat SDK through the camera service for QR code positioning and recognition, and can also perform distortion correction or zoom processing on it (such as according to Figure 6 The processed callback image frame 1 is sent to the WeChat SDK through the camera service for QR code positioning and recognition, which is not limited in this embodiment.

[0205] In each callback image frame received in sequence by the WeChat SDK recognition interface, Figure 6 The processing flow shown in the figure is used to process the callback image frame, and the image frame is processed as shown in the figure. Figure 6 The callback image frames processed by the processing flow shown appear alternately, and the recognition module of the WeChat SDK recognizes each callback image frame in turn according to the order in which the callback image frames are received, until the positioning and recognition operation of the QR code is completed according to a certain callback image frame.

[0206] It should be pointed out that the identification module of WeChat SDK may be based on Figure 6 The callback image frame N1 processed by the processing flow shown in FIG. 1 completes the positioning and recognition operation of the QR code. At this time, the QR code positioning result output by the recognition module is based on the callback image frame N1. The recognition module of WeChat SDK may also be based on the Figure 6 The processing flow shown in the figure completes the QR code positioning and recognition operation for callback image frame N2. The QR code positioning result output by the recognition module is based on callback image frame N2. After the WeChat SDK recognition module completes the QR code positioning and recognition operation based on a callback image frame, it will not perform the positioning and recognition operation on the next callback image frame.

[0207] aboutFigure 8 For any details not explained in detail in the scanning process shown, please refer to the previous Figure 5 The explanation of the scanning process shown will not be repeated here.

[0208] Thus, for two adjacent callback image frames, the image processing module uses different methods to process one callback image frame. Figure 6 After the process shown in the figure is processed, it is sent to the WeChat SDK for QR code positioning and recognition. Another callback image frame is sent directly to the WeChat SDK for QR code positioning and recognition (that is, it is processed using the native process). This can avoid the WeChat SDK's recognition module being unable to recognize the QR code. Figure 6 The callback image frame after the process shown in the figure will cause the QR code to be unsuccessful for a long time. Once the recognition module of WeChat SDK can recognize the QR code that has not been processed as shown in the figure, the QR code will be unsuccessful for a long time. Figure 6 After the callback image frame is processed by the process shown, the QR code can still be successfully recognized, thereby ensuring the success rate of the code scanning method.

[0209] Optionally, in this embodiment, after the image processing module processes the received callback image data to obtain the callback image frame n, the image processing module further performs distortion correction processing or zoom processing on it (such as according to Figure 6 The callback image frame after the callback image frame n, for example, callback image frame n+m1 (m1 is an integer greater than 1), is processed by the image processing module for processing the received callback image data to obtain callback image frame n+m1, and then the callback image frame n+m1 is directly sent to the WeChat SDK through the camera service for QR code positioning and recognition. Among them, the callback image frames between the callback image frame n and the callback image frame n+m1 are processed in the same way as the callback image frame n, and will not be repeated here.

[0210] For the callback image frame after the callback image frame n+m1, for example, the callback image frame n+m1+m2 (m2 is an integer greater than 1, and m1 and m2 may be equal or unequal), after the image processing module processes the received callback image data to obtain the callback image frame n+m1+m2, the image processing module continues to perform distortion correction processing or zoom processing on it (such as according to Figure 6 The processed callback image frames n+m1+m2 are sent to the WeChat SDK via the camera service for QR code location and recognition. The callback image frames between callback image frames n+m1 and n+m1+m2 are processed in the same way as callback image frame n+m1 and are not further described here.

[0211] The same can be analogized for other callback image frames. Thus, in the callback image frames received by the recognition interface of the WeChat SDK in turn, the callback image frames processed by the processing procedure shown in FIG. 8 and the callback image frames not processed by the processing procedure shown in FIG. 9 appear alternately, and the recognition module of the WeChat SDK recognizes each callback image frame in turn according to the receiving order of the callback image frames until the positioning and recognition of the two-dimensional code are completed according to a callback image frame. In this way, the problem that the recognition module of the WeChat SDK cannot recognize the callback image frame processed by the procedure shown in FIG. 8 and thus the long-time recognition of the two-dimensional code fails can be avoided. Figure 6 Figure 6 Figure 6

[0212] The embodiment further provides a computer storage medium, which stores computer instructions. When the computer instructions run on an electronic device, the electronic device executes the related method steps to implement the code scanning method in the above embodiment.

[0213] The embodiment further provides a computer program product. When the computer program product runs on a computer, the computer executes the related steps to implement the code scanning method in the above embodiment.

[0214] In addition, the embodiment of the present application further provides an apparatus, which can be a chip, a component or a module. The apparatus can include a processor and a memory connected to each other. The memory is used to store computer execution instructions. When the apparatus runs, the processor can execute the computer execution instructions stored in the memory, so that the chip executes the code scanning method in the above method embodiments.

[0215] The electronic device (such as a mobile phone) provided by the embodiment, the computer storage medium, the computer program product or the chip are used to execute the corresponding method provided above, and thus the beneficial effects achieved by the electronic device (such as a mobile phone) provided by the embodiment, the computer storage medium, the computer program product or the chip can refer to the beneficial effects of the corresponding method provided above, which will not be described herein again.

[0216] Through the above description of the implementation manner, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above.

[0217] ​​​In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the division of the apparatus embodiments is only a logical function division, and there can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different apparatuses can be indirect couplings or communication connections through some interfaces, apparatuses or units, and can be in electrical, mechanical or other forms.

[0218] The above merely use the embodiments to describe the technical solutions of the present application, but not to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A code scanning method, characterized in that: include: In response to an operation of enabling a code scanning function of a target application, sequentially acquiring a callback image frame and a preview image frame; Upon acquiring the first callback image frame, performing distortion correction processing on the QR code image in the first callback image frame to generate a second callback image frame corresponding to the first callback image frame, and sending the second callback image frame to the target application for code scanning and recognition; wherein the target application sequentially scans and recognizes each callback image frame in the order in which it is received until a QR code is recognized in one of the callback image frames; When a third callback image frame is acquired, the third callback image frame is sent to the target application for code scanning and recognition; wherein the third callback image frame is a callback image frame subsequent to the first callback image frame; When the fourth callback image frame is acquired, distortion correction processing is performed on the QR code image in the fourth callback image frame to generate a fifth callback image frame corresponding to the fourth callback image frame, and the fifth callback image frame is sent to the target application for code scanning and recognition; wherein the fourth callback image frame is a callback image frame subsequent to the third callback image frame; When the target application recognizes the QR code based on the received target callback image frame, a first preview image frame is displayed, on which a QR code positioning mark is displayed; wherein the QR code positioning mark is determined based on the position of the QR code image in the target callback image frame.

2. The method according to claim 1, characterized in that The third callback image frame is a callback image frame adjacent to the first callback image frame; The fourth callback image frame is a callback image frame adjacent to the third callback image frame.

3. The method according to claim 1 or 2, characterized in that Performing distortion correction processing on the two-dimensional code image in the first callback image frame to generate a second callback image frame corresponding to the first callback image frame includes: Extracting a first QR code image from the first callback image frame; Performing distortion correction processing on the first two-dimensional code image to obtain a second two-dimensional code image; Creating a blank image frame, determining a position of the first QR code image in the first callback image frame, and adding the second QR code image to the blank image frame to obtain the second callback image frame, such that the coordinates of the center point of the second QR code image in the second callback image frame are consistent with the coordinates of the center point of the first QR code image in the first callback image frame; The blank image frame and the first callback image frame have the same image size.

4. The method according to claim 3, characterized in that Performing distortion correction processing on the first two-dimensional code image in the first callback image frame includes: When the first two-dimensional code image in the first callback image frame does not meet the zoom condition, a distortion correction process is performed on the first two-dimensional code image in the first callback image frame.

5. The method according to claim 4, characterized in that Also includes: When the first two-dimensional code image in the first callback image frame meets a zoom condition, performing zoom processing on the first callback image frame to enlarge the two-dimensional code image to obtain a sixth callback image frame; The sixth callback image frame is sent to the target application for code scanning and recognition.

6. The method according to claim 3, characterized in that Performing distortion correction processing on the first two-dimensional code image, including: When the maximum rotation angle of the first two-dimensional code image is greater than a first threshold, performing distortion correction processing on the first two-dimensional code image in the first callback image frame according to perspective transformation; Among them, the maximum rotation angle is the maximum value of the first angle, the second angle, the third angle, and the fourth angle. The first angle is the angle between the left boundary of the first two-dimensional code image and the vertical direction, the second angle is the angle between the upper boundary of the first two-dimensional code image and the horizontal direction, the third angle is the angle between the right boundary of the first two-dimensional code image and the vertical direction, and the fourth angle is the angle between the lower boundary of the first two-dimensional code image and the horizontal direction.

7. The method according to claim 4, characterized in that Performing distortion correction processing on the first two-dimensional code image in the first callback image frame includes: Determining the coordinates of four vertices of the first two-dimensional code image; Determine the side length of the second two-dimensional code image according to the four vertex coordinates; Determine the coordinates of a vertex of the second two-dimensional code image, and calculate the coordinates of the other three vertices of the second two-dimensional code image based on the side length; Calculating a perspective transformation matrix according to the coordinates of the four vertices of the first two-dimensional code image and the coordinates of the four vertices of the second two-dimensional code image; Perform perspective transformation processing on the first two-dimensional code image according to the perspective transformation matrix to obtain the second two-dimensional code image.

8. The method according to claim 4 or 5, characterized in that The zoom conditions include: The image ratio of the QR code image in the callback image frame is less than the second threshold, and the target zoom ratio corresponding to the QR code is greater than the third threshold; The target zoom ratio is used to indicate the maximum zoom ratio allowed when zooming the callback image frame to enlarge the QR code image.

9. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the code scanning method as described in any one of claims 1-8.

10. A computer-readable storage medium comprising a computer program, characterized in that When the computer program runs on an electronic device, the electronic device executes the code scanning method according to any one of claims 1 to 8.

Images