Image processing method and device

Abstract

The present application provides an image processing method and a device. The method comprises: a first electronic device first acquiring a first picture displayed in a viewfinder frame of the first electronic device, wherein the first picture is obtained by a camera of the first electronic device capturing a picture displayed by a second electronic device, and the picture displayed by the second electronic device implicitly contains a graphic code used for information verification; detecting the degree of moire patterns on a first image of the first picture; on the basis of the degree of moire patterns on the first image, acquiring a second image obtained by the camera of the first electronic device capturing the picture displayed by the second electronic device; and finally, on the basis of the second image, extracting the graphic code. In the method, when a first electronic device captures a picture implicitly containing a graphic code, an image for extracting the graphic code is obtained on the basis of the degree of moire patterns appearing on the picture, thereby effectively improving the accuracy and efficiency of extracting graphic codes.

Description

Image processing method and device

[0001] Cross-reference to related applications

[0002] The present application claims priority to the Chinese patent application No. 202411324851.1, filed on September 23, 2024, and entitled “Image processing method and device”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of electronic devices, and in particular to an image processing method and device. BACKGROUND

[0004] The graphic code technology has been applied more and more widely, such as making payment by scanning a graphic code in an electronic payment application, or joining a friend by scanning a graphic code in a social application, or scanning a graphic code for authentication in an electronic device (such as a mobile phone), which can include new machine verification, bracelet pairing, account login, one-key screen projection, etc., or making entry and exit registration by scanning a graphic code in a travel application. Some commonly used graphic codes include two-dimensional codes, bar codes, ring codes, etc.

[0005] The commonly used graphic codes (such as two-dimensional codes and bar codes) are black and white patterns, which are difficult to meet the aesthetic needs. In this regard, in some code scanning application scenarios, an electronic device (such as a smart phone) with image acquisition and graphic code recognition functions can obtain a color image containing a graphic code by scanning / shooting, and extract the graphic code from the shot color image, and then decode (or analyze) the graphic code. After successful decoding, the information or content carried by the graphic code can be obtained, so that related operations can be performed. However, at a specific distance, the high-frequency interference of the light sensing elements of the camera of the electronic device (such as a smart phone) occurs, which causes the shot image to have high-frequency colored stripes, i.e., moire, which can cause the extraction of the graphic code from the image to fail, and further cause the decoding to fail or to require repeated scanning multiple times to successfully decode. As can be seen, the accuracy and efficiency of extracting the graphic code are key factors affecting the subsequent decoding and obtaining of corresponding information or content.

[0006] Therefore, how to improve the accuracy and efficiency of extracting the graphic code has become a problem to be solved at present. SUMMARY

[0007] The present application provides an image processing method and device for improving the accuracy and efficiency of extracting the graphic code.

[0008] In a first aspect, an embodiment of the present application provides an image processing method, which can be applied to, but is not limited to, a first electronic device (or a first terminal device). The first electronic device includes a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a tablet computer, etc.), a vehicle-mounted computer, a smart watch, a television, and other electronic devices with an acquisition device (such as a camera), etc. The method can be executed by the first electronic device or a component (such as a chip, a chip system, a circuit, etc.) corresponding to the first electronic device. Taking the method executed by the first electronic device as an example, the method includes: obtaining, by the first electronic device, a first picture displayed in a framing box of the first electronic device, the first picture being obtained by a camera of the first electronic device from a picture displayed by a second electronic device, and the picture displayed by the second electronic device implicitly containing a graphical code for information verification; detecting, by the first electronic device, a moire degree on a first image of the first picture; obtaining, by the first electronic device, a second image obtained by the camera of the first electronic device from the picture displayed by the second electronic device based on the moire degree on the first image; and extracting, by the first electronic device, the graphical code based on the second image.

[0009] In an embodiment of the present application, the picture displayed by the second electronic device can be a dynamic picture or a static image, which is not limited. The picture displayed by the second electronic device implicitly containing the graphical code for information verification can be understood as that a user or a machine cannot directly obtain the graphical code from the picture without performing multi-frame joint image processing on the picture. The graphical code can be a ring code, a two-dimensional code, a bar code, etc.

[0010] For example, the picture displayed by the second electronic device can be a starry sky ring picture.

[0011] The camera of the first electronic device can acquire the picture displayed by the second electronic device in real time and display the picture through a framing box (or a preview stream) on a screen of the first electronic device in real time.

[0012] The present application scheme can be applied to, but is not limited to, the first electronic device. In the present application scheme, after the first electronic device acquires the picture implicitly containing the graphical code, the first electronic device detects a moire degree on the picture, obtains an image (i.e., the second image) for extracting the graphical code based on the moire degree on the picture, so as to avoid or reduce the influence of the moire on the extraction of the graphical code. The present application scheme can effectively improve the accuracy of extracting the graphical code and improve the efficiency of extracting the graphical code.

[0013] With reference to the first aspect, in a possible implementation manner, the first electronic device detects the Moiré degree on the first image of the first picture, including: first, performing intra-frame difference processing based on the first image to obtain a third image after intra-frame difference processing; then, performing binaryzation processing based on the third image to obtain a fourth image after binaryzation processing; and then, based on the fourth image, counting a proportion of a first region in the fourth image, and determining the proportion of the first region in the fourth image as the Moiré degree on the first image; wherein the first region is a region in the fourth image in which pixel brightness exceeds a set threshold. Through this implementation manner, the Moiré degree on the first image of the first picture can be effectively detected.

[0014] With reference to the first aspect, in a possible implementation manner, the first electronic device detects the Moiré degree on the first image of the first picture, including: first, determining a difference between the distance between the first electronic device and the second electronic device and the Moiré occurrence distance according to the distance between the first electronic device and the second electronic device and the Moiré occurrence distance; and then, determining the Moiré degree on the first image according to the difference between the distance between the first electronic device and the second electronic device and the Moiré occurrence distance; wherein the difference and the Moiré degree on the first image are in an inverse relationship.

[0015] Through this implementation manner, the difference between the distance between the first electronic device and the second electronic device and the Moiré occurrence distance is effectively utilized to detect the Moiré degree on the first image of the first picture.

[0016] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device acquires binocular disparity information; and determines the distance between the first electronic device and the second electronic device according to the binocular disparity information; or the first electronic device acquires size information of the first picture; and determines the distance between the first electronic device and the second electronic device according to the size information of the first picture; or the first electronic device measures the distance between the first electronic device and the second electronic device through a ranging device of the first electronic device. Through this implementation manner, the distance between the first electronic device and the second electronic device can be effectively and accurately obtained.

[0017] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device acquires information of the second electronic device, the information of the second electronic device including pixels and / or size of a screen of the second electronic device; and the first electronic device determines the Moiré occurrence distance according to information of the first electronic device and information of the second electronic device; wherein the information of the first electronic device includes at least one of pixels, size of a screen of the first electronic device, and camera spatial resolution of the first electronic device. Through this implementation manner, the Moiré occurrence distance can be effectively and accurately obtained.

[0018] With reference to the first aspect, in a possible implementation manner, the first electronic device obtains, based on the Moiré degree on the first image, a second image obtained by the camera of the first electronic device capturing a second picture displayed by the second electronic device, including: when the Moiré degree on the first image exceeds a set threshold, adjusting the camera setting information of the first electronic device; obtaining the second image based on a second picture in the viewfinder, the second picture being obtained by the camera of the first electronic device capturing the picture displayed by the second electronic device after the adjustment; and when the Moiré degree on the first image does not exceed the set threshold, obtaining the second image based on the first picture. Through this implementation manner, it can be ensured that the Moiré degree on the image used for extracting the graphic code is weak or there is no Moiré, so that the accuracy and efficiency of subsequent extraction of the graphic code can be ensured.

[0019] With reference to the first aspect, in a possible implementation manner, the camera setting information includes a focusing distance and / or an aperture value; and the first electronic device adjusts the camera setting information of the first electronic device, including: adjusting a first focusing distance of the camera of the first electronic device to a second focusing distance according to a preset corresponding relationship between the image distance and the object distance; wherein the first focusing distance is a focusing distance obtained by automatic focusing of the camera, and a difference between the first focusing distance and the second focusing distance is determined according to a distance between the first electronic device and the second electronic device; and / or adjusting a first aperture value of the camera of the first electronic device to a second aperture value, the first aperture value being an aperture value obtained by automatic setting of the camera, and the second aperture value being smaller than the first aperture value. Through this implementation manner, the adjustment of the camera setting information can be effectively realized to achieve the purpose of reducing the Moiré degree (or eliminating the Moiré) on the first picture.

[0020] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device detecting that the Moiré degree on the fifth image of the second picture does not exceed the set threshold. Through this implementation manner, it can be ensured that the Moiré degree on the image obtained after the adjustment for extracting the graphic code is weak or there is no Moiré, so that the accuracy and efficiency of subsequent extraction of the graphic code can be ensured.

[0021] With reference to the first aspect, in a possible implementation manner, the second image is at least two continuous images; and the first electronic device performs extraction of the graphic code based on the second image, including: performing intra-frame difference processing and inter-frame difference processing based on the at least two images to obtain at least two corresponding gray-scale images after difference processing of the images; then performing binaryzation processing based on the at least two corresponding gray-scale images after difference processing of the images to obtain at least two corresponding images after binaryzation processing of the images; and then obtaining the graphic code based on the at least two corresponding images after binaryzation processing of the images. Through this implementation manner, an accurate graphic code can be effectively obtained.

[0022] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device obtaining sensor information of the first electronic device and sensor information of the second electronic device, the sensor information including inertial measurement unit (IMU) sensor information; and the first electronic device adjusting the camera setting information of the first electronic device according to the sensor information of the first electronic device and the sensor information of the second electronic device, and collecting the screen displayed by the second electronic device through the adjusted camera of the first electronic device, when the distance between the first electronic device and the second electronic device is the same as the distance at which moire occurs or the difference is less than a preset threshold. Through this implementation manner, the first electronic device can use the ranging capability to know the degree of moire on the subsequently collected screen in advance, so as to adjust the camera settings in advance to reduce the time required by the first electronic device to detect the degree of moire on the subsequently collected screen and / or adjust the camera setting information, thereby further improving the efficiency of extracting the graphical code.

[0023] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device sending device information of the first electronic device to the second electronic device, and receiving device information of the second electronic device from the second electronic device; and wherein the device information includes, but is not limited to, one or more of the following:

[0024] trigger information for starting the verification process, type of the device, version and model of the device.

[0025] Optionally, the implementation manner can be performed before the first electronic device obtains the first screen displayed in the viewfinder frame of the first electronic device.

[0026] Through this implementation manner, the first electronic device and the second electronic device can obtain the device information of each other to start the authentication or pairing process of the devices.

[0027] In a second aspect, the embodiments of the present application provide a device for image processing, which includes a plurality of functional modules or units (for example, a communication module, a storage module (optional), a display module (optional), a processing module, a collection module / camera module, etc.); the plurality of functional modules or units interact with each other to implement the method performed by the first electronic device in the first aspect and any possible implementation manner thereof. The plurality of functional modules or units can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of functional modules or units can be arbitrarily combined or divided based on specific implementation.

[0028] In a third aspect, an embodiment of the present application provides a device for image processing, the device comprising at least one processor; the at least one processor is coupled with at least one memory, and when the device is running, the at least one processor is configured to read a computer program or instructions stored in the at least one memory, so as to execute the method performed by the first electronic device in the first aspect and any possible implementation manner thereof.

[0029] In a possible design, the device further comprises the at least one memory.

[0030] In a fourth aspect, an embodiment of the present application provides a program product, which, when running on a device, causes the device to perform the method performed by the first electronic device in any of the aspects and any possible implementation manner thereof.

[0031] In a fifth aspect, an embodiment of the present application provides a readable storage medium, which stores a program, and when the program is executed by a device, causes the device to perform the method performed by the first electronic device in any of the aspects and any possible implementation manner thereof.

[0032] In a sixth aspect, an embodiment of the present application provides a chip, which is configured to read a program stored in a memory, and perform the method performed by the first electronic device in any of the aspects and any possible implementation manner thereof.

[0033] In a seventh aspect, an embodiment of the present application provides a chip system, which comprises a processor configured to support a device to implement the method performed by the first electronic device in any of the aspects and any possible implementation manner thereof. In a possible design, the chip system further comprises a memory configured to store necessary programs and data. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0034] It should be noted that the technical effects achieved by the second aspect to the seventh aspect or any possible implementation manner of the second aspect to the seventh aspect can be explained with reference to the technical effects achieved by the first aspect and any possible implementation manner thereof, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0035] FIG. 1 is a schematic diagram of superimposition of a sampling pixel;

[0036] FIG. 2 is a schematic diagram of a code scanning authentication scenario;

[0037] FIG. 3 is a schematic diagram of a system architecture to which the method provided by an embodiment of the present application is applicable;

[0038] FIG. 4 is a schematic diagram of a possible hardware structure of an electronic device according to an embodiment of the present application;

[0039] FIG. 5 is a schematic diagram of a possible software structure of an electronic device according to an embodiment of the present application;

[0040] FIG. 6A is a schematic diagram of an application interface for mobile phone scan code authentication according to an embodiment of the present application;

[0041] FIG. 6B is a schematic diagram of an application interface for mobile phone scan code authentication according to an embodiment of the present application;

[0042] FIG. 7 is a schematic diagram of a process of real-time detection of a picture and adjustment of camera settings in a scan authentication stage according to an embodiment of the present application;

[0043] FIG. 8A is a schematic diagram of a process of annular code (i.e. pattern code) extraction according to an embodiment of the present application;

[0044] FIG. 8B is a schematic diagram of three regions of interest (ROIs) according to an embodiment of the present application;

[0045] FIG. 8C is a schematic diagram of three gray scale images after intra-frame difference processing according to an embodiment of the present application;

[0046] FIG. 8D is a schematic diagram of three gray scale images after inter-frame difference processing according to an embodiment of the present application;

[0047] FIG. 8E is a schematic diagram of three images after binarization processing according to an embodiment of the present application;

[0048] FIG. 9 is a schematic diagram of a flow of an image processing method according to an embodiment of the present application;

[0049] FIG. 10 is a schematic diagram of a method flow according to an embodiment of the present application;

[0050] FIG. 11A is a schematic diagram of a corresponding process in a case where moire exists (or the degree of moire is relatively serious) according to an embodiment of the present application;

[0051] FIG. 11B is a schematic diagram of a corresponding process in a case where moire does not exist (or the degree of moire is relatively weak) according to an embodiment of the present application;

[0052] FIG. 12 is a schematic diagram of a curve of a corresponding relationship between image distance and object distance according to an embodiment of the present application;

[0053] FIG. 13 is a schematic diagram of a flow of performing pattern code extraction by adjusting camera settings according to an embodiment of the present application;

[0054] FIG. 14 is a schematic diagram of a method flow according to an embodiment of the present application;

[0055] FIG. 15 is another flow diagram for adjusting camera settings to perform QR code extraction according to the present disclosure. DETAILED DESCRIPTION

[0056] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0057] The names and related technical features involved in the embodiments of the present application will be explained first. It should be noted that these explanations are to make the embodiments of the present application easier to understand, and should not be regarded as limiting the scope of protection required by the present application.

[0058] 1) Electronic device: The electronic device (also referred to as terminal device) can include a mobile phone, a tablet computer, a notebook computer, a personal digital assistant (PDA), a point of sales (POS), a vehicle-mounted computer, a smart watch, a television, and other electronic devices with a display screen, etc.

[0059] Generally, the electronic device can support multiple applications. For example, one or more of the following applications: travel application, drawing application, presentation application, word processing application, video player application, shopping application, instant messaging application, photo management application, camera application, browser application, payment application, and health management application, etc. Among them, the instant messaging application can be various. For example, short message application, various email applications, WeChat, Instagram, DingTalk, etc. The user can send text, voice, picture, video file and other various files and other information to other contacts through the instant messaging application. The electronic device supports the above-mentioned applications, which can be native application programs or fast applications, and the embodiments of the present application do not make specific limitations. The electronic device can also support fast service cards corresponding to these applications, such as cards corresponding to order logistics services in the shopping application. In addition to supporting fast service cards corresponding to installed applications, the electronic device can also support fast service cards corresponding to uninstalled applications.

[0060] 2) Pixel: refers to a smallest unit in an image represented by a sequence of numbers, called a pixel.

[0061] Pixels refer to small squares composed of images, and each small square has a specific position and assigned color value. The color and position of the small squares determine the appearance of the image.

[0062] A pixel can be considered as an indivisible unit or element in the whole image. Indivisible means that it cannot be cut into smaller units or elements, and it is a small grid with a single color. Each dot matrix image contains a certain amount of pixels, which determines the size of the image on the screen.

[0063] 3) Frame: Generally, a frame is the basic unit of video and animation, which represents a certain moment in time of an image. In digital video, a frame is a still image, and these images are played continuously at a certain speed to form a dynamic picture.

[0064] 4) Moire: From the perspective of optical interference, moire is a high-frequency interference that occurs on the light-sensitive elements of digital cameras or scanners and other devices, which can cause the picture to appear colored high-frequency stripes. In simple terms, it is a wave interference phenomenon between different objects. Because it is irregular, there is no obvious shape rule. Moire is a manifestation of the beat principle, that is, two frequency close to the amplitude of the sine wave superimposed, the amplitude of the synthesized signal will change according to the difference between the two frequencies. When a digital camera or a mobile phone camera captures an object, this phenomenon occurs because the pixel arrangement of the object being photographed, such as a liquid crystal display, interferes with the pixel arrangement of the mobile phone. The composite light emitted by the projector is reflected by the screen to the camera of the mobile phone or camera, and interference occurs at the camera. Inside the device, the light passes through the lens, filter, and reaches the photodiode, and continues to interfere, resulting in the generation of moire.

[0065] For example, as shown in FIG. 1, (a) of FIG. 1 shows a screen being photographed, such as the screen of a mobile phone, or the screen of a projector, or the screen of a computer, etc.; (b) of FIG. 1 shows the sampling result of the mobile phone or camera sensor, when they are superimposed, that is, when the user is ready to take a picture, as shown in (c) of FIG. 1. The foregoing explains why different moire patterns are presented after adjusting the shooting angle. However, if a film camera without pixels is used for shooting, there is no picture as shown in (b) of FIG. 1, so no matter how the interlacing is, no irregular interference stripes will appear.

[0066] 5) Graphic code: A graphic code is a kind of information conversion into a simple graphic, which is used to identify and track goods, and can also be used to convey certain information or emotions, etc.

[0067] Specifically, for specific information, the information can be converted into a digital code, and then processed into a graphic to generate a special graphic that can be recognized by a machine. This special graphic can be called a graphic code.

[0068] For a certain information, the corresponding graphic code can be equivalent to a graphic obtained after encoding the information. In other words, the corresponding information can be obtained by decoding / analyzing the graphic code. In the embodiments of the present application, the graphic code can include a two-dimensional code, a bar code, a ring code, and the like.

[0069] 6) Interface: The interface can refer to a plane in the display screen of the electronic device that can be used to present images, videos, texts, information, and the like to the user. The visual style (such as shape, structure, size, and the like) of the interface can be fixed or can be flexibly set by the user.

[0070] The content displayed by the interface, such as images, texts, rich media, and the like, wherein the rich media refers to an information transmission method with animation, sound, video, and / or interactivity. The rich media can include one or a combination of several forms of streaming media, sound, Flash, and Java, Javascript, DHTML, and the like programming languages.

[0071] 7) Visual style: The visual style (referred to as "style" for short) can refer to the visual perception of a thing presented to a person, including the shape, structure, size, color, motion, and the like of the thing.

[0072] In the embodiments of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a and b, a and c, b and c, or a, b, and c, wherein a, b, and c can be single or multiple.

[0073] In addition, in the description of the present application, the words "first", "second", and the like, or the words "1", "2", and the like (except for special cases indicating numerical values) are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order. For example, the first electronic device and the second electronic device only represent two different electronic devices, and do not represent the relative importance between the two electronic devices. For another example, distance 1 (i.e., the distance between the two mobile phones) and distance 2 (i.e., the distance at which moire occurs) only represent two different distances, and do not represent the relative importance of the two distances.

[0074] The foregoing introduces some terms involved in the embodiments of the present application. The following introduces the technical background, application, and device involved in the embodiments of the present application.

[0075] The graphic code technology has been increasingly widely applied. For example, in an electronic payment application, payment is made by scanning a graphic code, or in a social application, a friend is added by scanning a graphic code, or in an electronic device (for example, a mobile phone) code scanning authentication application, which can include new machine verification, bracelet pairing, account login, one-key screen projection, etc., or in a travel application, entry and exit registration is made by scanning a graphic code. Some commonly used graphic codes include a two-dimensional code, a bar code, and a ring code, etc.

[0076] A commonly used graphic code (for example, a two-dimensional code or a bar code) is a black and white pattern, which is difficult to meet the aesthetic demand. In this regard, in some code scanning application scenarios, an electronic device (such as a smart phone) with image acquisition and graphic code recognition functions can obtain a color image containing a graphic code by scanning / shooting, and extract the graphic code from the shot color image, and then decode (or analyze) the graphic code. After successful decoding, the information or content carried by the graphic code can be obtained, so that related operations can be performed.

[0077] For example, the future of the Hongmeng ecological system can introduce a hidden code connection experience, realizing the use of an old mobile phone to scan a hidden code animation played by a new mobile phone to perform boot device authentication and accelerate the mobile phone cloning experience. The specific process includes: as shown in FIG. 2, after the new mobile phone is booted, a boot welcome is displayed, and then a broadcast is sent. After receiving the broadcast, the old mobile phone sends a confirmation indication of migration to the new mobile phone. After receiving the confirmation indication, if it is confirmed to accept the migration from the old mobile phone, the new mobile phone will proceed to the authentication stage of the new and old mobile phones.

[0078] In the authentication stage of the new and old mobile phones, a delicate animation such as a star river ring will be displayed on the screen of the old mobile phone to replace the traditional scanning of a two-dimensional code scheme, creating a high-end and delicate user experience. The new mobile phone scans the star river ring and extracts a sparse ring code from the star river ring to realize the authentication of the new and old mobile phones. After the new mobile phone is authenticated successfully, a high-speed communication connection can be established between the two mobile phones and data transmission services can be started, and data cloning can be performed. The old mobile phone quickly imports or transmits data to the new mobile phone.

[0079] However, at a specific distance, high-frequency interference occurs in the photosensitive elements of the camera of the electronic device (such as a mobile phone), causing colored high-frequency stripes, i.e., moire, in the shot picture, which can cause the extraction of the graphic code from the image to fail, and further cause decoding failure or repeated scanning multiple times to successfully decode. As can be seen, the accuracy and efficiency of extracting the graphic code are key factors affecting the subsequent decoding and obtaining of corresponding information or content.

[0080] Therefore, the image processing method provided in the embodiments of the present application can effectively improve the accuracy and efficiency of extracting the graphic code. The method and the device / apparatus in the embodiments of the present application are based on the same inventive concept. Since the principles of the method and the device / apparatus for solving problems are similar, the implementation of the device / apparatus and the method can be referred to each other, and the repeated parts will not be described herein.

[0081] The technical solutions in the embodiments of the present application can be applied to, but are not limited to, electronic devices, which can be any device capable of displaying an interface and / or having a photographing function. For example, the electronic device can be an electronic device such as a mobile phone, a foldable-screen mobile phone, a tablet computer, a wearable device (for example, a watch, a bracelet, etc.), a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart home device (for example, a smart television, etc.), a projector, etc. It can be understood that the embodiments of the present application do not make any limitation on the specific type of the electronic device. The electronic device to which the embodiments of the present application can be applied includes, but is not limited to, an electronic device carrying an Android operating system or other operating systems. For example, the electronic device can be the electronic device introduced in the following embodiments.

[0082] The technical solutions in the embodiments of the present application can be applied to various scenarios of sending a graphic code scan code that is not detected by the human eye by using a light-emitting array screen (i.e., the human eye can see the screen display, but cannot see the graphic code). For example, the technical solutions in the embodiments of the present application can be applied to scenarios such as a mobile phone camera scanning a mobile phone screen, a mobile phone camera scanning a tablet computer screen, a mobile phone camera scanning a personal computer (PC) screen, a mobile phone camera scanning a watch screen, a tablet computer camera scanning a mobile phone screen, a tablet computer camera scanning a tablet computer screen, a tablet computer camera scanning a PC screen, a tablet computer camera scanning a watch screen, etc.

[0083] ​FIG. 3 shows a system architecture diagram to which the method provided by the embodiments of the present application can be applied. Referring to FIG. 3, the system architecture can include a first electronic device and a second electronic device. In the embodiments of the present application, the first electronic device and the second electronic device can use wireless communication technology (for example, wireless fidelity (Wi-Fi), Bluetooth, star flash, near field communication (NFC), etc.) to interact with each other. The first electronic device and the second electronic device can also use wired communication technology to interact with each other.

[0084] In a case of a user demand, the user can hold the first electronic device to take a picture of the second electronic device and a picture displayed on the display screen of the second electronic device. The first electronic device includes a device capable of achieving photographing (photographing and / or video recording), for example, a camera. The second electronic device includes a device capable of displaying an interface, for example, a display screen (which can also be referred to as a screen of an electronic device). Optionally, the first electronic device can also include a device capable of displaying an interface, and the second electronic device can also include a device capable of achieving photographing (photographing and / or video recording).

[0085] In another case of a user demand, the user can hold the second electronic device to take a picture of the first electronic device and a picture displayed on the display screen of the first electronic device. The first electronic device includes a device capable of displaying an interface, for example, a display screen (which can also be referred to as a screen of an electronic device). The second electronic device includes a device capable of achieving photographing (photographing and / or video recording), for example, a camera. Optionally, the first electronic device can also include a device capable of achieving photographing (photographing and / or video recording), and the second electronic device can also include a device capable of displaying an interface.

[0086] The above-mentioned FIG. 3 is an example of a system architecture to which the embodiments of the present application can be applied. In actual applications, the embodiments of the present application do not limit the specific scenarios of the system architecture. For example, compared with the system architecture shown in FIG. 3, the system architecture to which the embodiments of the present application can be applied in actual applications can include more or fewer electronic devices.

[0087] Exemplarily, for the first electronic device or the second electronic device in the system shown in FIG. 3, FIG. 4 shows a possible hardware structure diagram of the electronic device according to the embodiments of the present application. Referring to FIG. 4, the electronic device 400 includes a power supply 410, a processor 420, a memory 430, an input module 440, a display module 450, an audio circuit 460, a communication interface 470, a camera 480, and the like. Those skilled in the art can understand that the hardware structure of the electronic device 400 shown in FIG. 4 does not constitute a limitation on the electronic device 400, and the electronic device 400 provided by the embodiments of the present application can include more or fewer components than those shown in the figure, can combine two or more components, or can have a different component configuration. The various components shown in FIG. 4 can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0088] The various constituent components of the electronic device 400 will be specifically introduced below in combination with FIG. 4:

[0089] The power supply 410 (such as a battery) is used to supply power to various components. Optionally, the power supply 410 can be logically connected to the processor 420 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption through the power management system.

[0090] The processor 420 is the control center of the electronic device 400, and connects various components through various interfaces and lines, executes various functions of the electronic device 400 and processes data by running or executing software programs and / or modules stored in the memory 430 and calling data stored in the memory 430, so as to realize various services based on the electronic device 400. In the embodiments of the present application, the processor 420 can be used to implement an image processing method provided by the embodiments of the present application.

[0091] The memory 430 can be used to store software programs and modules. The processor 420 executes various function applications and data processing of the electronic device 400 by running the software programs and modules stored in the memory 430. Optionally, the memory 430 can mainly include a program storage area and a data storage area. The program storage area can store an operating system (mainly including respective software programs or modules of a kernel layer, a system layer, an application program framework layer, and an application program layer, etc.). In addition, the memory 430 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage device.

[0092] The input module 440 can be configured to receive editing operations of various types of data objects such as digital or character information input by a user, and to generate key signal inputs related to user settings and function control of the electronic device 400. Optionally, the input module 440 can include a touch panel 441 and other input devices 442. The touch panel 441, which can also be referred to as a touch screen, can collect touch operations of a user thereon or adjacent thereto (such as operations of a user using a finger, a stylus, or any suitable object or accessory on or adjacent to the touch panel 441), and drive corresponding connection devices according to a pre-set program.

[0093] Optionally, the other input devices 442 can include, but are not limited to, one or more of a physical keyboard, an infrared sensor, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, a joystick, and the like. For example, the infrared sensor can be used to obtain a user's air gesture operation.

[0094] The display module 450 (which can also be referred to as a display screen or a display unit or a presentation unit) can be configured to display information input by a user or information provided to the user, as well as various menus of the electronic device 400. The display module 450 is a display system of the electronic device 400, and is configured to present an interface to realize human-computer interaction. The display module 450 can include a display panel 451. Optionally, the display panel 451 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0095] In embodiments of the present application, the display module 450 can be configured to display, but is not limited to, a picture / pattern captured by the camera 480.

[0096] In embodiments of the present application, if the electronic device 400 is a scanned / pictured electronic device, the display module 450 is included; if the electronic device 400 is a scanning / picturing electronic device, the display module 450 can be included or not included, and this is not limited.

[0097] The audio circuit 460, the microphone 461 and the speaker 462 can be used to provide an audio interface between a user and the electronic device 400. The audio circuit 460 can be used to convert audio data into a signal recognizable by the speaker 462 and transmit the signal to the speaker 462 for conversion into an audible signal output. The microphone 461 is used to collect external sound signals (such as human speech or other sounds) and convert the collected external sound signals into a signal recognizable by the audio circuit 460 and send the signal to the audio circuit 460. The audio circuit 460 can also be used to convert the signal sent by the microphone 461 into audio data and output the audio data to the RF circuit for transmission to, for example, another electronic device or to the memory 430 for subsequent further processing.

[0098] The electronic device 400 can be physically connected to other devices through the communication interface 470. Optionally, the communication interface 470 is connected to the communication interface of the other devices through a cable to achieve data transmission between the electronic device 400 and the other devices.

[0099] In the embodiments of the present application, the electronic device 400 can implement communication services and interact with other electronic devices, so the electronic device 400 needs to have a data transmission function. The electronic device 400 internally includes not only the communication interface 470 shown in FIG. 4, but also other communication modules, which are not listed one by one here.

[0100] In addition, in the embodiments of the present application, if the electronic device 400 is a scanning / shooting electronic device, the electronic device 400 can further include at least one camera 480 and at least one sensor, although the sensors are not shown in FIG. 4 and are not listed one by one here. The at least one sensor can include but is not limited to a pressure sensor, an air pressure sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a touch sensor, a temperature sensor, etc. If the electronic device 400 is a scanned / pictured electronic device, the electronic device 400 can further include at least one camera 480 and / or at least one sensor, or the electronic device 400 can not include at least one camera 480 and / or at least one sensor, which is not specifically limited.

[0101] The operating system (OS) involved in the embodiments of the present application is the most basic system software running on the electronic device 400. The software system of the electronic device 400 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take the operating system adopting a layered architecture as an example to exemplarily illustrate the software architecture of the electronic device 400.

[0102] It should be understood that the electronic device 400 shown in FIG. 4 is merely an example and does not constitute a specific limitation on the structure of the electronic device in the embodiments of the present application. The electronic device 400 can have more or fewer components than those shown in FIG. 4, can combine two or more components, or can have a different configuration of components. The various components shown in FIG. 4 can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0103] The software system of the electronic device 400 provided by the embodiments of the present application can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take the layered architecture as an example, and FIG. 5 shows a possible software structure diagram of the electronic device according to the embodiments of the present application.

[0104] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. As shown in FIG. 5, the software architecture can be divided into four layers, from top to bottom, application layer, application framework layer (FWK), system library, and kernel layer. In addition, the electronic device can also include a hardware layer.

[0105] The application layer is the uppermost layer of the operating system, including native applications of the operating system, and third-party applications, such as camera, gallery, calendar, Bluetooth, music, video, information, and the like.

[0106] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer can include some pre-defined functions. The application framework layer can include a window manager, a content provider, a view system, a resource manager, a notification manager, and the like.

[0107] The window manager is used to provide window management services (WMS). The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and take screenshots, etc. The window manager can provide multiple management functions to control the transparency, position, and size of windows or interfaces in the display screen, and the like.

[0108] The content provider is used to store and obtain data, and make the data accessible to applications. The data can include files (such as documents, videos, images, audio), text, and the like.

[0109] The view system includes visual controls, such as controls that display text, pictures, documents, and the like. The view system can be used to build an application. The interface in a display window can be composed of one or more controls. For example, a display interface that displays file icons can include controls that display text and controls that display pictures.

[0110] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (e.g., OpenGL ES), a two-dimensional graphics engine (e.g., SGL), an image processing library, and the like. The surface manager is used to manage the display subsystem and provides a fusion of two-dimensional and three-dimensional layers for a plurality of applications. The media library supports playback and recording of a plurality of commonly used audio, video formats, and static image files. The media library can support a plurality of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like. The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, composition, and layer processing. The two-dimensional graphics engine is a drawing engine for two-dimensional drawing.

[0111] The kernel layer provides core system services of the operating system, such as security, memory management, process management, network protocol stack, and a driver model, which are implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stack. This layer has many device-related drivers, including display drivers, keyboard drivers as input devices, Flash drivers based on memory technology devices, camera drivers, audio drivers, Bluetooth drivers, Wi-Fi drivers, and the like.

[0112] The hardware layer can include various sensors, display screens, cameras, and the like.

[0113] It should be noted that FIG. 5 is merely an example of a software structure diagram of an electronic device, and only some levels and software modules are simply listed. In actual applications, the operating system of the electronic device can also include other levels and each level can also include other software modules for implementing one or more functions or services. The embodiments of the present application do not make specific limitations on this. In addition, the embodiments of the present application do not limit the specific level of each software module.

[0114] It should be understood that the hardware structure of the electronic device in the embodiments of the present application can be as shown in FIG. 4, and the software system architecture of the electronic device can be as shown in FIG. 5, wherein the software program and / or module corresponding to the software system architecture in the electronic device can be stored in the memory 430, and the processor 420 can run the software program and application stored in the memory 430 to execute the flow of the image processing method provided by the embodiments of the present application.

[0115] The following describes the application effect achieved by the method provided in the embodiments of the present application from the perspective of the effect exhibited by the interface, taking the mobile phone 1 and the mobile phone 2 as examples of the first electronic device and the second electronic device based on the schematic diagram of the system architecture shown in FIG. 3. It should be noted that the screens presented by the interface of the electronic device are exemplary displays of the application effect of the embodiments of the present application, and cannot limit the actual presented screens and visual styles.

[0116] Example 1: FIGS. 6A-6B show an example of the application interface of the mobile phone scan code authentication according to the embodiments of the present application.

[0117] The mobile phone 1 and the mobile phone 2 have opened the communication functions such as Bluetooth and WLAN, as shown in 6A, the mobile phone 1 automatically starts the connection function, and the interface 601 of the mobile phone 1 shows a schematic diagram of device connection, searching for nearby devices; when the mobile phone 1 searches for the nearby mobile phone 2, the mobile phone 1 automatically starts the scan authentication function, at this time the screen of the mobile phone 1 displays a scan authentication viewfinder, as shown in interface 602; after the mobile phone 2 responds to the search of the mobile phone 1, the screen of the mobile phone 2 displays the interface 603 of the connection verification, and a dynamic starry sky ring picture is displayed in the interface 603. Then, the user can use the mobile phone 1 to scan the mobile phone 2, that is, the mobile phone 1 scans the starry sky ring picture displayed in the interface 603 of the mobile phone 2.

[0118] Further, as shown in (1) of FIG. 6B, the mobile phone 1 scans the starry sky ring picture displayed by the mobile phone 2, at this time the preview stream displayed on the screen of the mobile phone 1 displays the mobile phone 2 and the interface of the mobile phone 2 in real time, and the user can view the interface of the mobile phone 2 and the starry sky ring picture therein in real time through the preview stream of the mobile phone 1, and the viewfinder (ring-shaped circle) displayed on the screen of the mobile phone 1 can be used to real-time circle and display the starry sky ring picture in the interface of the mobile phone 2.

[0119] Example 2: FIG. 7 shows the process of real-time detection of the picture and adjustment of the camera settings in the scan authentication stage according to the embodiments of the present application.

[0120] In the embodiments of the present application, the mobile phone 1 can capture the starry sky ring picture displayed by the mobile phone 2 in the viewfinder in real time, and detect whether there is a moire on the starry sky ring picture. If the mobile phone 1 detects that there is a moire (or the degree of the moire is relatively serious, for example, the degree of the moire is higher than a set threshold) on the starry sky ring picture displayed in the viewfinder, the mobile phone 1 can automatically adjust the camera settings (for example, the focal length value or the aperture value of the camera) of the mobile phone 1 to eliminate the moire or reduce the degree of the moire, and then the mobile phone 1 extracts the ring code from the starry sky ring picture displayed in the viewfinder. If the mobile phone 1 detects that there is no moire (or the degree of the moire is relatively light, for example, the degree of the moire is not higher than the set threshold) on the starry sky ring picture displayed in the viewfinder, the mobile phone 1 can directly extract the ring code from the starry sky ring picture displayed in the viewfinder without adjusting the camera settings of the mobile phone 1.

[0121] For example, referring to FIG. 7, taking the image of a photographed frame of the mobile phone 2 scanned by the mobile phone 1 as an example, if it is detected that there is a moire on the image, as shown in (a) of FIG. 7, the mobile phone 1 can automatically adjust the camera settings (for example, the focal length value or the aperture value of the camera) of the mobile phone 1 to reduce the definition of the photographed picture. After adjusting the camera settings of the mobile phone 1, the mobile phone 1 captures the starry sky ring picture displayed in the viewfinder, as shown in (b) of FIG. 7, and detects whether there is a moire on the adjusted image. If it is detected that there is no moire or the degree of the moire is relatively light on the image, the process of extracting the ring code can be performed. If the mobile phone 1 detects that there is no moire (or the degree of the moire is relatively light) on the starry sky ring picture displayed in the viewfinder, as shown in (c) of FIG. 7, the mobile phone 1 can directly perform the process of extracting the ring code based on the starry sky ring picture displayed in the viewfinder.

[0122] Example Three: In the stage of extracting the ring code, FIG. 8A shows a schematic diagram of extracting the ring code (which is an example of the graphic code) according to the embodiments of the present application.

[0123] The mobile phone 1 scans the starry sky ring picture displayed by the mobile phone 2, and the mobile phone 1 obtains images of a plurality of photographed frames based on the starry sky ring picture displayed on the screen of the mobile phone 1. FIG. 8A shows a schematic diagram of the mobile phone 1 scanning the image of a photographed frame of the mobile phone 2. First, the circular ring region of each photographed frame is captured based on the image of each photographed frame, as shown in (1) of FIG. 8A. Then, the intra-frame difference and the inter-frame difference of the plurality of photographed frames are processed to obtain the difference results of the images of the plurality of photographed frames, as shown in (2) of FIG. 8A. Next, the difference results of the images of the plurality of photographed frames are binarized to obtain the corresponding binarized images, as shown in (3) of FIG. 8A. Finally, the binarized images of the plurality of photographed frames are used to obtain images for performing the extraction of the positioning points, as shown in (4) of FIG. 8A.

[0124] For example, taking the continuous 3 frames of images scanned by the mobile phone 1 from the mobile phone 2 as an example, as shown in FIG. 8B, three region of interest (ROI) images, i.e., image 1, image 2 and image 3, the process of ring code extraction includes the following steps:

[0125] Step 1: first, the three images are subjected to intra-frame difference processing to obtain three gray images after intra-frame difference processing, as shown in FIG. 8C, wherein image 1 corresponds to image 1 after intra-frame difference processing, image 2 corresponds to image 2 after intra-frame difference processing, and image 3 corresponds to image 3 after intra-frame difference processing.

[0126] Step 2: the three images are subjected to inter-frame difference processing to obtain three gray images after inter-frame difference processing, as shown in FIG. 8D, wherein image 1 corresponds to gray image 1, image 2 corresponds to gray image 2, and image 3 corresponds to gray image 3. The three gray images also need to be subjected to anti-shake alignment, which will not be described in detail here.

[0127] Step 3: the three gray images are subjected to binarization processing to obtain corresponding images after binarization processing, as shown in FIG. 8E(1), gray image 1 corresponds to image BW1 after binarization processing, gray image 2 corresponds to image BW2 after binarization processing, and gray image 3 corresponds to image BW3 after binarization processing.

[0128] Step 4: based on image BW1, image BW2 and image BW3, an image without (or with fewer) noise points is obtained by processing through an image processing algorithm, as shown in FIG. 8E(2).

[0129] Finally, the sparse ring code shown in FIG. 8E(2) is decoded / analyzed to obtain corresponding information or content.

[0130] It should be noted that in the embodiments of the present application, the mobile phone is not limited to automatically adjusting the camera settings in the case of detecting the presence of moire patterns, and the user can manually adjust the camera settings (such as the focal length value or aperture value of the camera) of the mobile phone, which is not limited.

[0131] The above is an example of a mobile phone as an electronic device to introduce the application effect achieved by the method of the embodiments of the present application. Of course, for other application scenarios of electronic devices, the corresponding interface display effects can also be achieved by referring to the above introduction, and further details will not be shown one by one.

[0132] The above is the application effect of the method provided by the embodiments of the present application from the perspective of the electronic device display. The following describes a method for processing an image provided by the present application from the perspective of technical implementation, i.e., how to use the method provided by the present application to achieve an interface effect without moire or with reduced moire degree, so as to improve the accuracy and efficiency of extracting a graphic code (or scanning a code).

[0133] FIG. 9 shows a flowchart of a method for processing an image provided by the embodiments of the present application. The method can be applied to an electronic device (also referred to as a terminal device), which can include but is not limited to a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a tablet computer, etc.), a vehicle-mounted computer, a smart watch, a television, and other electronic devices with image acquisition (shooting) functions, etc. The method can be executed by the electronic device, or by a component (such as a chip, a chip system, or a circuit, etc.) corresponding to the electronic device. Of course, the method can also be executed by other devices or apparatuses, for example, the method can be executed by a cloud (such as a server in the cloud). Therefore, the present application does not limit the subject executing the method and its specific structure and quantity, as long as the subject can execute the method provided by the embodiments of the present application by running a program in which the code of the method provided by the embodiments of the present application is recorded.

[0134] Exemplarily, the method is executed by a first electronic device (i.e., a shooting electronic device) as an example. The order of the steps in each of the following processes is only an example. In actual application, the order of the steps in each of the processes can be adjusted, and all or part of the steps described below can be adaptively executed. Referring to FIG. 9, the method can include the following steps:

[0135] S901: The first electronic device acquires a first image displayed in a viewfinder frame of the first electronic device. The first image is obtained by a camera of the first electronic device capturing a second image displayed by a second electronic device. The second image displayed by the second electronic device implicitly contains a graphic code for information verification.

[0136] In the embodiments of the present application, the second image displayed by the second electronic device can be a dynamic image or a static image, which is not limited. The camera of the first electronic device can capture the second image displayed by the second electronic device in real time, and display the second image in real time through the viewfinder frame (or a preview stream) in the screen of the first electronic device.

[0137] In addition, the second image displayed by the second electronic device implicitly contains a graphic code for information verification, which can be understood as follows: a user or a machine cannot directly obtain the graphic code from the image without performing multi-frame joint image processing on the image. The graphic code can be a ring code, a two-dimensional code, a bar code, etc.

[0138] In a possible implementation, the picture displayed by the second electronic device can be a starry sky picture.

[0139] S902: The first electronic device detects the Moiré degree on the first image of the first picture.

[0140] For example, the Moiré degree on the first image of the first picture can be detected and obtained by the processor of the first electronic device.

[0141] In a possible implementation, the first image can be an ROI image obtained by the first electronic device by taking a screenshot of a frame image of the first picture displayed in the frame of the first electronic device.

[0142] In the embodiments of the present application, when the first electronic device performs S902, the following implementation manners can be used, but are not limited thereto:

[0143] Implementation manner one: The first electronic device detects the Moiré degree on the first image of the first picture, including the following:

[0144] First, the first image is subjected to intra-frame difference processing to obtain a third image subjected to intra-frame difference processing; then, the third image is subjected to binarization processing to obtain a fourth image subjected to binarization processing; secondly, based on the fourth image, the proportion of a first region in the fourth image is counted, and the proportion of the first region in the fourth image is determined as the Moiré degree on the first image; wherein the first region is a region in the fourth image in which the pixel brightness exceeds a set threshold.

[0145] In the implementation manner one, the fourth image is obtained by the first image through the above several processing steps, and the proportion of the first region in the fourth image is taken as the Moiré degree on the first image. In the implementation manner one, the Moiré degree on the first image is quantified as the proportion of the first region in the fourth image.

[0146] Implementation manner two: The first electronic device detects the Moiré degree on the first image of the first picture, including: first, determining a difference between the distance between the first electronic device and the second electronic device and the distance at which the Moiré occurs according to the distance between the first electronic device and the second electronic device and the distance at which the Moiré occurs; and second, determining the Moiré degree on the first image according to the difference between the distance between the first electronic device and the second electronic device and the distance at which the Moiré occurs; wherein the difference and the Moiré degree on the first image are in an inverse relationship.

[0147] In the second implementation, the moire degree is determined according to a difference between the distance between the first electronic device and the second electronic device and the distance at which the moire occurs, and the smaller the difference is, the closer the distance between the first electronic device and the second electronic device is to the distance at which the moire occurs, and the greater the moire degree is; otherwise, the greater the difference is, the farther the distance between the first electronic device and the second electronic device is from the distance at which the moire occurs, and the smaller the moire degree is. In the second implementation, the moire degree is quantified as the difference between the distance between the first electronic device and the second electronic device and the distance at which the moire occurs.

[0148] In actual applications, there can be various ways to quantify the moire degree, and therefore, in the embodiments of the present application, the way in which the first electronic device detects the moire degree on the first image of the first picture is not limited to the above-mentioned several implementations.

[0149] Based on the above-mentioned second implementation, in a possible implementation, before the first electronic device detects the moire degree on the first image of the first picture, the method of the embodiments of the present application further includes: determining, by the first electronic device, the distance between the first electronic device and the second electronic device.

[0150] In the embodiments of the present application, the first electronic device can determine the distance between the first electronic device and the second electronic device in the following ways, but is not limited thereto:

[0151] The first electronic device acquires binocular disparity information, and determines the distance between the first electronic device and the second electronic device according to the binocular disparity information.

[0152] The first electronic device acquires size information of the first picture, and determines the distance between the first electronic device and the second electronic device according to the size information of the first picture.

[0153] For example, the first picture is a starry sky picture. The first electronic device can determine the distance between the first electronic device and the second electronic device according to size information of a starry sky ring displayed on the screen of the first electronic device and / or size information of a starry sky ring displayed on the screen of the second electronic device.

[0154] The first electronic device determines the distance between the first electronic device and the second electronic device through a ranging device of the first electronic device.

[0155] For example, the first electronic device determines the distance between the first electronic device and the second electronic device through a dtof device.

[0156] Based on the above-mentioned second implementation, in a possible implementation, before the first electronic device detects the moire degree on the first image of the first picture, the method of the embodiments of the present application further includes: determining, by the first electronic device, the distance at which the moire occurs.

[0157] The first electronic device determining the distance at which moire occurs can include: the first electronic device obtaining information of the second electronic device, the information of the second electronic device including but not limited to the pixels and / or size of the screen of the second electronic device; and the first electronic device determining the distance at which moire occurs according to the information of the first electronic device and the information of the second electronic device, wherein the information of the first electronic device includes at least one of the pixels, size of the screen of the first electronic device, and the camera spatial resolution of the first electronic device.

[0158] In the embodiments of the present application, the device for determining the distance between the first electronic device and the second electronic device (or the distance at which moire occurs) is not limited to the first electronic device itself, but can also be determined by other devices and provided to the first electronic device. For example, the distance between the first electronic device and the second electronic device (or the distance at which moire occurs) is measured / calculated and obtained by the second electronic device, and the second electronic device sends the distance to the first electronic device.

[0159] S903: The first electronic device obtains a second image of the second electronic device based on the degree of moire on the first image.

[0160] In a possible implementation, the first electronic device obtains a second image of the second electronic device based on the degree of moire on the first image, including the following cases:

[0161] Case one: the degree of moire on the first image exceeds a set threshold, and the first electronic device adjusts the camera setting information of the first electronic device; and the first electronic device obtains the second image based on the second screen in the viewfinder, the second screen being obtained by the adjusted camera of the first electronic device collecting the screen displayed by the second electronic device.

[0162] Case two: the degree of moire on the first image does not exceed the set threshold, and the first electronic device obtains the second image based on the first screen.

[0163] In the embodiments of the present application, for the picture (such as the first picture or the second picture) displayed by the first electronic device by collecting / shooting the picture displayed by the second electronic device, the moire appears on the displayed picture, and the degree (i.e., the moire degree) will affect the accuracy of subsequent extraction of the graphical code. In actual application, there can be various quantification methods of the moire degree (such as the implementation manner one and the implementation manner two in S902), which are not limited in the present application. Regardless of which quantification method is used, the moire degree on the displayed picture (such as the first picture or the second picture) of the first electronic device can be understood as the influence degree of the moire on the accuracy of extraction of the graphical code. For example, the weaker / smaller the moire degree, the higher the accuracy of extraction of the graphical code, and the more serious / larger the moire degree, the lower the accuracy of extraction of the graphical code.

[0164] In addition, the threshold value set above can be understood as a threshold value affecting the accuracy of extraction of the graphical code. The quantification method of the moire degree is different, and the definition and standard of the threshold value set above can also be different. However, in the present application, regardless of which quantification method is used to quantify the moire degree, the threshold value or threshold set (which can be understood as setting in the present application) should ensure that the accuracy of subsequent extraction of the graphical code is high. Moreover, the set threshold value can be a fixed value or a variable value. For example, for the detected moire degree in the implementation manner one and the implementation manner two in S902, the set threshold value for reference / comparison can change with the distance between the first electronic device and the second electronic device.

[0165] Based on the above case one, in a possible implementation manner, the camera setting information includes a focusing distance and / or an aperture value; the first electronic device adjusts the camera setting information of the first electronic device, including: adjusting the first focusing distance of the camera of the first electronic device to a second focusing distance according to a preset corresponding relationship between the image distance and the object distance; wherein the first focusing distance is the focusing distance obtained by the automatic focusing of the camera, and the adjustment difference between the first focusing distance and the second focusing distance is determined according to the distance between the first electronic device and the second electronic device; and / or

[0166] The first electronic device adjusts the first aperture value of the camera of the first electronic device to a second aperture value, and the first aperture value is the aperture value obtained by the automatic setting of the camera, and the second aperture value is smaller than the first aperture value.

[0167] Based on the above case two, in a possible implementation manner, before the first electronic device obtains the second image based on the first picture, the method of the embodiments of the present application can further include: the first electronic device detects that the moire degree on the fifth image of the second picture does not exceed the set threshold value.

[0168] If the first electronic device detects that the Moiré degree on the fifth image of the second picture exceeds a set threshold value, the camera setting information of the first electronic device can be continuously adjusted in the manner described above.

[0169] In a possible implementation, the first electronic device detecting the fifth image of the second picture can be that the first electronic device takes a screenshot of a frame of the second picture displayed in the viewfinder to obtain an ROI image.

[0170] S904: The first electronic device extracts the graphic code based on the second image.

[0171] In a possible implementation, the second image is at least two consecutive frames of images.

[0172] Specifically, for the first case in S903, the second image can refer to images obtained by the first electronic device taking screenshots of at least two consecutive frames of the first picture in the viewfinder. For the second case in S903, the second image can refer to images obtained by the first electronic device taking screenshots of at least two consecutive frames of the second picture in the viewfinder.

[0173] The first electronic device extracts the graphic code based on the second image can include: performing intra-frame difference processing and inter-frame difference processing based on the at least two frames of images to obtain a corresponding gray image after difference processing of the at least two frames of images; then performing binarization processing based on the corresponding gray image after difference processing of the at least two frames of images to obtain a corresponding image after binarization processing of the at least two frames of images after difference processing; and obtaining the graphic code based on the corresponding image after binarization processing of the at least two frames of images after difference processing.

[0174] In summary, the embodiment of the present application provides an image processing method, which can be applied to but not limited to the first electronic device, and the method includes: first, obtaining a first picture displayed in a viewfinder of the first electronic device, the first picture being obtained by a camera of the first electronic device collecting a picture displayed by a second electronic device, and the picture displayed by the second electronic device implicitly containing a graphic code for information verification; then detecting a Moiré degree on a first image of the first picture; then obtaining a second image collected by the camera of the first electronic device from the picture displayed by the second electronic device based on the Moiré degree on the first image; and finally extracting the graphic code based on the second image. In the method, when the first electronic device collects the picture implicitly containing the graphic code, the image used for extracting the graphic code is obtained based on the Moiré degree appearing in the picture, which not only effectively improves the accuracy and efficiency of extracting the graphic code, but also meets the aesthetic demand compared with the traditional black and white graphic code.

[0175] The following describes the method shown in FIG. 9 as an example applied to a mobile phone authentication scenario, and the method of the present application is described in detail through several specific embodiments.

[0176] In the following embodiments, the first electronic device is exemplified by mobile phone 1, the second electronic device is exemplified by mobile phone 2, and the requirement is exemplified by transferring / importing data stored in mobile phone 2 to mobile phone 1. When mobile phone 1 is a mobile phone to be verified by mobile phone 2, mobile phone 1 can extract a valid QR code from the starry ring picture displayed on the screen of mobile phone 2 by scanning the starry ring picture, and then obtain corresponding authentication information / pairing information by analyzing the QR code, so that mobile phone 1 can be paired and authenticated with mobile phone 2. After successful authentication, mobile phone 1 can establish a high-speed communication connection with mobile phone 2 and receive data from mobile phone 2.

[0177] Embodiment 1

[0178] In embodiment 1, it is described in detail how to detect the Moire fringe degree on the picture based on the detection method shown in embodiment 1 of S902 in the above authentication stage of mobile phone 1 scanning the starry ring picture displayed on the screen of mobile phone 2, and how to decide whether to adjust the camera settings based on the Moire fringe degree to improve the accuracy and efficiency of subsequent QR code extraction. Referring to FIG. 10, the method flow of embodiment 1 can include the following steps:

[0179] S1001: Mobile phone 1 and mobile phone 2 turn on the Bluetooth communication function and exchange device information through Bluetooth.

[0180] Mobile phone 1 and mobile phone 2 first exchange information (such as scanning code instructions or authentication instructions) through Bluetooth (or Wi-Fi, star flash, NFC, etc. Wireless communication technology), and trigger both sides to enter the starry ring display and scanning code process.

[0181] In an optional implementation, mobile phone 1 and mobile phone 2 can exchange device information through Bluetooth and other communication functions, for example, mobile phone 1 receives device type, version, and model information of mobile phone 2.

[0182] S1002: Mobile phone 1 scans the starry ring picture displayed on the screen of mobile phone 2 and displays the scanned starry ring picture on the screen of mobile phone 1 in real time (an example of the first picture shown in FIG. 9).

[0183] After mobile phone 1 turns on the scanning code authentication function, mobile phone 1 enters the scanning code application, that is, the camera function of mobile phone 1 is turned on, and a preview stream (also called a viewfinder frame) appears on the screen of mobile phone 1. The preview stream provides a real-time image / picture preview before mobile phone 1 takes a picture or video. At the same time, mobile phone 2 enters the scanning authentication process, that is, the starry ring picture is displayed on the screen of mobile phone 2.

[0184] The user can hold the camera of the mobile phone 1 to aim at the starry ring on the screen of the mobile phone 2, and the starry ring picture on the mobile phone 2 will be displayed in the preview stream of the mobile phone 1 in real time, and the processor of the mobile phone 1 detects whether the starry ring exists in the preview stream in real time.

[0185] In the embodiment of the present application, the way in which the mobile phone 1 detects whether the starry ring exists in the preview stream can be realized by using the existing way of detecting images / pictures, which will not be described in detail here.

[0186] S1003: The mobile phone 1 detects the moire degree of the scanned starry ring picture.

[0187] That is, the mobile phone 1 can automatically detect the moire degree of the starry ring picture displayed in the preview stream (or viewfinder frame) of the screen of the mobile phone 1 in real time.

[0188] In one possible implementation, the mobile phone 1 can capture (take a screenshot) the starry ring picture displayed in the preview stream to obtain an image of a starry ring ROI region, that is, image 1 (an example of the first image in the scheme shown in FIG. 9). Further, the mobile phone 1 can determine the moire degree of the starry ring picture scanned by the mobile phone 1 by detecting the moire degree existing in the image 1.

[0189] In this embodiment, the specific steps in which the mobile phone 1 detects the moire degree of the image 1 (an example of the first image in the scheme shown in FIG. 9) can include the following:

[0190] Step 1: Perform intra-frame difference processing on the starry ring ROI region in the image 1 to obtain a gray image 1.

[0191] For example, based on the image 1, according to the three values of RGB of each pixel, the following formula 1 is used to calculate a gray image (gray). Each pixel of the gray image = 2*B-1*G-1*R, Formula 1.

[0192] Wherein, “*” is a multiplication sign.

[0193] If there is moire on the image 1, the gray image can make the colorful moire obvious.

[0194] Step 2: Perform binaryzation processing on the gray image to obtain a binary image.

[0195] In the embodiment of the present application, in the binaryzation processing of the gray image, the threshold 1 used can be a statistical value, that is, the threshold 1 is a threshold that can filter out 90% of the information from the gray image histogram to obtain the binary image.

[0196] Step 3: Determine whether the density of the white region in the binary image is higher than a threshold 2.

[0197] The white region in the binary image can refer to a region in which the brightness of pixels in the image is higher than a certain threshold.

[0198] In one possible implementation, a ratio of the area of the white region in the binary image to the entire area of the image is calculated, and the ratio is taken as the density of the white region.

[0199] In the embodiments of the present application, the density of the white region in the binary image or the ratio of the area of the white region in the binary image to the entire area of the image can be determined as the moire degree on the image 1.

[0200] S1004: The mobile phone 1 determines whether the moire degree is higher than a set threshold.

[0201] For example, based on step three in S1003 described above, if the density of the white region in the binary image is higher than / over the threshold 2 (an example of the set threshold in the scheme shown in FIG. 9 described above), it is determined that the moire degree on the image 1 is relatively severe, and then S1005 described below is performed.

[0202] If the density of the white region in the binary image is not higher than / over the threshold 2 (an example of the set threshold in the scheme shown in FIG. 9 described above), it is determined that there is no moire on the image 1 or the moire degree is relatively weak, and then S1006 described below is performed.

[0203] In the above, the threshold 2 can change with the distance between the mobile phone 1 and the mobile phone 2, that is, the threshold 2 can be determined according to the distance between the mobile phone 1 and the mobile phone 2.

[0204] Exemplarily, FIG. 11A shows a corresponding processing schematic diagram in the case where there is moire (or the moire degree is relatively severe). Referring to FIG. 11A, (1) in FIG. 11A is the image 1 after the intra-frame difference processing, and (2) in FIG. 11A is the binary image. When it is determined that the density of the white region in the binary image is higher than the threshold 2, it is determined that there is moire (or the moire degree is relatively severe) on the image 1.

[0205] Exemplarily, FIG. 11B shows a corresponding processing schematic diagram in the case where there is no moire (or the moire degree is relatively weak). Referring to FIG. 11B, (1) in FIG. 11B is the image 1 after the intra-frame difference processing, and (2) in FIG. 11B is the binary image. When it is determined that the density of the white region in the binary image is not higher than the threshold 2, it is determined that there is no moire (or the moire degree is relatively weak) on the image 1.

[0206] S1005: When the moire degree is higher than the set threshold, the mobile phone 1 adjusts the setting of the camera of the mobile phone 1, and re-scans the starry sky ring picture displayed on the screen of the mobile phone 2.

[0207] That is, the mobile phone 1 adjusts the setting of the camera of the mobile phone 1 (for example, the focusing distance, the aperture, and the like of the camera), and after adjusting the setting of the camera, re-scans / acquires the starry sky ring picture displayed on the screen of the mobile phone 2, and displays the scanned / acquired starry sky ring picture in the screen preview stream of the mobile phone 1 in real time (the example of the second picture shown in FIG. 9).

[0208] In a possible implementation, the mobile phone 1 can reduce the definition of the image by adjusting the focusing distance of the camera of the mobile phone 1 (or the aperture of the camera, and the like), so as to reduce the moire degree of the acquired starry sky ring picture or eliminate the moire.

[0209] Exemplarily, when the mobile phone 1 reduces the moire degree of the acquired starry sky ring picture or eliminates the moire by adjusting the focal length value (that is, the focusing distance of the camera) of the camera of the mobile phone 1, the following several ways can be adopted:

[0210] Firstly, the mobile phone 1 determines a new focal length value c2 according to the focal length value c1 obtained by automatic focusing of the mobile phone 1 and the preset corresponding relationship between the focal length value and the object distance value, that is, adjusts the focal length value of the camera of the mobile phone 1 from c1 to c2. c2 can satisfy the following formula two: c2=f(d2); Formula two.

[0211] Wherein, d2=d1±Δd, since c1=f(d1), d1=f-1(c1) can be obtained; the value of Δd is related to the distance between the mobile phone 1 and the mobile phone 2 (that is, the value of Δd can be determined according to the distance between the mobile phone 1 and the mobile phone 2).

[0212] For example, FIG. 12 shows a curve diagram of the corresponding relationship between the image distance and the object distance (the distance between the mobile phone 1 and the mobile phone 2), as shown in FIG. 12, as the object distance increases, the image distance gradually decreases, and then tends to a value.

[0213] Secondly, the mobile phone 1 determines a new focal length value c2 according to the moire degree of the starry sky ring picture scanned / acquired in S1003 (that is, the moire degree of the image 1 detected by the mobile phone 1), and adjusts the focal length value of the camera of the mobile phone 1 from the focal length value c1 obtained by automatic focusing to c2.

[0214] For example, if the mobile phone 1 detects that the moire degree of the starry sky ring scanned by the mobile phone 1 is represented by a first value, the mobile phone 1 can determine a new focal length value c2 according to the first value and a preset mapping relationship, the preset mapping relationship representing the corresponding relationship between the moire degree and the focal length value, and the preset mapping relationship including the mapping relationship between the first value and c2.

[0215] Way three: the mobile phone 1 can dynamically adjust the focal length value of the camera according to the Moiré degree of the scanned / acquired starry sky ring picture, until the Moiré degree of the scanned / acquired starry sky ring picture is weak or there is no Moiré.

[0216] The mobile phone 1 can dynamically adjust the focal length value of the camera by trial and error, and detect the Moiré degree of the scanned / acquired starry sky ring picture after adjustment. When the mobile phone 1 detects that the Moiré degree of the scanned / acquired starry sky ring picture is serious (i.e., the Moiré degree is higher than the set threshold), it can continue to adjust the focal length value of the camera, until the Moiré degree of the scanned / acquired starry sky ring picture is weak or there is no Moiré (i.e., the Moiré degree is not higher than the set threshold), and then stop adjusting the focal length value of the camera to perform S1006 described below.

[0217] For example, as shown in the flowchart of FIG. 13, it includes the following steps: step a: the mobile phone 1 judges whether the Moiré degree of the scanned starry sky ring picture is higher than the set threshold; if yes, step b is performed, i.e., the mobile phone 1 adjusts the settings of its camera and re-scans the starry sky ring picture; if no, step c is performed, i.e., the mobile phone 1 extracts and decodes the ring code based on the scanned starry sky ring picture.

[0218] Of course, in the embodiments of the present application, the mobile phone 1 can also reduce the Moiré degree or eliminate the Moiré on the collected starry sky ring picture by adjusting the aperture or other camera settings or parameters of the camera. The way in which the mobile phone 1 adjusts the aperture or other camera settings or parameters of the camera can be implemented by referring to one or more of the ways in which the mobile phone 1 adjusts the focal length value of the camera described above, and will not be described again here.

[0219] In the embodiments of the present application, after adjusting the settings of the camera, the mobile phone 1 can re-detect the Moiré degree of the scanned / acquired starry sky ring picture. For example, after adjusting the settings of the camera, the mobile phone 1 takes a screenshot of the re-scanned / acquired starry sky ring picture to obtain an image of a starry sky ring ROI region (an example of the fifth image in the scheme shown in FIG. 9 described above), and then detects the Moiré degree of the image. The way and steps of detecting the Moiré degree of the image can be implemented by referring to the detection way and steps in S1003 described above, and will not be described again here.

[0220] S1006: The mobile phone 1 extracts and decodes the ring code based on the scanned starry sky ring picture of the mobile phone 1.

[0221] In one possible implementation, the mobile phone 1 can extract the ring code based on the scanned / acquired starry sky ring picture, which can include the following steps:

[0222] In the case that the mobile phone 1 displays the scanned / acquired starry sky ring picture in the preview stream of the mobile phone 1, the mobile phone 1 can capture (take a screenshot) the starry sky ring picture displayed in the preview stream, and obtain multiple images of the starry sky ring ROI region (for example, at least two images in the scheme shown in FIG. 9 described above), that is, multiple images containing the starry sky ring ROI region. Further, the mobile phone 1 can process the multiple images to obtain a final image, which is used to extract the ring code (that is, a kind of graphic code).

[0223] For example, the mobile phone 1 captures (takes a screenshot) the starry sky ring picture displayed in the preview stream to obtain three images of the starry sky ring ROI region (hereinafter referred to as three images, that is, image 1, image 2 and image 3) as an example. The mobile phone 1 extracts the ring code based on the three images, which can include the following steps:

[0224] Step 1: first perform intra-frame difference processing on the image 1, the image 2 and the image 3 to obtain three images after intra-frame difference processing, that is, image 11, image 22 and image 33.

[0225] Step 2: perform inter-frame difference processing on the image 11, the image 22 and the image 33 to obtain three images after inter-frame difference processing, that is, image 111, image 222 and image 333.

[0226] Step 3: perform binarization processing on the image 11, the image 22 and the image 33 to obtain three images after binarization processing, that is, image 1111, image 2222 and image 3333.

[0227] Step 3: process the image 1111, the image 2222 and the image 3333 using an image processing method and eliminate noise points to obtain the ring code.

[0228] The effect diagrams corresponding to the above extraction steps of the ring code can be referred to FIGS. 8B to 8E described above, which are not shown here.

[0229] In the first embodiment, the mobile phone 1 can detect the moire degree on the starry sky ring picture collected in the code scanning process in real time. In the case that the moire degree is heavy, the mobile phone 1 can timely adjust the settings of the camera to reduce the moire degree or eliminate the moire, so as to ensure that the mobile phone 1 can effectively and accurately extract the ring code from the collected starry sky ring picture, and then perform decoding and obtain the corresponding information, thereby avoiding the failure or low efficiency of the mobile phone 1 to extract the graphic code due to the influence of the moire. Therefore, the embodiment can effectively improve the accuracy of code scanning and improve the efficiency of code scanning of the mobile phone, that is, the mobile phone can successfully scan the code faster.

[0230] Second embodiment:

[0231] In the second embodiment, the mobile phone 1 is introduced in detail in the stage of scanning the starry ring displayed on the screen of the mobile phone 2 to perform authentication, how to detect the moire degree on the picture based on the detection mode shown in the second embodiment of S902 above, and based on the moire degree, whether to adjust the settings of the camera to improve the accuracy and efficiency of subsequent extraction of the graphic code. Referring to FIG. 14, the method flow of the second embodiment can include the following steps:

[0232] S1401: The mobile phone 1 and the mobile phone 2 start the Bluetooth communication function, and exchange the information through the Bluetooth interaction device.

[0233] S1402: The mobile phone 1 scans the starry ring picture displayed by the mobile phone 2, and displays the scanned / captured starry ring picture (the example of the first picture shown in FIG. 9 above) on the screen of the mobile phone 1 in real time.

[0234] S1401 and S1402 can be referred to the above S1001 and S1002 for detailed description, which will not be repeated here.

[0235] S1403: The mobile phone 1 detects the difference between the distance 1 between the mobile phone 1 and the mobile phone 2 and the distance 2 at which the moire occurs (which can be equivalent to the moire degree on the starry ring picture scanned by the mobile phone 1).

[0236] In the second embodiment, the mobile phone 1 detects the difference between the distance 1 between the mobile phone 1 and the mobile phone 2 and the distance 2 at which the moire occurs can include: first determining the distance 1 between the mobile phone 1 and the mobile phone 2, and the distance 2 at which the moire occurs; and then calculating the difference between the distance 1 and the distance 2.

[0237] Wherein, the mobile phone 1 can detect the distance 1 between the mobile phone 1 and the mobile phone 2 by using but not limited to the following ways:

[0238] First, the mobile phone 1 obtains the distance 1 between the mobile phone 1 and the mobile phone 2 according to the binocular parallax information.

[0239] Second, the mobile phone 1 obtains the starry ring size information displayed on the screen of the mobile phone 2 and / or the starry ring size information displayed on the screen of the mobile phone 1 to determine the distance between the mobile phone 1 and the mobile phone 2.

[0240] Third, the mobile phone 1 can measure the distance 1 between the mobile phone 1 and the mobile phone 2 through its own distance measuring device (such as a dtof device).

[0241] In a possible implementation, the mobile phone 1 determines the distance 2 at which moire occurs, including: obtaining the device information of the mobile phone 2 (for example, the screen pixels of the mobile phone 2, the size of the screen, etc.) through Bluetooth (or Wi-Fi, star flash, NFC, and other wireless communication technologies), and then determining the distance 2 at which moire occurs according to the device information of the mobile phone 1 (for example, the screen pixels of the mobile phone 1, the size of the screen, the camera spatial resolution, etc.) and / or the device information of the mobile phone 2 (for example, the screen pixels of the mobile phone 2, the size of the screen, etc.).

[0242] S1404: The mobile phone 1 determines whether the difference between the distance 1 between the mobile phone 1 and the mobile phone 2 and the distance 2 at which moire occurs is lower than a preset threshold value.

[0243] Based on the above S1403, if the mobile phone 1 detects that the difference between the distance 1 (the distance between the mobile phone 1 and the mobile phone 2) and the distance 2 (the distance at which moire occurs) is lower than the preset threshold value, it indicates that the distance between the mobile phone 1 and the mobile phone 2 is close to the distance at which moire occurs, that is, the degree of moire on the starry sky ring picture scanned / acquired by the mobile phone 1 exceeds the preset threshold value (the degree of moire is relatively serious), and then S1405 is performed.

[0244] Based on the above S1403, if the mobile phone 1 detects that the difference between the distance 1 (the distance between the mobile phone 1 and the mobile phone 2) and the distance 2 (the distance at which moire occurs) is not lower than the preset threshold value, it indicates that the distance between the mobile phone 1 and the mobile phone 2 is not close to the distance at which moire occurs, that is, the degree of moire on the starry sky ring picture scanned / acquired by the mobile phone 1 does not exceed the preset threshold value (the degree of moire is relatively weak or there is no moire), and then S1406 is performed.

[0245] S1405: The mobile phone 1 adjusts the settings of the camera itself and re-scans the starry sky ring picture.

[0246] S1405 can refer to the several ways of adjusting the settings of the camera in S1005 described above, which will not be repeated here.

[0247] In addition, in this embodiment two, after adjusting the settings of the camera itself, the mobile phone 1 can re-detect the degree of moire on the starry sky ring picture scanned / acquired, and the detection method can refer to the detection method of S1003 in the above-mentioned embodiment one, and refer to the way of reducing the degree of moire or eliminating the moire on the starry sky ring picture acquired by adjusting the focal length value (focusing distance) of the camera itself in S1005, which will not be repeated here.

[0248] Alternatively, after adjusting the settings of the camera itself, the mobile phone 1 can refer to the detection method of S1403 described above to determine the difference between the distance 1 between the mobile phone 1 and the mobile phone 2 and the distance 2 at which moire occurs, and when the difference between the distance 1 and the distance 2 is not lower than the preset threshold value, S1406 is performed.

[0249] For example, referring to the flow shown in FIG. 15, it includes: step a: the mobile phone 1 judges whether the difference between the distance 1 (i.e. the distance between the mobile phone 1 and the mobile phone 2) and the distance 2 (i.e. the distance at which the moire occurs) is lower than a set threshold value; if yes, step b is executed, i.e. the mobile phone 1 adjusts the setting of its own camera and re-scans the starry sky ring picture; if no, step c is executed, i.e. the mobile phone 1 extracts and decodes the ring code based on the starry sky ring picture scanned by itself.

[0250] S1406: the mobile phone 1 extracts and decodes the ring code based on the starry sky ring picture scanned by the mobile phone 1.

[0251] S1406 can be implemented in the manner described above in S1006, which will not be repeated here.

[0252] In the second embodiment, if the mobile phone 1 is equipped with a sensor capable of predicting distance, such as an inertial measurement unit (IMU), the mobile phone 1 can predict in advance whether the difference between the distance 1 (i.e. the distance between the mobile phone 1 and the mobile phone 2) and the distance 2 (i.e. the distance at which the moire occurs) is lower than a set threshold value, and in the case where the difference is lower than the set threshold value, the mobile phone 1 can adjust the setting of its own camera in advance.

[0253] For example, compared with S1403, the mobile phone 1 can predict in advance the distance 1 between itself and the mobile phone 2 when scanning the starry sky ring picture of the mobile phone 2 by means of the IMU sensor, and at the same time determine the distance 2 (i.e. the distance at which the moire occurs); compared with S1404, the mobile phone 1 can judge in advance whether the difference between the distance and the distance 2 is lower than a set threshold value; in the case where the difference is lower than the set threshold value, compared with S1405, the mobile phone 1 can adjust the setting of its own camera in advance and re-scan the starry sky ring picture. In the case where the difference is not lower than the set threshold value, compared with S1406, the mobile phone 1 can extract and decode the ring code based on the starry sky ring picture scanned by the mobile phone 1 in advance.

[0254] That is, the mobile phone 1 does not need to wait until after S1402 is executed before starting to execute S1403, therefore, the mobile phone 1 uses the sensor capable of predicting distance (such as the IMU sensor) to execute the above-mentioned S1403 to S1406 in advance, which can make the time for the mobile phone 1 to successfully extract the ring code (scan the code) faster.

[0255] It should be noted that in the present application, the mobile phone 1 can use the IMU sensor or the like to detect in advance whether the difference between the distance 1 and the distance 2 is lower than the set threshold value, and adjust the camera setting in the case where the difference is lower than the set threshold value. This is not limited to the above-mentioned S1403 and S1405. For example, before S1403, the mobile phone 1 can use the IMU sensor or the like to detect in advance whether the difference between the distance 1 and the distance 2 is lower than the set threshold value, and adjust the camera setting in the case where the difference is lower than the set threshold value. After the mobile phone 1 adjusts the camera setting in advance, it can continue to perform the subsequent detection (i.e., perform S1403 and S1404). If the difference is lower than the set threshold value, the mobile phone 1 can continue to adjust the camera setting (i.e., perform S1405). If the difference is not lower than the set threshold value, the mobile phone 1 can perform the extraction and decoding of the ring code (i.e., perform S1406).

[0256] Compared with the above-mentioned embodiment one, in the embodiment two, the mobile phone 1 detects the Moiré pattern degree on the starry sky ring picture in a different way, i.e., uses the ranging capability of the camera of the mobile phone to judge the Moiré pattern degree on the starry sky ring picture, and can use this ranging capability to adjust the camera setting in advance, reduce the time for subsequent scanning and extracting the ring code, not only improve the accuracy of code scanning, but also further improve the scanning efficiency, i.e., the time for the mobile phone 1 to successfully scan the code is faster.

[0257] For the above-mentioned embodiment one and embodiment two, it should be noted that:

[0258] (1) The above-mentioned embodiment two can be implemented separately from the embodiment one, or can be partially or wholly combined with the embodiment one. This is not limited in detail.

[0259] (2) The above-mentioned embodiment one and embodiment two mainly describe the differences between them. Except for the differences, the embodiment one and the embodiment two can be referred to each other.

[0260] (3) The step numbers of the above-mentioned embodiment one and embodiment two are only one example of the execution process, and do not constitute a limitation on the execution order of the steps. The steps in the present application have no time sequence dependency relationship between each other, and have no strict execution order. In addition, the steps shown in each flowchart are not all the steps that must be executed. Some steps can be added or deleted based on the actual needs on the basis of each flowchart.

[0261] Based on the above embodiments, the present application further provides a device, which comprises at least one processor coupled with at least one memory, and when the device is running, the at least one processor reads the computer program or instructions stored in the at least one memory to execute the functions performed by the first electronic device in the methods described in the embodiments and / or implementation manners of the present application. For example, the steps S901-S904 performed by the first electronic device in the embodiment shown in FIG. 9; for another example, the steps S1003-S1004 and S1006 performed by the mobile phone 1 in the implementation manner shown in FIG. 10. For another example, the steps a and b and c shown in FIG. 13; for another example, the steps S1403-S1404 and S1406 performed by the mobile phone 1 in the implementation manner shown in FIG. 14. For another example, the steps a and b and c shown in FIG. 15.

[0262] Based on the above embodiments, the present application further provides a system for image processing, which can comprise the first electronic device and the second electronic device in the above embodiments. In a possible scenario, the system can further comprise a flow transfer device of the electronic device.

[0263] Based on the above embodiments, the present application further provides a computer program product, which comprises a computer program (also can be referred to as code or instructions), when the computer program is running, causes a computer to execute the methods described in the embodiments and / or implementation manners of the present application.

[0264] Based on the above embodiments, the present application further provides a computer readable storage medium, which stores a computer program, when the computer program is executed by a computer, causes the computer to execute the methods described in the embodiments and / or implementation manners of the present application.

[0265] Based on the above embodiments, the present application further provides a chip for reading the computer program stored in the memory, and realizing the methods described in the embodiments and / or implementation manners of the present application.

[0266] Based on the above embodiments, the present application provides a chip system, which comprises a processor for supporting a computer device to implement the methods described in the embodiments and / or implementation manners of the present application. In a possible design, the chip system further comprises a memory for storing necessary programs and data of the computer device. The chip system can be composed of a chip, or can include a chip and other discrete devices. Those skilled in the art understand that the embodiments and / or implementation manners of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be implemented in a form of complete hardware, complete software, or a combination of software and hardware. Moreover, the present application can be implemented in a form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) including computer-usable program code.

[0267] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0268] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction devices, which implement the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0269] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0270] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first image displayed in a viewfinder of the first electronic device, the first image being obtained by a camera of the first electronic device capturing a screen displayed by a second electronic device, the screen displayed by the second electronic device implicitly containing a graphical code for information verification; detecting a moire degree on the first image of the first image; based on the moire degree on the first image, obtaining a second image obtained by the camera of the first electronic device capturing the screen displayed by the second electronic device; based on the second image, extracting the graphical code.

2. The method of claim 1, wherein, The detection of the moire degree on the first image of the first image comprises: performing intra-frame difference processing based on the first image to obtain a third image after intra-frame difference processing; performing binaryzation processing based on the third image to obtain a fourth image after binaryzation processing; based on the fourth image, statistics of a proportion of a first region in the fourth image is obtained, and the proportion of the first region in the fourth image is determined as the moire degree on the first image; wherein the first region is a region in the fourth image in which pixel brightness exceeds a set threshold.

3. The method of claim 2, wherein, The detection of the moire degree on the first image of the first image comprises: determining a difference between the distance between the first electronic device and the second electronic device and the distance at which moire occurs, according to the distance between the first electronic device and the second electronic device and the distance at which moire occurs; determining the moire degree on the first image according to the difference between the distance between the first electronic device and the second electronic device and the distance at which moire occurs; wherein the difference and the moire degree on the first image are inversely proportional.

4. The method of claim 3, wherein, The method further comprises: obtaining binocular disparity information; determining the distance between the first electronic device and the second electronic device according to the binocular disparity information; or obtaining size information of the first image; determining the distance between the first electronic device and the second electronic device according to the size information of the first image; or measuring the distance between the first electronic device and the second electronic device by a ranging device of the first electronic device.

5. The method of claim 3 or 4, wherein, The method further comprises: obtaining information of the second electronic device, the information of the second electronic device including pixels and / or size of a screen of the second electronic device; determining the distance at which moire occurs according to information of the first electronic device and information of the second electronic device; wherein the information of the first electronic device includes at least one of pixels, size of a screen of the first electronic device, and spatial resolution of a camera of the first electronic device.

6. The method of any one of claims 1-5, wherein, The obtaining of the second image based on the moire degree on the first image comprises: adjust camera setting information of the first electronic device when the moire degree on the first image exceeds a set threshold; and acquire the second image based on a second picture in the viewfinder, the second picture being obtained by the camera of the first electronic device after the adjustment collecting a picture displayed by the second electronic device; acquire the second image based on the first picture when the moire degree on the first image does not exceed the set threshold.

7. The method of claim 6, wherein, The camera setting information includes a focusing distance and / or an aperture value. The adjustment of the camera setting information of the first electronic device includes: adjusting a first focusing distance of the camera of the first electronic device to a second focusing distance according to a preset corresponding relationship between an image distance and an object distance, wherein the first focusing distance is a focusing distance obtained by automatic focusing of the camera, and an adjustment difference between the first focusing distance and the second focusing distance is determined according to a distance between the first electronic device and the second electronic device; and / or adjusting a first aperture value of the camera of the first electronic device to a second aperture value, the first aperture value being an aperture value obtained by automatic setting of the camera, and the second aperture value being smaller than the first aperture value.

8. The method of claim 6 or 7, wherein, The method further includes: detecting that the moire degree on a fifth image of the second picture does not exceed the set threshold.

9. The method of any one of claims 1-8, wherein, The second image is at least two continuous images. The extraction of the graphic code based on the second image includes: performing intra-frame difference processing and inter-frame difference processing based on the at least two images to obtain a gray image corresponding to the at least two images after difference processing; performing binaryzation processing based on the gray image corresponding to the at least two images after difference processing to obtain an image after binaryzation processing corresponding to the at least two images after difference processing; and obtaining the graphic code based on the image after binaryzation processing corresponding to the at least two images after difference processing.

10. The method of any one of claims 1-9, wherein, The method further includes: acquiring sensor information of the first electronic device and sensor information of the second electronic device, the sensor information including inertial measurement unit (IMU) sensor information; when the distance between the first electronic device and the second electronic device is the same as the distance at which moire occurs or the difference is less than a preset threshold, adjusting the camera setting information of the first electronic device and collecting a picture displayed by the second electronic device through the camera of the first electronic device after the adjustment.

11. The method of any one of claims 1-10, wherein, The method further includes: sending device information of the first electronic device to the second electronic device and receiving device information of the second electronic device from the second electronic device, the device information including one or more of the following: information triggering an opening verification process, a type of the device, and a version and model of the device.

12. The method of any one of claims 1-11, wherein, The picture displayed by the second electronic device is a starry sky ring picture.

13. The method of any one of claims 1-12, wherein, The graphic code is any one of the following: a ring code, a two-dimensional code, and a bar code.

14. An electronic device, comprising: comprising at least one processor coupled with at least one memory for reading a program stored in the at least one memory to perform the method of any one of claims 1 to 13.

15. A readable storage medium, characterized by, The readable storage medium has instructions stored therein, which, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 13.

16. A chip, characterized by The chip is coupled with a memory for executing program instructions in the memory to perform the method of any one of claims 1 to 13.

Images