Image processing method and device

Abstract

The embodiment of the invention provides an image processing method and device.The method comprises the steps that first electronic equipment obtains a first picture displayed in a viewing frame of the first electronic equipment, and the first picture is obtained by collecting a picture displayed by second electronic equipment through a camera of the first electronic equipment; a picture displayed by the second electronic equipment implicitly comprises a graphic code for information verification; then detecting the moire degree on the first image of the first picture; based on the moire degree on the first image, acquiring a second image obtained by acquiring a picture displayed by second electronic equipment through a camera of the first electronic equipment; and finally, extracting the graphic code based on the second image. According to the method, when the first electronic device collects the image containing the image line code in an implicit mode, the image used for extracting the graphic code is obtained based on the moire degree appearing on the image, and therefore the accuracy and efficiency of graphic code extraction can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic devices, and in particular to an image processing method and device. BACKGROUND

[0002] The graphic code technology has been applied more and more widely, such as making payment by scanning a graphic code in an electronic payment application, or joining a friend by scanning a graphic code in a social application, or scanning a code for authentication in an electronic device (such as a mobile phone), which can include new machine verification, bracelet pairing, account login, one-key screen projection, etc., or making entry and exit registration by scanning a graphic code in a travel application. Some commonly used graphic codes include two-dimensional codes, bar codes, ring codes, etc.

[0003] A commonly used graphic code (such as a two-dimensional code or a bar code) is a black and white pattern, which is difficult to meet the aesthetic needs. In this regard, in some code scanning application scenarios, an electronic device (such as a smart phone) with image acquisition and graphic code recognition functions can obtain a color image containing a graphic code by scanning / shooting, and extract the graphic code from the shot color image, and then decode (or analyze) the graphic code. After successful decoding, the information or content carried by the graphic code can be obtained, so that relevant operations can be performed. However, at a specific distance, the high-frequency interference of the light sensing element of the camera of the electronic device (such as a smart phone) occurs, so that the shot image appears colored high-frequency stripes, i.e., moire, which can cause the extraction of the graphic code from the image to fail, and further cause the decoding to fail or to need to be repeated multiple times to successfully decode. As can be seen, the accuracy and efficiency of extracting the graphic code are key factors affecting the subsequent decoding and obtaining of corresponding information or content.

[0004] Therefore, how to improve the accuracy and efficiency of extracting the graphic code has become a problem to be solved at present. SUMMARY

[0005] The present application provides an image processing method and device for improving the accuracy and efficiency of extracting a graphic code.

[0006] In a first aspect, embodiments of the present application provide an image processing method, which can be applied to, but is not limited to, a first electronic device (or a first terminal device). The first electronic device can include a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a tablet computer, etc.), a vehicle-mounted computer, a smart watch, a television, and other electronic devices with an acquisition device (such as a camera), etc. The method can be executed by the first electronic device or a component (such as a chip, a chip system, a circuit, etc.) corresponding to the first electronic device. Taking the method executed by the first electronic device as an example, the method includes: obtaining, by the first electronic device, a first picture displayed in a framing box of the first electronic device, the first picture being obtained by a camera of the first electronic device from a picture displayed by a second electronic device, and the picture displayed by the second electronic device implicitly containing a graphical code for information verification; detecting, by the first electronic device, a moire degree on a first image of the first picture; obtaining, by the first electronic device, a second image obtained by the camera of the first electronic device from the picture displayed by the second electronic device based on the moire degree on the first image; and extracting, by the first electronic device, the graphical code based on the second image.

[0007] In embodiments of the present application, the picture displayed by the second electronic device can be a dynamic picture or a static image, which is not limited. The picture displayed by the second electronic device implicitly containing the graphical code for information verification can be understood as that a user or a machine cannot directly obtain the graphical code from the picture without performing multi-frame joint image processing on the picture. The graphical code can be a ring code, a two-dimensional code, a bar code, etc.

[0008] Exemplarily, the picture displayed by the second electronic device can be a starry sky ring picture.

[0009] The camera of the first electronic device can acquire the picture displayed by the second electronic device in real time and display the picture through a framing box (or a preview stream) on a screen of the first electronic device in real time.

[0010] The present application scheme can be applied to, but is not limited to, the first electronic device. In the present application scheme, after the first electronic device acquires the picture implicitly containing the graphical code, the first electronic device detects a moire degree on the picture, obtains an image (i.e., the second image) for extracting the graphical code based on the moire degree on the picture, so as to avoid or reduce the influence of the moire on the extraction of the graphical code. The present application scheme can not only effectively improve the accuracy of extracting the graphical code, but also improve the efficiency of extracting the graphical code.

[0011] With reference to the first aspect, in a possible implementation manner, the first electronic device detects the Moiré degree on the first image of the first picture, including: first, performing intra-frame difference processing based on the first image to obtain a third image after intra-frame difference processing; then, performing binaryzation processing based on the third image to obtain a fourth image after binaryzation processing; and then, based on the fourth image, counting a proportion of a first region in the fourth image, and determining the proportion of the first region in the fourth image as the Moiré degree on the first image; wherein the first region is a region in the fourth image in which pixel brightness exceeds a set threshold. Through this implementation manner, the Moiré degree on the first image of the first picture can be effectively detected.

[0012] With reference to the first aspect, in a possible implementation manner, the first electronic device detects the Moiré degree on the first image of the first picture, including: first, determining a difference between the distance between the first electronic device and the second electronic device and the Moiré occurrence distance according to the distance between the first electronic device and the second electronic device and the Moiré occurrence distance; and then, determining the Moiré degree on the first image according to the difference between the distance between the first electronic device and the second electronic device and the Moiré occurrence distance; wherein the difference and the Moiré degree on the first image are in an inverse relationship.

[0013] Through this implementation manner, the difference between the distance between the first electronic device and the second electronic device and the Moiré occurrence distance is effectively utilized to detect the Moiré degree on the first image of the first picture.

[0014] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device acquires binocular disparity information; and determines the distance between the first electronic device and the second electronic device according to the binocular disparity information; or the first electronic device acquires size information of the first picture; and determines the distance between the first electronic device and the second electronic device according to the size information of the first picture; or the first electronic device measures the distance between the first electronic device and the second electronic device through a ranging device of the first electronic device. Through this implementation manner, the distance between the first electronic device and the second electronic device can be effectively and accurately obtained.

[0015] With reference to the first aspect, in a possible implementation manner, the method further includes: the first electronic device acquires information of the second electronic device, the information of the second electronic device including pixels and / or size of a screen of the second electronic device; and the first electronic device determines the Moiré occurrence distance according to information of the first electronic device and information of the second electronic device; wherein the information of the first electronic device includes at least one of pixels, size of a screen of the first electronic device, and camera spatial resolution of the first electronic device. Through this implementation manner, the Moiré occurrence distance can be effectively and accurately obtained.

[0016] In conjunction with the first aspect, in one possible implementation, the first electronic device acquires a second image obtained by its camera capturing the image displayed on the second electronic device, based on the moiré pattern intensity of the first image. This includes: adjusting the camera settings of the first electronic device when the moiré pattern intensity on the first image exceeds a set threshold; acquiring the second image based on a second view within the viewfinder, wherein the second image is captured by the adjusted camera of the first electronic device capturing the image displayed on the second electronic device; and acquiring the second image based on the first view when the moiré pattern intensity on the first image does not exceed the set threshold. This implementation ensures that the moiré pattern intensity on the image used to extract the graphic code is weak or absent, thereby ensuring high accuracy and efficiency in subsequent graphic code extraction.

[0017] In conjunction with the first aspect, in one possible implementation, the camera setting information includes a focus distance and / or an aperture value; the first electronic device adjusts its camera setting information by: adjusting a first focus distance of the camera of the first electronic device to a second focus distance according to a preset correspondence between image distance and object distance; wherein the first focus distance is the focus distance obtained by the camera's autofocus, and the difference between the first focus distance and the second focus distance is determined based on the distance between the first electronic device and the second electronic device; and / or adjusting a first aperture value of the camera of the first electronic device to a second aperture value, wherein the first aperture value is the aperture value obtained by the camera's autofocus, and the second aperture value is smaller than the first aperture value. This implementation effectively adjusts the camera setting information to reduce (or eliminate) the moiré pattern on the first image.

[0018] In conjunction with the first aspect, in one possible implementation, the method further includes: a first electronic device detecting that the moiré pattern intensity on the fifth image of the second frame does not exceed a set threshold. This implementation ensures that the adjusted image used for extracting the graphic code has weak or no moiré pattern, thereby ensuring high accuracy and efficiency in subsequent graphic code extraction.

[0019] In conjunction with the first aspect, in one possible implementation, the second image is at least two consecutive frames. The first electronic device extracts a graphic code based on the second image, including: first, performing intra-frame and inter-frame difference processing on the at least two frames to obtain a grayscale image corresponding to the difference between the at least two frames; then, performing binarization processing on the grayscale image corresponding to the difference between the at least two frames to obtain a binarized image corresponding to the difference between the at least two frames; and finally, obtaining the graphic code based on the binarized image corresponding to the difference between the at least two frames. This implementation can effectively obtain an accurate graphic code.

[0020] In conjunction with the first aspect, in one possible implementation, the method further includes: a first electronic device acquiring sensor information from both the first electronic device and a second electronic device, the sensor information including inertial measurement unit (IMU) sensor information; the first electronic device then predicts, based on the sensor information from both the first and second electronic devices, that when the distance between the first and second electronic devices is the same as or the difference between them is less than a preset threshold, it adjusts its camera settings and captures the image displayed by the second electronic device through the adjusted camera. Through this implementation, the first electronic device can utilize its ranging capability to anticipate the degree of moiré pattern on the subsequently captured image, thereby enabling it to adjust its camera settings in advance. This reduces the time required for the first electronic device to detect the degree of moiré pattern and / or adjust its camera settings, further improving the efficiency of graphic code extraction.

[0021] In conjunction with the first aspect, in one possible implementation, the method further includes: the first electronic device sending device information of the first electronic device to the second electronic device, and receiving device information of the second electronic device from the second electronic device; wherein the device information includes, but is not limited to, one or more of the following:

[0022] Information that triggers the start of the verification process, including the device type and device version / model.

[0023] Optionally, this implementation can be performed before the first electronic device acquires the first image displayed in the viewfinder of the first electronic device.

[0024] This implementation allows the first and second electronic devices to obtain each other's device information in order to initiate device authentication or pairing processes.

[0025] Secondly, embodiments of this application provide an image processing device, the device comprising multiple functional modules or units (e.g., a communication module, a storage module (optional), a display module (optional), a processing module, an acquisition module / camera module, etc.); the multiple functional modules or units interact to implement the method executed by the first electronic device in the first aspect and any possible implementation thereof. The multiple functional modules or units can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules or units can be arbitrarily combined or divided based on specific implementations.

[0026] Thirdly, embodiments of this application provide an image processing apparatus, the apparatus including at least one processor; the at least one processor is coupled to at least one memory, and when the apparatus is in operation, the at least one processor is configured to read computer programs or instructions stored in the at least one memory to execute the method executed by the first electronic device in the first aspect and any possible implementation thereof.

[0027] In one possible design, the device further includes the at least one memory.

[0028] Fourthly, embodiments of this application provide a program product that, when run on a device, causes the device to execute the method executed by the first electronic device in any of the above aspects and any possible implementations thereof.

[0029] Fifthly, embodiments of this application provide a readable storage medium storing a program that, when executed by a device, causes the device to perform the method executed by the first electronic device in any of the above aspects and embodiments.

[0030] Sixthly, embodiments of this application provide a chip for reading a program stored in a memory and executing the method executed by a first electronic device in any of the above aspects and any possible implementations thereof.

[0031] In a seventh aspect, embodiments of this application provide a chip system including a processor for supporting a device in implementing the method executed by a first electronic device in any of the above aspects and any possible implementations thereof. In one possible design, the chip system further includes a memory for storing the necessary programs and data. The chip system may be composed of chips or may include chips and other discrete devices.

[0032] It should be noted that the technical effects that can be achieved by any possible implementation or design of the second to seventh aspects or any of the second to seventh aspects can be referred to the description of the technical effects that can be achieved by the first aspect and any of the possible implementations, which will not be repeated here. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the superposition of sampled pixels;

[0034] Figure 2 This is a schematic diagram of a QR code authentication scenario;

[0035] Figure 3 A schematic diagram of a system architecture applicable to the method provided in the embodiments of this application;

[0036] Figure 4 This is a schematic diagram of a possible hardware structure of the electronic device in the embodiments of this application;

[0037] Figure 5 This is a schematic diagram of a possible software structure of an electronic device in an embodiment of this application;

[0038] Figure 6A A schematic diagram of the application interface for mobile phone QR code authentication provided in an embodiment of this application;

[0039] Figure 6B A schematic diagram of the application interface for mobile phone QR code authentication provided in an embodiment of this application;

[0040] Figure 7 This is a schematic diagram illustrating the process of real-time image detection and camera setting adjustment during the scanning and authentication stage, as described in an embodiment of this application.

[0041] Figure 8A This is a schematic diagram illustrating the process of extracting the ring code (i.e., graphic code) according to an embodiment of this application;

[0042] Figure 8B These are three schematic diagrams of regions of interest (ROIs) in embodiments of this application;

[0043] Figure 8C This is a schematic diagram of three grayscale images after intra-frame differential processing according to an embodiment of this application;

[0044] Figure 8D This is a schematic diagram of three grayscale images after inter-frame differential processing according to an embodiment of this application;

[0045] Figure 8E This is a schematic diagram of three images after binarization processing according to an embodiment of this application;

[0046] Figure 9 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram of the method flow of Embodiment 1 of this application;

[0048] Figure 11A This is a schematic diagram illustrating the processing for situations where moiré patterns (or severe moiré patterns) exist in the embodiments of this application.

[0049] Figure 11B This is a schematic diagram illustrating the processing in the case where there is no moiré pattern (or the moiré pattern is weak) in the embodiments of this application;

[0050] Figure 12A schematic diagram of the relationship between image distance and object distance provided for an embodiment of this application;

[0051] Figure 13 A schematic diagram illustrating a process for extracting image codes by adjusting camera settings, as provided in this application;

[0052] Figure 14 This is a schematic diagram of the method flow of Embodiment 2 of this application;

[0053] Figure 15 This application provides an alternative flowchart for performing graphic code extraction by adjusting camera settings. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0055] The names and related technical features involved in the embodiments of this application will be explained below. It should be noted that these explanations are for the purpose of making the embodiments of this application easier to understand, and should not be regarded as a limitation on the scope of protection claimed by this application.

[0056] 1) Electronic devices: Electronic devices (also known as terminal devices) can include mobile phones, tablets, laptops, personal digital assistants (PDAs), point of sale (POS) terminals, in-vehicle computers, smartwatches, televisions, and other electronic devices with display screens.

[0057] Typically, electronic devices can support a variety of applications. These may include one or more of the following: travel applications, drawing applications, presentation applications, word processing applications, video player applications, shopping applications, instant messaging applications, photo management applications, camera applications, browser applications, payment applications, and health management applications. Instant messaging applications can be diverse, including SMS applications, various email applications, WeChat, Instagram, and DingTalk. Users can send text, voice, images, video files, and other files to contacts through instant messaging applications. These supported applications can be native applications or quick apps; this application embodiment does not impose specific limitations. The electronic device can also support quick service cards corresponding to these applications, such as order logistics service cards in a shopping application. Furthermore, the electronic device can support quick service cards for both installed and uninstalled applications.

[0058] 2) Pixel: refers to the smallest unit in an image represented by a sequence of numbers.

[0059] A pixel is a small square that makes up an image. Each square has a specific location and a assigned color value. The color and position of the squares determine how the image appears.

[0060] A pixel can be considered an indivisible unit or element within an image. Indivisible means it cannot be further divided into smaller units or elements; it exists as a small square of a single color. Each bitmap image contains a certain number of pixels, which determine the size of the image displayed on the screen.

[0061] 3) Frame: A frame is usually the basic unit of video and animation, representing a single moment in time. In digital video, a frame is a still image; these images are played continuously at a certain speed to form a dynamic picture.

[0062] 4) Moiré patterns: From an optical interference perspective, moiré patterns are high-frequency interference occurring on the photosensitive elements of devices such as digital cameras or scanners, causing colored high-frequency stripes in images. Simply put, it's a waveform interference phenomenon between different objects. Due to its irregularity, it has no obvious shape pattern. Moiré patterns are a manifestation of the beat phenomenon, where the amplitude of the composite signal changes according to the frequency difference between two equal-amplitude sine waves with similar frequencies superimposed. This phenomenon occurs when digital cameras or mobile phone cameras capture objects because the pixel arrangement of the object being photographed, such as an LCD screen, interferes with the pixel arrangement of the phone. The composite light emitted by the projector is reflected by the screen to the camera of the phone or camera, where interference occurs. Inside the device, the light passes through lenses and filters to reach the photosensitive diode, where further interference occurs, resulting in moiré patterns.

[0063] For example, such as Figure 1 As shown, Figure 1 (a) shows a screen that has been photographed, such as a mobile phone screen, a projector screen, or a computer screen. Figure 1 (b) shows the sampling results from the phone or camera sensor. When these results are superimposed, i.e., when the user is about to take a picture, such as... Figure 1 As shown in (c). The foregoing explained why different moiré patterns appear after adjusting the shooting angle. However, if a film camera without pixels is used for shooting, there is no... Figure 1 The image shown in (b) exists, so no matter how they intersect, there will be no irregular interference fringes.

[0064] 5) Graphic Code: A graphic code is a way to convert information into a simple graphic, used to identify and track products, and can also be used to convey certain information or emotions.

[0065] Specifically, for specific information, the information can be converted into a digital code, and then processed graphically to generate a special graphic that can be recognized by machines. This special graphic can be called a graphic code.

[0066] For a specific piece of information, its corresponding graphic code can be considered as an image obtained by encoding that information; conversely, by decoding / analyzing the graphic code, the corresponding information can be obtained. In the embodiments of this application, the graphic code may include QR codes, barcodes, ring codes, etc.

[0067] 6) Interface: An interface refers to the two-dimensional surface on the display screen of an electronic device that is used to present images, videos, text, information, and other content to the user. The visual style of the interface (such as shape, structure, size, etc.) can usually remain fixed or can be flexibly set by the user.

[0068] The content displayed on the interface includes images, text, and rich media, where rich media refers to information dissemination methods that incorporate animation, sound, video, and / or interactivity. Rich media can include streaming media, audio, Flash, and one or a combination of programming languages ​​such as Java, Javascript, and DHTML.

[0069] 7) Visual style: Visual style (abbreviated as "style") can refer to the visual feeling that things present to people, including the shape, structure, size, color, movement and stillness of things.

[0070] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0071] Furthermore, in the description of this application, terms such as "first," "second," or "1," "2" (except in special cases indicating numerical values) are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order. For example, "first electronic device" and "second electronic device" simply indicate two different electronic devices, not their relative importance. Similarly, "distance 1" (the distance between two mobile phones) and "distance 2" (the distance at which moiré patterns occur) simply indicate two different distances, not their relative importance.

[0072] The preceding text introduced some of the terms used in the embodiments of this application. The following text introduces the technical background, applications, and devices involved in the embodiments of this application.

[0073] Image code technology has been increasingly widely used, for example in electronic payment applications for making payments, in social applications for adding friends, and in applications that require scanning images for authentication on electronic devices (such as mobile phones), including new device verification, smartwatch pairing, account login, and one-click screen mirroring. It can also be used in travel applications for entry and exit registration. Some commonly used image codes include QR codes, barcodes, and circular codes.

[0074] Commonly used graphic codes (such as QR codes and barcodes) are black and white patterns, which are difficult to meet aesthetic requirements. Therefore, in some scanning applications, electronic devices (such as smartphones) with image capture and graphic code recognition capabilities can obtain a color image containing the graphic code by scanning / capturing it, extract the graphic code from the captured color image, and then decode (or parse) the graphic code. Once decoded successfully, the information or content carried by the graphic code can be obtained, allowing for related operations.

[0075] For example, the HarmonyOS ecosystem may introduce a hidden code connection experience in the future, enabling users to scan a hidden code animation played on a new phone using an older phone for boot device authentication and to accelerate the phone cloning experience. Specific processes include: Figure 2 As shown, after the new phone is powered on, it displays a welcome message and then broadcasts a message. After receiving the broadcast, the old phone sends a migration confirmation instruction to the new phone. If the new phone accepts the migration from the old phone after receiving the confirmation instruction, it will then proceed with the authentication process between the old and new phones.

[0076] During the authentication phase between the old and new phones, the old phone's screen displays a sophisticated animation, such as a galaxy ring, replacing the traditional QR code scanning method and creating a high-end, refined user experience. The new phone scans the galaxy ring and extracts a sparse ring code from it for authentication. After successful authentication, a high-speed communication connection can be established between the two phones, and data transmission can begin, starting data cloning. The old phone quickly imports or transfers data to the new phone.

[0077] However, at certain distances, high-frequency interference in the image sensors of electronic devices (such as mobile phones) causes colored high-frequency stripes, or moiré patterns, to appear in the captured images. This leads to failure in extracting the graphic code from the image, resulting in decoding failure or requiring multiple scans to achieve successful decoding. Therefore, the accuracy and efficiency of graphic code extraction are crucial factors affecting subsequent decoding and the acquisition of corresponding information or content.

[0078] Therefore, this application provides an image processing method that can effectively improve the accuracy and efficiency of extracting graphic codes. In this application, the method and the device / apparatus are based on the same inventive concept. Since the methods and devices / apparatus solve problems in similar principles, the implementations of the devices / apparatus and methods can be mutually referred to, and repeated details will not be elaborated further.

[0079] The technical solutions in this application embodiment can be applied to, but are not limited to, electronic devices. An electronic device can be any device capable of displaying an interface and / or having a camera function. For example, an electronic device can be such as a mobile phone, a foldable phone, a tablet computer, wearable devices (e.g., watches, bracelets, etc.), in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices (e.g., smart TVs, etc.), projectors, and other electronic devices. It is understood that this application embodiment does not limit the specific type of electronic device. The electronic devices to which this application embodiment can be applied, exemplary embodiments of which include, but are not limited to, devices equipped with… Or an electronic device with another operating system. The electronic device may be, for example, the electronic device described in the following embodiments.

[0080] The technical solutions of this application can be applied to various scenarios where a screen using a light-emitting array sends out graphic codes that are imperceptible to the human eye (i.e., the human eye can see the image displayed on the screen, but cannot see the graphic code). For example, the technical solutions of this application can be applied to scenarios including: scanning a mobile phone screen with a mobile phone camera, scanning a tablet computer screen with a mobile phone camera, scanning a personal computer (PC) screen with a mobile phone camera, scanning a watch screen with a mobile phone camera, scanning a tablet computer screen with a tablet computer camera, scanning a PC screen with a tablet computer camera, and scanning a watch screen with a tablet computer camera.

[0081] Figure 3 A schematic diagram of a system architecture to which the method provided in the embodiments of this application can be applied is shown. See also Figure 3 As shown, the system architecture may include a first electronic device and a second electronic device. In embodiments of this application, the first electronic device and the second electronic device may interact with each other using wireless communication technologies (such as wireless fidelity (Wi-Fi), Bluetooth, Starlink, near field communication (NFC), etc.). The first electronic device and the second electronic device may also interact with each other using wired communication technologies.

[0082] In one scenario, a user can use a first electronic device to take pictures of the images displayed on a second electronic device and its screen. The first electronic device includes a device capable of taking pictures (photos and / or videos), such as a camera; the second electronic device includes a device capable of displaying an interface, such as a screen (also referred to as the screen of an electronic device). Optionally, the first electronic device may also include a device capable of displaying an interface, and the second electronic device may also include a device capable of taking pictures (photos and / or videos).

[0083] In another scenario where a user requires a second electronic device, the user can use the second electronic device to take pictures of the images displayed on the first electronic device and its screen. The first electronic device includes a device capable of displaying an interface, such as a display screen (also referred to as the screen of an electronic device); the second electronic device includes a device capable of taking pictures (photos and / or videos), such as a camera. Optionally, the first electronic device may also include a device capable of taking pictures (photos and / or videos), and the second electronic device may also include a device capable of displaying an interface.

[0084] The above Figure 3 This is an example of a system architecture to which the embodiments of this application may be applied. In practical applications, the embodiments of this application do not limit the specific scenario of the system architecture, for example, compared to Figure 3The system architecture shown in this application embodiment may contain more or fewer electronic devices in the actual system architecture used.

[0085] For example, for the above Figure 3 The first electronic device or the second electronic device in the system shown, Figure 4 A schematic diagram of a possible hardware structure of an electronic device according to an embodiment of this application is shown. See also Figure 4 As shown, the electronic device 400 includes components such as a power supply 410, a processor 420, a memory 430, an input module 440, a display module 450, an audio circuit 460, a communication interface 470, and a camera 480. Those skilled in the art will understand that... Figure 4 The hardware structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device 400. The electronic device 400 provided in the embodiments of this application may include more or fewer components than shown, may combine two or more components, or may have different component configurations. Figure 4 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0086] The following is combined with Figure 4 A detailed description of each component of the electronic device 400 is provided below:

[0087] The power supply 410 (such as a battery) is used to power various components. Optionally, the power supply 410 can be logically connected to the processor 420 through a power management system, thereby enabling the power management system to manage functions such as charging, discharging, and power consumption.

[0088] The processor 420 is the control center of the electronic device 400. It connects various components via various interfaces and lines, and executes software programs and / or modules stored in the memory 430, as well as calling data stored in the memory 430, to perform various functions and process data of the electronic device 400, thereby enabling various services based on the electronic device 400. In this embodiment, the processor 420 can be used to implement an image processing method provided in this embodiment.

[0089] The memory 430 can be used to store software programs and modules. The processor 420 executes various functional applications and data processing of the electronic device 400 by running the software programs and modules stored in the memory 430. Optionally, the memory 430 may mainly include a program storage area and a data storage area. The program storage area may store the operating system (mainly including the software programs or modules corresponding to the kernel layer, system layer, application framework layer, and application layer). In addition, the memory 430 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0090] The input module 440 can be used to receive editing operations on various types of data objects, such as numbers or characters, input by the user, and to generate key signal inputs related to user settings and function control of the electronic device 400. Optionally, the input module 440 may include a touch panel 441 and other input devices 442. The touch panel 441, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 441), and drive the corresponding connection devices according to a pre-set program.

[0091] Optionally, other input devices 442 may include, but are not limited to, one or more of the following: a physical keyboard, an infrared sensor, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick. For example, an infrared sensor can be used to detect the user's air gestures.

[0092] Display module 450 (also called display screen, display unit, or presentation unit) can be used to display information input by the user or information provided to the user, as well as various menus of electronic device 400. Display module 450 is the display system of electronic device 400, used to present the interface and realize human-computer interaction. Display module 450 may include display panel 451. Optionally, display panel 451 can be configured using liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar forms.

[0093] In this embodiment, the display module 450 can be used to display, but is not limited to, the images / patterns captured by the camera 480.

[0094] In the embodiments of this application, if the electronic device 400 is an electronic device being scanned / photographed, then it includes a display module 450; if the electronic device 400 is an electronic device being scanned / photographed, it may include a display module 450 or may not include a display module 450, and there is no limitation on this.

[0095] Audio circuitry 460, microphone 461, and speaker 462 provide an audio interface between the user and electronic device 400. Audio circuitry 460 converts audio data into signals recognizable by speaker 462 and transmits these signals to speaker 462, where they are converted into sound signals for output. Microphone 461 collects external sound signals (such as human speech or other sounds) and converts these signals into signals recognizable by audio circuitry 460, sending them to audio circuitry 460. Audio circuitry 460 can also convert the signals transmitted by microphone 461 into audio data and output the audio data to RF circuitry for transmission to, for example, another electronic device, or output the audio data to memory 430 for further processing.

[0096] The electronic device 400 can physically connect to other devices via the communication interface 470. Optionally, the communication interface 470 can be connected to the communication interfaces of the other devices via a cable to enable data transmission between the electronic device 400 and the other devices.

[0097] In this embodiment, the electronic device 400 is capable of communication services and interacting with other electronic devices. Therefore, the electronic device 400 needs to have data transmission capabilities. The electronic device 400 not only includes... Figure 4 The communication interface 470 shown may also contain other communication modules, which will not be described in detail here.

[0098] Furthermore, in this embodiment, if the electronic device 400 is used as a scanning / capturing electronic device, the electronic device 400 may also include at least one camera 480 and at least one sensor, etc., although Figure 4 Sensors are not shown in the diagram and will not be listed here. At least one sensor may include, but is not limited to, pressure sensors, barometric pressure sensors, accelerometers, distance sensors, fingerprint sensors, touch sensors, and temperature sensors. If the electronic device 400 is used as a scanned / photographed electronic device, it may also include at least one camera 480 and / or at least one sensor, or it may not include at least one camera 480 and / or at least one sensor; no specific limitations are imposed.

[0099] The operating system (OS) involved in this application embodiment is the most basic system software running on the electronic device 400. The software system of the electronic device 400 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes an operating system adopting a layered architecture as an example to illustrate the software architecture of the electronic device 400.

[0100] It should be understood that in practical applications, Figure 4 The electronic device 400 shown is merely an example and does not specifically limit the structure of the electronic device in the embodiments of this application. Furthermore, the electronic device 400 may have a more... Figure 4 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. Figure 4 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0101] The software system of the electronic device 400 provided in this application embodiment can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes a layered architecture as an example. Figure 5 A schematic diagram of a possible software structure of an electronic device according to an embodiment of this application is shown.

[0102] Layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example... Figure 5 As shown, this software architecture can be divided into four layers, from top to bottom: the application layer, the application framework layer (FWK), the system library, and the kernel layer. Additionally, electronic devices may also include a hardware layer.

[0103] The application layer is the top layer of the operating system, including the operating system's native applications as well as third-party applications, such as camera, gallery, calendar, Bluetooth, music, video, messaging, and so on.

[0104] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer may include predefined functions. It may include a window manager, content provider, view system, resource manager, notification manager, etc.

[0105] The window manager provides window management services (WMS). It can obtain the screen size, determine the presence of a status bar, lock the screen, and capture screenshots. The window manager offers various management functions to control parameters such as the transparency, position, and size of windows or interfaces on the screen.

[0106] Content providers are used to store and retrieve data, and make that data accessible to applications. This data may include files (e.g., documents, videos, images, audio), text, and other information.

[0107] A view system includes visual controls, such as controls that display text, images, documents, and other content. View systems can be used to build applications. The interface in a display window can consist of one or more controls. For example, a display interface showing a file icon can include controls for displaying text and controls for displaying images.

[0108] The system library can include multiple functional modules. For example: a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), and image processing libraries. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0109] The kernel layer provides the core system services of the operating system, such as security, memory management, process management, network protocol stack, and driver models, all of which are implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. This layer contains many drivers related to electronic devices, including: display drivers; keyboard drivers as input devices; Flash drivers for memory-based devices; camera drivers; audio drivers; Bluetooth drivers; and Wi-Fi drivers.

[0110] The hardware layer can include various sensors, displays, camera devices, etc.

[0111] It should be noted that, Figure 5This is merely an example of a software architecture diagram for an electronic device, simply listing some layers and software modules. In practical applications, the operating system of an electronic device may include other layers, and each layer may include other software modules for implementing one or more functions or services. This application does not specifically limit this aspect. Furthermore, this application does not limit the specific layer at which each software module resides.

[0112] It should be understood that the hardware structure of the electronic device in the embodiments of this application can be as follows: Figure 4 As shown, the software system architecture of electronic devices can be as follows: Figure 5 As shown, the software programs and / or modules corresponding to the software system architecture in the electronic device can be stored in the memory 430, and the processor 420 can run the software programs and applications stored in the memory 430 to execute the flow of an image processing method provided in this application embodiment.

[0113] Based on the above Figure 3 The schematic diagram of the system architecture shown uses mobile phone 1 and mobile phone 2 as examples of the first and second electronic devices, respectively, to illustrate the application effects that the method provided in this application embodiment can achieve from the perspective of its interface display effect. It should be noted that the screens presented by the interfaces of the electronic devices described below are exemplary demonstrations of the application effects of this application embodiment and do not constitute a limitation on the actual screens and visual styles presented.

[0114] Example 1: Figure 6A-Figure 6B An example of the application interface for mobile phone QR code authentication according to an embodiment of this application is shown.

[0115] Mobile phones 1 and 2 have Bluetooth and WLAN communication functions enabled, as shown in Figure 6A. Mobile phone 1 automatically enables the connection function, and a device connection diagram is displayed on screen 601, searching for nearby devices. When mobile phone 1 finds nearby mobile phone 2, it automatically initiates the scanning and authentication function. At this time, the screen of mobile phone 1 displays a scanning and authentication viewfinder, as shown in screen 602. After mobile phone 2 responds to the search of mobile phone 1, the screen of mobile phone 2 displays a connection verification screen 603, which shows a dynamic starry sky image. Then, the user can use mobile phone 1 to scan mobile phone 2, that is, mobile phone 1 scans the starry sky image displayed in screen 603 of mobile phone 2.

[0116] Furthermore, such as Figure 6BAs shown in Figure (1), mobile phone 1 scans the galaxy ring screen displayed on mobile phone 2. At this time, mobile phone 2 and its interface are displayed in real time in the preview stream on the screen of mobile phone 1. Users can view the interface of mobile phone 2 and the galaxy ring screen in it in real time through the preview stream of mobile phone 1. The viewfinder (circular circle) displayed on the screen of mobile phone 1 can be used to select and display the galaxy ring screen in the interface of mobile phone 2 in real time.

[0117] Example 2: Figure 7 This illustrates the process of real-time image detection and camera setting adjustment during the scanning and authentication phase in an embodiment of this application.

[0118] In this embodiment, mobile phone 1 can capture the starry ring image displayed by mobile phone 2 in its viewfinder in real time and detect whether there are moiré patterns on the starry ring image. If mobile phone 1 detects moiré patterns on the starry ring image displayed in its viewfinder (or the moiré patterns are severe, for example, the moiré patterns are higher than a set threshold), it can automatically adjust its camera settings (such as the camera's focal length or aperture value) to eliminate or reduce the moiré patterns. After that, mobile phone 1 extracts the ring code from the starry ring image displayed in its viewfinder. If mobile phone 1 detects no moiré patterns on the starry ring image displayed in its viewfinder (or the moiré patterns are mild, for example, the moiré patterns are not higher than a set threshold), mobile phone 1 can directly extract the ring code from the starry ring image displayed in its viewfinder without adjusting its camera settings.

[0119] For example, see Figure 7 As shown, taking an image captured by phone 1 and then scanning it with phone 2 as an example, if moiré patterns are detected in the image, such as... Figure 7 As shown in (a), mobile phone 1 can automatically adjust its camera settings (such as the camera's focal length or aperture value) to reduce the sharpness of the captured image. After adjusting the camera settings, mobile phone 1 takes a screenshot of the starry ring image displayed in its viewfinder, as shown in (a). Figure 7 As shown in (b), the system detects whether there are moiré patterns on the adjusted image. If no moiré patterns or only slight moiré patterns are detected, the ring code extraction process can be performed. If mobile phone 1 detects that there are no moiré patterns (or only slight moiré patterns) on the starry ring image displayed in its viewfinder, such as... Figure 7 As shown in (c), mobile phone 1 can directly perform the ring code extraction process based on the starry ring image displayed in its viewfinder.

[0120] Example 3: In the ring code extraction stage, Figure 8A A schematic diagram of the extraction of a ring code (i.e., an example of a graphic code) according to an embodiment of this application is shown.

[0121] Phone 1 scans the galaxy ring image displayed on Phone 2, and based on the galaxy ring image displayed on its own screen, Phone 1 obtains multiple frames of images. Figure 8A This diagram illustrates how mobile phone 1 scans a frame captured by mobile phone 2. First, a circular region is cropped based on the image of each captured frame, as shown below. Figure 8A As shown in (1). Then, based on the images of multiple captured frames, intra-frame difference and inter-frame difference processing are performed to obtain the difference results of the images of each captured frame, as shown in Figure (1). Figure 8A As shown in (2). Next, binarization is performed based on the difference results of the images of each captured frame to obtain the corresponding binarized image, as shown in (2). Figure 8A As shown in (3). Finally, based on the binarized images of multiple shooting frames, an image for performing localization point extraction is obtained, as shown in [image 1]. Figure 8A As shown in (4).

[0122] For example, taking three consecutive frames of images obtained by scanning phone 2 with phone 1 as an example, such as... Figure 8B The three region of interest (ROI) images shown, namely image 1, image 2, and image 3, are used for the ring code extraction process, which includes the following steps:

[0123] Step 1: First, perform intra-frame differencing on these three images to obtain three grayscale images after intra-frame differencing, such as... Figure 8C As shown, image 1 corresponds to image 1 after intra-frame differential processing, image 2 corresponds to image 2 after intra-frame differential processing, and image 3 corresponds to image 3 after intra-frame differential processing.

[0124] Step 2: Perform inter-frame differencing on these three images to obtain three grayscale images after inter-frame differencing, as shown below. Figure 8D As shown, image 1 corresponds to grayscale image 1, image 2 corresponds to grayscale image 2, and image 3 corresponds to grayscale image 3. These three grayscale images also need to be aligned and stabilized, which will not be described in detail here.

[0125] Step 3: Binarize these three grayscale images to obtain the corresponding binarized images, such as... Figure 8E As shown in (1), grayscale image 1 corresponds to the binarized image BW1, grayscale image 2 corresponds to the binarized image BW2, and grayscale image 3 corresponds to the binarized image BW3.

[0126] Step 4: Based on images BW1, BW2, and BW3, process them using image processing algorithms to remove noise, obtaining an image with no noise (or very few noise), such as... Figure 8E As shown in (2).

[0127] Finally, forFigure 8E The sparse ring code shown in (2) can be decoded / parsed to obtain the corresponding information or content.

[0128] It should be noted that the embodiments of this application are not limited to the mobile phone automatically adjusting the camera settings when it detects the presence of moiré patterns. Users can also manually adjust the camera settings of the mobile phone (such as the camera's focal length or aperture value). There is no limitation on this.

[0129] The above example uses a mobile phone as an electronic device to illustrate the application effect achieved by the method of this application embodiment. Of course, for other electronic device application scenarios, the corresponding interface display effect can also be achieved by referring to the above description, and will not be shown one by one.

[0130] The above describes the possible application effects of the method provided by the embodiments of this application from the perspective of electronic device display. The following describes an image processing method provided by this application from the perspective of technical implementation, that is, how to use the method provided by this application to achieve an interface effect with no moiré pattern or reduced moiré pattern, so as to improve the accuracy and efficiency of extracting graphic codes (or scanning codes).

[0131] Figure 9 This document illustrates a flowchart of an image processing method provided in an embodiment of this application. This method can be applied to electronic devices (also known as terminal devices), including but not limited to personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, tablets, etc.), in-vehicle computers, smartwatches, televisions, and other electronic devices with acquisition (capture) functions. The method can be executed by the electronic device, or by a corresponding component of the electronic device (such as a chip, chip system, or circuit, etc.). Of course, the method can also be executed by other devices or apparatuses, for example, by the cloud (such as a cloud server). Therefore, this application does not limit the subject executing the method, its specific structure, or its number, as long as it can be processed according to the method provided in the embodiments of this application by running a program that records the code of the method.

[0132] For example, this method is described using a first electronic device (i.e., the electronic device taking the picture). The order of the steps in the following processes is merely an example. In practical applications, the execution order of the steps in each process can be adjusted, and all or some of the steps described below can be executed adaptively. See also Figure 9 As shown, the method may include the following steps:

[0133] S901: The first electronic device acquires the first image displayed in the viewfinder of the first electronic device. The first image is acquired by the camera of the first electronic device from the image displayed by the second electronic device. The image displayed by the second electronic device implicitly contains a graphic code for information verification.

[0134] In this embodiment, the screen displayed by the second electronic device can be a dynamic screen or a static image, and there is no limitation thereto. The camera of the first electronic device can capture the screen displayed by the second electronic device in real time and display it in real time through the viewfinder (or preview stream) on the screen of the first electronic device.

[0135] Furthermore, the image displayed by the second electronic device implicitly contains a graphic code for information verification. This can be understood as follows: without multi-frame joint image processing of the image, the user or machine cannot directly obtain the graphic code from the image. The graphic code can be a ring code, QR code, barcode, etc.

[0136] In one possible implementation, the second electronic device may display a galaxy ring image.

[0137] S902: The first electronic device detects the degree of moiré pattern on the first image of the first screen.

[0138] For example, the degree of moiré pattern on the first image of the first frame can be detected and obtained by the processor of the first electronic device.

[0139] In one possible implementation, the first image can be a ROI image obtained by the first electronic device taking a screenshot of a frame of the first scene displayed in its own viewfinder.

[0140] In this embodiment of the application, when the first electronic device executes S902, it may adopt, but is not limited to, the following implementation methods:

[0141] Implementation Method 1: The first electronic device detects the degree of moiré pattern on the first image of the first screen, including the following:

[0142] First, intra-frame difference processing is performed on the first image to obtain the third image after intra-frame difference processing; then, binarization processing is performed on the third image to obtain the fourth image after binarization processing; next, based on the fourth image, the proportion of the first region in the fourth image is counted, and the proportion of the first region in the fourth image is determined as the moiré pattern degree on the first image; wherein, the first region is the region in the fourth image where the pixel brightness exceeds a set threshold.

[0143] In implementation method one, based on the first image, a fourth image is obtained through the above processing steps, and the proportion of the first region in the fourth image is used as the moiré pattern intensity on the first image. In implementation method one, it is equivalent to quantifying the moiré pattern intensity on the first image as the proportion of the first region in the fourth image.

[0144] Implementation Method 2: The first electronic device detects the degree of moiré pattern on the first image of the first screen, including: firstly, determining the difference between the distance between the first electronic device and the second electronic device and the distance at which the moiré pattern occurs, based on the distance between the first electronic device and the second electronic device and the distance at which the moiré pattern occurs; then, determining the degree of moiré pattern on the first image based on the difference between the distance between the first electronic device and the second electronic device and the distance at which the moiré pattern occurs; wherein, the difference is inversely proportional to the degree of moiré pattern on the first image.

[0145] In implementation method two, the degree of moiré pattern is determined based on the difference between the distance between the first and second electronic devices and the distance at which the moiré pattern occurs. The smaller this difference, the closer the distance between the first and second electronic devices is to the distance at which the moiré pattern occurs, and therefore the greater the degree of moiré pattern. Conversely, the larger the difference, the further the distance between the first and second electronic devices is from the distance at which the moiré pattern occurs, and therefore the less the degree of moiré pattern. In implementation method two, the degree of moiré pattern is essentially quantified as the difference between the distance between the first and second electronic devices and the distance at which the moiré pattern occurs.

[0146] In practical applications, there can be various ways to quantify the degree of moiré pattern. Therefore, in this embodiment, the method by which the first electronic device detects the degree of moiré pattern on the first image is not limited to the above-mentioned implementation methods.

[0147] Based on the above-mentioned second implementation method, in one possible implementation method, before the first electronic device detects the degree of moiré pattern on the first image of the first screen, the method of this application embodiment further includes: the first electronic device determining the distance between the first electronic device and the second electronic device.

[0148] In this embodiment of the application, the first electronic device determines the distance between the first electronic device and the second electronic device, which can be achieved in, but is not limited to, the following ways:

[0149] Method 1: The first electronic device acquires binocular parallax information; based on the binocular parallax information, the distance between the first electronic device and the second electronic device is determined.

[0150] Method 2: The first electronic device acquires the size information of the first screen; based on the size information of the first screen, the distance between the first electronic device and the second electronic device is determined.

[0151] For example, the first screen displays a galaxy ring. The first electronic device can determine the distance between the first electronic device and the second electronic device based on the size information of the galaxy ring displayed on the screen of the first electronic device and / or the size information of the galaxy ring displayed on the screen of the second electronic device.

[0152] Method 3: The first electronic device measures the distance between the first electronic device and the second electronic device using its ranging device.

[0153] For example, the first electronic device measures the distance between the first electronic device and the second electronic device using a dtof device.

[0154] Based on the above-described second implementation method, in one possible implementation method, before the first electronic device detects the degree of moiré pattern on the first image of the first screen, the method of this application embodiment further includes: the first electronic device determining the distance at which the moiré pattern occurs.

[0155] Determining the distance where moiré patterns occur may involve the first electronic device acquiring information from a second electronic device, the information of which includes, but is not limited to, the pixel count and / or size of the screen of the second electronic device; and the first electronic device then determining the distance where moiré patterns occur based on the information from the first electronic device and the information from the second electronic device; wherein the information from the first electronic device includes at least one of the following: the pixel count and size of the screen of the first electronic device, and the spatial resolution of the camera of the first electronic device.

[0156] In this embodiment, the device used to determine the distance (or the distance at which moiré patterns occur) between the first electronic device and the second electronic device is not limited to the first electronic device itself; it may also be determined by other devices and provided to the first electronic device. For example, the second electronic device may measure / calculate and obtain the distance (or the distance at which moiré patterns occur) between the first electronic device and the second electronic device, and then send this distance to the first electronic device.

[0157] S903: The first electronic device acquires a second image obtained by the camera of the first electronic device capturing the image displayed by the second electronic device based on the moiré pattern on the first image.

[0158] In one possible implementation, the first electronic device acquires a second image obtained by its camera capturing the image displayed by the second electronic device, based on the moiré pattern intensity of the first image. This includes the following scenarios:

[0159] Scenario 1: If the moiré pattern on the first image exceeds a set threshold, the first electronic device adjusts its camera settings. Based on the second image in the viewfinder, a second image is acquired. The second image is obtained by the camera of the first electronic device capturing the image displayed on the second electronic device after the adjustment.

[0160] Scenario 2: If the moiré pattern on the first image does not exceed the set threshold, the first electronic device acquires the second image based on the first image.

[0161] In this embodiment, regarding the image displayed by the first electronic device by capturing / capturing the screen displayed by the second electronic device (such as the first screen or the second screen), moiré patterns appear on the displayed image. The degree of these moiré patterns (i.e., the moiré pattern intensity) affects the accuracy of subsequent graphic code extraction. In practical applications, there can be multiple ways to quantify the moiré pattern intensity (such as implementation method one and implementation method two in S902 above). This application is not limited to any one of these methods. Regardless of the quantization method used, the moiré pattern intensity on the image displayed by the first electronic device (such as the first screen or the second screen) can be understood as the degree of influence of the moiré pattern on the accuracy of graphic code extraction. For example, the weaker / smaller the moiré pattern intensity, the higher the accuracy of graphic code extraction; the more severe / larger the moiré pattern intensity, the lower the accuracy of graphic code extraction.

[0162] Furthermore, the threshold set above can be understood as a limit affecting the accuracy of the extracted graphic code. Different quantization methods for moiré pattern intensity may lead to different definitions and standards for the threshold. However, in this application, regardless of the quantization method used to quantify the moiré pattern intensity, the corresponding threshold or limit set (in this application, "set" can be understood as "determined") should ensure high accuracy in subsequent graphic code extraction. Moreover, the set threshold can be a fixed value or a variable value. For example, for the moiré pattern intensity detected in implementation methods one and two in S902 above, the threshold set for reference / comparison can vary with the distance between the first electronic device and the second electronic device.

[0163] Based on the above scenario one, in one possible implementation, the camera setting information includes focus distance and / or aperture value; the first electronic device adjusts its camera setting information by: adjusting a first focus distance of the camera of the first electronic device to a second focus distance according to a preset correspondence between image distance and object distance; wherein, the first focus distance is the focus distance obtained by the camera's autofocus, and the difference between the first focus distance and the second focus distance is determined based on the distance between the first electronic device and the second electronic device; and / or

[0164] The first electronic device adjusts the first aperture value of the camera of the first electronic device to a second aperture value. The first aperture value is the aperture value automatically set by the camera, and the second aperture value is smaller than the first aperture value.

[0165] Based on the above second scenario, in one possible implementation, before the first electronic device acquires the second image based on the first screen, the method of this application embodiment may further include: the first electronic device detecting that the moiré pattern degree on the fifth image of the second screen does not exceed a set threshold.

[0166] If the moiré pattern on the fifth image of the second frame exceeds the set threshold, the first electronic device can continue to adjust its camera settings in the manner described above.

[0167] In one possible implementation, the fifth image detected by the first electronic device in the second screen can be a ROI image obtained by the first electronic device taking a screenshot of a frame of the second screen displayed in the viewfinder.

[0168] S904: The first electronic device extracts the graphic code based on the second image.

[0169] In one possible implementation, the second image is at least two consecutive frames;

[0170] Specifically, in case one of S903 above, the second image can refer to an image obtained by the first electronic device taking screenshots of at least two consecutive frames of the first view in the viewfinder. In case two of S903 above, the second image can refer to an image obtained by the first electronic device taking screenshots of at least two consecutive frames of the second view in the viewfinder.

[0171] The first electronic device extracts a graphic code based on the second image, which may include: first performing intra-frame difference processing and inter-frame difference processing on the at least two frames of images to obtain a grayscale image corresponding to the difference between the at least two frames of images; then performing binarization processing on the grayscale image corresponding to the difference between the at least two frames of images to obtain a binarized image corresponding to the difference between the at least two frames of images; and finally obtaining a graphic code based on the binarized image corresponding to the difference between the at least two frames of images.

[0172] In summary, this application provides an image processing method that can be applied to, but is not limited to, a first electronic device. The method includes: firstly, acquiring a first image displayed in the viewfinder of the first electronic device, wherein the first image is captured by the camera of the first electronic device from an image displayed by a second electronic device, and the image displayed by the second electronic device implicitly contains a graphic code for information verification; then, detecting the degree of moiré pattern on the first image of the first image; then, based on the degree of moiré pattern on the first image, acquiring a second image obtained by the camera of the first electronic device from the image displayed by the second electronic device; and finally, extracting the graphic code based on the second image. In this method, when the first electronic device captures an image implicitly containing a graphic code, the image used for extracting the graphic code is obtained based on the degree of moiré pattern present in the image. This not only effectively improves the accuracy and efficiency of graphic code extraction but also better meets aesthetic requirements compared to traditional black and white graphic codes.

[0173] The following is based on the above. Figure 9 The method shown is applied to a mobile phone authentication scenario as an example. The method of this application embodiment will be described in detail through several specific implementation methods.

[0174] In the following implementation, the first electronic device is mobile phone 1, and the second electronic device is mobile phone 2. Taking the need to transfer / import data stored in mobile phone 2 to mobile phone 1 as an example, when mobile phone 1 is the mobile phone to be verified by mobile phone 2, mobile phone 1 can scan the starry sky ring screen displayed on the screen of mobile phone 2 to extract a valid graphic code. Then, by parsing the graphic code, the corresponding authentication information / pairing information can be obtained, so that mobile phone 1 can pair and authenticate with mobile phone 2. After successful authentication, mobile phone 1 can establish a high-speed communication connection with mobile phone 2 and receive data from mobile phone 2.

[0175] Implementation Method 1:

[0176] In Implementation Method 1, it is described in detail how, during the authentication stage where mobile phone 1 scans the starry ring displayed on the screen of mobile phone 2, it detects the degree of moiré pattern on the screen based on the detection method shown in Implementation Method 1 of S902 above, and decides whether to adjust the camera settings based on the degree of moiré pattern to improve the accuracy and efficiency of subsequent graphic code extraction. See also Figure 10 As shown, the method flow of Implementation Method 1 may include the following steps:

[0177] S1001: Mobile phone 1 and mobile phone 2 enable Bluetooth communication and exchange information between the devices via Bluetooth.

[0178] Mobile phone 1 and mobile phone 2 first exchange information (such as scanning instructions or authentication instructions) via Bluetooth (or wireless communication technologies such as Wi-Fi, StarFlash, NFC, etc.), and trigger the process of both parties entering the Star Ring display and scanning.

[0179] In one alternative implementation, mobile phone 1 and mobile phone 2 can exchange device information through communication functions such as Bluetooth. For example, mobile phone 1 can receive information such as the device type and version model of mobile phone 2.

[0180] S1002: Mobile phone 1 scans the starry sky image displayed on mobile phone 2 and displays the scanned starry sky image on the screen of mobile phone 1 in real time (as described above). Figure 9 (Example of the first screen shown).

[0181] After phone 1 enables the QR code authentication function, it enters the QR code scanning application, which means that phone 1's camera function is activated, and a preview stream (also known as a viewfinder) appears on phone 1's screen. The preview stream provides a real-time preview of the image / view before phone 1 takes a picture or video. At the same time, phone 2 enters the scanning authentication process, and a starry sky ring image is displayed on phone 2's screen.

[0182] Users can point the camera of phone 1 at the galaxy ring on the screen of phone 2. The galaxy ring on phone 2 will be displayed in real time in the preview stream of phone 1, and the processor of phone 1 will detect whether the galaxy ring exists in the preview stream in real time.

[0183] In this embodiment of the application, the method by which mobile phone 1 detects whether there is a galaxy ring in the preview stream can be implemented using existing image / screen detection methods, which will not be described in detail here.

[0184] S1003: The degree of moiré pattern on the scanned star ring image detected by mobile phone 1.

[0185] That is, mobile phone 1 can automatically detect the moiré pattern of the starry ring image displayed in real time in the preview stream (or viewfinder) on its own screen.

[0186] In one possible implementation, mobile phone 1 can capture (screenshot) the galaxy ring image displayed in the preview stream to obtain an image of the galaxy ring ROI region, i.e., image 1 (as described above). Figure 9 (Example of the first image in the illustrated scheme). Furthermore, mobile phone 1 can determine the degree of moiré pattern on the galaxy ring image scanned by mobile phone 1 by detecting the degree of moiré pattern present on image 1.

[0187] In this first embodiment, mobile phone 1 detects image 1 (as described above). Figure 9 The specific steps for assessing the moiré pattern on the example of the first image in the illustrated scheme may include the following:

[0188] Step 1: Perform intra-frame difference processing on the galaxy ring ROI region in image 1 to obtain grayscale values. Figure 1 .

[0189] For example, based on image 1, a grayscale image is obtained by using the following formula 1 based on the three RGB values ​​of each pixel.

[0190] Each pixel in a grayscale image = 2*B-1*G-1*R, Formula 1;

[0191] In this context, "*" represents the multiplication sign.

[0192] If moiré patterns exist in image 1, the grayscale image can make the colored moiré patterns stand out.

[0193] Step 2: Binarize the grayscale image to obtain the binary image.

[0194] In this embodiment of the application, in the binarization process of the grayscale image, the threshold 1 used can be a statistical value, that is, the threshold 1 is a threshold that can filter out 90% of the information from the grayscale image histogram to obtain the binarized image.

[0195] Step 3: Determine whether the density of the white region in the binarized image is higher than the threshold 2.

[0196] In the binarized image, the white area can refer to the region where the brightness of the pixels in the image is higher than a certain threshold.

[0197] In one possible implementation, the ratio of the area of ​​the white region in the binarized image to the total area of ​​the image is calculated, and this ratio is used as the density of the white region.

[0198] In this embodiment of the application, the density of the white region in the binarized image, or the ratio of the area of ​​the white region in the binarized image to the total area of ​​the image, can be determined as the moiré pattern degree on image 1.

[0199] S1004: Mobile phone 1 determines whether the moiré pattern intensity is higher than the set threshold.

[0200] For example, based on step three in S1003 above, if the density of the white region in the binarized image is higher than / exceeds a threshold of 2 (as described above) Figure 9 If the threshold set in the scheme shown is an example, then if it is determined that the moiré pattern on image 1 is severe, then the following S1005 is executed.

[0201] If the density of the white region in the binarized image is not higher than / does not exceed the threshold 2 (as mentioned above) Figure 9 If the threshold set in the scheme shown is used as an example, and it is determined that there is no moiré pattern or the moiré pattern is weak on the image 1, then the following S1006 is executed.

[0202] In the above, the threshold 2 can change with the distance between mobile phone 1 and mobile phone 2, that is, the threshold 2 can be determined based on the distance between mobile phone 1 and mobile phone 2.

[0203] For example, Figure 11A This diagram illustrates the processing steps for cases with moiré patterns (or severe moiré patterns). See also... Figure 11A As shown, Figure 11A Image (1) is the image after intra-frame difference processing of image 1. Figure 11A In (2), the image is binarized. When the density of the white area in the binarized image is higher than the threshold 2, it is determined that there is moiré pattern (or the degree of moiré pattern is more serious) on the image 1.

[0204] For example, Figure 11B This diagram illustrates the processing steps for cases where there is no moiré pattern (or a very weak moiré pattern). See also... Figure 11B As shown, Figure 11B Image (1) is the image after intra-frame difference processing of image 1. Figure 11B (2) is the image after binarization. When it is determined that the density of the white area in the image after binarization is not higher than the threshold 2, it is determined that there is no moiré pattern (or the degree of moiré pattern is weak) on the image 1.

[0205] S1005: When the moiré pattern exceeds the set threshold, mobile phone 1 adjusts its camera settings and rescans the star ring image.

[0206] That is, phone 1 adjusts its own camera settings (such as camera focus distance, aperture, etc.), and after adjusting the camera settings, rescans / captures the starry sky image displayed on phone 2's screen, and displays the scanned / captured starry sky image in real time in phone 1's screen preview stream (as described above). Figure 9 (Example of the second screen shown).

[0207] In one possible implementation, mobile phone 1 can reduce the image sharpness by adjusting the focus distance of its own camera (or the aperture of the camera, etc.), thereby reducing or eliminating the moiré pattern on the captured starry ring image.

[0208] For example, when mobile phone 1 reduces or eliminates the moiré pattern on the captured starry sky image by adjusting the focal length value (i.e., the camera's focusing distance), it can achieve this in the following ways:

[0209] Method 1: Based on the focal length value c1 obtained by mobile phone 1 through autofocus and the preset correspondence between focal length and object distance, mobile phone 1 determines a new focal length value c2, that is, adjusts the focal length value of mobile phone 1's camera from c1 to c2. c2 can satisfy the following formula:

[0210] c2 = f(d2); Formula 2;

[0211] Where d2=d1±Δd, since c1=f(d1), we can get d1=f-1(c1); the value of Δd is related to the distance between mobile phone 1 and mobile phone 2 (that is, the value of Δd can be determined according to the distance between mobile phone 1 and mobile phone 2).

[0212] For example, Figure 12 A schematic diagram showing the relationship between image distance and object distance (the distance between mobile phone 1 and mobile phone 2) is provided. (See attached diagram.) Figure 12 As shown, as the object distance increases, the image distance gradually decreases and then tends to a certain value.

[0213] Method 2: Based on the degree of moiré pattern on the scanned / acquired star ring image detected in S1003 above (i.e., the degree of moiré pattern on image 1 detected by mobile phone 1 above), mobile phone 1 determines a new focal length value c2, and adjusts the focal length value of the camera of mobile phone 1 from the focal length value c1 obtained by autofocus to c2.

[0214] For example, if mobile phone 1 detects that the degree of moiré pattern on the galaxy ring it scans is represented by a first value, mobile phone 1 can determine a new focal length value c2 based on the first value and a preset mapping relationship. The preset mapping relationship represents the correspondence between the degree of moiré pattern and the focal length value, and the preset mapping relationship includes the mapping relationship between the first value and c2.

[0215] Method 3: Mobile phone 1 can dynamically adjust the camera's focal length based on the intensity of moiré patterns on the scanned / collected star ring image in real time, until the intensity of moiré patterns on the scanned / collected star ring image is weak or there are no moiré patterns.

[0216] Mobile phone 1 can dynamically adjust the camera's focal length using a trial-and-error method, and detect the degree of moiré pattern on the scanned / acquired galaxy ring image after adjustment. If the degree of moiré pattern on the scanned / acquired galaxy ring image is severe (i.e., the degree of moiré pattern is higher than the set threshold), mobile phone 1 can continue to adjust the camera's focal length until the degree of moiré pattern on the scanned / acquired galaxy ring image is weak or there is no moiré pattern (i.e., the degree of moiré pattern is not higher than the set threshold). At this point, the adjustment of the camera's focal length can be stopped, and the following step S1006 can be executed.

[0217] For example, such as Figure 13The flowchart shown includes: Step a: Mobile phone 1 determines whether the moiré pattern on the galaxy ring image it scans is higher than the set threshold; if yes, then step b is executed, that is, mobile phone 1 adjusts its camera settings and rescans the galaxy ring image; if no, then step c is executed, that is, mobile phone 1 extracts and decodes the ring code based on the galaxy ring image it scans.

[0218] Of course, in this embodiment, mobile phone 1 can also reduce or eliminate moiré patterns on the captured starry sky image by adjusting the camera's aperture or other camera settings or parameters. The method by which mobile phone 1 adjusts the camera's aperture or other camera settings or parameters can be implemented by referring to one or more of the methods described above for adjusting the camera's focal length, and will not be detailed here.

[0219] In this embodiment, after adjusting the camera settings, mobile phone 1 can re-detect the moiré pattern intensity on the scanned / captured galaxy ring image. For example, after adjusting the camera settings, mobile phone 1 can take a screenshot of the re-scanned / captured galaxy ring image to obtain an image of the galaxy ring ROI region (as described above). Figure 9 (Example of the fifth image in the scheme shown) Then detect the degree of moiré pattern on the image. The specific method and steps for detecting the degree of moiré pattern on the image can be implemented by referring to the detection method and steps in S1003 above, and will not be repeated here.

[0220] S1006: Mobile phone 1 extracts and decodes the ring code based on the Galaxy Ring image scanned by mobile phone 1.

[0221] In one possible implementation, the extraction of the ring code by mobile phone 1 based on the scanned / acquired Galaxy Ring image may include the following steps:

[0222] When mobile phone 1 displays the scanned / captured galaxy ring image in its own preview stream, mobile phone 1 can capture (screenshot) the galaxy ring image displayed in the preview stream to obtain multiple frames of images of the galaxy ring ROI region (as described above). Figure 9 (Example of at least two frames in the illustrated scheme), that is, multiple images containing the galaxy ring ROI region. Further, mobile phone 1 can process these multiple images to obtain a final image, which is used to extract the ring code (i.e., a type of graphic code).

[0223] For example, taking a screenshot of the galaxy ring displayed in the preview stream using mobile phone 1, and obtaining three frames of images of the galaxy ring ROI region (hereinafter referred to as the three frames, i.e., image 1, image 2, and image 3) as an example, mobile phone 1 can extract the ring code based on these three frames, which may include the following steps:

[0224] Step 1: First, perform intra-frame difference processing on images 1, 2, and 3 to obtain three images after intra-frame difference processing, namely image 11, image 22, and image 33.

[0225] Step 2: Perform inter-frame difference processing on images 11, 22, and 33 to obtain three images after inter-frame difference processing, namely image 111, image 222, and image 333.

[0226] Step 3: Binarize images 11, 22, and 33 to obtain three binarized images, namely image 1111, image 2222, and image 3333.

[0227] Step 3: Process images 1111, 2222, and 3333 using image processing methods to remove noise and obtain the ring code.

[0228] The corresponding effect diagrams for the above-mentioned ring code extraction steps can be found in the above images. Figure 8B to Figure 8E As shown, it will not be illustrated here.

[0229] In Implementation Method 1, during the scanning process, mobile phone 1 can detect the degree of moiré pattern on the captured starry ring image in real time. If the moiré pattern is severe, the camera settings can be adjusted to reduce or eliminate it, ensuring that mobile phone 1 can effectively and accurately extract the ring code from the captured starry ring image, then perform decoding to obtain the corresponding information. This avoids the failure or low efficiency of mobile phone 1 in extracting the graphic code due to the influence of moiré patterns. Therefore, this implementation method can effectively improve the accuracy of scanning and increase the efficiency of mobile phone scanning, meaning that the mobile phone can successfully scan codes faster.

[0230] Implementation Method Two:

[0231] In Implementation Method Two, it is described in detail how, during the authentication stage where mobile phone 1 scans the starry ring displayed on the screen of mobile phone 2, the degree of moiré pattern on the screen is detected based on the detection method shown in Implementation Method Two in S902 above, and based on the degree of moiré pattern, a decision is made on whether to adjust the camera settings to improve the accuracy and efficiency of subsequent graphic code extraction. See also Figure 14 As shown, the method flow of Implementation Method Two may include the following steps:

[0232] S1401: Mobile phone 1 and mobile phone 2 enable Bluetooth communication and exchange information between the devices via Bluetooth.

[0233] S1402: Mobile phone 1 scans the galaxy ring image displayed on mobile phone 2, and displays the scanned / collected galaxy ring image on the screen of mobile phone 1 in real time (as described above). Figure 9 (Example of the first screen shown).

[0234] S1401 and S1402 can be referred to one-to-one with the descriptions in S1001 and S1002 above, and will not be repeated here.

[0235] S1403: Mobile phone 1 detects the difference between the distance 1 between mobile phone 1 and mobile phone 2 and the distance 2 where the moiré pattern occurs (which can be equivalent to the degree of moiré pattern on the starry ring image scanned by mobile phone 1).

[0236] In the second implementation method, the difference between the distance 1 between mobile phone 1 and mobile phone 2 and the distance 2 where the moiré pattern occurs can be detected by mobile phone 1: first determine the distance 1 between mobile phone 1 and mobile phone 2 and the distance 2 where the moiré pattern occurs; then calculate the difference between distance 1 and distance 2.

[0237] Among them, mobile phone 1 can detect the distance 1 between mobile phone 1 and mobile phone 2 in the following ways, but not limited to:

[0238] Method 1: Mobile phone 1 obtains binocular parallax information and determines the distance 1 between mobile phone 1 and mobile phone 2 based on the binocular parallax information.

[0239] Method 2: Mobile phone 1 obtains the size information of the galaxy ring displayed on the screen of mobile phone 2 and / or the size information of the scanned galaxy ring displayed on the screen of mobile phone 1 to determine the distance between mobile phone 1 and mobile phone 2.

[0240] Method 3: Mobile phone 1 can measure the distance 1 between itself and mobile phone 2 using its own ranging device (such as a DTof device).

[0241] In one possible implementation, mobile phone 1 determines the distance 2 where the moiré pattern occurs by: obtaining device information of mobile phone 2 (such as screen pixels, screen size, etc.) via Bluetooth (or wireless communication technologies such as Wi-Fi, StarFlash, NFC, etc.), and then determining the distance 2 where the moiré pattern occurs based on the device information of mobile phone 1 (such as screen pixels, screen size, camera spatial resolution, etc.) and / or the device information of mobile phone 2 (such as screen pixels, screen size, etc.).

[0242] S1404: Mobile phone 1 determines whether the difference between the distance 1 between mobile phone 1 and mobile phone 2 and the distance 2 where moiré occurs is lower than a set threshold.

[0243] Based on the above S1403, if the difference between the detection distance 1 (distance between mobile phone 1 and mobile phone 2) and the distance 2 (distance where moiré patterns occur) is lower than the set threshold, it means that the distance between mobile phone 1 and mobile phone 2 is relatively close to the distance where moiré patterns occur, which is equivalent to the degree of moiré patterns on the star ring screen scanned / collected by mobile phone 1 exceeding the set threshold (the degree of moiré patterns is more severe), then the following S1405 is executed.

[0244] Based on the above S1403, if the difference between the detection distance 1 (distance between mobile phone 1 and mobile phone 2) and the distance 2 (distance where moiré patterns occur) is not lower than the set threshold, it means that the distance between mobile phone 1 and mobile phone 2 is not close to the distance where moiré patterns occur, which is equivalent to the degree of moiré patterns on the star ring screen scanned / collected by mobile phone 1 not exceeding the set threshold (the degree of moiré patterns is weak or non-existent), then the following S1406 is executed.

[0245] S1405: Phone 1 adjusts its camera settings and rescans the starry sky.

[0246] The S1405 can be adjusted using the same methods as the S1005 described above, and will not be repeated here.

[0247] Furthermore, in this second embodiment, after adjusting the settings of its own camera, the mobile phone 1 can re-detect the degree of moiré pattern on the scanned / collected star ring image. The detection method can refer to the detection method in S1003 of the first embodiment, and refer to the method in S1005 to reduce the degree of moiré pattern on the collected star ring image or eliminate the moiré pattern by adjusting the focal length value (focus distance) of its own camera. It will not be described again here.

[0248] Alternatively, after adjusting its own camera settings, mobile phone 1 can refer to the detection method of S1403 above to determine the difference between the distance 1 between mobile phone 1 and mobile phone 2 and the distance 2 where the moiré pattern occurs. When the difference between distance 1 and distance 2 is not lower than a preset threshold, the following S1406 is executed.

[0249] For example, see Figure 15 The process shown includes: Step a: Mobile phone 1 determines whether the difference between distance 1 (i.e., the distance between mobile phone 1 and mobile phone 2) and distance 2 (the distance where the moiré pattern occurs) is lower than a set threshold; if yes, then step b is executed, i.e., mobile phone 1 adjusts its own camera settings and rescans the star ring image; if no, then step c is executed, i.e., mobile phone 1 extracts and decodes the ring code based on the star ring image it scanned.

[0250] S1406: Mobile phone 1 extracts and decodes the ring code based on the Galaxy Ring image scanned by mobile phone 1.

[0251] S1406 can be implemented in the same way as described in S1006 above, and will not be repeated here.

[0252] In the second implementation, if mobile phone 1 is equipped with a sensor capable of predicting distance, such as an inertial measurement unit (IMU), mobile phone 1 can predict in advance whether the difference between distance 1 (i.e., the distance between mobile phone 1 and mobile phone 2) and distance 2 (i.e., the distance where moiré patterns occur) is lower than a set threshold. If the predicted difference is lower than the set threshold, mobile phone 1 can adjust its camera settings in advance.

[0253] For example, compared to S1403, phone 1 can use its IMU sensor to predict in advance the distance 1 between itself and phone 2 when scanning the starry ring image of phone 2, and simultaneously determine distance 2 (i.e., the distance at which the moiré pattern occurs); compared to S1404, phone 1 can determine in advance whether the difference between distance and distance 2 is lower than a set threshold; if the difference is lower than the set threshold, compared to S1405, phone 1 can adjust its camera settings in advance and rescan the starry ring image. If the difference is not lower than the set threshold, compared to S1406, phone 1 can extract and decode the ring code in advance based on the starry ring image scanned by phone 1.

[0254] That is, mobile phone 1 does not need to wait until S1402 is executed before starting to execute S1403. Therefore, mobile phone 1 uses a sensor that can predict distance (such as an IMU sensor) to execute the above S1403 to S1406 in advance, which can make mobile phone 1 successfully extract the ring code (scan the code) faster.

[0255] It should be noted that in this application, the step of mobile phone 1 using an IMU sensor or similar device to detect in advance whether the difference between distance 1 and distance 2 is lower than a set threshold, and adjusting the camera settings if it is lower than the set threshold, can be independent of the aforementioned S1403 and S1405, and is not limited in this respect. For example, before S1403, mobile phone 1 can use an IMU sensor or similar device to detect in advance whether the difference between distance 1 and distance 2 is lower than a set threshold, and adjust the camera settings in advance if it is lower than the set threshold. After mobile phone 1 adjusts the camera settings in advance, it can continue to perform subsequent detection (i.e., execute S1403 and S1404). If the difference is lower than the set threshold, mobile phone 1 can continue to adjust its own camera settings (i.e., execute S1405). If the difference is not lower than the set threshold, mobile phone 1 can perform the extraction and decoding of the ring code (i.e., execute S1406).

[0256] Compared to the first implementation method described above, in the second implementation method, the method by which mobile phone 1 detects the degree of moiré pattern on the captured starry sky image is different. That is, the ranging capability of the mobile phone's camera is used to determine the degree of moiré pattern on the captured starry sky image, and the camera settings can be adjusted in advance using this ranging capability to reduce the time for subsequent scanning and extraction of graphic codes. This not only improves the accuracy of scanning, but also further improves the efficiency of scanning, meaning that mobile phone 1 can successfully scan codes faster.

[0257] Regarding the above-described implementation methods one and two, it should be noted that:

[0258] (1) The above-described implementation method 2 can be implemented separately from implementation method 1, or it can be implemented in part or in whole, without any specific limitation.

[0259] (2) The above focuses on describing the differences between Implementation Method 1 and Implementation Method 2. Apart from the differences, Implementation Method 1 and Implementation Method 2 can be referred to each other.

[0260] (3) The step numbers of the flowcharts described in Embodiment 1 and Embodiment 2 above are merely examples of the execution flow and do not constitute a restriction on the order of execution of the steps. There are no temporal dependencies between the steps in the various implementations of this application, and there is no strict execution order between them. In addition, not all the steps shown in the flowcharts are mandatory steps, and some steps can be added or deleted based on the actual needs of each flowchart.

[0261] Based on the above embodiments, this application also provides a device including at least one processor coupled to at least one memory. When the device is in operation, the at least one processor reads computer programs or instructions stored in the at least one memory to perform the functions of the first electronic device in the methods described in the embodiments and / or implementations of this application. For example, performing... Figure 9 Steps S901 to S904 are executed by the first electronic device in the illustrated embodiment; for example, steps S901 to S904 are executed. Figure 10 In the illustrated embodiment, mobile phone 1 executes S1003-S1004 and S1006. For example, it executes... Figure 13 Steps a, b, and c are shown; for example, executing... Figure 14 In the illustrated embodiment, mobile phone 1 executes S1403-S1404 and S1406. For example, it executes... Figure 15 Steps a, b, and c are shown.

[0262] Based on the above embodiments, this application also provides an image processing system, which may include the first electronic device and the second electronic device as described in the foregoing embodiments. In one possible scenario, the system may further include a transfer device for the electronic device.

[0263] Based on the above embodiments, this application also provides a computer program product, which includes a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the methods described in the embodiments and / or implementation methods of this application.

[0264] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the methods described in the embodiments and / or implementation methods of this application.

[0265] Based on the above embodiments, this application also provides a chip for reading computer programs stored in a memory to implement the methods described in the embodiments and / or implementation methods of this application.

[0266] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments and / or implementations of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete devices. Those skilled in the art will understand that the embodiments and / or implementations of this application can be provided as methods, systems, or computer program products. Therefore, this application can be implemented entirely in hardware, entirely in software, or in a combination of software and hardware aspects. Furthermore, this application can be implemented as a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0267] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure One One or more processes and / or boxes Figure OneA device that provides the functions specified in one or more boxes.

[0268] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure One One or more processes and / or boxes Figure One The function specified in one or more boxes.

[0269] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure One One or more processes and / or boxes Figure One The steps of the function specified in one or more boxes.

[0270] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image processing method, characterized in that, include: The first image displayed in the viewfinder of the first electronic device is obtained. The first image is captured by the camera of the first electronic device from the image displayed by the second electronic device. The image displayed by the second electronic device implicitly contains a graphic code for information verification. Detect the degree of moiré pattern on the first image of the first frame; Based on the degree of moiré pattern on the first image, a second image is obtained by capturing the image displayed on the second electronic device from the camera of the first electronic device; Based on the second image, the graphic code is extracted.

2. The method as described in claim 1, characterized in that, The detection of the moiré pattern intensity on the first image of the first frame includes: Intra-frame difference processing is performed on the first image to obtain the third image after intra-frame difference processing. The third image is binarized to obtain the fourth image after binarization. Based on the fourth image, the proportion of the first region in the fourth image is calculated, and the proportion of the first region in the fourth image is determined as the degree of moiré pattern on the first image; The first region is the area in the fourth image where the pixel brightness exceeds a set threshold.

3. The method as described in claim 2, characterized in that, The detection of the moiré pattern intensity on the first image of the first frame includes: Based on the distance between the first electronic device and the second electronic device and the distance at which the moiré pattern occurs, determine the difference between the distance between the first electronic device and the second electronic device and the distance at which the moiré pattern occurs; The degree of moiré pattern on the first image is determined based on the difference between the distance between the first electronic device and the second electronic device and the distance at which the moiré pattern occurs; wherein the difference is inversely proportional to the degree of moiré pattern on the first image.

4. The method as described in claim 3, characterized in that, The method further includes: Obtain binocular parallax information; determine the distance between the first electronic device and the second electronic device based on the binocular parallax information; or Obtain the size information of the first image; determine the distance between the first electronic device and the second electronic device based on the size information of the first image; or The distance between the first electronic device and the second electronic device is measured using the ranging device of the first electronic device.

5. The method as described in claim 3 or 4, characterized in that, The method further includes: Obtain information about the second electronic device, including the pixels and / or size of the screen of the second electronic device; The distance at which the moiré pattern occurs is determined based on the information of the first electronic device and the information of the second electronic device; wherein the information of the first electronic device includes at least one of the following: the number of pixels and the size of the screen of the first electronic device, and the spatial resolution of the camera of the first electronic device.

6. The method according to any one of claims 1-5, characterized in that, The step of obtaining a second image from the view displayed on the second electronic device by the camera of the first electronic device, based on the moiré pattern intensity on the first image, includes: When the moiré pattern on the first image exceeds a set threshold, the camera settings of the first electronic device are adjusted; based on the second image in the viewfinder, the second image is acquired, which is obtained by the adjusted camera of the first electronic device capturing the image displayed on the second electronic device; When the moiré pattern on the first image does not exceed a set threshold, the second image is obtained based on the first image.

7. The method as described in claim 6, characterized in that, The camera settings include focus distance and / or aperture value; The adjustment of the camera settings information of the first electronic device includes: Based on a preset correspondence between image distance and object distance, the first focusing distance of the camera of the first electronic device is adjusted to a second focusing distance; wherein, the first focusing distance is the focusing distance obtained by the camera's autofocus, and the difference between the first focusing distance and the second focusing distance is determined based on the distance between the first electronic device and the second electronic device; and / or The first aperture value of the camera of the first electronic device is adjusted to a second aperture value, wherein the first aperture value is the aperture value automatically set by the camera, and the second aperture value is smaller than the first aperture value.

8. The method as described in claim 6 or 7, characterized in that, The method further includes: The degree of moiré pattern on the fifth image of the second frame is detected to be no more than the set threshold.

9. The method according to any one of claims 1-8, characterized in that, The second image consists of at least two consecutive frames; The extraction of the graphic code based on the second image includes: Intra-frame difference processing and inter-frame difference processing are performed on the at least two frames of images to obtain the grayscale image corresponding to the at least two frames of images after difference. Binarization is performed on the grayscale images corresponding to the at least two frames of images after difference to obtain the binarized images corresponding to the at least two frames of images after difference. The graphic code is obtained based on the binarized image corresponding to the at least two frames of images after differential processing.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: The sensor information of the first electronic device and the sensor information of the second electronic device are acquired, and the sensor information includes the sensor information of the inertial measurement unit (IMU). Based on the sensor information of the first electronic device and the sensor information of the second electronic device, when it is predicted that the distance between the first electronic device and the second electronic device is the same as or the difference is less than a preset threshold, the camera settings of the first electronic device are adjusted, and the image displayed by the second electronic device is captured by the camera of the first electronic device after adjustment.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: Sending device information of the first electronic device to the second electronic device, and receiving device information of the second electronic device from the second electronic device; the device information includes one or more of the following: Information that triggers the start of the verification process, including the device type and device version / model.

12. The method according to any one of claims 1-11, characterized in that, The second electronic device displays a galaxy ring image.

13. The method according to any one of claims 1-12, characterized in that, The graphic code is any one of the following: Ring code, QR code, barcode.

14. An electronic device, characterized in that, It includes at least one processor coupled to at least one memory, the at least one processor being configured to read a program stored in the at least one memory to perform the method as described in any one of claims 1 to 13.

15. A readable storage medium, characterized in that, The readable storage medium contains instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 13.

16. A chip, characterized in that, The chip is coupled to a memory for executing program instructions in the memory to perform the method as described in any one of claims 1 to 13.

Images