Image processing method and device

Abstract

An image processing method and device are used for accurately displaying a pattern based on an image shot by a camera device of a vehicle when the angle between the normal of the plane where the pattern to be shot is located and the orientation of the head of the vehicle is greater than or equal to a first angle threshold. The method comprises the steps that a first image of a first space area is acquired based on a camera device, the first image comprises a first pattern, and the first pattern corresponds to a second pattern in the first space area; and displaying a third pattern, wherein the third pattern is a pattern obtained after the first pattern is processed. On the basis of the method, under the condition that the angle between the normal of the plane where the second pattern is located and the head orientation of the vehicle is larger than or equal to the first angle threshold value, the pattern can be accurately displayed based on the image shot by the camera device of the vehicle without support of a special system and the like, and application scenes are wide.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and particularly to an image processing method and device. Background Art

[0002] The currently common ways of paying for leaving a parking lot are as follows: Way 1: The user binds the license plate through a mobile application and opens an agreement. When the vehicle enters and exits the parking lot, the license plate is recognized by the camera in the parking lot, and when the vehicle exits the parking lot, the parking fee is automatically deducted based on the passive payment system in the parking lot. Way 2: The parking lot constructs a full-automatic electronic toll collection (ETC) system, and when the vehicle exits the parking lot, the parking fee is collected through the ETC system.

[0003] However, the above ways require the parking lot to support specialized systems, such as the above passive payment system or ETC system, resulting in relatively limited application scenarios. Summary of the Invention

[0004] This application provides an image processing method and device, which are used to accurately display a corresponding pattern by capturing an image based on the imaging device of a mobile device when the angle between the normal line of the plane where the pattern to be captured is located and the moving direction of the mobile device is greater than or equal to a first angle threshold.

[0005] In a first aspect, this application provides an image processing method. This method can be applied to a mobile device, or a module in a mobile device (such as a processor, an operating system, a chip, or a chip system, etc.), and at least one imaging device is installed on the mobile device. For example, the mobile device can be a vehicle, etc. Taking the application to a vehicle as an example, this method can include: obtaining a first image of a first spatial region based on the imaging device, the first image including a first pattern, the first pattern corresponding to a second pattern in the first spatial region, and the angle between the normal line of the plane where the second pattern is located and the front direction of the vehicle being greater than or equal to a first angle threshold; displaying a third pattern, the third pattern being the pattern obtained after processing the first pattern.

[0006] Based on the above method, when the angle between the normal line of the plane where the second pattern is located and the front direction of the vehicle is greater than or equal to the first angle threshold, an image can be captured based on the imaging device of the vehicle to accurately display the third pattern, without the support of a specialized system, etc., and the application scenario is relatively wide. For example, when a vehicle pays in a parking lot, it can be done without the parking lot supporting a passive payment system or an ETC system, etc., but by capturing a two-dimensional code through the vehicle's own imaging device, and then the vehicle processes to obtain an accurate two-dimensional code to complete the payment.

[0007] In a possible design, before obtaining the first image of the first spatial region based on the imaging device, a first instruction may be obtained, and based on the first instruction, a first mode is started; wherein, the first instruction is used to trigger the opening of the first mode, and the first mode is used to trigger obtaining the first image of the first spatial region based on the imaging device. This can accurately start the first mode so as to subsequently capture the first image based on the imaging device.

[0008] In a possible design, the clarity of the first pattern is lower than a first preset clarity, and / or the first pattern is deformed relative to the second pattern; before displaying the third pattern, the first image may be subjected to image correction and / or deblurring processing to obtain a second image; and then the third pattern is obtained based on the second image, and the clarity of the third pattern is higher than or equal to the first preset clarity and / or the third pattern is not deformed relative to the second pattern. This can accurately obtain the third pattern based on the distortion degree and / or blurring degree of the first image so as to present an accurate pattern to the user subsequently.

[0009] In a possible design, for image correction of the first image, the method may include: determining the position of the first pattern in the first image; and performing optical correction on the first image according to the position of the first pattern in the first image. Combining the position of the first pattern in the first image, a suitable optical correction method can be selected, and then the first image can be accurately optically corrected.

[0010] In a possible design, for deblurring processing of the first image, the method may include: performing deblurring processing on the first image according to the spatial information of the first spatial region and the motion information of the vehicle. This can accurately remove the blur of the first image.

[0011] In a possible design, the motion information may include at least one of the following: motion speed, motion acceleration, or motion direction, etc.

[0012] In a possible design, for deblurring processing of the first image according to the spatial information of the first spatial region and the motion information of the vehicle, the method may include: determining the spatial position corresponding to the first image according to the spatial information of the first spatial region; determining the spatial position corresponding to the adjacent frame of the first image according to the motion information; and performing deblurring processing on the first image according to the spatial position corresponding to the first image and the spatial position corresponding to the adjacent frame; wherein, the adjacent frame of the first image is an image before and / or after the first image obtained based on the imaging device. This can accurately remove the blur of the first image.

[0013] In a possible design, obtaining the third pattern based on the second image, the method may include: performing a perspective transformation process on the second image to obtain a third image, where the third image includes the third pattern; and cropping the third image to obtain the third pattern. This can accurately obtain the third pattern.

[0014] In a possible design, before displaying the third pattern, it may be determined that the third pattern meets a preset condition according to the first information in the first image; the distance between the first information and the first pattern is within a preset range. In this way, a third pattern that meets the conditions can be displayed subsequently, meeting the user's needs.

[0015] In a possible design, the preset condition may include: the third pattern is used for payment.

[0016] In a possible design, before displaying the third pattern, when it is determined that there are multiple third patterns and the multiple third patterns are the same, perform a duplicate removal process on the multiple third patterns; or, when it is determined that there are multiple third patterns and the multiple third patterns are different, select one third pattern. This can avoid repeated display of the same pattern, or can select and display the pattern that meets the user's needs.

[0017] In a possible design, the third pattern may include a QR code or a barcode.

[0018] In a second aspect, the present application further provides a mobile device, and the mobile device has the function of implementing the method in the above first aspect or each possible design example of the first aspect. The function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0019] In a possible design, the structure of the mobile device may include units or modules for executing the method in the above first aspect or each possible design example of the first aspect.

[0020] In a possible design, the structure of the mobile device includes a display screen, at least one camera device, and one or more processors. Optionally, the mobile device may further include one or more memories, and optionally, the mobile device may further include a transceiver. The transceiver is used for receiving and transmitting information, etc., and for communicating and interacting with other devices. The processor is configured to support the mobile device in executing the corresponding functions in the above first aspect or each possible design example of the first aspect. The memory is coupled to the processor and stores the necessary program instructions and data of the device.

[0021] In a third aspect, the present application provides a computer-readable storage medium storing program instructions that, when run on a computer, cause the computer to execute the method described in the first aspect and any possible design thereof. Exemplarily, the computer-readable storage medium can be any available medium accessible by the device. By way of example but not limitation: the readable medium may include a non-transitory computer-readable medium, random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disc storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by the device.

[0022] In a fourth aspect, the present application further provides a program product that may include instructions that, when run on a computer, cause the method described in the first aspect or any possible design of the first aspect to be executed.

[0023] In a fifth aspect, the present application further provides a chip or chip system including a processor coupled to a memory for reading and executing program instructions stored in the memory to enable the device to implement the method described in the first aspect or any possible design of the first aspect.

[0024] For the various aspects in the second to fifth aspects above and the possible technical effects achieved by each aspect, please refer to the description of the technical effects that can be achieved by the first aspect or various possible solutions in the first aspect above, and details will not be repeated here. Description of the Drawings

[0025] Figure 1 It is a structural diagram of a mobile device provided by the present application;

[0026] Figure 2 It is a schematic diagram of the software structure of a mobile device provided by the present application;

[0027] Figure 3 It is a flowchart of an image processing method provided by the present application;

[0028] Figure 4 It is a schematic diagram of the installation position of an in-vehicle camera provided by the present application;

[0029] Figure 5 It is a schematic diagram of an image captured by a DVR camera directly in front of a vehicle provided by the present application;

[0030] Figure 6 A schematic diagram showing the distribution and shooting characteristics of two-dimensional codes in a scenario where a vehicle scans a code for payment when leaving a parking lot provided by this application;

[0031] Figure 7 A schematic diagram showing the position of a two-dimensional code obtained by a vehicle performing region segmentation and semantic understanding on a first image provided by this application;

[0032] Figure 8 A schematic diagram showing optical correction provided by this application;

[0033] Figure 9 A schematic diagram showing a mobile device selecting different optical correction methods provided by this application;

[0034] Figure 10 A schematic flowchart showing the process of eliminating motion blur provided by this application;

[0035] Figure 11 A schematic diagram showing the elimination of motion blur provided by this application;

[0036] Figure 12 A schematic diagram showing different perspective transformation examples provided by this application;

[0037] Figure 13 A schematic flowchart showing the process of a vehicle processing an image provided by this application;

[0038] Figure 14 A schematic diagram showing that a vehicle can combine the surrounding information of multiple two-dimensional codes to de-duplicate and classify the two-dimensional codes, and finally display one or more two-dimensional codes provided by this application;

[0039] Figure 15 A schematic diagram showing the display of a two-dimensional code provided by this application;

[0040] Figure 16 A schematic diagram showing a user clicking on an in-vehicle screen to select a two-dimensional code provided by this application;

[0041] Figure 17 A schematic diagram of the structure of a mobile device provided by this application. Detailed implementation manners

[0042] The present application will be further described in detail below with reference to the accompanying drawings.

[0043] Embodiments of the present application provide an image processing method and device, which are used to accurately display a corresponding pattern by capturing an image based on an imaging device of a mobile device when an angle between a normal line of a plane where a pattern to be captured is located and a moving direction of the mobile device is greater than or equal to a first angle threshold. Among them, the method and device in the present application are based on the same technical concept. Since the principles for the method and device to solve problems are similar, the implementation of the device and the method can be referred to each other, and repeated parts will not be elaborated.

[0044] In the description of the present application, terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0045] In the description of the present application, "at least one (kind)" means one (kind) or more than one (kind), and more than one (kind) means two (kinds) or more than two (kinds). "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or, a and b and c, where a, b, and c can be single or multiple.

[0046] In the description of the present application, "and / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. " / " represents "or", for example, a / b represents a or b.

[0047] In order to more clearly describe the technical solutions of the embodiments of the present application, the image processing method and device provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0048] Embodiments of the present application provide a mobile device, which can implement the image processing method provided by the embodiments of the present application. The mobile device can be a device or apparatus with data connectivity function, data calculation and processing, and interface display function. Exemplarily, the mobile device in the present application can be a vehicle, or other mobile tools with image processing function, such as a drone, an automatic delivery device, a food delivery robot, etc. Optionally, the mobile device can also be a mobile phone, an in-vehicle terminal device, an in-vehicle computer, a wearable device (such as a smart watch, a smart bracelet, smart glasses, etc.). Of course, the mobile device can also be various portable devices. The present application does not limit the specific form of the electronic device.

[0049] The following refers to Figure 1 , and introduces the structure of the mobile device provided by the embodiments of the present application.

[0050] AsFigure 1 As shown, the mobile device 100 may, but is not limited to, include at least one of the following: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera device 193, a display screen 194, and a SIM card interface 195, etc.

[0051] The processor 110 may include one or more processing units. For example, the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. Among them, the controller may be the nerve center and command center of the mobile device 100. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0052] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from this memory. By providing the memory, the number of times the processor 110 accesses the internal memory 121 for data can be reduced, the waiting time of the processor 110 is reduced, and thus the efficiency of the system is improved.

[0053] The image processing method provided by the embodiments of the present application can be controlled by the processor 110 or other components can be called to complete it. For example, the software program of the embodiments of the present application stored in the internal memory 121 is called, and the display screen 194 is controlled to display the user interface. The processor 110 can include different devices. For example, when the CPU and GPU are integrated, the CPU and GPU can cooperate to execute the image processing method provided by the embodiments of the present application. For example, some internal algorithms in this method are executed by the CPU, and another part of the algorithms related to the user interface display are executed by the GPU to obtain a faster processing efficiency.

[0054] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The display screen 194 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, the display screen 194 can display windows, photos, videos, web pages, or files, etc. In addition, it can be understood that in some embodiments, the status bar may also include a Bluetooth icon, a Wi-Fi icon, an external device icon, etc. In some embodiments, when the processor 110 detects a touch operation of the user's finger (or a stylus, etc.) on the user interface, in response to the touch operation, corresponding instructions are executed, and an updated user interface can also be displayed on the display screen 194.

[0055] In the embodiments of the present application, after the processor 110 runs the image processing method provided by the embodiments of the present application, the processor 110 can control the display screen 194 to display patterns, etc.

[0056] In some embodiments, part or all of the display screen 194 can be used to display the user interface. For example, the mobile device 100 can have multiple display modes, and each display mode has a corresponding display area. The shapes or sizes of the display areas corresponding to different display modes may vary. It should be noted that the display areas corresponding to different display modes may overlap.

[0057] Of course, the mobile device 100 can also have only one display mode. In this case, the mobile device 100 can display the user interface through the entire area of the display screen 194, or can display the user interface through a partial area of the display screen 194.

[0058] The imaging device 193 is used to capture images or videos. Optionally, the mobile device 100 can include 1 to N imaging devices 193. Generally, the imaging device 193 can include a photosensitive element such as a lens group and an image sensor. Among them, the lens group includes multiple lenses (convex lenses or concave lenses), which are used to collect the optical signals reflected by the object to be photographed and transmit the collected optical signals to the image sensor. The image sensor generates the original image of the object to be photographed according to the optical signals.

[0059] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the mobile device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, program codes of at least one application program, etc. The operating system can be, but is not limited to, including etc. The data storage area can store the data created during the use of the mobile device 100, etc.

[0060] The internal memory 121 can also store one or more computer programs for executing the image processing method provided in the embodiments of the present application. The one or more computer programs are stored in the above internal memory 121 and are configured to be executed by one or more processors 110. The one or more computer programs include instructions, and the above instructions can be used to execute the respective steps in the following embodiments.

[0061] In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0062] The sensor module 180 may include, but is not limited to, at least one of the following: fingerprint sensor, touch sensor, pressure sensor, magnetic sensor, ambient light sensor, barometric pressure sensor, bone conduction sensor, etc.

[0063] The touch sensor, also known as the "touch panel". The touch sensor can be disposed on the display screen 194. The touch sensor is used to detect a touch operation acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor can be integrated with the display screen 194 to form a touch display screen. In other embodiments, the touch sensor can also be disposed on the surface of the mobile device 100, at a different position from that of the display screen 194.

[0064] The wireless communication function of the mobile device 100 can be implemented by the antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor, and baseband processor, etc.

[0065] The antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: The antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0066] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied on the mobile device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be disposed in the same device.

[0067] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0068] The wireless communication module 160 can provide solutions for wireless communications applied to the mobile device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive the signals to be transmitted from the processor 110, frequency-modulate them, amplify them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0069] In addition, the mobile device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor. The mobile device 100 can receive inputs from the keys 190 and generate key signal inputs related to the user settings and function controls of the mobile device 100. The mobile device 100 can use the motor 191 to generate vibration prompts. The indicator 192 in the mobile device 100 can be an indicator light, which can be used to indicate the charging state, the change in battery power, or can also be used to indicate messages, notifications, etc. The SIM card interface 195 in the mobile device 100 is used to connect the SIM card. The SIM card can be in contact with and separated from the mobile device 100 by being inserted into or removed from the SIM card interface 195.

[0070] It should be understood that in practical applications, Figure 1 the mobile device 100 shown is only an example and does not constitute a limitation on the mobile device. Moreover, the mobile device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0071] The software system of the mobile device 100 provided by the embodiments of the present application may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Taking the layered architecture as an example in the embodiments of the present application, reference is made to Figure 2 exemplarily illustrate the software structure of the mobile device.

[0072] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers is through software interfaces. As Figure 2 shown, the software architecture can be divided into four layers, from top to bottom are the application layer, the application framework layer (framework, FWK), the system library, and the kernel layer. In addition, the mobile device may further include a hardware layer.

[0073] The application layer is the top layer of the operating system, including the native applications of the operating system and third-party applications, such as applications like camera, gallery, calendar, Bluetooth, music, video, information, and so on.

[0074] The application framework layer provides application programming interfaces (application programming interface, API) and programming frameworks for the applications in the application layer. The application framework layer may include some predefined functions. The application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, etc.

[0075] The window manager is used to provide window management services (window manager service, WMS). The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The window manager can provide multiple management functions to control parameters such as the transparency, position, and size of windows or interfaces in the display screen.

[0076] The content provider is used to store and obtain data and make this data accessible to applications. The data may include files (such as documents, videos, images, audio), text, and other information.

[0077] The view system includes visual controls, such as controls for displaying text, pictures, documents, and other content. The view system can be used to build applications. The interface in the display window can be composed of one or more controls. For example, a display interface showing a file icon can include a control for displaying text and a control for displaying pictures.

[0078] The system library can include multiple functional modules. For example: the surface manager, media libraries, 3D graphics processing libraries (such as OpenGL ES), 2D graphics engines (such as SGL), image processing libraries, etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media libraries support the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media libraries can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing.

[0079] The Kernel layer provides the core system services of the operating system. Services such as security, memory management, process management, network protocol stack, and driver model are all implemented based on the Kernel layer. The Kernel layer also serves as an abstraction layer between the hardware and software stacks. There are many driver programs related to mobile devices in this layer. The main drivers are: display driver; keyboard driver as an input device; Flash driver based on memory technology devices; camera device driver; audio driver; Bluetooth driver; WiFi driver, etc.

[0080] The hardware layer can include various sensors, display screens, camera devices, etc.

[0081] It should be noted that Figure 2 Only as an example of the software structure diagram of a mobile device, only some levels and software modules are simply listed. In actual applications, the operating system of the mobile device can also include other levels and each level can also include other software modules for implementing one or more functions or services. In this regard, the embodiments of this application do not make specific limitations. In addition, the embodiments of this application do not limit the specific levels where each software module is located.

[0082] The image processing method provided by the embodiments of the present application can be applied to scenarios where the angle between the normal line of the plane where the pattern to be captured by the mobile device is located and the moving direction of the mobile device is greater than or equal to a preset threshold, resulting in a large distortion degree of the captured image or a blurred problem in the captured image. For example, the present application can be applied to scenarios where a vehicle is in an area such as a parking lot or other surrounding environment with two-dimensional code signs. The vehicle uses an on-vehicle camera device to scan the two-dimensional code passing by, and after parsing and processing the scanned two-dimensional code, presents it to the user. For another example, the present application can also be applied to other mobile tools with image recording, such as drones, automatic delivery, food delivery robots, and so on. For another example, the present application can be applied to other scenarios of automatically processing images, such as positioning, spatial recognition, etc.

[0083] In the present application, the moving direction of the mobile device does not limit the mobile device during movement. The moving direction of the mobile device can also be understood as the direction in which the mobile device is about to move. That is to say, the mobile device can also be in a stationary state, and the present application does not make a limitation. For example, when the mobile device is a vehicle, the moving direction of the mobile device is the front direction of the vehicle, and the vehicle can be stationary or moving, and the present application does not make a limitation.

[0084] Based on the above description, Figure 3 FIG. shows an image processing method provided by an embodiment of the present application. This method can be executed by a mobile device or a chip, component, or system module inside the mobile device, etc. The following embodiments will be described by taking the vehicle as an example of the mobile device.

[0085] Exemplarily, referring to Figure 3 FIG., this method may include the following steps:

[0086] Step 301: The vehicle acquires a first image of a first spatial region based on a camera device; the first image includes the first pattern, and the first pattern corresponds to a second pattern in the first spatial region, and the angle between the normal line of the plane where the second pattern is located and the front direction of the vehicle is greater than or equal to a first angle threshold.

[0087] That the first pattern corresponds to the second pattern in the first spatial region can be understood as that the camera device captures the second pattern to obtain the first pattern in the first image.

[0088] Wherein, the vehicle is equipped with at least one camera device. The camera device can also be called a camera, video camera, camera, etc. Exemplarily, the camera device can include a monocular camera, a binocular camera, or a structured light camera, etc.

[0089] The present application does not limit the installation position of at least one camera device in the vehicle. For example, the camera device installed on the vehicle can also be called an on-vehicle camera, etc. The on-vehicle camera can be installed at least at one position in the front, rear, left, or right of the vehicle. AsFigure 4 As shown, taking the in-vehicle cameras installed in the front, left, and right sides of the vehicle as an example. The in-vehicle camera optionally installed in the front of the vehicle can be a camera with a normal field of view (FOV), such as a digital video recorder (DVR), etc. The in-vehicle cameras installed on the left and right sides of the vehicle can be cameras with a normal FOV or panoramic cameras with a large FOV. It should be understood that the foregoing is only an exemplary illustration and does not limit the present application.

[0090] In some examples, the second pattern may include a two-dimensional code or a bar code, etc., or may be other patterns, which is not limited in the present application. The first pattern is the pattern obtained by the imaging device capturing the second pattern (such as a two-dimensional code or a bar code, etc.).

[0091] The first region can be understood as the region of interest (ROI), and the first spatial region may include the region where the second pattern is located.

[0092] In some embodiments, before the vehicle acquires the first image of the first spatial region based on the imaging device, a first instruction is acquired, and the first mode can be enabled according to the first instruction. The first instruction is used to trigger the opening of the first mode, and the first mode is used to trigger the acquisition of the first image of the first spatial region based on the imaging device.

[0093] Optionally, when the second pattern includes a two-dimensional code or a bar code, the first mode can be called the code scanning mode.

[0094] For example, the opening of the first mode (illustrated by taking the code scanning mode as an example) can have the following multiple ways, or it can be understood that there are the following multiple ways for the vehicle to acquire the first instruction:

[0095] Way 1: An operation icon is displayed on the central control display screen of the vehicle, and the user triggers the first instruction by clicking the icon for enabling the code scanning mode on the icon, so that the vehicle enables the code scanning mode.

[0096] Optionally, after the vehicle is started (i.e., ignited), the vehicle can actively display the icon for enabling the code scanning mode on the central control display screen.

[0097] Way 2: A trigger for the code scanning function is set on the vehicle hardware (such as the steering wheel, shift lever, buttons, knobs, window, etc.) used for interacting with the user in the vehicle. The user triggers the corresponding function on the vehicle hardware, that is, triggers the first instruction, so that the vehicle enables the code scanning mode.

[0098] Way 3: The user triggers the first instruction by voice, so that the vehicle enables the code scanning mode.

[0099] Optionally, after the vehicle is started (i.e., ignited), the vehicle can start playing the voice "Do you want to enable the QR code scanning mode", and then the user can reply with the voice "Enable the QR code scanning mode", thereby triggering the first instruction.

[0100] It should be understood that the above methods are only exemplary descriptions, and there can be many other methods, which will not be listed one by one in this application.

[0101] After the first mode is enabled, the vehicle can take pictures based on at least one camera device to obtain an image of the vehicle's surrounding environment, such as the first image of the first spatial area. For example, Figure 5 The figure shows a schematic diagram of an image taken by a DVR camera directly in front of the vehicle.

[0102] In some scenarios, the angle between the normal of the plane where the second pattern is located and the head orientation of the vehicle (which can also be understood as the movement direction or driving direction of the vehicle) is greater than or equal to the first angle threshold. For example, in the scenario where the vehicle leaves the parking lot to scan the code for payment, the QR codes in the parking lot are usually distributed on the wall, on the pillar, or on the side of the exit (such as Figure 6 shown), and the angle between the driving direction of the vehicle and the normal of the plane where the QR codes in the parking lot are distributed is usually large. The position of the vehicle-mounted camera is fixed (such as Figure 6 shown, the mounting position of the vehicle-mounted camera in the figure is directly in front of the vehicle or on the left and right sides), so the QR codes captured by the vehicle may have the following characteristics: the QR codes captured by the camera directly in front of the vehicle have a large angle deflection at the left and right edges; the QR codes captured by the cameras on the left and right sides of the vehicle are blurred because the camera orientations on both sides are at 90 degrees to the vehicle's driving direction. Therefore, to obtain an accurate QR code, it is necessary to perform image processing on the distortion degree and / or blur degree of the image captured by the vehicle, so as to accurately obtain the QR code. The specific processing process can be seen in the following description.

[0103] In this application, blurring can include motion blurring, etc.

[0104] Among them, the first angle threshold can be a preset angle such as 60 degrees, 70 degrees, etc., which is not limited in this application.

[0105] In an alternative embodiment, the clarity of the first pattern is lower than the first preset clarity, and / or the first pattern is deformed relative to the second pattern; the vehicle can perform image correction and / or deblurring processing on the first image to obtain a second image; and then the vehicle obtains a third pattern according to the second image, and the third pattern is the pattern obtained after processing the first pattern; the clarity of the third pattern is higher than or equal to the first preset clarity and / or the third pattern is not deformed relative to the second pattern. It should be understood that image correction eliminates image deformation, and deblurring improves image clarity.

[0106] In one possible way, the vehicle can correct the first image by the following method: the vehicle determines the position of the first pattern in the first image, and then can perform optical correction on the first image according to the position of the first pattern in the first image.

[0107] In some embodiments, the vehicle can determine the position of the first pattern in the first image by performing region segmentation and / or image understanding on the first image.

[0108] Among them, image understanding can also be understood as semantic understanding.

[0109] For example, still taking the scenario of a vehicle scanning a QR code as an example, Figure 7 The figure shows a schematic diagram of the vehicle performing region segmentation and semantic understanding on the first image to obtain the position of the QR code in the first image.

[0110] It should be understood that Figure 7 This is only an example and does not limit the present application.

[0111] In addition to region segmentation and / or image understanding, the present application can also obtain the position of the first pattern in the first image through a variety of other methods, which will not be elaborated here one by one.

[0112] Optical correction (lens undistortion) is to perform lens distortion removal operation based on the internal parameters of the imaging device according to specific pixel mapping methods. After optical correction, the image size (number of pixels) may change. The following are two common optical correction methods: 1) Regular size lens undistortion: Keep the center of the lens unchanged and perform transformation, and the pixels of the processed image (image size) are the same as the original image; 2) Fullsize lens undistortion: Correct all pixel points of the original image and fill the image (usually black pixel points, etc.) to obtain an image larger than the original image. As Figure 8 shown, the figure shows a schematic diagram of optical correction by the above two optical correction methods.

[0113] Optionally, after the vehicle determines the position of the first pattern in the first image, the vehicle can select different optical correction methods according to the different positions of the first pattern in the first image. As Figure 9As shown, the vehicle can determine whether the first pattern is at the edge position of the first image. If the first pattern is at the edge position of the first image, the vehicle can perform pixel-consistent optical correction on the first image, that is, keep the lens center unchanged and perform optical correction, and the pixels (image size) of the corrected image are the same as those of the original first image; if the first pattern is not at the edge position of the first image, the vehicle can perform pixel-lossless optical correction on the first image, that is, correct all pixel points of the original first image and fill the image to obtain an image larger than the original first image.

[0114] Among them, the first pattern being at the edge position of the first image can be understood as the distance between the center of the first pattern and the center of the first image being greater than or equal to a preset distance threshold. The value of the preset distance threshold is not limited in this application.

[0115] In a possible way, the vehicle can perform deblurring processing on the first image through the following method: the vehicle can perform deblurring processing on the first image according to the spatial information of the first spatial region and the motion information of the vehicle.

[0116] Exemplarily, the spatial information of the first spatial region can include the spatial coordinates of the first spatial region.

[0117] The motion information of the vehicle can include but is not limited to at least one of the following: motion speed, motion acceleration, or motion direction, etc.

[0118] In some embodiments, as Figure 10 shown, the vehicle can determine the spatial position corresponding to the first image according to the spatial information of the first spatial region, and determine the spatial position corresponding to the adjacent frame of the first image according to the motion information; then the vehicle performs deblurring processing on the first image according to the spatial position corresponding to the first image and the spatial position corresponding to the adjacent frame. Among them, the adjacent frame of the first image is an image before and / or after the first image obtained based on the imaging device. Based on this method, the originally blurred image can be clearly displayed, for example Figure 11 shown.

[0119] Optionally, the deblurring method adopted in this application can include but is not limited to the following methods: the deblurring method using a convolutional neural network, such as the method of constructing a blur kernel; the method of applying non-blind deconvolution to eliminate motion blur; or the method of eliminating blur based on a deep learning network, such as a generative adversarial network (GAN).

[0120] Among them, the blur kernel or other models constructed to eliminate blur in this application can be trained based on the image data captured by the vehicle's imaging device through sampling.

[0121] In some embodiments, the vehicle may perform the above optical correction process on the first image to obtain a second image, or the vehicle may perform the above blur elimination process on the first image to obtain a second image, or the vehicle may perform the above optical correction process and blur elimination process on the first image to obtain a second image.

[0122] Optionally, in the case where the vehicle performs the above optical correction process and motion blur elimination process on the first image to obtain a second image, the vehicle may first perform the optical correction process on the first image and then perform the blur elimination process on the corrected image to obtain the second image; or the vehicle may also first perform the blur elimination process on the first image and then perform the optical correction process on the image after blur elimination to obtain the second image. This application does not make any limitations in this regard.

[0123] Further, the vehicle may perform a perspective transformation process on the second image to obtain a third image, where the third image includes the third pattern; then crop the third image to obtain the third pattern. Through perspective transformation, the shape of the third pattern (such as a QR code) can be changed into a recognizable shape (such as a rectangle, etc.).

[0124] Among them, perspective transformation is to project an image onto a new viewing plane, and perspective transformation is also called projective mapping. The general transformation formula of the perspective transformation method can be the following formula one:

[0125]

[0126] Among them, u and v are the original image coordinates before perspective transformation, and the image coordinates after perspective transformation are x' and y' are intermediate parameters of perspective transformation, and w' and w are scaling factors. is a common transformation matrix, which can be split into four parts, represents a linear transformation, [a 31 a 32 is used for translation, [a 13 a 23 T is used to generate perspective transformation.

[0127] The image after perspective transformation is usually not a parallelogram, or the image changes from a parallelogram to a parallelogram after perspective transformation, such as Figure 12 ​Examples 1, 2, and 3 are shown as different perspective transformation examples.

[0128] In this application, in the process of performing perspective transformation on the second image to obtain the third image, the second image is the original image before perspective transformation, and the third image is the image after perspective transformation. Combining Formula 1, the image coordinates of the third image can be obtained based on the image coordinates of the second image.

[0129] In some embodiments, it can be as Figure 9 shown. After the vehicle optically corrects the first image, the vehicle can perform perspective transformation on the optically corrected image, crop the perspectively transformed image to obtain a fourth pattern, and the fourth pattern corresponds to the first pattern. Further optionally, if the image is blurred, the vehicle can perform deblurring processing on the fourth pattern to obtain the third pattern.

[0130] It should be understood that this application does not limit the order of perspective transformation in the image processing process, and corresponding possible solutions will not be described one by one in this application.

[0131] Step 302: The vehicle displays the third pattern.

[0132] The description and obtaining method of the third pattern can be referred to the description in the foregoing step 301.

[0133] In some embodiments, before the vehicle displays the third pattern, it can determine that the third pattern meets a preset condition according to the first information in the first image; wherein, the distance between the first information and the first pattern is within a preset range.

[0134] The first information can be understood as the peripheral information of the first pattern. For example, in the vehicle parking lot code scanning scenario, the first information can be the peripheral information of the QR code, such as text, pictures, etc. For example, the first information can include at least one of the following "scan code", "charge", "payment", "leave the field" and other texts corresponding to parking payment; the first information can also include at least one of the following "add", "friend", "public account" and other texts corresponding to non-payment types; the first information can also include texts such as ETC.

[0135] In an alternative embodiment, the preset condition may include: the third pattern is for payment. For example, the third pattern can be a QR code for user payment, etc. Optionally, payment can also be described as charging, etc. For example, it can be understood that the third pattern can be a QR code for the payee to charge.

[0136] For example, taking the vehicle parking lot code scanning scenario as an example, as Figure 13 shown, when the vehicle turns on the code scanning mode, based on at least one camera device to capture the surrounding environment, after processing the captured image to obtain a QR code, the vehicle detects and analyzes the QR code.

[0137] Optionally, the vehicle may classify and analyze the two-dimensional code in combination with the first information. Exemplarily, the analysis result of the two-dimensional code may include, but is not limited to, the payee, the amount, the payment method (APP, electronic bank, etc.). The classification of the two-dimensional code may include, but is not limited to, the classification of paid (which may also be described as charged) and non-paid (which may also be described as non-charged) two-dimensional codes, or the classification of different paid APPs or banks.

[0138] Further, if the vehicle successfully analyzes the two-dimensional code, it displays the two-dimensional code. As Figure 13 shown, the vehicle may display a paid two-dimensional code after successful analysis, so as to present the two-dimensional code related to the parking fee to the user.

[0139] Optionally, the vehicle may also display other types of patterns. For example, the vehicle may also present the non-paid two-dimensional code to the user for the user to select the two-dimensional code of interest to participate in activities, etc., or receive coupons, etc.

[0140] In some possible cases, the vehicle may fail to analyze the two-dimensional code. For example, no two-dimensional code is analyzed, or what is analyzed is not a two-dimensional code. As Figure 13 shown, when the vehicle fails to analyze the two-dimensional code, it may prompt the vehicle user to slow down and continue to capture the surrounding environment based on at least one camera device.

[0141] In some embodiments, before the vehicle displays the third pattern, when it is determined that the third pattern includes multiple and the multiple third patterns are the same, the vehicle may perform a duplicate removal process on the multiple third patterns to avoid repeated display.

[0142] Alternatively, when it is determined that the third pattern includes multiple and the multiple third patterns are different, the vehicle may select one third pattern to display, or may also select multiple third patterns to display. For example, when the vehicle selects one third pattern to display, it may select from the multiple third patterns based on the preset conditions described above.

[0143] For example, Figure 14 shown, in the vehicle parking lot scanning code scenario, the vehicle may perform duplicate removal and classification on the two-dimensional code in combination with the surrounding information of multiple two-dimensional codes, and finally display one or more two-dimensional codes. For example Figure 15 shown, it may display the paid two-dimensional codes of different paid APPs after duplicate removal. Optionally, it may also display the non-paid two-dimensional codes after duplicate removal.

[0144] Optionally, when the vehicle can display the two-dimensional code, it may display the two-dimensional code through an in-vehicle screen, an instrument panel, a head-up display (HUD), etc.

[0145] Furthermore, after the vehicle displays the third pattern, the user can interact with the vehicle based on the displayed third pattern. For example, after the vehicle displays a QR code, the user can select the QR code, such as by pressing a button on the vehicle, clicking on the vehicle screen, or by voice interaction. Figure 16 A schematic diagram showing a user clicking on a car screen to select a QR code.

[0146] For another example, the vehicle can be pre-associated with user information. After the vehicle displays the QR code, the vehicle can use electronic payment methods based on the associated user information and the QR code to make payments.

[0147] It should be understood that in the description of the above embodiments, the image can also be described as a picture, etc., and this application does not limit this.

[0148] Based on the above image processing method, when the angle between the normal of the plane containing the second pattern and the vehicle's front heading is greater than or equal to a first angle threshold, the third pattern can be accurately displayed based on the image captured by the vehicle's camera, without the need for specialized systems or other support, and has a wide range of application scenarios. For example, when a vehicle pays for parking, it can use the vehicle's own camera to capture the QR code without the parking lot supporting a contactless payment system or ETC system. Instead, the vehicle can process the QR code accurately to complete the payment.

[0149] It should be understood that the above description only uses vehicles as an example, and other mobile devices, such as drones, automatic delivery robots, food delivery robots, etc. can be used as references in the same way.

[0150] Based on the above embodiments, the present application also provides a mobile device, see Figure 17 As shown, mobile device 1700 may include at least one camera 1701, at least one processor 1702, and at least one display screen 1703. Optionally, mobile device 1700 may further include at least one memory. The memory may be internal or external to mobile device 1700. Optionally, mobile device 1700 may further include a transceiver. The processor 1702 may control operations performed by the transceiver. The transceiver may include a communication interface, etc.

[0151] The camera device 1701 can be used to capture images, such as the first image of the first spatial area described above.

[0152] Specifically, the processor 1702 can be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor 1702 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0153] Among them, the imaging device 1701, the processor 1702, and the display screen 1703 are interconnected with each other. Optionally, the imaging device 1701, the processor 1702, and the display screen 1703 are interconnected through a bus 1704; the bus 1704 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 17 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0154] In an optional implementation manner, the memory is used to store programs, etc. Specifically, the program can include program codes, and the program codes include computer operation instructions. The memory may include a RAM, and may also include a non-volatile memory, such as one or more disk memories. The processor 1702 executes the application programs stored in the memory to implement the above functions, thereby implementing the functions of the mobile device 1700.

[0155] For example, the processor 1702 can be used to control the imaging device to obtain a first image of a first spatial region, etc.

[0156] The display screen 1703 can be used to display the aforementioned third pattern.

[0157] In addition, the processor 1702 may implement other operations performed by the mobile device in the above embodiments. For specific descriptions, reference may be made to the relevant descriptions in the above embodiments, and details are not elaborated here.

[0158] Exemplarily, the above-mentioned processor 1702 may specifically be Figure 1 the processor 110 shown in the figure. The above-mentioned memory may specifically be Figure 1 the internal memory 121 shown in the figure and / or an external memory connected to the external memory interface 120 of the mobile device. The transceiver may be Figure 1 the mobile communication module 150 and / or the wireless communication module 160 shown in the figure. The above-mentioned display screen 1703 may specifically be Figure 1 the display screen 194 shown in the figure. The embodiments of the present application do not impose any restrictions on this.

[0159] Based on the same concept, the embodiments of the present application further provide a mobile device, which may include units or modules for performing each step in the method provided in the above embodiments. Optionally, the mobile device may include a camera device, a display screen, and a processing unit. Optionally, the mobile device may further include a communication unit. Among them, the display screen is used to display images; the communication unit is used for communication; the processing unit is used to execute the method provided in the above embodiments.

[0160] For the specific functions of the above-mentioned various modules, reference may be made to the above embodiments, and details are not repeated here.

[0161] Based on the above embodiments, the embodiments of the present application further provide a computer program product, which includes a computer program; when the computer program runs on a computer, the computer is caused to execute the method provided in the above embodiments.

[0162] Based on the above embodiments, the embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer is caused to execute the method provided in the above embodiments.

[0163] Optionally, the above-mentioned computer may but is not limited to including a mobile device.

[0164] Among them, the storage medium may be any available medium that can be accessed by a computer. Taking this as an example but not limited to: the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0165] Based on the above embodiments, the embodiments of the present application further provide a chip, which is used to read the computer program stored in the memory and implement the method provided in the above embodiments. Optionally, the chip may include a processor and a memory, and the processor is coupled to the memory for reading the computer program stored in the memory and implementing the method provided in the above embodiments.

[0166] Based on the above embodiments, the embodiments of the present application provide a chip system, which includes a processor for supporting a computer device to implement the functions involved in the mobile device in the above embodiments. In a possible design, the chip system further includes a memory for storing the necessary programs and data of the computer device. The chip system may be composed of chips or may include chips and other discrete devices.

[0167] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0168] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more flows and / or Figure 1 blocks or multiple blocks.

[0169] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one or more flows and / or Figure 1 blocks or multiple blocks.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the steps specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 in a block or a plurality of blocks.

[0171] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that, Applied to a vehicle, the vehicle is equipped with a camera device, and the method includes: Obtaining a first image of a first spatial region based on the camera device; the first image includes a first pattern, the first pattern corresponding to a second pattern in the first spatial region, and the angle between the normal of the plane where the second pattern is located and the front direction of the vehicle is greater than or equal to a first angle threshold; Displaying a third pattern, the third pattern being a pattern obtained after processing the first pattern.

2. The method according to claim 1, wherein Before obtaining the first image of the first spatial region based on the camera device, the method further includes: Obtaining a first instruction for triggering the start of a first mode, the first mode being used to trigger the obtaining of the first image of the first spatial region based on the camera device; Based on the first instruction, starting the first mode.

3. The method according to claim 1 or 2, characterized in that, The clarity of the first pattern is lower than a first preset clarity, and / or the first pattern is deformed relative to the second pattern; Before displaying the third pattern, the method further includes: Performing image correction and / or deblurring processing on the first image to obtain a second image; Based on the second image, obtaining the third pattern, the clarity of the third pattern being higher than or equal to the first preset clarity and / or the third pattern not being deformed relative to the second pattern.

4. The method according to claim 3, wherein Performing image correction on the first image includes: Determining the position of the first pattern in the first image; Performing optical correction on the first image according to the position of the first pattern in the first image.

5. The method according to claim 3 or 4, characterized in that, Performing deblurring processing on the first image includes: Performing deblurring processing on the first image according to the spatial information of the first spatial region and the motion information of the vehicle.

6. The method according to claim 5, wherein Performing deblurring processing on the first image according to the spatial information of the first spatial region and the motion information of the vehicle includes: Determining the spatial position corresponding to the first image according to the spatial information of the first spatial region; Determining the spatial positions corresponding to adjacent frames of the first image according to the motion information; the adjacent frames of the first image are images before and / or after the first image obtained based on the camera device; Performing deblurring processing on the first image according to the spatial position corresponding to the first image and the spatial positions corresponding to the adjacent frames.

7. The method according to any one of claims 3 to 6, characterized in that Obtaining the third pattern based on the second image includes: Performing perspective transformation processing on the second image to obtain a third image, the third image including the third pattern; Cropping the third image to obtain the third pattern.

8. The method according to any one of claims 1 to 7, characterized in that, Before displaying the third pattern, the method further includes: Determining that the third pattern meets a preset condition according to first information in the first image; the distance between the first information and the first pattern is within a preset range.

9. The method according to claim 8, wherein The preset condition includes: the third pattern is used for payment.

10. The method according to any one of claims 1-9, characterized in that, Before displaying the third pattern, the method further includes: When it is determined that there are multiple third patterns and the multiple third patterns are the same, performing a duplicate removal process on the multiple third patterns; or When it is determined that the third pattern includes multiple ones and the multiple third patterns are different, select one third pattern.

11. The method according to any one of claims 1 to 10, characterized in that, The third pattern includes a QR code or a bar code.

12. A mobile device, characterized in that, The mobile device includes: At least one camera device for capturing an image; A display screen for displaying an image; At least one processor for supporting the mobile device to implement the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a computer, the method according to any one of claims 1-11 is implemented.

14. A computer program product comprising instructions, characterized in that, When the instructions run on a computer, the computer is caused to execute the method according to any one of claims 1-11.

15. A chip, characterized in that, It includes a processor, and the processor is used to implement the method according to any one of claims 1-11.

Images