Image processing method and electronic device

Abstract

An image processing method and an electronic device relate to the field of image processing technology. After acquiring a first image, the electronic device determines a target sub-image based on the image content of the first image and the image content of N preset sub-images, and each sub-image of the N sub-images includes a partial or complete image of a preset target object. The electronic device determines a first local area in the first image that matches the target sub-image, and renders the first local area based on the position information of the first local area in the first image and the preset rendering rule of the target object to obtain a second image, and displays the second image. The proportion of the first local area in the target object is less than a first preset threshold. In this way, even if the first image only includes a small area of ​​the target object, the electronic device can identify and track the target object and render the first local area.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and electronic equipment. Background Art

[0002] An electronic device with an application that implements augmented reality (AR) technology will identify whether the image includes a preset target object after capturing an image. If the captured image includes the target object, virtual content such as images, videos, and text will be added to the target object in the image (i.e., rendering will be performed), so that the real environment and virtual information are superimposed on the same screen or space in real time, effectively improving the user experience.

[0003] In the prior art, when the captured image includes half or more of the target object, the electronic device can accurately identify and track the target object and overlay virtual content on the target object area included in the captured image. Figure 1 As shown, Figure 1 (A) in FIG. 1 shows a target object, which is a satellite image of the earth. Figure 1 (B) in FIG. 1 shows an image including the target object. Figure 1 The image shown in (B) includes the complete area of ​​the target object, so the electronic device can accurately identify and track the satellite image of the earth and overlay virtual content on the satellite image.

[0004] However, if the captured image includes a small area of ​​the target object, the existing methods cannot accurately identify the target object in the image, and thus cannot render the image of the target object in such an image. Figure 1 The image shown in (C) includes Figure 1 The target object shown in (A) in FIG. Figure 1 The image shown in (C) includes a small area of ​​the target object, and the electronic device cannot accurately identify and track the satellite image of the earth. Figure 1 The satellite image shown in (C) is rendered, which reduces the user experience. Summary of the invention

[0005] The present application provides an image processing method and an electronic device, which can solve the problem that when an image includes a small area of ​​the target object, the electronic device cannot identify and track the target object, nor can it render the image of the target object.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides an image processing method, wherein after acquiring a first image, the electronic device determines a target sub-image according to the image content of the first image and the image content of preset N (N is an integer greater than or equal to 2) sub-images. Afterwards, the electronic device determines a first local area in the first image that matches the target sub-image, and renders the first local area according to the position information of the first local area in the first image and the preset rendering rule of the target object to obtain a second image, and displays the second image. Here, the target sub-image is: among the N sub-images, the sub-image with the highest similarity to the image content of the first image, and the similarity is greater than a preset value. The division granularity of at least two sub-images among the N sub-images is different. Each sub-image in the N sub-images includes a partial or entire image of a preset target object. The proportion of the first local area in the target object is less than a first preset threshold.

[0008] Since the division granularity of at least two sub-images among the N sub-images is different, and each sub-image includes a partial or full image of a preset target object, the images of the target object included in different sub-images are also different, and the features of the target objects in different sub-images are different. In this way, even if the first image only includes a small area of ​​the target object (such as the first local area), the electronic device can identify and track the target object and render the first local area.

[0009] Generally, the rendering function is often implemented by an application for implementing AR services. Therefore, the electronic device in the present application is installed with an application capable of implementing AR services. The execution subject of the image processing method provided in the present application is essentially an application in the electronic device.

[0010] In combination with the first aspect, in a possible design, the method for the electronic device to obtain the first image includes: the electronic device obtains the first image instantly captured by the user, or selects the first image from stored images.

[0011] In practical applications, there are many ways for the electronic device to obtain the first image, which is not limited in this application.

[0012] In combination with the first aspect, in another possible design method, the rendering rules of the above-mentioned target object include: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, and the first preset content includes at least one of text, pictures, video or audio.

[0013] In combination with the first aspect, in another possible design method, the rendering rules of the above-mentioned target object also include: using a second preset method to render other areas in the target object except the first local area, or superimposing a second preset content on other areas in the target object except the first local area, and the second preset content includes at least one of text, pictures, video or audio.

[0014] In combination with the first aspect, in another possible design method, the above-mentioned N sub-images are sub-images in M ​​sub-image groups, and the M sub-image groups are pre-obtained based on M (M is an integer greater than or equal to 2) preset division granularities and target objects. The M preset division granularities are different from each other. Each sub-image group in the M sub-image groups uniquely corresponds to a preset division granularity. All sub-images in each sub-image group constitute the entire area or the first area of ​​the target object, and the ratio of the first area to the target object is greater than a second preset threshold.

[0015] In the image processing method provided by the present application, M sub-image groups are obtained in advance according to the target object and M preset division granularities. Each sub-image group uniquely corresponds to a preset division granularity, and all sub-images in each sub-image group constitute the entire area or the first area of ​​the target object. In this way, the electronic device can select a corresponding sub-image from the M sub-image groups according to actual needs to identify and track the target object in the first image.

[0016] In combination with the first aspect, in another possible design, each sub-image of the N sub-images includes a partial or complete image of the first local area.

[0017] The N sub-images in the present application may be all the sub-images in the M sub-image group, or may be part of the sub-images in the M sub-image group.

[0018] In combination with the first aspect, in another possible design, the N sub-images include a target sub-image determined when the electronic device processes the previous frame of the first image, and all adjacent sub-images of the target sub-image. For a certain sub-image, the adjacent sub-images of the sub-image include: in the M sub-image groups, a sub-image whose distance from the sub-image in the target object is less than or equal to a preset distance, a sub-image containing all or part of the image of the sub-image, and a sub-image contained by the sub-image.

[0019] In practical applications, the electronic device will capture multiple frames of images in a short period of time, and the image content of the multiple frames of images will not change much. For each frame of image acquired, the electronic device can perform image recognition and tracking based on all sub-images in the M sub-image groups, or based on the target sub-image determined when processing the previous frame of image, and all adjacent sub-images of the target sub-image.

[0020] Combined with the definition of adjacent sub-images, it can be seen that all adjacent sub-images of the “target sub-image determined when processing the previous frame of image” include part or all of the image of the “target sub-image determined when processing the previous frame of image”. The “target sub-image determined when processing the previous frame of image” is the sub-image with the highest similarity between the image content of the previous frame of image among all pre-stored sub-images, and the similarity is greater than a preset value. Therefore, the electronic device can accurately and more efficiently complete image recognition and tracking based on the target sub-image determined when processing the previous frame of image, and all adjacent sub-images of the target sub-image, thereby improving the efficiency of image rendering.

[0021] In a second aspect, the present application provides an electronic device, which includes a storage unit, an acquisition unit, a processing unit, and a display unit.

[0022] Specifically, the acquisition unit is used to acquire a first image. The processing unit is used to determine a target sub-image according to the image content of the first image acquired by the acquisition unit and the image content of the preset N sub-images stored in the storage unit; the target sub-image is: among the N sub-images, the sub-image with the highest similarity to the image content of the first image, and the similarity is greater than a preset value, at least two sub-images in the N sub-images have different division granularities, each sub-image in the N sub-images includes a part or all of the image of the preset target object, and N is an integer greater than or equal to 2. The processing unit is also used to determine a first local area in the first image that matches the target sub-image, the proportion of the first local area in the target object is less than a first preset threshold, and the first local area is rendered according to the position information of the first local area in the first image and the preset rendering rule of the target object to obtain a second image. The display unit is used to display the second image obtained by the processing unit.

[0023] In combination with the first aspect, in a possible design, the acquisition unit is specifically used to acquire a first image captured instantly by a user, or to select the first image from stored images.

[0024] In combination with the first aspect, in another possible design method, the rendering rules of the target object include: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, and the first preset content includes at least one of text, pictures, video or audio.

[0025] In combination with the first aspect, in another possible design method, the rendering rules of the target object also include: using a second preset method to render other areas in the target object except the first local area, or superimposing second preset content on other areas in the target object except the first local area, and the second preset content includes at least one of text, pictures, video or audio.

[0026] In combination with the first aspect, in another possible design method, the N sub-images are sub-images in M ​​sub-image groups, the M sub-image groups are pre-obtained based on M preset division granularities and target objects, and the M preset division granularities are different from each other; each sub-image group in the M sub-image groups uniquely corresponds to a preset division granularity; all sub-images in each sub-image group constitute the entire area or the first area of ​​the target object, the ratio of the first area to the target object is greater than a second preset threshold, and M is an integer greater than or equal to 2.

[0027] In combination with the first aspect, in another possible design, each sub-image of the N sub-images includes a partial or complete image of the first local area.

[0028] In combination with the first aspect, in another possible design, the N sub-images include a target sub-image determined by the processing unit when processing the previous frame of the first image, and all adjacent sub-images of the target sub-image. For a certain sub-image, the adjacent sub-images of the sub-image include: in the M sub-image groups, a sub-image whose distance from the sub-image in the target object is less than or equal to a preset distance, a sub-image containing all or part of the image of the sub-image, and a sub-image contained by the sub-image.

[0029] In a third aspect, the present application provides an electronic device, the electronic device comprising a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store a computer program code, the computer program code comprising a computer instruction, and when the processor executes the computer instruction, the electronic device performs the image processing method as described in the first aspect and any possible design thereof.

[0030] In a fourth aspect, the present application provides a chip system, which is applied to an electronic device including a display screen; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through a line; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the image processing method as described in the first aspect and any possible design thereof.

[0031] In a fifth aspect, the present application provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the image processing method as described in the first aspect and any possible design thereof.

[0032] In a sixth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the image processing method as described in the first aspect and any possible design thereof.

[0033] For the specific description of the second to sixth aspects and their various implementations in the present application, reference may be made to the detailed description in the first aspect and its various implementations; and for the beneficial effects of the second to sixth aspects and their various implementations, reference may be made to the beneficial effects analysis in the first aspect and its various implementations, which will not be repeated here.

[0034] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a target object and an image including the target object;

[0036] Figure 2 This is a schematic diagram of the structure of the image processing system in the embodiment of the present application;

[0037] Figure 3 Schematic diagram of the hardware structure of the electronic device 21 in the embodiment of the present application;

[0038] Figure 4 The process diagram of the image processing method in the embodiment of the present application is as follows: Figure 1 ;

[0039] Figure 5 A schematic diagram of a target object in an embodiment of the present application;

[0040] Figure 6 The structure of the sub-diagram in the embodiment of this application is shown in FIG. Figure 1 ;

[0041] Figure 7 A schematic diagram of the structure of the sub-graph provided in the embodiment of the present application Figure 2 ;

[0042] Figure 8 A schematic diagram of the structure of the sub-graph provided in the embodiment of the present application Figure 3 ;

[0043] Fig. 9 This is a schematic diagram of the structure of the sub-graph database in the embodiment of the present application;

[0044] Fig. 10A This is the tree structure 1 of the subgraph database in the embodiment of the present application;

[0045] Fig. 10B This is the second tree structure of the subgraph database in the embodiment of the present application;

[0046] Fig.11A A schematic diagram of a multi-resolution hierarchical model in an embodiment of the present application;

[0047] Fig. 11B The distribution of sub-graphs in the multi-resolution hierarchical model in the embodiment of the present application;

[0048] Fig.12 A schematic diagram of the positional relationship between a target object and a target subgraph in an embodiment of the present application;

[0049] Fig.13 A schematic diagram of an application scenario to which the image processing method provided in an embodiment of the present application is applicable;

[0050] Fig.14 This is a schematic diagram of an image acquired by a mobile phone in an embodiment of the present application;

[0051] Fig.15 Schematic diagram of the image processing method provided in the embodiment of the present application Figure 2 ;

[0052] Fig.16 This is an interface diagram of the mobile phone after rendering data in image 140 in the embodiment of the present application;

[0053] Fig.17 This is an interface diagram after the mobile phone renders data in the image 142 in the embodiment of the present application;

[0054] Fig.18 Schematic diagram of the image processing method provided in the embodiment of the present application Figure 3 ;

[0055] Fig.19 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0056] Fig. 20 A schematic diagram of the structure of the chip system provided in the embodiment of the present application;

[0057] Fig.21 A schematic diagram of the structure of a computer program product provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0059] Augmented reality (AR) is a technology that calculates the position of a target object in a captured image in real time and then adds virtual content such as virtual images, videos, and text at the corresponding positions. Based on AR, a virtual scene can be combined with the real environment for effective interaction.

[0060] For example, a user uses an AR application installed on an electronic device (such as a smartphone) to identify whether the captured image contains a target object (such as a world map). When it is recognized that the captured image contains the above target object, the electronic device can render an animation or an image to bring a festive feeling to the user.

[0061] Another example: After a red envelope initiator发起 a real-world red envelope based on地理位置 and target object through an AR application, the user who grabs the red envelope needs to go to the above地理位置 and scan the target object according to the clue map to grab the above real-world red envelope. When the scanned object matches the object prompted by the clue map, the electronic device renders and displays the scene graph of the "3D red envelope villain", and the "3D red envelope villain" can nod and bow or wave to greet the user. When the user touches and slides the 3D red envelope with a finger, the 3D red envelope can rotate.

[0062] Generally, the target object is preset by developers. The target object involved in the embodiments of this application is also preset, which refers to a specific object, or objects of the same type with almost the same characteristics (it is difficult to detect subtle changes), such as a world map, a huge poster, etc. A local area of the target object in the embodiments of this application can uniquely identify the target object. For the situation where the local area cannot identify the target object, the embodiments of this application do not provide protection. For example, the Chinese character "福" is composed of multiple strokes, and the combination of some strokes cannot uniquely identify the Chinese character "福".

[0063] The above AR services are mainly implemented based on image recognition technology and image tracking technology. Specifically, image recognition technology is used to recognize the target object in the captured image, and image tracking technology is used to determine the position of the target object in the image and superimpose virtual content according to the determined position (i.e., perform a rendering operation).

[0064] Image tracking refers to calculating the accurate position of a target object in an image (for the convenience of description, this application refers to this image as a field of view image) through information such as image content and features.

[0065] Generally, an electronic device uses image feature matching technology to achieve tracking of the target object. Specifically, this technical solution mainly includes the following steps S1-S5.

[0066] S1. The electronic device extracts the feature information of the target object and the field of view image.

[0067] Feature information refers to the information obtained after extracting features from an image using a feature extraction algorithm. Usually, feature information is represented by a vector of n0 dimensions, where n0 is a positive integer.

[0068] Feature information includes feature points and feature descriptors. Feature points are used to indicate the location of the feature on the image. Feature descriptors are used to indicate the local features of the location on the image, and are generally represented by vectors.

[0069] The electronic device may extract features of the target object and the field of view image using a feature extraction algorithm, thereby obtaining feature information of the target object and the field of view image. Among them, the feature extraction algorithms used by different manufacturers may be different, and the data dimensions of the feature information of the target object and the field of view image obtained accordingly may also be different.

[0070] Feature extraction algorithms are algorithms used to implement feature extraction. For example: deep residual network (resnet), Google convolutional neural network (GoogLeNet), perceptron neural network (PNN), Hopfield network, radial basis function (RBF) neural network, convolutional neural network (CNN), feedback neural network, speeded-uprobust features (SURF) feature extraction algorithm, scale-invariant feature transform (SIFT) feature extraction algorithm, fast feature point extraction and description algorithm (oriented FAST and rotated BRIEF, ORB) feature extraction algorithm, etc.

[0071] S2. The electronic device determines feature information that matches the feature information of the target object and the feature information of the field of view image.

[0072] The electronic device may use a feature point matching algorithm to determine mutually matching feature information. The feature point matching algorithm is an algorithm used to implement feature matching, such as a brute-force matching algorithm, a nearest neighbor matching algorithm, and the like.

[0073] Mutually matching feature information means that the value of the similarity between the feature information is higher than a preset value. Here, the similarity can be represented by the distance between the feature information, and the distance can be Hamming distance, cosine distance, Euler distance or asymmetric distance.

[0074] It can be understood that, for two images, the higher the similarity value is, the more the amount of feature information that matches between the two images is, or the smaller the distance between the feature information of the two images is.

[0075] S3. The electronic device calculates a homography matrix from the target object to the field of view image based on the mutually matched feature information.

[0076] Homography refers to the mapping relationship between a point on a plane (3D homogeneous vector) and another plane, and is a linear transformation of a 3D homogeneous vector. The homography matrix is ​​the transformation equation from a point on a plane to another plane, which can be represented by a 3×3 non-singular matrix H.

[0077] For example, if (u1, v1, 1) T is a pixel point in the target object, (u2, v2, 1) T is a pixel in the field of view image, and these two pixels correspond to each other, which can be expressed as:

[0078]

[0079] The specific calculation method of the homography matrix can refer to the method of determining the homography matrix in the prior art, and will not be described in detail here.

[0080] S4. The electronic device determines whether the field of view image includes the target object.

[0081] The electronic device may determine whether the field of view image includes the target object by using the following first implementation manner or second implementation manner.

[0082] A first implementation method: the electronic device determines the proportion of the amount of "mutually matching feature information obtained in S2" in the feature information of the target object, and determines whether the field of view image includes the target object based on the proportion.

[0083] If the proportion is greater than or equal to the preset proportion, it is determined that the field of view image includes the target object. Otherwise, it is determined that the field of view image does not include the target object. The preset proportion can be a user-defined setting on the user side or the system side.

[0084] A second implementation manner: the electronic device determines the number of feature information that conforms to the homography matrix obtained in S3, and determines whether the field of view image includes the target object according to the number.

[0085] If the number of feature information that conforms to the homography matrix is ​​greater than or equal to a first threshold, it is determined that the field of view image includes the target object. Otherwise, it is determined that the field of view image does not include the target object. The first threshold can be customized by the user side or the system side.

[0086] If the electronic device determines that the visual field image does not include the target object, the electronic device terminates tracking of the target object. If the electronic device determines that the visual field image includes the target object, the electronic device continues to perform the following step S5 to achieve tracking of the target object.

[0087] S5. The electronic device determines the positions of the four vertices of the target object on the field of view image according to the homography matrix and the coordinate values ​​of the four vertices of the target object (generally a rectangle).

[0088] In practical applications, the field of view image will be affected by various external factors, such as: illumination change, motion blur, hardware noise, etc. Therefore, when matching the feature information of the target object with the feature information of the field of view image, some feature information may be mismatched.

[0089] In order to reduce the possibility of image tracking failure due to mismatched feature information, the field of view image needs to include half or more of the target object. In this way, the electronic device can obtain enough correctly matched feature information to accurately identify and track the target object, and then overlay virtual content on the target object in the field of view image.

[0090] For example, in the above scenario of scanning the world map, if Figure 1 As shown in (B), when the image acquired by the electronic device contains all areas of the satellite image of the earth, the electronic device superimposes a virtual image on the satellite image in the acquired image. In the above-mentioned real-life red envelope scenario, when the object image scanned by the electronic device is mostly or completely consistent with the target object image used to hide the red envelope, the electronic device superimposes virtual content on the scanned object image.

[0091] However, if the field of view image captured by the electronic device only contains a small area of ​​the target object, the electronic device cannot track the target object and thus cannot complete the rendering operation, which reduces the user experience. Figure 1 As shown in (C) in FIG. 1 , when the image acquired by the electronic device includes a small area of ​​the satellite image, the electronic device cannot accurately identify and track the satellite image of the earth, and thus cannot accurately identify and track the satellite image of the earth. Figure 1 Rendering of the satellite image shown in (C).

[0092] To this end, an embodiment of the present application provides an image processing method that can be applied to the process of rendering a target object by an electronic device. By adopting the image processing method provided by the embodiment of the present application, the electronic device can still accurately identify and track the target object in a scene where the field of view image only includes a local area of ​​the target object, and render the local area of ​​the target object in the field of view image, and display the rendered image.

[0093] The image processing method provided in the embodiment of the present application can be applied to rendering scenarios in the field of augmented reality (AR), and can also be applied to other scenarios that require image recognition and tracking. In the rendering scenario in the AR field, the electronic device uses the image processing method provided in the embodiment of the present application to determine that the field of view image includes an area of ​​the target object (such as the first local area). After that, the electronic device renders the area of ​​the target object in the field of view image according to the location information of the area and the preset rendering rules of the target object. For details, please refer to the subsequent Fig.15 Description.

[0094] The image processing method provided in the embodiment of the present application is applicable to an image processing system. Figure 2 FIG. 4 shows a structure of the image processing system. Figure 2 As shown, the image processing system includes a server 20 and an electronic device 21. In practical applications, the image processing system usually includes multiple electronic devices. Figure 2 Only a single electronic device 21 is shown.

[0095] Among them, the server 20 stores multiple sub-images of the target object (specifically refer to the description of the sub-images in the subsequent "data preparation" process) and the characteristic information of each sub-image, and each sub-image is a partial or complete area of ​​the target object. The electronic device 21 can obtain the multiple sub-images and the characteristic information of each sub-image from the server 20. Subsequently, after the electronic device 21 obtains the first image (the first image is the field of view image described in this application) (captured in real time or saved by the electronic device 21), it can determine the first local area in the first image that matches the target sub-image based on the multiple sub-images and the characteristic information of each sub-image, that is, it realizes the image tracking of the target object, and renders the first local area in the first image based on the position information of the first local area in the first image and the preset rendering rules of the target object, and displays the rendered image (corresponding to the second image in the embodiment of the present application).

[0096] The proportion of the above-mentioned first local area in the target object is less than or equal to the first preset threshold. It can be understood that the first local area is a small area of ​​the target object. In this way, the solution provided by the embodiment of the present application can accurately identify the target object even if the field of view image includes a small area of ​​the target object, and render and display the small area. Here, the first preset threshold can be pre-set by the developer, and the embodiment of the present application does not limit this.

[0097] The sub-image in the embodiment of the present application is a partial or full area of ​​the target object. The electronic device uses the sub-image to track the target object. Therefore, regardless of whether the area of ​​the target object included in the first image accounts for a large or small proportion of the target object, the electronic device 21 can accurately track the target object.

[0098] Of course, the electronic device 21 in the embodiment of the present application can also autonomously generate multiple sub-graphs of the target object and obtain feature information of each sub-graph. After that, the electronic device 21 saves the multiple sub-graphs and the feature information of each sub-graph.

[0099] For ease of understanding, the embodiment of the present application is mainly described by taking the server 20 storing multiple subgraphs of a target object and feature information of each subgraph as an example.

[0100] The electronic device 21 in the embodiment of the present application can be various devices equipped with a camera (such as a front camera or a rear camera), such as: a mobile phone, a vehicle-mounted device (such as a vehicle-mounted navigator), a tablet computer, a desktop, a laptop, a handheld computer, a wearable device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, an AR\virtual reality (VR) device, and other devices with image tracking functions. Of course, the electronic device 21 can also be Figure 2 The server 20 in the embodiment of the present application does not impose any special restrictions on the specific form of the computing device.

[0101] An application for implementing AR services is installed in the electronic device 21. The application may be a built-in application of the electronic device 21 itself, or an application provided by a third-party service provider and installed by the user, which is not limited in the present embodiment of the application.

[0102] Please refer to Figure 3 , taking the electronic device 21 as a mobile phone as an example, a hardware structure diagram of the electronic device 21 is shown. Figure 3As shown, the electronic device 21 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0103] Among them, the sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0104] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 21. In other embodiments of the present application, the electronic device 21 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0105] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0106] The controller may be the nerve center and command center of the electronic device 21. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0107] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0108] In some embodiments, the processor 210 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0109] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The I2S interface can be used for audio communication. The PCM interface can also be used for audio communication to sample, quantize and encode analog signals. The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. The MIPI interface can be used to connect the processor 210 with peripheral devices such as the display screen 294 and the camera 293. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. The USB interface 230 is an interface that complies with the USB standard specification, and can specifically be a Mini USB interface, a MicroUSB interface, a USB Type C interface, etc. The USB interface 230 can be used to connect a charger to charge the electronic device 21, and can also be used to transmit data between the electronic device 21 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

[0110] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 21. In other embodiments of the present application, the electronic device 21 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0111] The charging management module 240 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, the internal memory 221, the external memory, the display 294, the camera 293, and the wireless communication module 260. The power management module 241 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc.

[0112] The wireless communication function of the electronic device 21 can be implemented through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor and the baseband processor.

[0113] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 21 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0114] The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 21. The mobile communication module 250 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the same device as at least some of the modules of the processor 210. The modulation and demodulation processor may include a modulator and a demodulator.

[0115] The wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), GNSS, frequency modulation (FM), nearfield communication (NFC), infrared (IR), etc., which are applied to the electronic device 21. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and filters it, and sends the processed signal to the processor 210. The wireless communication module 260 can also receive the signal to be sent from the processor 210, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0116] Exemplarily, the GNSS in the embodiments of the present application may include: GPS, GLONASS, BDS, QZSS, SBAS, and / or GALILEO, etc.

[0117] The electronic device 21 implements the display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information. The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel.

[0118] The electronic device 21 can realize the shooting function through ISP, camera 293, video codec, GPU, display screen 294 and application processor. ISP is used to process the data fed back by camera 293. Camera 293 is used to obtain static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos.

[0119] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic devices 21 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0120] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 21. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0121] The internal memory 221 may be used to store computer executable program codes, which include instructions. The processor 210 executes the instructions stored in the internal memory 221 to execute various functional applications and data processing of the electronic device 21 .

[0122] The electronic device 21 can implement audio functions such as music playing and recording through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone jack 270D, and the application processor.

[0123] The audio module 270 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The speaker 270A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 270B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 270C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 270D is used to connect wired headphones. The headphone jack 270D can be a USB interface 230, or it can be a 3.2mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0124] The button 290 includes a power button, a volume button, etc. The button 290 can be a mechanical button. It can also be a touch button. The electronic device 21 can receive the button input and generate a key signal input related to the user settings and function control of the electronic device 21. The motor 291 can generate a vibration prompt. The motor 291 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 292 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 295 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 295, or pulled out from the SIM card interface 295 to achieve contact and separation with the electronic device 21. The electronic device 21 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0125] The image processing method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0126] The image processing method provided in the embodiment of the present application includes a process in which a server generates M (M is an integer greater than or equal to 2) sub-image groups (referred to as the "data preparation" process) and a process in which an electronic device acquires an image, identifies and tracks a target object included in the image, and renders an area of ​​the target object in the image (referred to as the "image processing" process).

[0127] The following first describes the "data preparation" process.

[0128] like Figure 4 As shown, the “data preparation” process may include S401 to S403 .

[0129] S401. The server 20 obtains multiple subgraphs according to M preset division granularities and target objects to obtain M subgraph groups.

[0130] The M preset division granularities are different from each other. The division granularities are all customized by the system side or the user side, and this embodiment does not limit this.

[0131] Specifically, for each of the M preset division granularities, the server 20 obtains one or more sub-images (the sub-image is the entire image or a partial area of ​​the target object) according to the preset division granularity and the target object to obtain a sub-image group. The division granularity of each sub-image in the sub-image group is equal to the preset division granularity, and all sub-images in the sub-image group constitute the entire area or the first area of ​​the target object. The proportion of the first area to the target object is greater than a second preset threshold, and the second preset threshold is predetermined by the system side or the user side. For example: the second preset threshold is 90% of all areas of the target object or 95% of all areas of the target object, so that all sub-images in the sub-image group constitute the entire area or most of the area of ​​the target object.

[0132] In the process of acquiring the sub-graph, the server 20 may acquire the sub-graph from the target object in a certain order (such as from left to right or from top to bottom), or may acquire the sub-graph from the target object randomly, and the embodiment of the present application does not specifically limit this.

[0133] In addition, the embodiment of the present application does not limit the spatial position relationship between different sub-images in the sub-image group. Any two sub-images in the same sub-image group may or may not have overlapping areas.

[0134] In one example, Figure 5 The map image shown is the target object. When the preset division granularity is all areas of the target object, the first sub-image group obtained by the server 20 may include: Figure 5 The image shown.

[0135] In another example, Figure 5 The map image shown is the target object. When the preset division granularity is 50% of all areas of the target object, the second sub-image group obtained by the server 20 may include: Figure 6 Sub-pictures 61 to 63 are shown. Figure 6 In the example, each of sub-images 61 to 63 occupies 50% of the entire area of ​​the target object. Figure 6 Sub-images 61 to 63 in the image can be spliced ​​to present most of the target object. There are overlapping areas between sub-images 61 to 63.

[0136] In another example, Figure 5The map image shown is the target object. When the preset division granularity is 25% of all areas of the target object, the third sub-image group obtained by the server 20 may include: Figure 7 Sub-pictures 71 to 74 are shown. Figure 7 In the figure, each of sub-images 71 to 74 occupies 25% of the entire area of ​​the target object. Figure 7 After the sub-images 71 to 74 in the image are spliced ​​together, the entire area of ​​the target object can be presented. The sub-images 71 to 74 are sequentially connected on the target object.

[0137] In another example, Figure 5 The map image shown is the target object. When the preset division granularity is 15% of all areas of the target object, the fourth sub-image group obtained by the server 20 may include: Figure 8 Sub-pictures 81 to 85 are shown. Figure 8 In the example, each of sub-images 81 to 85 occupies 15% of the entire area of ​​the target object. Figure 8 After the sub-images 81 to 85 in the image are spliced ​​together, most of the target object can be presented. There are gaps between the sub-images 81 to 85.

[0138] The embodiment of the present application does not specifically limit the number of sub-graphs in the same sub-graph group. In combination with the above example, when the preset division granularity is 15% of all areas of the target object, the server 20 can obtain 6 sub-graphs. Regardless of the number of sub-graphs in the same sub-graph group, it is only necessary that all sub-graphs in the sub-graph group constitute the entire area or the first area of ​​the target object.

[0139] The embodiment of the present application does not limit the above-mentioned division granularity. The server 20 can obtain the sub-graph from the target object according to the size of the sub-graph, or can obtain the sub-graph from the target object according to other factors. For example: if the target object is a world map, the server 20 can obtain the sub-graph from the world map according to the size of the sub-graph, the granularity of "continent", "country", "province", "city", "urban area", etc. Correspondingly, the shape of the sub-graph can be a rectangle, a circle, a polygon, etc., or it can be the map shape of each continent / country / province / city / urban area, and the embodiment of the present application does not specifically limit this.

[0140] For ease of description, the embodiments of the present application are mainly described by taking the sub-image as a rectangle as an example.

[0141] S402: The server 20 obtains feature information of each sub-image in each sub-image group according to a preset feature extraction algorithm.

[0142] The method for the server 20 to obtain the characteristic information of each sub-image may refer to the description of obtaining the characteristic information of an image in the prior art, and will not be described in detail here.

[0143] S403: The server 20 stores the sub-images in each of the M sub-image groups and feature information of each sub-image.

[0144] The server 20 may store the sub-graphs in each sub-graph group and the feature information of each sub-graph in a sub-graph database.

[0145] For each subgraph, the subgraph database may further include information of adjacent subgraphs of the subgraph, for example, identifiers (specifically, numbers) of the adjacent subgraphs.

[0146] In the embodiment of the present application, the adjacent sub-images of a sub-image include: in the target object, a sub-image whose distance from the sub-image is less than or equal to a preset distance (which can be zero or a very small value), a sub-image that contains all or part of the image of the sub-image, and a sub-image that is contained in the sub-image.

[0147] Combining the above examples, Figure 5 The map image shown is the target object. For the target object, the server 20 obtains four sub-image groups, which are: the first sub-image group, the second sub-image group, the third sub-image group, and the fourth sub-image group. For sub-image 81, since the spatial distance between sub-image 82 and sub-image 81 is less than or equal to the preset distance (such as the distance of several pixels), sub-image 82 is an adjacent sub-image of sub-image 81. In addition, since sub-image 61 and sub-image 71 both contain the entire image of sub-image 81, sub-image 61 and sub-image 71 are also adjacent sub-images of sub-image 81.

[0148] For example, in combination with the above Figures 5 to 8 , Fig. 9 The sub-image database 90 shown includes data of sub-images in the first sub-image group, data of multiple sub-images in the second sub-image group, data of multiple sub-images in the third sub-image group, and data of multiple sub-images in the fourth sub-image group. Fig. 9 As shown, the data of the plurality of sub-images in the fourth sub-image group may include: data of sub-image 81, data of sub-image 82, data of sub-image 83, data of sub-image 84, and data of sub-image 85. Taking sub-image 81 as an example, Fig. 9 As shown, the data of sub-image 81 may include: feature information of sub-image 81 , and identifiers of sub-image 82 , sub-image 61 , and sub-image 71 .

[0149] Optionally, the subgraph database can be represented by a tree structure.

[0150] Exemplarily, the subgraph database in the embodiment of the present application can be used Fig. 10A The retrieval tree shown is shown. Fig. 10A As shown in , the search tree includes M layers, each of which corresponds to a subgraph group. Fig. 10A As shown in , the first level of the search tree corresponds to subgraph group 1, the second level corresponds to subgraph group 2, and so on. Each level includes one or more nodes, such as Fig. 10A As shown, the first layer includes nodes 1a, etc., and the second layer includes nodes 2a, 2b, 2c, etc. These nodes represent a subgraph respectively. All nodes in each layer constitute the entire area or the first area of ​​the target object.

[0151] For any two adjacent layers in the M layers, the "size of the subgraph represented by the upper layer node" may be larger than the "size of the subgraph represented by the lower layer subgraph".

[0152] For example: Combining the above Figures 5 to 8 , Fig. 10B A four-layer retrieval tree is shown, and the four layers correspond to the first sub-graph group, the second sub-graph group, the third sub-graph group, and the fourth sub-graph group. Fig. 10B Directly use subgraphs to represent nodes. The first layer includes Figure 5 The target object is shown. The second layer includes sub-image 61, sub-image 62, and sub-image 63. Since the target object belongs to the adjacent sub-images of sub-image 61, sub-image 62, and sub-image 63, the image of the target object is connected to sub-image 61, sub-image 62, and sub-image 63. The third layer includes sub-image 71, sub-image 72, sub-image 73, and sub-image 74. The fourth layer includes sub-image 81, sub-image 82, sub-image 83, sub-image 84, and sub-image 85. The connection relationship between the sub-images in the third layer and the sub-images in the second layer, and the connection relationship between the sub-images in the fourth layer and the sub-images in the third layer can refer to the relevant description of "the image of the target object is connected to sub-image 61, sub-image 62, and sub-image 63", which will not be repeated here.

[0153] Optionally, the sub-image database can be regarded as a multi-resolution hierarchical model. The model includes M layers, each of which corresponds to a sub-image group, and each sub-image group includes one or more sub-images. From the top layer to the bottom layer, the resolution of the sub-images is getting higher and higher.

[0154] For example, Fig.11A A multi-resolution hierarchical model is shown. Each layer in the model is a target object drawn according to a sub-image. Figure 6 , Figure 7 and Fig.11A ,like Fig. 11B As shown, the second layer of the model may include sub-images 61, 62, and 63, and the third layer may include sub-images 71, 72, 73, and 74. The size of the sub-images in the third layer is smaller than that of the sub-images in the second layer, and conversely, the resolution of the sub-images in the third layer is higher than that of the sub-images in the second layer.

[0155] It can be seen that the higher the level of the multi-resolution hierarchical model, the more sub-images there are, and the more detailed the images of the target objects displayed by the sub-images of this level are. Therefore, the larger M is, the clearer the images of the target objects displayed by all the sub-images of the Mth layer are.

[0156] The sub-images and feature information of each sub-image stored in the server 20 provide support for image tracking, that is, for the subsequent "image processing" process. In the "image processing" process, for the electronic device 21, the M sub-image groups and feature information of each sub-image are pre-set.

[0157] Next, the “image processing” process is described.

[0158] The "image processing" process is based on the "data preparation" process. Figure 4 As shown, after S403, it also includes: S404~S411.

[0159] S404 , the electronic device 21 obtains the sub-images in each of the M sub-image groups and feature information of each sub-image from the server 20 .

[0160] In one implementation, the electronic device 21 may actively send a data request to the server 20 to request to obtain data related to the target object from the server 20, that is, to request to obtain the sub-images in each of the M sub-image groups and the feature information of each sub-image. In response to receiving the data request, the server 20 may send the sub-images in each of the M sub-image groups and the feature information of each sub-image to the electronic device 21. For example, an application with the function of rendering the target object is installed in the electronic device 21. After the user inputs an operation in the application to trigger the acquisition of data related to the target object, the electronic device 21 sends the above data request to the server 20.

[0161] In another implementation, the server 20 may actively send the sub-images in each of the M sub-image groups and the feature information of each sub-image to the electronic device 21. The electronic device 21 may receive the sub-images in each of the M sub-image groups and the feature information of each sub-image sent by the server 20. For example, the electronic device 21 is installed with an application having a function of rendering a target object. During the installation process, the server 20 sends the sub-images in each of the M sub-image groups and the feature information of each sub-image to the electronic device 21.

[0162] Exemplarily, the application having the function of rendering the target object is an AR application.

[0163] S405. The electronic device 21 stores the sub-images in each of the M sub-image groups and feature information of each sub-image.

[0164] S406: The electronic device 21 receives a first operation of the user in the application (used to trigger the application to start a rendering function), and acquires a first image in response to the first operation.

[0165] The first image may be captured by the camera of the electronic device 21 in real time, or may be selected from stored images by the electronic device 21. The images stored by the electronic device 21 may include: images downloaded by the electronic device 21 from a network server, images received by the electronic device 21 from other devices (such as Bluetooth transmission), images received by the electronic device 21 from social applications (such as WeChat transmission, QQ transmission), images taken by the electronic device 21, etc., which are not limited in the embodiments of the present application.

[0166] S407: The electronic device 21 determines whether the first image includes an area of ​​the target object according to the image content of the first image and the image contents of N (N is an integer greater than or equal to 2) sub-images.

[0167] The image content includes characteristic information of the image. This embodiment is described by taking the image content as the characteristic information of the image as an example.

[0168] After acquiring the first image, the electronic device 21 acquires feature information of the first image. The method for the electronic device 21 to acquire feature information of the first image can refer to the description of S1 above, which will not be repeated here.

[0169] The electronic device 21 determines whether the first image includes a region of the target object according to the feature information of the first image and the feature information of the N sub-images. For details, please refer to the description of S4 above, which will not be repeated here. The first image is the field of view image above.

[0170] Optionally, the N sub-images may be all sub-images in the above-mentioned M sub-image groups.

[0171] If the electronic device 21 determines that the first image does not include the target object area, the electronic device 21 terminates the recognition and tracking of the target object. If the electronic device 21 determines that the first image includes the target object area, the electronic device 21 continues to execute S408.

[0172] S408: The electronic device 21 determines a target sub-graph from the N sub-graphs.

[0173] When the first image includes an area of ​​the target object, the number of sub-images whose feature information matches the feature information of the first image may be multiple. Mutually matching feature information means that the value of the similarity between the feature information is higher than the preset value. Among them, the calculation of the similarity between the feature information can refer to the description of S2 above, which is not repeated here. In other words, there may be multiple sub-images in the N sub-images, and the similarity between the feature information of each sub-image in the multiple sub-images and the feature information of the first image is greater than the preset value.

[0174] The electronic device selects a sub-image with the highest similarity value from the N sub-images as a target sub-image.

[0175] S409: The electronic device 21 determines a first local area in the first image that matches the target sub-image.

[0176] The target sub-image is the sub-image with the highest similarity to the feature information of the first image among the N sub-images, and the similarity is greater than a preset value. Therefore, the target sub-image is the sub-image that best matches the first image among the N sub-images. In this way, the electronic device 21 can determine the first local area in the first image that matches the target sub-image. The first local area is the local area of ​​the target object.

[0177] The embodiment of the present application is mainly described by taking the case where the proportion of the first local area in the target object is less than the first preset threshold as an example. Of course, the area of ​​the target object in the first image may also be the entire or most area of ​​the target object.

[0178] After determining the first local area, the electronic device 21 can determine the position information of the first local area in the first image.

[0179] Since the target sub-image is a sub-image with a similarity value higher than a preset value and the highest similarity value among the multiple sub-images, the target sub-image has a larger amount of feature information that matches the first image.

[0180] In this way, the electronic device 21 can calculate the homography matrix H from the target sub-image to the first image based on the matching feature information in the target sub-image and the first image (this process can refer to the calculation method of the homography matrix in the prior art, which will not be repeated here). Further, the electronic device 21 calculates the position information of the first local area in the first image based on the calculated homography matrix H and the coordinates of the target sub-image.

[0181] The electronic device 21 may also calculate the position information of the target object in the first image according to the calculated homography matrix H and the position information of the target sub-image in the target object.

[0182] Specifically, the electronic device 21 may calculate the coordinates of the target object in the coordinate system of the target sub-graph according to the position information of the target sub-graph in the target object.

[0183] In one example, if Figure 8 As shown, if the width of the target object is W, the height is H, the target subgraph is subgraph 81, the width of subgraph 81 is a, and the height is b. The upper left vertex of the target object is the origin of the coordinate system. Then in the coordinate system of the target subgraph, the coordinates of the four vertices of the target object in the clockwise direction are (0, 0), (W, 0), (W, H), (0, H).

[0184] In another example, Fig.12 As shown in , if the width of the target object is W and the height is H, the width of the target subgraph is a and the height is b. The vertex q1 of the target subgraph is the origin of the coordinate system, the coordinates of vertex q1 are (0, 0), the coordinates of vertex q2 are (a, 0), the coordinates of q3 are (a, b), and the coordinates of q4 are (0, b). Fig.12 In the coordinate system shown, the distance between the target object and the x-axis is Dy, and the distance between the target object and the y-axis is Dx. Then the four vertices of the target object are represented in the coordinate system of the target subgraph as: n1 = (-Dx, -Dy), n2 = (W-Dx, -Dy), n3 = (W-Dx, H-Dy), n4 = (-Dx, H-Dy).

[0185] After determining the coordinates of the target object in the coordinate system of the target sub-image, the electronic device 21 calculates the position information of the target object in the first image, that is, the coordinates of the target object in the coordinate system of the first image, based on the “coordinates of the target object in the coordinate system of the target sub-image” and the homography matrix H.

[0186] Specifically, in the clockwise direction, if the vertices of the target object in the coordinate system of the target subgraph are represented as q1, q2, q3, and q4, and the vertices of the target object in the coordinate system of the first image are represented as n1, n2, n3, and n4, then n1 = q1×H, n2 = q2×H, n3 = q3×H, and n4 = q4×H.

[0187] S410: The electronic device 21 renders the first local area according to the position information of the first local area in the first image and a preset rendering rule of the target object to obtain a second image.

[0188] S411. The electronic device 21 displays a second image.

[0189] In one implementation, after determining the position information of the target object in the first image, the electronic device 21 renders all regions of the target object according to the position information and a preset rendering rule of the target object. Since the first image only includes the first local region, after rendering all regions of the target object, the electronic device 21 only displays the rendered first local region.

[0190] The rendering rule of the target object includes: rendering the first local area in a first preset manner, or superimposing a first preset content on the first local area. The first preset content includes at least one of text, picture, video or audio. In addition, the rendering rule of the target object also includes: rendering other areas of the target object except the first local area in a second preset manner, or superimposing a second preset content on other areas of the target object except the first local area, and the second preset content includes at least one of text, picture, video or audio.

[0191] In another implementation, after determining the position information of the first local area in the first image, the electronic device 21 renders the first local area according to the position information and a preset rendering rule of the target object, and displays the rendered first local area.

[0192] The rendering rule of the target object includes: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, wherein the first preset content includes at least one of text, picture, video or audio.

[0193] The first preset mode or the second preset mode may be to display in 3D effect, to display in a highlighted manner, or to add a background color, etc., which is not limited in the embodiment of the present application.

[0194] Exemplarily, the electronic device 21 may superimpose a picture on the first local area, or may superimpose text for guiding user operations on the first local area, or may display the first local area with a 3D effect, or may display the first local area in a highlighted manner, or may add a border to the first local area, or may add a background color to the first partial image, etc.

[0195] Optionally, the rendering rule of the target object may include M (1≤M) sub-rules, each of which corresponds to one or more sub-image groups. That is, the rendering rules corresponding to different sub-image groups may be the same or different. This embodiment of the present application does not limit this.

[0196] If the rendering rules corresponding to different sub-image groups are different, the electronic device 21 renders the first local area according to the rendering rule corresponding to the target sub-image.

[0197] In summary, the electronic device 21 determines the target sub-image with the highest similarity to the first image from the multiple sub-images, and implements image tracking of the target object based on the target sub-image. Since the embodiment of the present application does not limit the size of the target sub-image, even if the first image only includes a small area of ​​the target object, the electronic device 21 can accurately identify and track the target object and then render it.

[0198] To facilitate understanding, the following description is given with reference to specific examples.

[0199] The electronic device 21 is a mobile phone, the application installed in the electronic device 21 is application 1, and the target object is Figure 5 The world map shown is illustrated by taking an example in which user XX uses the mobile phone to shoot the world map and renders data in the shot image.

[0200] like Fig.13 As shown, in the scenario where user XX uses a mobile phone to take a picture of a world map, the image obtained by the mobile phone can be Fig.14 The image 140 shown in (a) may also be Fig.14 The image 141 shown in (b) can also be Fig.14 Image 142 is shown in (c).

[0201] Image 140 includes a local area of ​​the target object, such as a map of China. Image 141 includes the entire area of ​​the target object. Image 142 includes the area at the upper left corner of the target object and also includes other images, such as an image of a flower.

[0202] It is easy to understand that image 140 only includes a local area of ​​the target object, while image 141 includes the entire area of ​​the target object. Therefore, "the distance between the user and the world map in the real environment when image 140 is captured" is smaller than "the distance between the user and the world map in the real environment when image 141 is captured".

[0203] Taking the image 140 obtained by the mobile phone as an example, after obtaining the image 140, the mobile phone uses the image processing method provided in the embodiment of the present application to determine the position information of the world map in the image 140, and renders the image 140 according to the position information and the rendering rule of the world map, and displays as follows: Fig.16 The user interface shown in (A) or (B).

[0204] Fig.16 The interface shown in (A) in FIG. 172 is a diagram showing an image 171 of the Temple of Heaven superimposed on the location "Beijing" shown in image 140, and an image 170 of the Big Wild Goose Pagoda is displayed at the location "Xi'an". In this way, user XX views Fig.16The user interface shown in (A) can directly understand the landmark buildings of the specific geographical location in the image 140. In this scenario, the developer pre-sets the locations to be rendered in the world map and the images of the landmark buildings superimposed for each location to be rendered. In this way, when the field of view image includes a certain location to be rendered and the mobile phone determines the location information of the world map in the field of view image, the image of the corresponding landmark building can be superimposed for the location to be rendered and the rendered image can be displayed.

[0205] Fig.16 The interface shown in (B) in FIG. 1 shows image 173, which is a text "Here is the map of China" superimposed on image 140. In this way, user XX can view Fig.16 The user interface shown in (B) can directly understand which country the map shown in image 140 belongs to. In this scenario, the developer divides the world map by country and determines the text superimposed on the map of each country. In this way, when the field of view image includes all or part of the map of a certain country and the mobile phone determines the location information of the world map in the field of view image, the corresponding text can be superimposed on the map included in the field of view image and the rendered image can be displayed.

[0206] Fig.16 (A) and (B) are merely examples and are not intended to limit the interface displayed after rendering the image. In actual applications, a mobile phone may be displayed in a similar manner.

[0207] Taking the image 142 obtained by the mobile phone as an example, after obtaining the image 142, the mobile phone uses the image processing method provided in the embodiment of the present application to determine the position information of the world map in the image 142, and renders the area of ​​the world map in the image 142 according to the position information and the rendering rule of the world map, and displays as follows: Fig.17 The user interface shown in (A) or (B).

[0208] Fig.17 The interface shown in (A) in FIG. 1 shows image 191. Image 191 is a text “This belongs to Europe” superimposed on image 142. In this way, user XX views Fig.17 The user interface shown in (A) can directly understand which continent the map shown in image 142 belongs to. In this scenario, the developer divides the world map into continents and determines the text superimposed on the map of each continent. In this way, when the field of view image includes all or part of the map of a continent and the mobile phone determines the location information of the world map in the field of view image, the corresponding text can be superimposed on the map included in the field of view image and the rendered image can be displayed.

[0209] Fig.17The interface shown in (B) in FIG. 1 shows image 192. Image 192 is a frame and text "Want to know here, please click" superimposed on image 142. In this way, user XX can see Fig.17 When the user interface shown in (B) is displayed, subsequent operations can be performed according to the text "Want to know more about this place? Please click", such as clicking on the map shown in the interface, and the mobile phone displays an introduction to the place clicked by the user.

[0210] With the above Fig.16 similar, Fig.17 (A) and (B) are merely examples and are not intended to limit the interface displayed after rendering the image. In actual applications, a mobile phone may be displayed in a similar manner.

[0211] The method provided in this embodiment can meet the needs of users more conveniently and quickly, and has strong interactivity; for the terminal, it can track the target object more accurately and perform rendering operations.

[0212] Fig.15 The above steps from "acquiring image 142" to "displaying Fig.17 The process flow of the interface shown in the figure is as follows. Fig.15 As shown, the image processing method provided in this embodiment includes:

[0213] S1500 . The mobile phone receives an operation input by a user for triggering application 1 to start a rendering function, and acquires image 142 in response to the operation.

[0214] Optionally, the mobile phone obtains image 140 through a camera (front camera or rear camera).

[0215] S1501. The mobile phone obtains feature information of the image 142.

[0216] The method for the mobile phone to obtain the characteristic information of the image 142 can refer to the description of S1 above, which will not be repeated here.

[0217] S1502: The mobile phone determines, based on the feature information of the image 142 and the feature information of the N sub-images, that the image 142 includes an area of ​​the target object (specifically, a world map).

[0218] S1503 . The mobile phone determines a target sub-image from the N sub-images, and determines a first local area in the image 142 that matches the target sub-image.

[0219] Please refer to the description of S408 above, which will not be repeated here.

[0220] S1504: The mobile phone renders the first local area according to the position information of the first local area in the image 142 and a preset rendering rule of the target object, and displays the rendered image.

[0221] The specific process of the mobile phone rendering image 142 can refer to the description of rendering in the prior art, which will not be repeated here.

[0222] After the mobile phone renders the image 142, the interface displayed on the mobile phone is as follows Fig.17 shown. Fig.17 The description can refer to the above description and will not be repeated here.

[0223] In real life, Fig.13 In the application scenario shown, user XX usually uses a mobile phone to continuously shoot images of a world map, such as a video. In this way, the mobile phone can obtain multiple continuous images (specifically, image frames). In this case, for the current image, the mobile phone can refer to the previous frame of the current image to complete image tracking.

[0224] The following is combined with the above Figure 4 , the process is described by taking the electronic device 21 as an example. Figure 4 ,like Fig.18 As shown, after S411, the image processing method provided in the embodiment of the present application further includes:

[0225] S180: The electronic device 21 acquires a third image.

[0226] The third image is a next frame image of the first image.

[0227] In actual applications, in a short period of time, the position of the electronic device 21 may not change or may be moving slowly, so that the image content of the third image is partially or completely the same as that of the first image.

[0228] S181. The electronic device 21 determines an adjacent subgraph of a target subgraph.

[0229] The adjacent subgraphs may refer to the description of S403 above, which will not be repeated here.

[0230] As can be seen from the description of the adjacent sub-images in S403 above, the adjacent sub-images of the target sub-image include a part or all of the image of the first local area.

[0231] S182. The electronic device 21 obtains image content of adjacent sub-images of the target sub-image, image content of the target sub-image, and image content of the third image.

[0232] S183. The electronic device 21 determines a first sub-image according to image contents of adjacent sub-images of the target sub-image, image contents of the target sub-image, and image contents of the third image.

[0233] The first sub-image is an adjacent sub-image of the target sub-image and a sub-image in the target sub-image that has the highest similarity with the image content of the third image and a similarity greater than a preset value. It can be simply understood that: in the process of processing the third image, a target sub-image matching the third image is determined.

[0234] S184: The electronic device 21 determines a second local area in the third image that matches the first sub-image.

[0235] S185. The electronic device 21 renders the second local area according to the position information of the second local area in the third image and a preset rendering rule of the target object to obtain a fourth image.

[0236] S186. The electronic device 21 displays a fourth image.

[0237] The process of the electronic device 21 rendering the second partial area and displaying the fourth image is similar to the process of the electronic device 21 rendering the first partial area and displaying the second image, which will not be repeated here.

[0238] The electronic device 21 completes the recognition and tracking of the target object in the third image based on the adjacent sub-images of the target sub-image and the target sub-image. Therefore, the electronic device 21 determines the second local area faster and more accurately, effectively improving the processing performance of the electronic device.

[0239] In combination with the description of S180 to S186, it is easy to understand that if the first image in the embodiment of the present application is the first frame image acquired by the electronic device 21, the above-mentioned N sub-images may be all sub-images in the M sub-image group. If the first image is the i-th (i≠1) frame image acquired by the electronic device 21, the above-mentioned N sub-images may be the target sub-image determined by the electronic device 21 when processing the i-1th frame image and all adjacent sub-images of the target sub-image.

[0240] In summary, even if the image acquired by the electronic device 21 only includes a small area of ​​the target object, the electronic device 21 can accurately identify and track the target object.

[0241] It should be noted that the above description is only described by taking one target object as an example. In practical applications, the number of preset target objects can be multiple, and each target object corresponds to a sub-image database, so that the electronic device 21 stores multiple sub-image databases. The sub-image database stores multiple sub-images of the corresponding target objects and the feature information of each sub-image. In this scenario, after acquiring the first image, the electronic device 21 can select sub-image databases one by one from multiple sub-image databases, and process the first image according to the selected sub-image database. Of course, if the hardware of the electronic device 21 is sufficient to support parallel processing, the electronic device 21 can also process the first image in parallel according to multiple sub-image databases.

[0242] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0243] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0244] like Fig.19 FIG. 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device is used to identify a target object in an image and render the target object in the image, for example, to perform Figure 4 , Fig.15 or Fig.18 The electronic device may include an acquisition unit 190 , a processing unit 191 , a storage unit 192 , and a display unit 193 .

[0245] The acquisition unit 190 is used to acquire a first image. The processing unit 191 is used to determine a target sub-image according to the image content of the first image acquired by the acquisition unit 190 and the image content of the preset N sub-images stored in the storage unit 192; the target sub-image is: among the N sub-images, the sub-image with the highest similarity to the image content of the first image, and the similarity is greater than a preset value, at least two sub-images among the N sub-images have different division granularities, each sub-image among the N sub-images includes a part or all of the image of the preset target object, and N is an integer greater than or equal to 2. The processing unit 191 is also used to determine a first local area in the first image that matches the target sub-image, the proportion of the first local area in the target object is less than a first preset threshold, and the first local area is rendered according to the position information of the first local area in the first image and the preset rendering rule of the target object to obtain a second image. The display unit 193 is used to display the second image obtained by the processing unit.

[0246] For example, combined with Figure 4 The acquisition unit 190 may be used to execute S406, the processing unit 191 may be used to execute S407 to S410, the display unit 193 may be used to execute S411, and the storage unit 192 is used to store N sub-images.

[0247] Optionally, the acquisition unit 190 is specifically configured to acquire a first image captured instantly by the user, or to select the first image from stored images.

[0248] Optionally, in a possible implementation of the present application, the rendering rules of the target object include: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, and the first preset content includes at least one of text, pictures, video or audio.

[0249] Optionally, in another possible implementation of the present application, the rendering rules of the target object also include: rendering other areas in the target object except the first local area using a second preset method, or superimposing second preset content on other areas in the target object except the first local area, and the second preset content includes at least one of text, pictures, video or audio.

[0250] Optionally, in another possible implementation of the present application, the N sub-images are sub-images in M ​​sub-image groups, the M sub-image groups are pre-obtained based on M preset division granularities and target objects, and the M preset division granularities are different from each other; each sub-image group in the M sub-image groups uniquely corresponds to a preset division granularity; all sub-images in each sub-image group constitute the entire area or the first area of ​​the target object, the ratio of the first area to the target object is greater than a second preset threshold, and M is an integer greater than or equal to 2.

[0251] Optionally, in another possible implementation of the present application, each sub-image of the N sub-images includes a partial or complete image of the first local area.

[0252] As an example, combining Figure 3 The function implemented by the acquisition unit 190 in the electronic device is similar to Figure 3 The functions of the camera 293, antenna 1 or antenna 2 are the same, and the functions implemented by the processing unit 191 are the same as those of the Figure 3 The functions of the processor 210 are the same as those of the storage unit 192. Figure 3 The function of the internal memory 221 is the same as that of the display unit 193. Figure 3 The function of display screen 294 in is the same.

[0253] The explanation of the relevant contents in this embodiment can refer to the above method embodiment, which will not be repeated here.

[0254] The present application also provides a chip system, such as Fig. 20 As shown, the chip system includes at least one processor 2001 and at least one interface circuit 2002. The processor 2001 and the interface circuit 2002 can be interconnected via lines. For example, the interface circuit 202 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 2002 can be used to send signals to other devices (such as a processor 2001 or a display screen of an electronic device). Exemplarily, the interface circuit 2002 can read instructions stored in the memory and send the instructions to the processor 2001. When the instructions are executed by the processor 2001, the electronic device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0255] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions. When the instructions are executed on an identification device, the identification device executes each step performed by the identification device in the method flow shown in the above method embodiment.

[0256] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0257] Fig.21 A conceptual partial view of a computer program product provided by an embodiment of the present application is schematically shown, wherein the computer program product includes a computer program for executing a computer process on a computing device.

[0258] In one embodiment, the computer program product is provided using a signal bearing medium 210. The signal bearing medium 210 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 4 , Fig.15 or Fig.18 Thus, for example, reference to Figure 4 One or more features of S407 to S411 may be undertaken by one or more instructions associated with the signal bearing medium 210. In addition, Fig.21 The program instructions in also describe example instructions.

[0259] In some examples, signal bearing medium 210 may include computer readable medium 211, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.

[0260] In some implementations, signal bearing medium 210 may include computer recordable medium 212 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.

[0261] In some implementations, signal bearing medium 210 may include communication medium 213 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0262] The signal bearing medium 210 may be communicated by a wireless form of communication medium 213 (eg, a wireless communication medium conforming to the IEEE 802.11 standard or other transmission protocol). The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0263] In some examples, such as for Figure 4 , Fig.15 or Fig.18 The described electronic devices may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable media 211 , computer-recordable media 212 , and / or communication media 213 .

[0264] It should be understood that the arrangement described here is only for illustrative purposes. Thus, it will be appreciated by those skilled in the art that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements can be omitted together according to the desired result. In addition, many of the described elements can be implemented as discrete or distributed components, or in any appropriate combination and position to combine other components to implement functional entities.

[0265] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer execution instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0266] The above is only a specific implementation of the present application. Those skilled in the art may conceive of changes or substitutions based on the specific implementation provided by the present application, which should all be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: include: acquiring a first image; Determining a target sub-image according to the image content of the first image and the image contents of the preset N sub-images; The target sub-image is: among the N sub-images, the sub-image with the highest similarity to the image content of the first image, and the similarity is greater than a preset value, at least two of the N sub-images have different division granularities, each of the N sub-images includes a partial or complete image of a preset target object, and N is an integer greater than or equal to 2; Determine a first local area in the first image that matches the target sub-image, wherein a proportion of the first local area in the target object is less than a first preset threshold; Rendering the first local area according to the position information of the first local area in the first image and a preset rendering rule of the target object to obtain a second image; The rendering rule of the target object includes: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, wherein the first preset content includes at least one of text, picture, video or audio; The rendering rule of the target object further includes: rendering other areas of the target object except the first local area in a second preset manner, or superimposing second preset content on other areas of the target object except the first local area, wherein the second preset content includes at least one of text, picture, video or audio; The second image is displayed.

2. The image processing method according to claim 1, characterized in that: The acquiring of the first image comprises: The first image captured instantly by the user is acquired, or the first image is selected from stored images.

3. The image processing method according to any one of claims 1 to 2, characterized in that: The N sub-images are sub-images in M ​​sub-image groups, and the M sub-image groups are pre-obtained based on M preset division granularities and the target object, and the M preset division granularities are different from each other; each sub-image group in the M sub-image groups uniquely corresponds to a preset division granularity; all sub-images in each sub-image group constitute the entire area or the first area of ​​the target object, and the proportion of the first area to the target object is greater than a second preset threshold, and M is an integer greater than or equal to 2.

4. The image processing method according to any one of claims 1 to 2, characterized in that: Each of the N sub-images includes a partial or complete image of the first local area.

5. An electronic device, characterized in that: include: An acquisition unit, configured to acquire a first image; a processing unit, configured to determine a target sub-image according to the image content of the first image acquired by the acquisition unit and the image content of the preset N sub-images stored in the storage unit; the target sub-image is: a sub-image among the N sub-images having the highest similarity with the image content of the first image and a similarity greater than a preset value, at least two of the N sub-images having different division granularities, each of the N sub-images including a partial or complete image of a preset target object, and N being an integer greater than or equal to 2; The processing unit is further configured to determine a first local area in the first image that matches the target sub-image, the proportion of the first local area in the target object being less than a first preset threshold, and render the first local area according to position information of the first local area in the first image and a preset rendering rule of the target object to obtain a second image; The rendering rule of the target object includes: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, wherein the first preset content includes at least one of text, picture, video or audio; The rendering rule of the target object further includes: rendering other areas of the target object except the first local area in a second preset manner, or superimposing second preset content on other areas of the target object except the first local area, wherein the second preset content includes at least one of text, picture, video or audio; the rendering rule of the target object includes: rendering the first local area in a first preset manner, or superimposing first preset content on the first local area, wherein the first preset content includes at least one of text, picture, video or audio; The rendering rule of the target object further includes: rendering other areas of the target object except the first local area in a second preset manner, or superimposing second preset content on other areas of the target object except the first local area, wherein the second preset content includes at least one of text, picture, video or audio; A display unit is used to display the second image obtained by the processing unit.

6. The electronic device according to claim 5, characterized in that: The acquisition unit is specifically used to acquire the first image captured instantly by the user, or to select the first image from stored images.

7. The electronic device according to any one of claims 5 to 6, characterized in that: The N sub-images are sub-images in M ​​sub-image groups, and the M sub-image groups are pre-obtained based on M preset division granularities and the target object, and the M preset division granularities are different from each other; each sub-image group in the M sub-image groups uniquely corresponds to a preset division granularity; all sub-images in each sub-image group constitute the entire area or the first area of ​​the target object, and the proportion of the first area to the target object is greater than a second preset threshold, and M is an integer greater than or equal to 2.

8. The electronic device according to any one of claims 5 to 6, characterized in that: Each of the N sub-images includes a partial or complete image of the first local area.

9. A chip system, characterized in that: The chip system is applied to an electronic device including a display screen; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory of the electronic device and send the signal to the processor, the signal including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the image processing method as described in any one of claims 1 to 4.

10. A computer storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the image processing method according to any one of claims 1 to 4.

11. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the image processing method according to any one of claims 1 to 4.

Images