A method for identifying functions of a mobile application based on image recognition

Abstract

The present invention relates to the field of image recognition, and particularly to a method for identifying functions of a mobile application based on image recognition, including: collecting structural feature parameters of an interface icon; determining the priority of interface icon recognition according to the structural feature parameters; performing dimension feature recognition on the interface icon in the order corresponding to the priority; restoring the original three-dimensional structure corresponding to the interface icon according to the dimension feature recognition result; matching the software function corresponding to the interface icon stored in the database according to the original three-dimensional structure; if the original three-dimensional structure cannot be restored, determining the comparison and recognition method of the interface icon according to the resolution of the image; the present invention realizes accurate recognition of the functions of different mobile applications.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and particularly to a method for identifying functions of a mobile application based on image recognition. Background Art

[0002] In today's digital and intelligent era, the number and variety of mobile applications have shown an explosive growth. Users' demands for mobile applications are becoming increasingly diverse and personalized, which requires developers to provide a more intelligent, convenient, and efficient application experience. As an important branch of the field of artificial intelligence, image recognition technology has made remarkable progress in recent years. Traditional methods for identifying functions of mobile applications often rely on users' manual operations and text descriptions. This method is not only inefficient but also prone to errors, while the technology based on image recognition provides a new way to solve these problems. With the continuous improvement of the hardware performance of mobile devices, such as the improvement of the image classification standard value of cameras and the acceleration of processor speed, etc., it makes real-time processing of image recognition in mobile applications possible. In such a technical background, the function recognition of mobile applications based on image recognition has emerged. It aims to quickly and accurately identify the functions of an application through automatic analysis of the application interface image, and provide more intelligent services for users, such as automatically classifying and recommending applications, assisting users to quickly find the required functions, etc.

[0003] Chinese Patent Publication No.: CN108062748A discloses an image recognition system and an image recognition method, including: a cloud computing platform for receiving a test card photo uploaded by a user device, generating and sending down the components and indicators of the test object included in the test card photo according to the color of the test card in the test card photo; a user device having a wireless communication module and a photographing module for photographing the test card photo, uploading the photographed test card photo, and receiving and displaying the components and indicators of the test object. The present invention can facilitate users to complete some health indicators and food component detections that previously needed to be completed on professional equipment anytime and anywhere, avoiding the inconvenience of carrying various detection instruments; at the same time, the cloud computing platform integrates all detection functions, can quickly generate detection results, is intuitive, and improves the detection efficiency; in addition, the cloud computing platform also has fingerprint and face recognition functions, which can solve the problems of identity authentication and security. Thus, there is a problem that the application functions cannot be accurately identified according to the application icon. Summary of the Invention

[0004] Therefore, the present invention provides a method for identifying functions of a mobile application based on image recognition to overcome the problem in the existing image recognition technology that the application functions cannot be accurately identified according to the application icon.

[0005] To achieve the above object, the present invention provides a method for identifying functions of a mobile application based on image recognition, including:

[0006] Structural feature parameters of the acquisition interface icon;

[0007] Determine the priority of interface icon recognition according to the structural feature parameters;

[0008] Perform dimensional feature recognition on the interface icons in the order corresponding to the priority;

[0009] Restore the original three-dimensional structure corresponding to the interface icon according to the dimensional feature recognition result;

[0010] Match the software functions corresponding to the interface icons stored in the database according to the original three-dimensional structure;

[0011] If the original three-dimensional structure cannot be restored, determine the comparison and recognition method of the interface icon according to the resolution of the image;

[0012] Among them, the comparison and recognition method includes comparing with the graphic information in the database to determine the function of the interface icon, or comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function, or training the database to generate a function prediction model to determine the function of the interface icon.

[0013] Furthermore, classifying the interface icons according to the structural feature parameters to determine the priority of interface icon recognition includes:

[0014] Calculate the image classification standard value according to the structural feature parameters;

[0015] Determine the recognition priority of the corresponding interface icons in descending order according to the image classification standard value;

[0016] Among them, the image classification standard value is positively correlated with the recognition priority; the calculation formula of the image classification standard value is:

[0017] S = α×A + β×B

[0018] Among them, A is the superposition quantity of the basic geometric figures constituting the interface icon, α is the weight coefficient of the superposition quantity, B is the quantity of the basic geometric figure types constituting the interface icon, β is the weight coefficient of the quantity of the basic geometric figure types, and α + β = 1.

[0019] Furthermore, performing dimensional feature recognition on the interface icons in the order corresponding to the priority includes:

[0020] Collect the image of the interface icon;

[0021] Identify the dimensional feature data of the interface icon in the image;

[0022] Compare the dimension feature data with the standard dimension feature data stored in the structure database to determine the dimension type of the interface icon;

[0023] Wherein, if the types of the dimension feature data and the standard dimension feature data correspond, determine the dimension type of the interface icon as the corresponding type.

[0024] Further, restore the original three-dimensional structure corresponding to the interface icon, including:

[0025] Determine the constituent elements of the interface icon according to the dimension type of the interface icon;

[0026] Retrieve a number of corresponding geometric structure models in the structure database according to the constituent elements of the interface icon;

[0027] Connect the points of the retrieved geometric structure models to construct an original three-dimensional restoration model;

[0028] Wherein, the constituent elements of the interface icon include two-dimensional basic geometric figures and three-dimensional basic geometric figures.

[0029] Further, the structural feature parameters include the superposition quantity of the basic geometric figures constituting the interface icon and the quantity of the types of the basic geometric figures constituting the interface icon.

[0030] Further, the comparison with the graphic information in the database to determine the function of the interface icon includes: if the image resolution of the interface icon is greater than or equal to the preset image resolution, determine the function of the interface icon by comparing with the graphic information in the database.

[0031] Further, compare the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function, including:

[0032] If the image resolution of the interface icon is less than the preset image resolution and there are variant features in the interface icon, compare the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function;

[0033] Directly compare the non-variant features with the graphic information in the database to preliminarily determine the function of the interface icon to be recognized;

[0034] Compare the feature parts of the variant features separately with the graphic information in the database to further determine the function of the interface icon to be recognized.

[0035] Further, if the image resolution of the interface icon is less than the preset image resolution and there are no variant features in the interface icon, a function prediction model is generated by training the database to determine the function, including:

[0036] Data training is performed on a number of structural feature parameters and a number of interface icon functions in the database to generate a function prediction model;

[0037] The structural feature parameters of the interface icon are input into the function prediction model to output the function of the interface icon;

[0038] Among them, the function prediction model updates the function prediction model by learning the relationship between the structural feature parameters and functions of the interface icon.

[0039] Further, the non-variant feature is an image feature composed of the basic geometric figure or text.

[0040] Further, the variant feature is an image feature obtained by performing several shape transformations and artistic processing on the basic geometric figure or text on the basis of the non-variant feature.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: by collecting the structural feature parameters of the interface icon and classifying them to determine the priority, the present invention can perform the recognition work more efficiently and targeted, optimize the recognition process, and improve the recognition efficiency; by performing dimension feature recognition in the order of priority and restoring the original three-dimensional structure, it helps to more accurately understand and analyze the features of the interface icon, thereby improving the accuracy of the interface icon function recognition; by setting a method for determining the comparison and recognition of the interface icon according to the image classification standard value, the flexibility and adaptability of the icon recognition are increased; in the case where the icon cannot be recognized by the comparison and recognition method, further recognition is performed by combining the surrounding chart information, and the association relationship between the icons is utilized to ensure the accuracy and integrity of the interface icon function recognition.

[0042] Further, the method of the present invention simplifies the complexity of the restoration and improves the restoration efficiency by setting a method for restoring the original three-dimensional structure corresponding to the interface icon and clarifying the basic geometric bodies used when the interface icon is formed; by selecting a perspective similar to the icon for comparison and determining the restored three-dimensional structure, the consistency and accuracy of the restored result and the original icon in terms of vision are ensured, so as to better serve the subsequent function recognition work and improve the quality of the function recognition of the entire mobile application.

[0043] Furthermore, the method of the present invention determines the function by setting a method of comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively, and makes the determination according to the recognition priority order of the interface icon, which improves the efficiency and accuracy of the comparison; directly compares the non-variant features with the graphic information in the database to preliminarily determine the function of the interface icon to be recognized, and realizes a step-by-step processing method to quickly screen out relatively clear function information.

[0044] Furthermore, the method of the present invention determines the function by setting a method of training the database to generate a function prediction model. For interface icons whose functions cannot be determined by conventional comparison and recognition methods, a mapping model generation unit is established separately for database training, which can specifically solve the problems of complex and special icon recognition, and avoid the situation that the functions of these icons cannot be determined due to inability to match; the prediction result of the mapping model generated by training is used as the recognition result, which expands the way of function recognition and improves the coverage rate and accuracy of overall function recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is the overall flowchart of the method for identifying the functions of mobile application programs based on image recognition according to the embodiments of the present invention;

[0046] Figure 2 is the specific flowchart of identifying the dimensional features of the interface icon according to the corresponding order of the priority in the method for identifying the functions of mobile application programs based on image recognition according to the embodiments of the present invention;

[0047] Figure 3 is the specific flowchart of restoring the original three-dimensional structure corresponding to the interface icon in the method for identifying the functions of mobile application programs based on image recognition according to the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to make the objectives and advantages of the present invention clearer, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and do not limit the protection scope of the present invention.

[0050] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in this specification means the presence of features, integers, steps, operations, elements / components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements / components. It should be understood that when we say that a module is "connected" or "coupled" to another module, it can be directly connected or coupled to other modules, or there may also be intermediate units. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling.

[0051] Please refer to Figure 1 、 Figure 2 and Figure 3 as shown, which are respectively the overall flowchart of the method for identifying the functions of a mobile application based on image recognition according to an embodiment of the present invention, the specific flowchart for identifying the dimensional features of interface icons in the order corresponding to the priority, and the specific flowchart for restoring the original three-dimensional structure corresponding to the interface icons. A method for identifying the functions of a mobile application based on image recognition according to the present invention includes:

[0052] Collecting the structural feature parameters of the interface icons;

[0053] Determining the priority of interface icon recognition according to the structural feature parameters;

[0054] Identifying the dimensional features of the interface icons in the order corresponding to the priority;

[0055] Restoring the original three-dimensional structure corresponding to the interface icons according to the dimensional feature recognition result;

[0056] Matching the software functions corresponding to the interface icons stored in the database according to the original three-dimensional structure;

[0057] If the original three-dimensional structure cannot be restored, determining the comparison and recognition method of the interface icons according to the resolution of the image;

[0058] Among them, the comparison and recognition method includes comparing with the graphic information in the database to determine the function of the interface icon, or comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function, or training the database to generate a function prediction model to determine the function of the interface icon.

[0059] In implementation, the present invention can perform the recognition work more efficiently and targeted by collecting the structural feature parameters of the interface icons and classifying them to determine the priority, optimizing the recognition process and improving the recognition efficiency; by performing dimensional feature recognition in the order of priority and restoring the original three-dimensional structure, it helps to more accurately understand and analyze the features of the interface icons, thereby improving the accuracy of interface icon function recognition; by setting the method of comparative recognition of the interface icons according to the image classification standard value, the flexibility and adaptability of icon recognition are increased; in the case where the comparative recognition method cannot recognize the icon, further recognition is performed by combining the surrounding chart information, and the association relationship between the icons is utilized to ensure the accuracy and integrity of the interface icon function recognition.

[0060] Specifically, classifying the interface icons according to the structural feature parameters to determine the priority of interface icon recognition includes:

[0061] Calculating the image classification standard value according to the structural feature parameters;

[0062] Determining the recognition priority of the corresponding interface icons in descending order according to the image classification standard value;

[0063] Among them, the image classification standard value is positively correlated with the recognition priority; the calculation formula of the image classification standard value is:

[0064] S = α×A + β×B

[0065] Wherein, A is the superposition quantity of the basic geometric figures constituting the interface icon, α is the weight coefficient of the superposition quantity, B is the quantity of the basic geometric figure types constituting the interface icon, β is the weight coefficient of the quantity of the basic geometric figure types, and α + β = 1.

[0066] Specifically, the preferred embodiment of the weight coefficient of the superposition quantity of the basic geometric figures constituting the interface icon is 0.3, and the preferred embodiment of the weight coefficient of the quantity of the basic geometric figure types constituting the interface icon is 0.7.

[0067] Specifically, the basic geometric figure types include triangle, rectangle, circle, ellipse, trapezoid.

[0068] Specifically, the superposition quantity of the basic geometric figures refers to the quantity of the basic geometric figures that are superposed together to form the overall planar figure.

[0069] Specifically, the larger the image classification standard value, the higher the complexity of the interface icon.

[0070] Specifically, performing dimensional feature recognition on the interface icons in the order corresponding to the priority includes:

[0071] Collect the image of the interface icon;

[0072] Identify the dimensional feature data of the interface icon in the image;

[0073] Compare the dimensional feature data with the standard dimensional feature data stored in the structure database to determine the dimensional type of the interface icon;

[0074] Wherein, if the types corresponding to the dimensional feature data and the standard dimensional feature data are the same, the dimensional type of the interface icon is determined as the corresponding type.

[0075] Specifically, the dimensional types of the interface icon include two-dimensional interface icons, three-dimensional interface icons, and interface icons combining two dimensions and three dimensions.

[0076] Specifically, the dimensional feature data includes the shadow area, perspective information, and the number of colors with color gradient.

[0077] Specifically, restoring the original three-dimensional structure corresponding to the interface icon includes:

[0078] Determine the constituent elements of the interface icon according to the dimensional type of the interface icon;

[0079] Retrieve a number of corresponding geometric structure models in the structure database according to the constituent elements of the interface icon;

[0080] Connect the points of the retrieved geometric structure models to construct an original three-dimensional restoration model;

[0081] Wherein, the constituent elements of the interface icon include two-dimensional basic geometric figures and three-dimensional basic geometric figures.

[0082] In implementation, the method of the present invention simplifies the complexity of restoration and improves the restoration efficiency by setting a method for restoring the original three-dimensional structure corresponding to the interface icon and clarifying the basic geometric bodies used in the composition of the interface icon; by selecting a perspective similar to the icon for comparison and determining the restored three-dimensional structure, it ensures the visual consistency and accuracy between the restoration result and the original icon, thus better serving the subsequent function recognition work and improving the quality of function recognition of the entire mobile application.

[0083] Specifically, the structural feature parameters include the superposition quantity of the basic geometric figures constituting the interface icon and the quantity of the types of the basic geometric figures constituting the interface icon.

[0084] Specifically, the structural feature parameters further include the type of the basic geometric figure of the interface icon, the position and quantity of the line segments of the interface icon, and the colors of each point, each line segment, and each face of the interface icon.

[0085] Specifically, comparing with the graphic information in the database to determine the function of the interface icon includes: if the image resolution of the interface icon is greater than or equal to the preset image resolution, comparing with the graphic information in the database to determine the function of the interface icon.

[0086] Specifically, comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function includes:

[0087] If the image resolution of the interface icon is less than the preset image resolution and there are variant features in the interface icon, comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function;

[0088] Directly comparing the non-variant features with the graphic information in the database to preliminarily determine the function of the interface icon to be recognized;

[0089] Separately comparing the feature part of the variant features with the graphic information in the database to further determine the function of the interface icon to be recognized.

[0090] In implementation, the method of the present invention determines the function by setting a method of comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively, and makes a determination according to the recognition priority order of the interface icon, improving the efficiency and accuracy of the comparison; directly comparing the non-variant features with the graphic information in the database to preliminarily determine the function of the interface icon to be recognized, realizing a step-by-step processing method to quickly screen out relatively clear function information.

[0091] Specifically, if the image resolution of the interface icon is less than the preset image resolution and there are no variant features in the interface icon, training the database to generate a function prediction model to determine the function, including,

[0092] Training the data of several structural feature parameters and several interface icon functions in the database to generate a function prediction model;

[0093] Inputting the structural feature parameters of the interface icon into the function prediction model to output the function of the interface icon;

[0094] Wherein, the function prediction model updates the function prediction model by learning the relationship between the structural feature parameters of the interface icon and the function.

[0095] Specifically, the generation and update process of the function prediction model is a conventional technical means well-known to those skilled in the art, so the generation and update process of the function prediction model will not be elaborated herein.

[0096] In implementation, the method of the present invention sets a method for determining functions by training a database to generate a function prediction model. For interface icons whose functions cannot be determined by conventional comparison and recognition methods, a separate mapping model generation unit is established to perform database training. This can specifically solve complex and special icon recognition problems and avoid the situation where the functions of these icons cannot be determined due to matching. The prediction results of the mapping model generated by training are used as recognition results, which expands the means of function recognition and improves the coverage and accuracy of overall function recognition.

[0097] Specifically, the non-variant features are image features formed by the basic geometric figures or characters, and the variant features are image features obtained by performing several shape transformations and artistic processing on the basic geometric figures or characters based on the non-variant features.

[0098] In implementation, shape transformation includes distorting a basic geometric figure such as a square so that the edge of the square appears wavy, enlarging or reducing the square, cutting the square to form an irregular geometric shape combination, and superimposing multiple basic figures such as circles, triangles and squares to form a complex pattern;

[0099] The artistic processing of text includes font deformation, such as turning straight lines into curves, breaking text into basic strokes and recombining them into new structures, and adding shadows and depth to text to increase the three-dimensional effect of text.

[0100] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for identifying mobile application functions based on image recognition, characterized in that: include: Collect structural characteristic parameters of interface icons; Determining the priority of interface icon recognition according to the structural feature parameters; Performing dimensional feature recognition on the interface icons in an order corresponding to the priorities; Restoring the original three-dimensional structure corresponding to the interface icon according to the dimensional feature recognition result; Matching software functions corresponding to the interface icons stored in a database according to the original three-dimensional structure; If the original three-dimensional structure cannot be restored, determining a comparison and recognition method of the interface icon according to the resolution of the image; Wherein, the comparison and identification method includes comparing with the graphic information in the database to determine the function of the interface icon, or comparing the variant features and non-variant features of the interface icon with the graphic information in the database to determine the function, or training the database to generate a function prediction model to determine the function of the interface icon; Classifying the interface icons according to the structural feature parameters to determine the priority of interface icon recognition includes: Calculate the image classification standard value according to the structural feature parameters; Determining the recognition priority of the corresponding interface icon in descending order according to the image classification standard value; The image classification standard value is positively correlated with the recognition priority; the calculation formula of the image classification standard value is: S = α × A + β × B Among them, A is the number of superpositions of basic geometric figures constituting the interface icon, α is the weight coefficient of the superposition number, B is the number of basic geometric figure types constituting the interface icon, β is the weight coefficient of the number of basic geometric figure types, α+β=1; The non-variant feature is an image feature composed of the basic geometric figures or text; The variant feature is an image feature that undergoes several shape transformations and artistic processing on basic geometric figures or texts based on the non-variant feature.

2. The method for identifying mobile application functions based on image recognition according to claim 1, characterized in that: The dimensional features of the interface icons are identified in the order corresponding to the priorities, including: Collecting an image of the interface icon; Identifying dimensional feature data of an interface icon in the image; Comparing the dimension feature data with standard dimension feature data stored in a structure database to determine the dimension type of the interface icon; If the dimensional feature data and the standard dimensional feature data are of the same type, the dimensional type of the interface icon is determined to be the corresponding type.

3. The method for identifying mobile application functions based on image recognition according to claim 2, characterized in that: Restoring the original three-dimensional structure corresponding to the interface icon includes: Determining constituent elements of the interface icon according to the dimension type of the interface icon; Retrieving corresponding geometric structure models in the structure database according to the constituent elements of the interface icon; Connecting the retrieved geometric structure models point by point to construct an original three-dimensional restoration model; Among them, the constituent elements of the interface icon include two-dimensional basic geometric figures and three-dimensional basic geometric figures.

4. The method for identifying mobile application functions based on image recognition according to claim 3, characterized in that: The structural characteristic parameters include the number of superpositions of basic geometric figures constituting the interface icon and the number of types of basic geometric figures constituting the interface icon.

5. The method for identifying mobile application functions based on image recognition according to claim 4, characterized in that: The comparing with the graphic information in the database to determine the function of the interface icon includes: if the image resolution of the interface icon is greater than or equal to the preset image resolution, then comparing with the graphic information in the database to determine the function of the interface icon.

6. The method for identifying mobile application functions based on image recognition according to claim 5, characterized in that: The variant features and the non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function, including: If the image resolution of the interface icon is less than the preset image resolution and there are variant features in the interface icon, the variant features and non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function; Compare the non-variant features directly with the graphic information in the database to preliminarily determine the function of the interface icon to be identified; The characteristic part of the variant feature is compared with the graphic information in the database to further determine the function of the interface icon to be identified.

7. The method for identifying mobile application functions based on image recognition according to claim 6, characterized in that: If the image resolution of the interface icon is less than the preset image resolution and there is no variant feature in the interface icon, training the database to generate a function prediction model to determine the function, including: Performing data training on a number of structural feature parameters and a number of interface icon functions in the database to generate a function prediction model; Inputting the structural characteristic parameters of the interface icon into the function prediction model to output the function of the interface icon; The function prediction model updates the function prediction model by learning the relationship between the structural feature parameters and functions of the interface icons.

Images