Mobile application program function identification method based on image identification

Abstract

The invention relates to the field of image recognition, in particular to a mobile application program function recognition method based on image recognition, which comprises the following steps: acquiring structural feature parameters of interface icons; determining the priority of interface icon recognition according to the structural feature parameters; performing dimension feature recognition on the interface icons according to a sequence corresponding to the priorities; restoring an original three-dimensional structure corresponding to the interface icon according to the dimension feature recognition result; matching software functions corresponding to the interface icons stored in a database according to the original three-dimensional structure; if the original three-dimensional structure cannot be restored, determining a comparison recognition mode of the interface icon according to the resolution of the image; according to the invention, accurate identification of functions of different mobile application programs is realized.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method for identifying functions of mobile application programs based on image recognition. Background Art

[0002] In today's digital and intelligent era, the number and types of mobile applications are growing explosively. Users' demands for mobile applications are becoming increasingly diversified and personalized, which requires developers to provide a more intelligent, convenient and efficient application experience. As an important branch of artificial intelligence, image recognition technology has made significant progress in recent years. Traditional mobile application function recognition methods often rely on users' manual operations and text descriptions, which is not only inefficient but also prone to errors. Image recognition-based technology provides a new way to solve these problems. With the continuous improvement of mobile device hardware performance, such as the increase in camera image classification standards and faster processor speeds, real-time processing of image recognition in mobile applications is possible. Against this technical background, mobile application function recognition based on image recognition has emerged. It aims to quickly and accurately identify the functions of an application through automatic analysis of the application interface image, and provide users with more intelligent services, such as automatic classification and recommendation of applications, and assisting users to quickly find the required functions.

[0003] Chinese Patent Publication No.: CN108062748A discloses an image recognition system and an image recognition method, including: a cloud computing platform, which is used to receive test card photos uploaded by user devices, and generate and send test object components and indicators contained in the test card photos according to the color of the test card in the test card photos; a user device with a wireless communication module and a shooting module, which is used to take test card photos, upload the taken test card photos, and receive and display the test object components and indicators. The present invention can facilitate users to complete the detection of some health indicators and food ingredients that previously required professional equipment anytime and anywhere, avoiding the inconvenience of carrying various detection instruments; at the same time, the cloud computing platform integrates all detection functions, can quickly generate detection results, is intuitive, and improves detection efficiency; in addition, the cloud computing platform also has fingerprint and face recognition functions, which can solve identity authentication and security issues. It can be seen that there is a problem that the application function cannot be accurately identified based on the application icon. Summary of the invention

[0004] To this end, the present invention provides a method for identifying mobile application functions based on image recognition, so as to overcome the problem that the existing image recognition technology cannot accurately identify application functions based on application icons.

[0005] To achieve the above object, the present invention provides a method for identifying mobile application functions based on image recognition, comprising:

[0006] Collect structural characteristic parameters of interface icons;

[0007] Determining the priority of interface icon recognition according to the structural feature parameters;

[0008] Performing dimensional feature recognition on the interface icons in an order corresponding to the priorities;

[0009] Restoring the original three-dimensional structure corresponding to the interface icon according to the dimensional feature recognition result;

[0010] Matching software functions corresponding to the interface icons stored in a database according to the original three-dimensional structure;

[0011] If the original three-dimensional structure cannot be restored, determining a comparison and recognition method of the interface icon according to the resolution of the image;

[0012] Among them, the comparison and identification method includes comparing with the graphic information in the database to determine the function of the interface icon, or comparing the variant features and non-variant features of the interface icon with the graphic information in the database to determine the function, or training the database to generate a function prediction model to determine the function of the interface icon.

[0013] Further, classifying the interface icons according to the structural feature parameters to determine the priority of interface icon recognition includes:

[0014] Calculate the image classification standard value according to the structural feature parameters;

[0015] Determining the recognition priority of the corresponding interface icon in descending order according to the image classification standard value;

[0016] The image classification standard value is positively correlated with the recognition priority; the calculation formula of the image classification standard value is:

[0017] S = α × A + β × B

[0018] Among them, A is the number of superimposed basic geometric figures constituting the interface icon, α is the weight coefficient of the superimposed number, B is the number of basic geometric figure types constituting the interface icon, β is the weight coefficient of the number of basic geometric figure types, α+β=1.

[0019] Furthermore, the dimensional features of the interface icons are identified in the order corresponding to the priorities, including:

[0020] Collecting an image of the interface icon;

[0021] Identifying dimensional feature data of an interface icon in the image;

[0022] Comparing the dimension feature data with standard dimension feature data stored in a structure database to determine the dimension type of the interface icon;

[0023] If the dimensional feature data and the standard dimensional feature data are of the same type, the dimensional type of the interface icon is determined to be the corresponding type.

[0024] Furthermore, restoring the original three-dimensional structure corresponding to the interface icon includes:

[0025] Determining constituent elements of the interface icon according to the dimension type of the interface icon;

[0026] Retrieving corresponding geometric structure models in the structure database according to the constituent elements of the interface icon;

[0027] Connecting the retrieved geometric structure models point by point to construct an original three-dimensional restoration model;

[0028] Among them, the constituent elements of the interface icon include two-dimensional basic geometric figures and three-dimensional basic geometric figures.

[0029] Furthermore, the structural characteristic parameters include the number of superpositions of basic geometric figures constituting the interface icon and the number of types of basic geometric figures constituting the interface icon.

[0030] Furthermore, the comparing with the graphic information in the database to determine the function of the interface icon includes: if the image resolution of the interface icon is greater than or equal to the preset image resolution, then comparing with the graphic information in the database to determine the function of the interface icon.

[0031] Further, the variant features and the non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function, including:

[0032] If the image resolution of the interface icon is less than the preset image resolution and there are variant features in the interface icon, the variant features and non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function;

[0033] Compare the non-variant features directly with the graphic information in the database to preliminarily determine the function of the interface icon to be identified;

[0034] The characteristic part of the variant feature is compared with the graphic information in the database to further determine the function of the interface icon to be identified.

[0035] Furthermore, if the image resolution of the interface icon is less than the preset image resolution and there is no variant feature in the interface icon, the database is trained to generate a function prediction model to determine the function, including:

[0036] Performing data training on a number of structural feature parameters and a number of interface icon functions in the database to generate a function prediction model;

[0037] Inputting the structural characteristic parameters of the interface icon into the function prediction model to output the function of the interface icon;

[0038] The function prediction model updates the function prediction model by learning the relationship between the structural feature parameters and functions of the interface icons.

[0039] Furthermore, the non-variant feature is an image feature composed of the basic geometric figures or texts.

[0040] Furthermore, the variant feature is an image feature obtained by performing several shape transformations and artistic processing on basic geometric figures or texts based on the non-variant feature.

[0041] Compared with the prior art, the beneficial effects of the present invention lie in that, by collecting the structural feature parameters of the interface icons and classifying them to determine the priority, the present invention can perform the recognition work more efficiently and specifically, optimize the recognition process, and improve the recognition efficiency; by performing dimensional feature recognition and restoring the original three-dimensional structure in order of priority, it helps to more accurately understand and analyze the characteristics of the interface icons, thereby improving the accuracy of the interface icon function recognition; by setting a comparative recognition method for determining the interface icons according to the image classification standard value, the flexibility and adaptability of icon recognition are increased; when the comparative recognition method cannot identify the icon, further recognition is performed in combination with the surrounding chart information, and the association relationship between icons is utilized to ensure the accuracy and completeness of the interface icon function recognition.

[0042] Furthermore, the method of the present invention clarifies the basic geometric bodies used in the construction of the interface icon by setting up a method for restoring the original three-dimensional structure corresponding to the interface icon, which helps to simplify the complexity of restoration and improve the efficiency of restoration; by selecting a perspective similar to the icon for comparison and determining the restored three-dimensional structure, the visual consistency and accuracy of the restoration result with the original icon is ensured, thereby better serving the subsequent function identification work and improving the quality of the entire mobile application function identification.

[0043] Furthermore, the method of the present invention improves the efficiency and accuracy of comparison by setting up a method for comparing variant features and non-variant features of interface icons with graphic information in a database to determine functions, and makes judgments in order of recognition priority of interface icons; the function of the interface icon to be identified is preliminarily determined by directly comparing non-variant features with graphic information in a database, thereby realizing a step-by-step processing method to quickly screen out clearer functional information.

[0044] Furthermore, the method of the present invention sets a method for determining functions by training a database to generate a function prediction model. For interface icons whose functions cannot be determined by conventional comparison and recognition methods, a separate mapping model generation unit is established to perform database training. This can specifically solve complex and special icon recognition problems and avoid the situation where the functions of these icons cannot be determined due to matching. The prediction results of the mapping model generated by training are used as recognition results, which expands the means of function recognition and improves the coverage and accuracy of overall function recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is an overall flow chart of a method for identifying a function of a mobile application program based on image recognition according to an embodiment of the present invention;

[0046] Figure 2 A specific flow chart of the method for identifying mobile application function based on image recognition according to an embodiment of the present invention, in which dimensional features of interface icons are identified in the order corresponding to the priorities;

[0047] Figure 3 The present invention is a specific flow chart of restoring the original three-dimensional structure corresponding to the interface icon according to the method for identifying mobile application functions based on image recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0050] It will be understood by those skilled in the art that, unless otherwise stated, the singular forms "a", "an" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in this specification refers to the presence of features, integers, steps, operations, elements / components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements / components. It should be understood that when we refer to a module as being "connected" or "coupled" to another module, it may be directly connected or coupled to the other module, or there may be an intermediate unit. In addition, "connected" or "coupled" used herein may include wireless connection or wireless coupling.

[0051] See also Figure 1 , Figure 2 as well as Figure 3 As shown, they are respectively an overall flow chart of a method for identifying a function of a mobile application based on image recognition according to an embodiment of the present invention, a specific flow chart for identifying dimensional features of an interface icon in the order corresponding to the priority, and a specific flow chart for restoring the original three-dimensional structure corresponding to the interface icon. A method for identifying a function of a mobile application based on image recognition according to the present invention comprises:

[0052] Collect structural characteristic parameters of interface icons;

[0053] Determining the priority of interface icon recognition according to the structural feature parameters;

[0054] Performing dimensional feature recognition on the interface icons in an order corresponding to the priorities;

[0055] Restoring the original three-dimensional structure corresponding to the interface icon according to the dimensional feature recognition result;

[0056] Matching software functions corresponding to the interface icons stored in a database according to the original three-dimensional structure;

[0057] If the original three-dimensional structure cannot be restored, determining a comparison and recognition method of the interface icon according to the resolution of the image;

[0058] Among them, the comparison and identification method includes comparing with the graphic information in the database to determine the function of the interface icon, or comparing the variant features and non-variant features of the interface icon with the graphic information in the database to determine the function, or training the database to generate a function prediction model to determine the function of the interface icon.

[0059] In implementation, the present invention can perform recognition work more efficiently and specifically by collecting structural feature parameters of interface icons and classifying them to determine priorities, thereby optimizing the recognition process and improving recognition efficiency; by performing dimensional feature recognition and restoring the original three-dimensional structure in order of priority, it helps to more accurately understand and analyze the characteristics of interface icons, thereby improving the accuracy of interface icon function recognition; by setting a comparative recognition method for determining the interface icons according to image classification standard values, the flexibility and adaptability of icon recognition are increased; when the comparative recognition method cannot identify the icon, further recognition is performed in combination with surrounding chart information, and the association relationship between icons is utilized to ensure the accuracy and completeness of interface icon function recognition.

[0060] Specifically, classifying the interface icons according to the structural feature parameters to determine the priority of interface icon recognition includes:

[0061] Calculate the image classification standard value according to the structural feature parameters;

[0062] Determining the recognition priority of the corresponding interface icon in descending order according to the image classification standard value;

[0063] The image classification standard value is positively correlated with the recognition priority; the calculation formula of the image classification standard value is:

[0064] S = α × A + β × B

[0065] Among them, A is the number of superimposed basic geometric figures constituting the interface icon, α is the weight coefficient of the superimposed number, B is the number of basic geometric figure types constituting the interface icon, β is the weight coefficient of the number of basic geometric figure types, α+β=1.

[0066] Specifically, a preferred embodiment of the weight coefficient of the number of superpositions of basic geometric figures constituting the interface icon is 0.3, and a preferred embodiment of the weight coefficient of the number of types of basic geometric figures constituting the interface icon is 0.7.

[0067] Specifically, basic geometric figure types include triangle, rectangle, circle, ellipse, and trapezoid.

[0068] Specifically, the stacking number of basic geometric figures refers to the number of basic geometric figures that are stacked together to form an overall plane figure.

[0069] Specifically, the larger the image classification standard value is, the higher the complexity of the interface icon is.

[0070] Specifically, the dimensional features of the interface icons are identified in the order corresponding to the priorities, including:

[0071] Collecting an image of the interface icon;

[0072] Identifying dimensional feature data of an interface icon in the image;

[0073] Comparing the dimension feature data with standard dimension feature data stored in a structure database to determine the dimension type of the interface icon;

[0074] If the dimensional feature data and the standard dimensional feature data are of the same type, the dimensional type of the interface icon is determined to be the corresponding type.

[0075] Specifically, the dimensional types of interface icons include two-dimensional interface icons, three-dimensional interface icons, and interface icons that are a combination of two-dimensional and three-dimensional.

[0076] Specifically, the dimensional feature data includes shadow area, perspective information, and the number of colors in the color gradient.

[0077] Specifically, restoring the original three-dimensional structure corresponding to the interface icon includes:

[0078] Determining constituent elements of the interface icon according to the dimension type of the interface icon;

[0079] Retrieving corresponding geometric structure models in the structure database according to the constituent elements of the interface icon;

[0080] Connecting the retrieved geometric structure models point by point to construct an original three-dimensional restoration model;

[0081] Among them, the constituent elements of the interface icon include two-dimensional basic geometric figures and three-dimensional basic geometric figures.

[0082] In implementation, the method of the present invention clarifies the basic geometric bodies used in the composition of the interface icon by setting a method for restoring the original three-dimensional structure corresponding to the interface icon, which helps to simplify the complexity of restoration and improve the efficiency of restoration; by selecting a perspective similar to the icon for comparison and determining the restored three-dimensional structure, the visual consistency and accuracy of the restoration result with the original icon is ensured, thereby better serving the subsequent function recognition work and improving the quality of the entire mobile application function recognition.

[0083] Specifically, the structural characteristic parameters include the number of superpositions of basic geometric figures constituting the interface icon and the number of types of basic geometric figures constituting the interface icon.

[0084] Specifically, the structural feature parameters also include the type of basic geometric figures of the interface icon, the position and number of line segments of the interface icon, and the colors of each point, each line segment, and each surface of the interface icon.

[0085] Specifically, the comparing with the graphic information in the database to determine the function of the interface icon includes: if the image resolution of the interface icon is greater than or equal to the preset image resolution, then comparing with the graphic information in the database to determine the function of the interface icon.

[0086] Specifically, the variant features and non-variant features of the interface icon are compared with the graphic information in the database to determine the function, including:

[0087] If the image resolution of the interface icon is less than the preset image resolution and there are variant features in the interface icon, the variant features and non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function;

[0088] Compare the non-variant features directly with the graphic information in the database to preliminarily determine the function of the interface icon to be identified;

[0089] The characteristic part of the variant feature is compared with the graphic information in the database to further determine the function of the interface icon to be identified.

[0090] In implementation, the method of the present invention improves the efficiency and accuracy of comparison by setting a method for comparing the variant features and non-variant features of the interface icon with the graphic information in the database respectively to determine the function, and makes judgments according to the recognition priority order of the interface icon; the non-variant features are directly compared with the graphic information in the database to preliminarily determine the function of the interface icon to be identified, realizing a step-by-step processing method to quickly screen out clearer functional information.

[0091] Specifically, if the image resolution of the interface icon is less than the preset image resolution and there is no variant feature in the interface icon, the database is trained to generate a function prediction model to determine the function, including:

[0092] Performing data training on a number of structural feature parameters and a number of interface icon functions in the database to generate a function prediction model;

[0093] Inputting the structural characteristic parameters of the interface icon into the function prediction model to output the function of the interface icon;

[0094] The function prediction model updates the function prediction model by learning the relationship between the structural feature parameters and functions of the interface icons.

[0095] Specifically, the generation and update process of the function prediction model is a conventional technical means well known to those skilled in the art, so the generation and update process of the function prediction model will not be described in detail here.

[0096] In implementation, the method of the present invention sets a method for determining functions by training a database to generate a function prediction model. For interface icons whose functions cannot be determined by conventional comparison and recognition methods, a separate mapping model generation unit is established to perform database training. This can specifically solve complex and special icon recognition problems and avoid the situation where the functions of these icons cannot be determined due to matching. The prediction results of the mapping model generated by training are used as recognition results, which expands the means of function recognition and improves the coverage and accuracy of overall function recognition.

[0097] Specifically, the non-variant features are image features formed by the basic geometric figures or characters, and the variant features are image features obtained by performing several shape transformations and artistic processing on the basic geometric figures or characters based on the non-variant features.

[0098] In implementation, shape transformation includes distorting a basic geometric figure such as a square so that the edge of the square appears wavy, enlarging or reducing the square, cutting the square to form an irregular geometric shape combination, and superimposing multiple basic figures such as circles, triangles and squares to form a complex pattern;

[0099] The artistic processing of text includes font deformation, such as turning straight lines into curves, breaking text into basic strokes and recombining them into new structures, and adding shadows and depth to text to increase the three-dimensional effect of text.

[0100] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for identifying mobile application functions based on image recognition, characterized in that: include: Collect structural characteristic parameters of interface icons; Determining the priority of interface icon recognition according to the structural feature parameters; Performing dimensional feature recognition on the interface icons in an order corresponding to the priorities; Restoring the original three-dimensional structure corresponding to the interface icon according to the dimensional feature recognition result; Matching software functions corresponding to the interface icons stored in a database according to the original three-dimensional structure; If the original three-dimensional structure cannot be restored, determining a comparison and recognition method of the interface icon according to the resolution of the image; Among them, the comparison and identification method includes comparing with the graphic information in the database to determine the function of the interface icon, or comparing the variant features and non-variant features of the interface icon with the graphic information in the database to determine the function, or training the database to generate a function prediction model to determine the function of the interface icon.

2. The method for identifying mobile application functions based on image recognition according to claim 1, characterized in that: Classifying the interface icons according to the structural feature parameters to determine the priority of interface icon recognition includes: Calculate the image classification standard value according to the structural feature parameters; Determining the recognition priority of the corresponding interface icon in descending order according to the image classification standard value; The image classification standard value is positively correlated with the recognition priority; the calculation formula of the image classification standard value is: S=α×A+β×B Among them, A is the number of superimposed basic geometric figures constituting the interface icon, α is the weight coefficient of the superimposed number, B is the number of basic geometric figure types constituting the interface icon, β is the weight coefficient of the number of basic geometric figure types, α+β=1.

3. The method for identifying mobile application functions based on image recognition according to claim 2, characterized in that: The dimensional features of the interface icons are identified in the order corresponding to the priorities, including: Collecting an image of the interface icon; Identifying dimensional feature data of an interface icon in the image; Comparing the dimension feature data with standard dimension feature data stored in a structure database to determine the dimension type of the interface icon; If the dimensional feature data and the standard dimensional feature data are of the same type, the dimensional type of the interface icon is determined to be the corresponding type.

4. The method for identifying mobile application functions based on image recognition according to claim 3, characterized in that: Restoring the original three-dimensional structure corresponding to the interface icon includes: Determining constituent elements of the interface icon according to the dimension type of the interface icon; Retrieving corresponding geometric structure models in the structure database according to the constituent elements of the interface icon; Connecting the retrieved geometric structure models point by point to construct an original three-dimensional restoration model; Among them, the constituent elements of the interface icon include two-dimensional basic geometric figures and three-dimensional basic geometric figures.

5. The method for identifying mobile application functions based on image recognition according to claim 4, characterized in that: The structural characteristic parameters include the number of superpositions of basic geometric figures constituting the interface icon and the number of types of basic geometric figures constituting the interface icon.

6. The method for identifying mobile application functions based on image recognition according to claim 5, characterized in that: The comparing with the graphic information in the database to determine the function of the interface icon includes: if the image resolution of the interface icon is greater than or equal to the preset image resolution, then comparing with the graphic information in the database to determine the function of the interface icon.

7. The method for identifying mobile application functions based on image recognition according to claim 6, characterized in that: The variant features and the non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function, including: If the image resolution of the interface icon is less than the preset image resolution and there are variant features in the interface icon, the variant features and non-variant features of the interface icon are respectively compared with the graphic information in the database to determine the function; Compare the non-variant features directly with the graphic information in the database to preliminarily determine the function of the interface icon to be identified; The characteristic part of the variant feature is compared with the graphic information in the database to further determine the function of the interface icon to be identified.

8. The method for identifying mobile application functions based on image recognition according to claim 7, characterized in that: If the image resolution of the interface icon is less than the preset image resolution and there is no variant feature in the interface icon, training the database to generate a function prediction model to determine the function, including: Performing data training on a number of structural feature parameters and a number of interface icon functions in the database to generate a function prediction model; Inputting the structural characteristic parameters of the interface icon into the function prediction model to output the function of the interface icon; The function prediction model updates the function prediction model by learning the relationship between the structural feature parameters and functions of the interface icons.

9. The method for identifying mobile application functions based on image recognition according to claim 8, characterized in that: The non-variant features are image features formed by the basic geometric figures or texts.

10. The method for identifying mobile application functions based on image recognition according to claim 9, characterized in that: The variant feature is an image feature that undergoes several shape transformations and artistic processing on basic geometric figures or texts based on the non-variant feature.

Images