Website asset identification method and device, model training method and electronic equipment

Abstract

The present application relates to the field of website asset identification, and discloses a website asset identification method, a model training method, a website asset identification device, an electronic device, and a computer-readable storage medium. Among them, the website asset identification method includes: collecting website page data of the target website, the website page data includes visual element data of the target website; generating key point data according to the visual element data; generating a website asset heat map of the target website according to the key point data, the website asset heat map includes several key points corresponding to the key point data; using a preset recognition model to automatically annotate the website asset heat map to obtain the asset information of the key points. Compared with the prior art, the website asset identification method, model training method, website asset identification device, electronic device, and computer-readable storage medium provided in the embodiments of the present application have the advantage of being able to improve the accuracy of website asset identification results.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a website asset identification method, a model training method, a website asset identification device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of Internet technology, the number and types of website assets are increasing, including but not limited to links, buttons, forms, pictures, videos, etc. These assets are the basis for the realization of website functions, and are also the key to website user experience and business operations. Therefore, effective identification and management of website assets is of great significance to ensuring website security, optimizing user experience and improving operational efficiency.

[0003] At present, website asset recognition technologies mainly rely on traditional image processing and pattern recognition methods. These methods usually include steps such as image acquisition, feature extraction, asset classification and recognition. Although these existing website asset recognition technologies can realize the recognition of website assets, the accuracy of website asset recognition results of these existing website asset recognition technologies is low. Summary of the invention

[0004] The purpose of this application is to provide a website asset identification method, a model training method, a website asset identification device, an electronic device and a computer-readable storage medium, which can improve the accuracy of website asset identification results.

[0005] In a first aspect, an embodiment of the present application provides a website asset identification method, comprising: collecting website page data of a target website, the website page data including visual element data of the target website; generating key point data based on the visual element data; generating a website asset heat map of the target website based on the key point data, the website asset heat map including a number of key points corresponding to the key point data; and automatically annotating the website asset heat map using a preset recognition model to obtain asset information of the key points.

[0006] Compared with the related art, in the website asset identification method provided in the embodiment of the present application, after collecting the website page data of the target website, key point data is generated by feature extraction of the visual element data in the website page data, and a website asset heat map of the target website is generated based on the key point data. Since the website asset heat map displays network assets more intuitively and obviously, the website asset heat map is automatically labeled using a preset recognition model, and the accuracy of the asset information of the key points obtained is also greatly improved; in addition, the more intuitive website asset heat map can also make the staff more efficient in monitoring, analyzing and optimizing network assets.

[0007] In an optional embodiment, generating key point data according to the visual element data includes: mapping the visual element data to a multidimensional space and performing a convolution operation to generate a plurality of feature maps; and selecting a target feature map from the plurality of feature maps as the key point data according to the importance of each of the feature maps. Since different websites have different focuses on website assets, by performing a convolution operation on the visual element data in the form of a multidimensional space mapping, features of the visual element data in different dimensions can be extracted. Therefore, the generated plurality of feature maps can also correspond to features of the visual element data in different dimensions. By extracting the features of the visual element data in different dimensions and selecting them according to their importance, the versatility and accuracy of extracting key point data for different websites can be improved.

[0008] In an optional embodiment, generating a website asset heat map of the target website based on the key point data includes: inputting the key point data into a pre-trained heat map model for feature analysis; obtaining an output result of the heat map model, the output result including the color type, color intensity and position data of the key point corresponding to each of the key points; generating the website asset heat map based on the position data, the color type and the color intensity corresponding to all the key points.

[0009] In an optional embodiment, the convolution operation on the visual element data includes: the heat map model includes a feature extraction layer and a target detection layer; for each of the key points, the feature extraction layer is used to generate a feature map of the corresponding scale according to the key point data and size information; the target detection layer is used to determine the position data, color type and color intensity of the key point according to a number of the feature maps, and generate the website asset heat map. The heat map model generates feature maps of different scales for website assets of various sizes, so that the heat map model can adapt to website assets of different sizes and reduce the omission of website assets; in addition, the feature map analysis of website assets at different scales can also make the color type and color intensity corresponding to each key point finally output by the heat map model more accurate, improve the accuracy of the website asset heat map subsequently established, and then improve the accuracy of the recognition result of the website asset heat map for website asset recognition.

[0010] In an optional embodiment, the website page data also includes user behavior data of the target website, and the website asset identification method also includes: inputting the user behavior data and the key point data into the heat map model for feature analysis, obtaining the output result of the heat map model and generating the website asset heat map. When generating the website asset heat map, the user behavior data and the key point data are also input into the heat map model for feature analysis, so that the final website asset heat map can be associated with the user behavior data, such as the user behavior data can be associated with different colors in the website asset heat map, and the user's operation behavior can be more intuitively displayed in the website asset heat map, which is convenient for developers to understand the interaction process between users and websites. In addition, it is also easier to perform cluster analysis on user behavior data.

[0011] In an optional embodiment, the website asset heat map is automatically annotated using a preset recognition model to obtain the asset information of the key points, including: classifying the heat map using the recognition model, obtaining the type of each key point and the interaction characteristics between the user and each key point, forming the asset information, and annotating the asset information at the corresponding key points in the website asset heat map.

[0012] In a second aspect, an embodiment of the present application provides a model training method, comprising: obtaining a training data set, the training data set comprising multiple website data subsets, each of the website data subsets comprising a training website screenshot of the training website, user behavior data of the training website, a training heat map corresponding to the training website, and annotation information of the training website, wherein the annotation information comprises the website asset type and location data of the training website; performing model training on an initial recognition model according to the training data set to obtain a trained recognition model.

[0013] Compared with the related art, in the model training method provided in the embodiment of the present application, training website screenshots, user behavior data, training heat maps, and annotation information of multiple training websites are used as training data sets to train the initial recognition model, so that the trained recognition model can generate corresponding website asset heat maps based on the website screenshots and user behavior data of the target website, and mark the website asset type and location of the website assets in the website asset heat map.

[0014] In a third aspect, an embodiment of the present application provides a website asset identification device, comprising: a data acquisition module, the data acquisition module is used to collect website page data of a target website, the website page data includes visual element data of the target website; a key point extraction module, the key point extraction module is used to extract key point data based on the visual element data; a heat map generation module, the heat map generation module is used to generate a website asset heat map of the target website based on the key point data, the website asset heat map includes a number of key points corresponding to the key point data; a website asset identification module, the website asset identification module is used to automatically annotate the website asset heat map using a preset recognition model to obtain the asset information of the key points.

[0015] Compared with the related art, in the website asset identification device provided in the embodiment of the present application, after the data collection module collects the website page data of the target website, the key point extraction module generates key point data by extracting features from the visual element data in the website page data, and the heat map generation module generates a website asset heat map of the target website based on the key point data. Since the website asset heat map displays network assets more intuitively and obviously, the website asset identification module automatically annotates the website asset heat map using a preset recognition model, and the accuracy of the asset information of the key points obtained is also greatly improved; in addition, the more intuitive website asset heat map can also make the staff more efficient in monitoring, analyzing and optimizing network assets.

[0016] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the website asset identification method as described above.

[0017] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the aforementioned website asset identification method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can also be obtained based on these drawings.

[0019] Figure 1 A flowchart of a website asset identification method provided in Embodiment 1 of the present application;

[0020] Figure 2 Extracting visual element data from website page data in the website asset identification method provided in the first embodiment of the present application;

[0021] Figure 3 A schematic diagram of a process for extracting key point data from visual element data in a website asset identification method provided in Embodiment 1 of the present application;

[0022] Figure 4 A flowchart of the model training method provided in Example 2 of the present application;

[0023] Figure 5 This is a schematic diagram of the structure of the website asset identification device provided in the third embodiment of the present application;

[0024] Figure 6 This is a schematic diagram of the structure of the electronic device provided in Example 4 of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application as claimed, but is merely representative of selected embodiments of the present application.

[0027] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0028] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0029] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0030] The embodiments of the present application provide a website asset identification method, a model training method, a website asset identification device, an electronic device, and a computer-readable storage medium. Among them, the website asset identification device and the electronic device provided in the embodiments of the present application can realize the identification of website assets in a website by executing the website asset identification method provided in the embodiments of the present application. Below, the website asset identification method, the model training method, the website asset identification device, the electronic device, and the computer-readable storage medium will be specifically described respectively.

[0031] Embodiment 1

[0032] Please refer to Figure 1 , Embodiment 1 of the present application provides a website asset identification method, comprising the following steps.

[0033] Step S101: collecting website page data of a target website, where the website page data includes visual element data of the target website.

[0034] The target website is the website that needs to be identified with website assets, which can be a single website or multiple websites, and can be set according to different application scenarios.

[0035] In this step, the website page data of the target website can be collected by an automated tool, for example, by accessing the target website through a web crawler and crawling the target website according to a set crawling rule to obtain the website page data of the target website.

[0036] Optionally, in some embodiments of the present application, after obtaining the website page data, the website page data can also be screened to remove invalid data, erroneous data, duplicate data and other interfering data such as advertising scripts and tracking codes in the website page data, thereby improving the data quality of the website page data. Specifically, for example, the HTML (Hyper Text Markup Language) tags of the website page data can be screened by regular expressions to remove HTML tag errors or HTML non-standard data to achieve the effect of reducing interfering data. Or in other embodiments of the present application, for example, the website page data can be deduplicated by using a CSS (Cascading Style Sheets) selector to extract structured data from the website page data, and other methods can be used to reduce interfering data. The specific settings can be made according to actual needs.

[0037] Furthermore, in some embodiments of the present application, the format of the website page data can also be standardized, that is, the encoding format of the website page data from different sources and different encoding standards can be unified, so that all the website page data are converted into a unified data format, which facilitates the storage and management of the website page data, and also provides convenience for the subsequent analysis and processing of the website page data, thereby improving the efficiency of the subsequent analysis and processing of the website page data.

[0038] The visual element data is the data related to the display of the website assets in the website page, such as the color and outline of the website assets. Specifically, in addition to the visual element data, the collected website page data may also include other data such as user behavior data. Please refer to Figure 2 ,In this step, visual element data can be extracted from the website page data through the following steps.

[0039] Step S201: Perform edge detection on website page data to determine contour data of visual elements in the target website.

[0040] In step S201, for example, the contours of the visual elements in the target website may be identified by an edge detection algorithm to obtain the contour data of the visual elements.

[0041] Optionally, before using the edge detection algorithm to perform edge detection, the image data of the corresponding image and the text data of the corresponding text in the website page data can be binarized, and then the binarized website page data can be edge detected. The binarization process can significantly enhance the visual contrast between different visual elements in the target website, provide clearer boundary information for the edge detection algorithm, and improve the accuracy of the contour data obtained by edge detection.

[0042] Step S202: determining range data of the visual element according to the outline data, and dividing the target website into a plurality of visual element regions according to the range data.

[0043] In this step S202, for example, the range of the visual element can be determined based on the determined contour data by a region growing algorithm to obtain the range data of the visual element, and finally the target website is divided into multiple visual element regions according to the range data of each visual element.

[0044] It is understood that in the embodiments of the present application, the outline of a visual element refers to the outer boundary or edge of the visual element, which is the boundary between the visual element and the background or other visual elements; and the range of a visual element refers to the entire area of ​​the visual element, which includes not only the outline but also all the contents inside the visual element. Taking a button as an example of a visual element, the outline of the visual element is the border line of the button, and the range of the visual element is the entire button area including the border, background color and text content.

[0045] Optionally, in some embodiments of the present application, in the process of determining the range of a visual element based on already determined contour data through a region growing algorithm, small noise points and abnormal areas in the range can also be removed through morphological operations, thereby improving the accuracy of the range data of the visual element.

[0046] Step S203: identifying visual element features of each visual element region as visual element data.

[0047] In this step S203, data processing methods such as color analysis and / or texture analysis can be used to process data of each visual element area, and each visual element area obtained is input into a color analysis model and / or a texture analysis model for color analysis and / or texture analysis. The color analysis results and / or texture analysis results output by the color analysis model and / or the texture analysis model are used as visual element features of the visual element area, that is, all the color analysis results and / or texture analysis results obtained are used as visual element data.

[0048] Step S102: Generate key point data according to the visual element data.

[0049] In this step, the key point data is the feature data obtained by mapping the visual element data through the convolution function. For details, please refer to Figure 3 In this step, extracting key point data from visual element data may specifically include the following steps.

[0050] Step S301: Map the visual element data to a multi-dimensional space and perform a convolution operation to generate a number of feature maps.

[0051] Among them, the main body performing the convolution operation can be a convolution layer, that is, a neural network layer that can perform convolution operations, and the convolution operation on the visual element data can be performed by inputting the visual element data into the convolution layer. The convolution layer can map the visual element data to a multidimensional space and generate a new feature set through the convolution operation. The mapping relationship between the generated feature set and the visual element data is the generated feature map, so each feature map can correspond to one and / or several new feature data.

[0052] In different embodiments of the present application, the convolution layer can be pre-set or generated according to the visual element data when performing the convolution operation. In addition, in different embodiments of the present application, in the process of performing the convolution operation on the visual element data, the number of convolution layers can be one or more, and the specific number of convolution layers and the setting method of the convolution layers can be set according to the actual application scenario. For example, in one embodiment of the present application, it can be specifically to classify the visual element data, such as dividing the visual element data into two different types of color data and texture data, and then constructing a corresponding number of full-dimensional dynamic convolution (ODConv) layers according to the type classification results. For example, by adjusting the convolution kernel size of the standard full-dimensional dynamic convolution layer, a full-dimensional dynamic convolution layer corresponding to different data types is generated, and the corresponding convolution operation on the visual element data can be input into each of the several full-dimensional dynamic convolution layers for convolution operation. Setting multiple full-dimensional dynamic convolution layers corresponding to the data types for convolution operations can make each full-dimensional dynamic convolution layer focus on the corresponding data type for feature mapping. Multiple full-dimensional dynamic convolution layers can also map the visual element data to the multi-dimensional space, realize the comprehensive capture and mapping of the multi-scale features of the visual element data, perform more in-depth feature mapping and extraction, and improve the accuracy of the subsequently established website asset heat map, thereby improving the accuracy of the recognition results of the website asset heat map for website asset recognition.

[0053] Furthermore, the convolution kernels of multiple full-dimensional dynamic convolution layers can also dynamically adjust the size of the convolution kernels according to the importance of the data type to highlight important features and suppress noise. This method of dynamically adjusting the convolution kernels of the full-dimensional dynamic convolution layer can more accurately capture key point data, thereby improving recognition accuracy.

[0054] Step S302: selecting a target feature map from a plurality of feature maps as key point data according to the importance of each feature map.

[0055] In this step S302, the importance of each feature map can be reflected by different parameters of different feature maps. For example, in this embodiment, the selection of key point data can be specifically to use the principal component analysis (PCA) technology to use the feature map with the largest data variability as the key point data. It can be understood that the aforementioned use of the principal component analysis (PCA) technology to use the feature map with the largest data variability as the key point data is only an example of the specific method of selecting key point data in this embodiment. In some other embodiments of the present application, the selection of key point data can also be, for example, using a mathematical model such as a random forest model, ranking the importance of feature maps according to the contribution of feature maps to the prediction results, and selecting the feature map with the first importance ranking as the key point data; calculating the variance or information gain of each feature map, and selecting the feature with the highest variance or the highest information gain as the key point data; using L1 regularization in model training, automatically selecting key point data, etc. to select key point data. It can be understood that in the actual application process, the various key point data selection methods mentioned above can be used in combination so that the selected key point data has both statistical significance and meets the actual application requirements.

[0056] Step S103: Generate a website asset heat map of the target website according to the key point data, where the website asset heat map includes a number of key points corresponding to the key point data.

[0057] Specifically, in this step, the pre-trained heat map model can be used to generate the website asset heat map of the target website. That is, the key point data is input into the pre-trained heat map model for feature analysis, and the color type, color intensity and position data of each key point output by the heat map model are obtained; finally, the position of each key point in the website asset heat map is determined according to the position data corresponding to all key points, and each key point is colored according to the color type and color intensity corresponding to each key point, and finally the corresponding website asset heat map is generated.

[0058] Specifically, the heat map model includes a feature extraction layer and a target detection layer. For each key point, the feature extraction layer is used to generate a feature map of the corresponding scale based on the key point data and size information; the target detection layer is used to determine the position data, color type and color intensity of the key point based on several feature maps, and generate a website asset heat map.

[0059] Furthermore, in some embodiments of the present application, the number of feature extraction layers in the heat map model can be multiple layers. The multiple feature extraction layers can determine the size of the website asset corresponding to each key point based on the key point data, and generate multiple feature maps of different scales for website assets of each size. The multiple feature maps of different scales can be used to determine the color type and color intensity corresponding to each key point. That is, for website assets of multiple different sizes, multiple feature maps of different scales are generated for each website asset of different sizes through multiple feature extraction layers to determine the color type and color intensity, thereby improving the accuracy of the color type and color intensity of website assets of different sizes determined by the heat map model.

[0060] Among them, scale refers to the spatial resolution of the generated feature map, while size mainly refers to the size of the website assets. Specifically, feature maps of different scales and website assets of different sizes can be divided into the following categories:

[0061] Large-scale feature map: Maintaining a high spatial resolution, which may be close to the original input size, the large-scale feature map can capture more detailed information.

[0062] Medium-scale feature maps: obtained by moderate downsampling, capable of capturing medium-scale features and patterns.

[0063] Small-scale feature map: After multiple downsampling, it has a larger receptive field and can capture more global information.

[0064] Large-size assets: such as large banners, background images, video players, etc.

[0065] Medium-sized assets: such as article content areas, sidebars, navigation menus, etc.

[0066] Small-sized assets: such as buttons, icons, small text boxes, etc.

[0067] Furthermore, when training the multi-layer feature extraction layer and target detection layer included in the heat map model, the diversity and distribution of asset types can be identified by analyzing the existing data set to understand the representation of different assets in the web page. Furthermore, the size range of the website assets is analyzed to determine the thresholds of large-sized assets, medium-sized assets, and small-sized assets, so that the trained heat map model can be used to determine the size of the website assets corresponding to each key point based on the key point data.

[0068] In addition, during the model training of the heat map model, you can also apply the strategies in the CCDC (Continuous Change Detection and Classification) algorithm, such as geometric scaling, non-geometric scaling, and rotation, to transform the website assets in the existing dataset and create synthetic data to increase the diversity of the website assets in the existing dataset. Using an existing dataset with higher diversity to train the heat map model can improve the heat map model's ability to identify different website assets after training.

[0069] In addition, when designing a heat map model, a hierarchical rotation angle loss function can also be constructed in the heat map model, so that the heat map model can also identify the rotation angle of website assets. Correspondingly, the CCDC algorithm is applied to rotate the website assets in the existing data set to expand the existing data set. The existing data set can also be used to optimize the parameters of the hierarchical rotation angle loss function in the heat map model to improve the model training effect of the heat map model.

[0070] In this embodiment, the position data of each key point in the website asset heat map can be determined according to the key point data in the following manner. For example, a coordinate system can be established for the website asset heat map, and then the coordinate position of each key point in the target website is obtained from the key point data, and finally the coordinate position of the key point in the target website is mapped to the coordinate system corresponding to the website asset heat map by coordinate mapping to determine the position data of each key point.

[0071] It is understandable that the above is only an example of a specific method for determining the position data of each key point in the website asset heat map in this embodiment. In some other embodiments of the present application, other methods can also be used to determine the position data of each key point, and they can be selected or used in combination according to the actual application scenario in the specific application process. For example, in some other embodiments of the present application, the position data of each key point can also be determined by grid division, that is, the target website is divided into a number of grid units, and the corresponding relationship between each grid unit and the position of the key point is determined, that is, which one or more key points are included in each grid unit, and which grid unit each key point is located in, and then each grid unit of the target website is mapped to the corresponding position in the website asset heat map to determine the position data of each key point. Or in some other embodiments of the present application, the position data of each key point can also be determined based on the HTML structure and CSS attributes of the key point.

[0072] Finally, the heat map model in this embodiment includes multiple multi-layer feature extraction layers and target detection layers of different scales, which can perform data analysis of website assets of different sizes and scales corresponding to key points, and generate feature maps of different scales for website assets of each size, so that the heat map model can adapt to website assets of different sizes and reduce the omission of website assets; in addition, performing feature map analysis of website assets at different scales can also make the color type and color intensity corresponding to each key point finally output by the heat map model more accurate, thereby improving the accuracy of the subsequently established website asset heat map, and thereby improving the accuracy of the recognition results of website asset identification based on the website asset heat map.

[0073] Further, in some other embodiments of the present application, the website page data may also include user behavior data of the target website. User behavior data may include, for example, click behavior, mouse movement trajectory, dwell time, scroll depth, form filling, content selection, page jump, search query and other data generated by user operations. On this basis, when generating a website asset heat map, the user behavior data and key point data may also be input into the heat map model for feature analysis, the output result of the heat map model is obtained and the website asset heat map is generated. When generating a website asset heat map, the user behavior data and key point data are also input into the heat map model for feature analysis, so that the final website asset heat map can be associated with the user behavior data. For example, the frequency of user behavior can be represented in the website asset heat map by color gradients of different color types and different color intensities. For example, in the network asset heat map, red can be used to represent areas with high user operation frequency, while blue can represent areas with low user operation frequency; in the network asset heat map, higher color intensity can be used to represent areas with high user operation frequency, while lower color intensity can represent areas with low user operation frequency. Generating a website asset heat map of the target website based on user behavior data can more intuitively display the user's operation behavior in the website asset heat map, making it easier for developers to understand the interaction process between users and websites. In addition, it can also make it easier to perform cluster analysis on user behavior data.

[0074] Before generating the website asset heat map, the color type and color intensity of the key points are adjusted according to the user behavior data, so that the user behavior data is associated with the different colors in the website asset heat map. The user's operation behavior can be displayed more intuitively in the website asset heat map, which is convenient for developers to understand the interaction process between users and websites. In addition, it can also make it easier to perform cluster analysis on user behavior data.

[0075] Step S104: Automatically annotate the website asset heat map using a preset recognition model to obtain asset information of key points.

[0076] In this step, a deep learning model such as a convolutional neural network can be used as a recognition model to automatically annotate the website asset heat map. The recognition model can specifically identify the distribution characteristics, size characteristics, color characteristics and other characteristics of each key point on the website asset heat map to classify each key point on the website asset heat map, obtain the type of each key point as the asset information corresponding to each key point; and then annotate these asset information at the corresponding key points in the website asset heat map.

[0077] In addition, in the website asset heat map in which the color type and color intensity of key points are adjusted according to user behavior data as described above, the recognition model can also obtain the interaction characteristics between the user and each key point as the asset information of the website.

[0078] Specifically, the recognition model can perform cluster analysis on user behavior data based on the website asset heat map, cluster similar user behavior data together to form multiple cluster data sets, and then perform statistical analysis on each cluster data set. Clustering of similar user behavior data can be based on one or more of the following clustering dimensions, using a K-means clustering algorithm or predefined clustering rules. Clustering dimensions include: A: Behavior type: such as click, scroll, stay, etc.; B: Interactive elements: such as buttons, links, forms, etc.; C: Page area: such as the top, middle, and bottom of the page; D: Time period: such as morning and evening peaks, weekdays / weekends, etc.; E: User characteristics: such as new users / old users, geographic location, etc.; F: Behavior sequence: consider the order of user behavior.

[0079] Furthermore, after identifying the website assets, a classification algorithm can be applied to classify the identified website assets. For example, different types of website assets such as links, forms, buttons, etc. can be classified separately. Then, based on the classification results, the website assets of different categories corresponding to the key points in the website asset heat map are marked, and the asset type, location, size and other information corresponding to each website asset are attached to the key points corresponding to the website asset in the website asset heat map.

[0080] Optionally, after classifying the results of website asset identification of the website asset heat map, the identification results can also be stored in a preset website asset database. That is, the content marked at each key point in the website asset heat map, the size and type of the website assets and other data are summarized and organized into a format that is easy to understand and operate, and then added to the preset website asset database. Among them, the preset website asset database is a pre-set database for storing various website assets. When performing website asset identification later, the website assets stored in the preset website asset database can be directly used as samples for identification, which is convenient for subsequent website content management, user experience optimization, market analysis and business decision support operations. The website asset identification results of each target website are continuously classified and stored in the preset website asset database. Finally, a huge website asset database can be built to facilitate developers to perform website design optimization and user experience improvement.

[0081] After building the website asset database, website asset identification can also be achieved through the following steps:

[0082] S1: Feature extraction: extract features of the assets to be identified, using the same feature extraction method as that used in building the website asset database, such as the feature extraction method provided in the aforementioned steps S101 to S103.

[0083] S2: Feature comparison, comparing the extracted features with the website assets in the website asset database. The comparison methods may include but are not limited to: a. Euclidean distance calculation, b. cosine similarity calculation, c. machine learning model, d. deep learning model (such as convolutional neural network), etc.

[0084] S3: Similarity sorting: Based on the comparison results, the website assets in the website asset database are sorted by similarity, and several candidate website assets with the highest similarity are selected.

[0085] S4: Decision rule application, applying predefined decision rules to determine the final recognition result. This may include threshold screening, multi-feature fusion, contextual information consideration, etc.

[0086] S5: Result output, outputting the identification results, including information such as asset type, location, possible use, etc. of the website assets.

[0087] Compared with the related art, in the website asset identification method provided in the first embodiment of the present application, after collecting the website page data of the target website, key point data is generated by feature extraction of the visual element data in the website page data, and a website asset heat map of the target website is generated based on the key point data. Since the website asset heat map displays network assets more intuitively and obviously, the website asset heat map is automatically labeled using a preset recognition model, and the accuracy of the asset information of the key points obtained is also greatly improved; in addition, the more intuitive website asset heat map can also make the staff more efficient in monitoring, analyzing and optimizing network assets.

[0088] In addition, in some other embodiments of the present application, when the website page data changes, the website asset heat map can be updated according to the changed website page data, and a new round of website asset identification can be performed according to the updated website asset heat map. As the target website and the website page data corresponding to the target website change, the corresponding website asset heat map also changes accordingly, so that the website asset heat map and further website asset identification results can change dynamically with the target website, and the change process of the website assets in the target website can be more intuitively displayed.

[0089] Embodiment 2

[0090] Please refer to Figure 4 , Embodiment 2 of the present application provides a model training method, comprising the following steps:

[0091] Step S401: Obtain a training data set.

[0092] Step S402: Perform model training on the initial recognition model according to the training data set to obtain a trained recognition model.

[0093] The training data set may specifically include multiple website data subsets, each of which includes a training website screenshot of the training website, user behavior data of the training website, a training heat map corresponding to the training website, and annotation information of the training website, and the annotation information includes website asset type and location data of the training website.

[0094] On this basis, in step S402, the initial recognition model is trained according to the training data set, specifically by inputting the training website screenshots and user behavior data of the training website into the initial recognition model, obtaining the website asset heat map, website asset type and location data output by the initial recognition model, and converging the output results of the initial recognition model according to the training heat map corresponding to the training website and the annotation information of the training website, thereby obtaining a trained recognition model.

[0095] Furthermore, in this embodiment, strategies in the CCDC (Continuous Change Detection and Classification) algorithm, such as geometric scaling, non-geo-ratio scaling, and rotation, can be applied to transform the website assets in the existing data set and create synthetic data to increase the diversity of the website assets in the existing data set. Using a training data set with higher diversity to train the initial recognition model can improve the training effect and training efficiency of the recognition model.

[0096] Compared with the related art, in the model training method provided in the second embodiment, the training website screenshots, user behavior data, training heat maps, and annotation information of multiple training websites are used as training data sets to train the initial recognition model, so that the trained recognition model can generate a corresponding website asset heat map based on the website screenshots and user behavior data of the target website, and annotate the website asset type and location of the website assets in the website asset heat map.

[0097] Embodiment 3

[0098] Please refer to Figure 5 Embodiment 3 of the present application provides a website asset identification device, including: a data acquisition module 101, the data acquisition module 101 is used to collect website page data of a target website, the website page data includes visual element data of the target website; a key point extraction module 102, the key point extraction module 102 is used to extract key point data according to the visual element data; a heat map generation module 103, the heat map generation module 103 is used to generate a website asset heat map of the target website according to the key point data, the website asset heat map includes a number of key points corresponding to the key point data; a website asset identification module 104, the website asset identification module 104 is used to automatically annotate the website asset heat map using a preset recognition model to obtain asset information of the key points.

[0099] It can be understood that the website asset identification device provided in the third embodiment of the present application is an embodiment of the device corresponding to the aforementioned first embodiment, and steps S101 to S104 in the aforementioned first embodiment correspond to the data acquisition module 101, the key point extraction module 102, the heat map generation module 103, and the website asset identification module 104 in the third embodiment for execution.

[0100] Compared with the related art, in the website asset identification device provided in the third embodiment of the present application, after the data collection module 101 collects the website page data of the target website, the key point extraction module 102 generates key point data by extracting features from the visual element data in the website page data, and the heat map generation module 103 generates a website asset heat map of the target website based on the key point data. Since the website asset heat map displays the network assets more intuitively and obviously, the website asset identification module 104 automatically annotates the website asset heat map using a preset recognition model, and the accuracy of the asset information of the key points obtained is also greatly improved; in addition, the more intuitive website asset heat map can also make the staff more efficient in monitoring, analyzing and optimizing network assets.

[0101] Embodiment 4

[0102] Please refer to Figure 6 Embodiment 4 of the present application relates to an electronic device, comprising: at least one processor 201; and a memory 202 communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201, and the instructions are executed by the at least one processor 201 so that the at least one processor 201 can execute the methods in the above embodiments.

[0103] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0104] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0105] Embodiment 5

[0106] Embodiment 5 of the present application relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the above method embodiment is implemented.

[0107] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0108] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A website asset identification method, characterized in that: include: Collecting website page data of a target website, wherein the website page data includes visual element data of the target website, and the website page data also includes user behavior data of the target website; Generate key point data according to the website page data; Generating a website asset heat map of the target website according to the key point data, wherein the website asset heat map includes a number of key points corresponding to the key point data; Automatically annotating the website asset heat map using a preset recognition model to obtain asset information of the key points; Generating a website asset heat map of the target website according to the key point data includes: Inputting the user behavior data and the key point data into a pre-trained heat map model for feature analysis, obtaining an output result of the heat map model and generating the website asset heat map; The method of automatically labeling the website asset heat map using a preset recognition model to obtain asset information of the key points includes: The recognition model is used to classify the heat map, the type of each key point and the interaction characteristics between the user and each key point are obtained, the asset information is formed, and the asset information is marked at the corresponding key points in the website asset heat map.

2. The website asset identification method according to claim 1, characterized in that: The generating key point data according to the website page data includes: Mapping the visual element data to a multi-dimensional space and performing a convolution operation to generate a plurality of feature maps; A target feature map is selected from the plurality of feature maps as the key point data according to the importance of each of the feature maps.

3. The website asset identification method according to claim 1, characterized in that: The step of obtaining the output result of the heat map model and generating a website asset heat map of the target website includes: Obtaining an output result of the heat map model, wherein the output result includes a color type and a color intensity corresponding to each of the key points and position data of the key points; The website asset heat map is generated according to the location data, the color types, and the color intensities corresponding to all the key points.

4. The website asset identification method according to claim 3, characterized in that: The thermal map model includes a feature extraction layer and a target detection layer; For each of the key points, the feature extraction layer is used to generate a feature map of a corresponding scale according to the key point data and size information; The target detection layer is used to determine the position data, color type and color intensity of the key point according to the plurality of feature maps, and generate the website asset heat map.

5. A website asset identification device, characterized in that: include: A data collection module, the data collection module is used to collect website page data of a target website, the website page data includes visual element data of the target website, and the website page data also includes user behavior data of the target website; A key point extraction module, the key point extraction module is used to extract key point data according to the website page data; A heat map generation module, the heat map generation module is used to input the user behavior data and the key point data into a pre-trained heat map model for feature analysis, obtain the output result of the heat map model and generate a website asset heat map, the website asset heat map includes a number of key points corresponding to the key point data; A website asset identification module, the website asset identification module is used to automatically annotate the website asset heat map using a preset identification model to obtain asset information of the key points; The website asset identification module is used to automatically annotate the website asset heat map using a preset identification model to obtain the asset information of the key points, including: the website asset identification module is used to classify the heat map using the identification model, obtain the type of each key point and the interaction characteristics between the user and each key point, form the asset information, and annotate the asset information at the corresponding key points in the website asset heat map.

6. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the website asset identification method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: The computer program is executed by a processor to implement the website asset identification method described in any one of claims 1 to 4.

Images