Interface analysis method and device, storage medium and electronic equipment

Abstract

The invention discloses an interface analysis method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the steps that a development user interface and a design user interface are acquired; performing feature extraction on the development user interface by adopting a first branch network in a twin convolutional neural network model to obtain a first interface feature; performing feature extraction on the design user interface by adopting a second branch network in the twin convolutional neural network model to obtain a second interface feature; and performing similarity analysis according to the first interface feature and the second interface feature to obtain a reduction degree between the development user interface and the design user interface. According to the method, the analysis accuracy of the development user interface on the reduction degree of the design user interface can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to an interface analysis method, device, storage medium and electronic device. Background Art

[0002] User interface (UI) development is usually the process in which developers develop a developed user interface (development draft) based on the actual design user interface (i.e. design draft). The degree of restoration of the developed user interface to the designed user interface is very important.

[0003] At present, the degree of restoration of the developed user interface to the designed user interface is usually analyzed through subjective human eye recognition or contour extraction for alignment comparison. Due to the particularity of the user interface (UI), which is different from traditional image comparison analysis, the accuracy of the analysis of the degree of restoration of the developed user interface to the designed user interface is poor under the current method. Summary of the invention

[0004] The embodiment of the present application provides a solution that can effectively improve the accuracy of analyzing the degree of restoration of the developed user interface to the designed user interface.

[0005] The embodiments of the present application provide the following technical solutions:

[0006] According to one embodiment of the present application, an interface analysis method includes: obtaining a development user interface and a design user interface; using a first branch network in a twin convolutional neural network model to perform feature extraction on the development user interface to obtain a first interface feature; using a second branch network in the twin convolutional neural network model to perform feature extraction on the design user interface to obtain a second interface feature; performing a similarity analysis based on the first interface feature and the second interface feature to obtain a degree of restoration between the development user interface and the design user interface.

[0007] In some embodiments of the present application, the first branch network in the twin convolutional neural network model is used to extract features of the development user interface to obtain a first interface feature, including: using the first branch network to perform convolution and pooling operations on the development user interface to obtain a first same-domain feature; processing the first same-domain feature based on a coordinate attention mechanism to obtain a first coordinate attention feature; performing a full connection operation and activation processing on the first coordinate attention feature to obtain the first interface feature.

[0008] In some embodiments of the present application, the second branch network in the twin convolutional neural network model is used to extract features of the designed user interface to obtain a second interface feature, including: using the second branch network to perform convolution and pooling operations on the designed user interface to obtain a second same-domain feature; processing the second same-domain feature based on a coordinate attention mechanism to obtain a second coordinate attention feature; performing a full connection operation and activation processing on the second coordinate attention feature to obtain the second interface feature.

[0009] In some embodiments of the present application, the first same-domain feature is processed based on the coordinate attention mechanism to obtain the first coordinate attention feature, including: averaging the first same-domain feature in the horizontal direction and the vertical direction to obtain two first pooled features; fusing the two first pooled features to obtain a first fused feature; performing convolution compression on the first fused feature to obtain a first convolution compressed feature; encoding the spatial information in the vertical and horizontal directions of the first convolution compressed feature through batch normalization to obtain two first feature maps; and normalizing and weighting the two first feature maps to obtain the first coordinate attention feature.

[0010] In some embodiments of the present application, the second same-domain feature is processed based on the coordinate attention mechanism to obtain a second coordinate attention feature, including: averaging the second same-domain feature in the horizontal direction and the vertical direction to obtain two second pooled features; fusing the two second pooled features to obtain a second fused feature; performing convolution compression on the second fused feature to obtain a second convolution compressed feature; encoding the spatial information in the vertical and horizontal directions of the second convolution compressed feature through batch normalization to obtain two second feature maps; and normalizing and weighting the two second feature maps to obtain the second coordinate attention feature.

[0011] In some embodiments of the present application, the similarity analysis is performed based on the first interface feature and the second interface feature to obtain the degree of restoration between the development user interface and the design user interface, including: calculating the feature absolute value difference between the first interface feature and the second interface feature; performing a full connection operation on the feature absolute value difference to obtain a fully connected feature; and performing calculations based on the fully connected feature to obtain the similarity between the development user interface and the design user interface as the degree of restoration.

[0012] In some embodiments of the present application, the first coordinate attention feature is fully connected and activated to obtain the first interface feature, including: performing maximum pooling and probability distribution conversion on the first coordinate attention feature to obtain a first conversion feature; performing full connection and activation on the first conversion feature to obtain the first interface feature.

[0013] According to one embodiment of the present application, an interface analysis device includes: an acquisition module for acquiring a development user interface and a design user interface; a first extraction module for using a first branch network in a twin convolutional neural network model to perform feature extraction on the development user interface to obtain a first interface feature; a second extraction module for using a second branch network in the twin convolutional neural network model to perform feature extraction on the design user interface to obtain a second interface feature; and an analysis module for performing a similarity analysis based on the first interface feature and the second interface feature to obtain a degree of restoration between the development user interface and the design user interface.

[0014] In some embodiments of the present application, the first extraction module is used to: use the first branch network to perform convolution and pooling operations on the development user interface to obtain a first same-domain feature; process the first same-domain feature based on the coordinate attention mechanism to obtain a first coordinate attention feature; perform a full connection operation and activation processing on the first coordinate attention feature to obtain the first interface feature.

[0015] In some embodiments of the present application, the second extraction module is used to: use the second branch network to perform convolution and pooling operations on the designed user interface to obtain a second same-domain feature; process the second same-domain feature based on the coordinate attention mechanism to obtain a second coordinate attention feature; perform a full connection operation and activation processing on the second coordinate attention feature to obtain the second interface feature.

[0016] In some embodiments of the present application, the first extraction module is used to: perform average pooling on the first same-domain features in the horizontal and vertical directions respectively to obtain two first pooled features; perform feature fusion on the two first pooled features to obtain a first fused feature; perform convolution compression on the first fused feature to obtain a first convolution compressed feature; encode the spatial information in the vertical and horizontal directions of the first convolution compressed feature through batch normalization to obtain two first feature maps; and perform normalization and weighting on the two first feature maps to obtain the first coordinate attention feature.

[0017] In some embodiments of the present application, the second extraction module is used to: perform average pooling on the second same-domain features in the horizontal direction and the vertical direction respectively to obtain two second pooled features; perform feature fusion on the two second pooled features to obtain second fused features; perform convolution compression on the second fused features to obtain second convolution compressed features; encode the spatial information in the vertical direction and the horizontal direction of the second convolution compressed features through batch normalization to obtain two second feature maps; and perform normalization and weighting on the two second feature maps to obtain the second coordinate attention feature.

[0018] In some embodiments of the present application, the analysis module is used to: calculate the feature absolute value difference between the first interface feature and the second interface feature; perform a full connection operation on the feature absolute value difference to obtain a fully connected feature; and perform calculations based on the fully connected feature to obtain the similarity between the development user interface and the design user interface as the degree of restoration.

[0019] In some embodiments of the present application, the first extraction module is used to: perform maximum pooling processing and probability distribution conversion on the first coordinate attention feature to obtain a first conversion feature; perform full connection operation and activation processing on the first conversion feature to obtain the first interface feature.

[0020] According to another embodiment of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by a processor of a computer, the computer executes the method described in the embodiment of the present application.

[0021] According to another embodiment of the present application, an electronic device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiment of the present application.

[0022] According to another embodiment of the present application, a computer program product or a computer program includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0023] In an embodiment of the present application, a development user interface and a design user interface are obtained; a first branch network in a twin convolutional neural network model is used to perform feature extraction on the development user interface to obtain a first interface feature; a second branch network in the twin convolutional neural network model is used to perform feature extraction on the design user interface to obtain a second interface feature; and a similarity analysis is performed based on the first interface feature and the second interface feature to obtain a degree of restoration between the development user interface and the design user interface.

[0024] In this way, the first branch network in the twin convolutional neural network model is used to extract features of the development user interface to obtain the first interface features; the second branch network in the twin convolutional neural network model is used to extract features of the design user interface to obtain the second interface features; based on the similarity analysis of the first interface features and the second interface features, the similarity between the development user interface and the design user interface can be accurately obtained to accurately represent the degree of restoration of the development user interface to the design user interface, which can effectively improve the overall analysis accuracy of the degree of restoration of the development user interface to the design user interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flow chart of an interface analysis method according to an embodiment of the present application is shown.

[0027] Figure 2 A schematic diagram of a model structure according to an embodiment of the present application is shown.

[0028] Figure 3 A schematic diagram of a model structure according to another embodiment of the present application is shown.

[0029] Figure 4 A block diagram of an interface analysis device according to an embodiment of the present application is shown.

[0030] Figure 5 A block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are only used to explain the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are partial embodiments for implementing the present disclosure, rather than providing all embodiments for implementing the present disclosure. In the absence of conflict, the technical solutions recorded in the embodiments of the present disclosure can be implemented in any combination.

[0032] It should be noted that, in the embodiments of the present disclosure, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or apparatus. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements (such as steps in a method or units in a device, for example, a unit may be a portion of a circuit, a portion of a processor, a portion of a program or software, etc.) in the method or apparatus including the element.

[0033] For example, the interface analysis method provided in the embodiment of the present disclosure includes a series of steps, but the interface analysis method provided in the embodiment of the present disclosure is not limited to the recorded steps. Similarly, the interface analysis device provided in the embodiment of the present disclosure includes a series of units, but the device provided in the embodiment of the present disclosure is not limited to including the units explicitly recorded, and may also include units required to obtain relevant information or perform processing based on the information.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0035] It is understandable that in the specific implementation of this application, relevant data is involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0036] Figure 1 The flowchart of the interface analysis method according to an embodiment of the present application is schematically shown. The execution subject of the interface analysis method can be any device and / or server with processing capabilities, wherein the device is such as a computer, a mobile phone, a television, a smart watch, and a home appliance, etc., and the server is such as a cloud server or a physical server, etc.

[0037] like Figure 1As shown, the interface analysis method may include steps S110 to S140.

[0038] Step S110, obtaining a development user interface and a design user interface; step S120, using the first branch network in the twin convolutional neural network model to perform feature extraction on the development user interface to obtain first interface features; step S130, using the second branch network in the twin convolutional neural network model to perform feature extraction on the design user interface to obtain second interface features; step S140, performing similarity analysis based on the first interface features and the second interface features to obtain the degree of restoration between the development user interface and the design user interface.

[0039] The twin convolutional neural network model is a pre-trained model based on the twin convolutional neural network. The twin convolutional neural network model has two branches, including the first branch network and the second branch network. The first branch network and the second branch network use the same network structure and share network parameters to obtain image features in the same domain. Figure 2 The twin convolutional neural network model 200 includes a first branch network 210, a second branch network 220 and a similarity analysis network 230.

[0040] The development user interface and the design user interface can be input into the first branch network 210 and the second branch network 220 in the twin convolutional neural network model respectively. The first branch network in the twin convolutional neural network model is used to extract features of the development user interface to obtain first interface features; the second branch network in the twin convolutional neural network model is used to extract features of the design user interface to obtain second interface features.

[0041] Finally, by using the similarity analysis network 230, similarity analysis can be performed based on the first interface feature and the second interface feature to obtain the similarity between the first interface feature and the second interface feature, which can accurately represent the degree of restoration of the developed user interface to the designed user interface.

[0042] In this way, the first branch network in the twin convolutional neural network model is used to extract features of the development user interface to obtain the first interface features; the second branch network in the twin convolutional neural network model is used to extract features of the design user interface to obtain the second interface features; based on the similarity analysis of the first interface features and the second interface features, the similarity between the development user interface and the design user interface can be accurately obtained to accurately represent the degree of restoration of the development user interface to the design user interface, which can effectively improve the overall analysis accuracy of the degree of restoration of the development user interface to the design user interface.

[0043] Described below Figure 1When performing interface analysis under the embodiments, further optional specific embodiments are provided for each step performed.

[0044] In one embodiment, the use of the first branch network in the twin convolutional neural network model to extract features from the development user interface to obtain a first interface feature may include: using the first branch network to perform convolution and pooling operations on the development user interface to obtain a first same-domain feature; processing the first same-domain feature based on a coordinate attention mechanism to obtain a first coordinate attention feature; and performing a full connection operation and activation processing on the first coordinate attention feature to obtain the first interface feature.

[0045] The first branch network may include a convolution and pooling layer, a coordinate attention mechanism layer, and a fully connected and activated layer; the development user interface is subjected to convolution and pooling operations through the convolution and pooling layer to obtain a first same-domain feature; the first same-domain feature is processed based on the coordinate attention mechanism through the coordinate attention mechanism layer to obtain a first coordinate attention feature; the first coordinate attention feature is subjected to a fully connected operation and activation processing through the fully connected and activated layer to obtain a first interface feature. In particular, the weight of the feature-rich area can be increased through the coordinate attention mechanism layer, and the first interface feature extracted in the entire process can accurately represent the development user interface.

[0046] See also Figure 3 In a specific implementation, the convolution and pooling layers in the first branch network 210 may include two convolution layers 211, a maximum pooling layer 212, two convolution layers 213, a maximum pooling layer 214, three convolution layers 215, a maximum pooling layer 216, three convolution layers 217, a maximum pooling layer 218, and three convolution layers 219, which are arranged in sequence. Each convolution layer may include a sub-convolution layer and a sub-activation layer (such as a ReLU layer) for convolution and activation processing, respectively, and a maximum pooling layer (max pooling) is used for maximum pooling processing. The convolution and pooling layers may perform convolution operations and pooling operations on the development user interface in sequence through layers 211 to 219 to obtain the first same-domain feature.

[0047] Furthermore, the first branch network 210 may include a coordinate attention mechanism layer 2110, through which the first same-domain feature is processed based on the coordinate attention mechanism, the weight of the feature-rich area can be increased, and the first coordinate attention feature of the focused area can be obtained to be more prominent.

[0048] Furthermore, the fully connected and activated layer in the first branch network 210 may include a fully connected layer 2113, which may specifically include a sub-fully connected layer (fully connected) and an activation layer (Sigmoid). The first coordinate attention feature is fully connected and activated through the fully connected and activated layer to obtain the first interface feature.

[0049] Correspondingly, in one embodiment, the use of the second branch network in the twin convolutional neural network model to extract features from the design user interface to obtain a second interface feature may include: using the second branch network to perform convolution and pooling operations on the design user interface to obtain a second same-domain feature; processing the second same-domain feature based on a coordinate attention mechanism to obtain a second coordinate attention feature; performing a full connection operation and activation processing on the second coordinate attention feature to obtain the second interface feature.

[0050] The second branch network may include a convolution and pooling layer, a coordinate attention mechanism layer, and a fully connected and activated layer; the convolution and pooling layer performs convolution and pooling operations on the design user interface to obtain a second same-domain feature; the coordinate attention mechanism layer processes the second same-domain feature based on the coordinate attention mechanism to obtain a second coordinate attention feature; the fully connected and activated layer performs a fully connected operation and activation processing on the second coordinate attention feature to obtain a second interface feature. In particular, the coordinate attention mechanism layer can increase the weight of the feature-rich area, and the second interface feature extracted by the entire process can accurately represent the design user interface.

[0051] For further information, see Figure 3 In a specific implementation, the convolution and pooling layers in the second branch network 220 may include two convolution layers 221, a maximum pooling layer 222, two convolution layers 223, a maximum pooling layer 224, three convolution layers 225, a maximum pooling layer 226, three convolution layers 227, a maximum pooling layer 228, and three convolution layers 229, which are arranged in sequence. Each convolution layer may include a sub-convolution layer and a sub-activation layer (such as a ReLU layer) for convolution and activation processing, respectively, and a maximum pooling layer (max pooling) is used for maximum pooling processing. The convolution and pooling layers may perform convolution operations and pooling operations on the design user interface through layers 221 to 229 in sequence to obtain a second same-domain feature.

[0052] Furthermore, the second branch network 220 may include a coordinate attention mechanism layer 2210, through which the second same-domain features are processed based on the coordinate attention mechanism, the weight of the feature-rich area can be increased, and a more significant second coordinate attention feature of the focused area can be obtained.

[0053] Furthermore, the fully connected and activated layer in the second branch network 220 may include a fully connected layer 2213, which may specifically include a sub-fully connected layer (fully connected) and an activation layer (Sigmoid). The second coordinate attention feature is fully connected and activated through the fully connected and activated layer to obtain the second interface feature.

[0054] In one embodiment, the processing of the first same-domain feature based on the coordinate attention mechanism to obtain the first coordinate attention feature may include: averaging the first same-domain feature in the horizontal direction and the vertical direction to obtain two first pooled features; fusing the two first pooled features to obtain a first fused feature; performing convolution compression on the first fused feature to obtain a first convolution compressed feature; encoding the spatial information in the vertical and horizontal directions of the first convolution compressed feature through batch normalization to obtain two first feature maps; and normalizing and weighting the two first feature maps to obtain the first coordinate attention feature.

[0055] The first same-domain feature is average-pooled in the horizontal and vertical directions respectively to obtain the first pooled feature A1 in the horizontal direction and the first pooled feature A2 in the vertical direction; then, the first pooled feature A1 in the horizontal direction and the first pooled feature A2 in the vertical direction can be fused in the spatial dimension (for example, A1 and A2 are connected in series or superimposed, etc.) to obtain the first fused feature; the first fused feature is convolutionally compressed to compress the channel to obtain the first convolutionally compressed feature; then, the first convolutionally compressed feature is batch-normalized to encode the spatial information in the vertical and horizontal directions respectively to obtain the first feature map B1 in the vertical direction and the first feature map B2 in the horizontal direction; finally, the first feature map B1 in the vertical direction and the first feature map B2 in the horizontal direction are normalized and weighted, so as to accurately increase the weight of the feature-rich area to obtain the first coordinate attention feature.

[0056] In one embodiment, the second same-domain feature is processed based on a coordinate attention mechanism to obtain a second coordinate attention feature, including: averaging the second same-domain feature in the horizontal direction and the vertical direction to obtain two second pooled features; fusing the two second pooled features to obtain a second fused feature; performing convolution compression on the second fused feature to obtain a second convolution compressed feature; encoding the spatial information in the vertical and horizontal directions of the second convolution compressed feature through batch normalization to obtain two second feature maps; and normalizing and weighting the two second feature maps to obtain the second coordinate attention feature.

[0057] The second same-domain features are average-pooled in the horizontal and vertical directions respectively to obtain the second pooled features C1 in the horizontal direction and the second pooled features C2 in the vertical direction; then, the second pooled features C1 in the horizontal direction and the second pooled features C2 in the vertical direction can be fused in the spatial dimension (for example, C1 and C2 are connected in series or superimposed, etc.) to obtain the second fused features; the second fused features are convolutionally compressed to compress the channels to obtain the second convolutionally compressed features; then, the second convolutionally compressed features are batch-normalized to encode the spatial information in the vertical and horizontal directions respectively to obtain the second feature map D1 in the vertical direction and the second feature map D2 in the horizontal direction; finally, the second feature map D1 in the vertical direction and the second feature map D2 in the horizontal direction are normalized and weighted, so as to accurately increase the weights of the feature-rich areas to obtain the second coordinate attention features.

[0058] Furthermore, in one embodiment, performing a full connection operation and activation processing on the first coordinate attention feature to obtain the first interface feature may include: performing a maximum pooling operation and a probability distribution conversion on the first coordinate attention feature to obtain a first conversion feature; performing a full connection operation and activation processing on the first conversion feature to obtain the first interface feature.

[0059] See also Figure 3 The fully connected and activated layers in the first branch network 210 may also include a maximum pooling layer 2111, three probability distribution layers (softmax) 2112, and finally connected to a fully connected layer 2113. The fully connected layer 2113 may specifically include a sub-fully connected layer (fully connected) and an activation layer (Sigmoid).

[0060] Through the maximum pooling layer 2111 and the three probability distribution layers (softmax) 2112, the first coordinate attention feature can be subjected to maximum pooling processing and probability distribution conversion in turn to obtain the first conversion feature; then, the first coordinate attention feature is subjected to full connection operation and activation processing through the fully connected layer 2113 to obtain the first interface feature.

[0061] In one embodiment, the second coordinate attention feature is fully connected and activated to obtain the second interface feature, including: performing maximum pooling and probability distribution conversion on the second coordinate attention feature to obtain the second conversion feature; performing full connection and activation on the second conversion feature to obtain the second interface feature.

[0062] See also Figure 3 The fully connected and activated layers in the second branch network 220 may also include a maximum pooling layer 2211, three probability distribution layers (softmax) 2212, and finally connected to a fully connected layer 2213. The fully connected layer 2213 may specifically include a sub-fully connected layer (fully connected) and an activation layer (Sigmoid).

[0063] Through the maximum pooling layer 2211 and the three probability distribution layers (softmax) 2212, the second coordinate attention feature can be subjected to maximum pooling processing and probability distribution conversion in turn to obtain the second conversion feature; then, through the fully connected layer 2213, the second coordinate attention feature is subjected to full connection operation and activation processing to obtain the second interface feature.

[0064] Furthermore, in one embodiment, the similarity analysis is performed based on the first interface feature and the second interface feature to obtain the degree of restoration between the development user interface and the design user interface, including: calculating the feature absolute value difference between the first interface feature and the second interface feature; performing a full connection operation on the feature absolute value difference to obtain a fully connected feature; and performing calculations based on the fully connected feature to obtain the similarity between the development user interface and the design user interface as the degree of restoration.

[0065] See also Figure 3 , the first interface feature h(x1) can be extracted through the first branch network 210, and the second interface feature h(x2) can be extracted through the second branch network 220. Then, the feature absolute value difference between the first interface feature h(x1) and the second interface feature h(x2) can be calculated in the similarity analysis network 230 according to the formula |h(x1)-h(x2)|. Then, the feature absolute value difference |h(x1)-h(x2)| is fully connected through the fully connected layer 232 in the similarity analysis network 230 to obtain the fully connected feature and the activation function (Sigmoid) is used to calculate based on the fully connected feature to obtain the similarity h(x1-x2) between the development user interface and the design user interface, wherein the fully connected layer 232 may specifically include a sub-fully connected layer (fully connected) and an activation layer (Sigmoid).

[0066] Furthermore, in an embodiment of the present application, when the twin convolutional neural network model is pre-trained, the first branch network and the second branch network use the VGGNet network as the backbone and reuse the training parameters of the VGGNet network on the ImageNet dataset using transfer learning, which can further ensure that the model has stronger feature extraction capabilities.

[0067] In order to facilitate better implementation of the interface analysis method provided in the embodiment of the present application, the embodiment of the present application also provides an interface analysis device based on the above interface analysis method. The meanings of the terms are the same as those in the above interface analysis method, and the specific implementation details can refer to the description in the method embodiment. Figure 4 A block diagram of an interface analysis device according to an embodiment of the present application is shown.

[0068] like Figure 4 As shown, the interface analysis device 400 may include: an acquisition module 410 can be used to acquire a development user interface and a design user interface; a first extraction module 420 can be used to use a first branch network in a twin convolutional neural network model to perform feature extraction on the development user interface to obtain a first interface feature; a second extraction module 430 can be used to use a second branch network in the twin convolutional neural network model to perform feature extraction on the design user interface to obtain a second interface feature; an analysis module 440 can be used to perform a similarity analysis based on the first interface feature and the second interface feature to obtain the degree of restoration between the development user interface and the design user interface.

[0069] In some embodiments of the present application, the first extraction module is used to: use the first branch network to perform convolution and pooling operations on the development user interface to obtain a first same-domain feature; process the first same-domain feature based on the coordinate attention mechanism to obtain a first coordinate attention feature; perform a full connection operation and activation processing on the first coordinate attention feature to obtain the first interface feature.

[0070] In some embodiments of the present application, the second extraction module is used to: use the second branch network to perform convolution and pooling operations on the designed user interface to obtain a second same-domain feature; process the second same-domain feature based on the coordinate attention mechanism to obtain a second coordinate attention feature; perform a full connection operation and activation processing on the second coordinate attention feature to obtain the second interface feature.

[0071] In some embodiments of the present application, the first extraction module is used to: perform average pooling on the first same-domain features in the horizontal and vertical directions respectively to obtain two first pooled features; perform feature fusion on the two first pooled features to obtain a first fused feature; perform convolution compression on the first fused feature to obtain a first convolution compressed feature; encode the spatial information in the vertical and horizontal directions of the first convolution compressed feature through batch normalization to obtain two first feature maps; and perform normalization and weighting on the two first feature maps to obtain the first coordinate attention feature.

[0072] In some embodiments of the present application, the second extraction module is used to: perform average pooling on the second same-domain features in the horizontal direction and the vertical direction respectively to obtain two second pooled features; perform feature fusion on the two second pooled features to obtain second fused features; perform convolution compression on the second fused features to obtain second convolution compressed features; encode the spatial information in the vertical direction and the horizontal direction of the second convolution compressed features through batch normalization to obtain two second feature maps; and perform normalization and weighting on the two second feature maps to obtain the second coordinate attention feature.

[0073] In some embodiments of the present application, the analysis module is used to: calculate the feature absolute value difference between the first interface feature and the second interface feature; perform a full connection operation on the feature absolute value difference to obtain a fully connected feature; and perform calculations based on the fully connected feature to obtain the similarity between the development user interface and the design user interface as the degree of restoration.

[0074] In some embodiments of the present application, the first extraction module is used to: perform maximum pooling processing and probability distribution conversion on the first coordinate attention feature to obtain a first conversion feature; perform full connection operation and activation processing on the first conversion feature to obtain the first interface feature.

[0075] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0076] In addition, the present application also provides an electronic device, such as Figure 5 As shown, Figure 5 A block diagram of an electronic device according to an embodiment of the present application is shown, specifically:

[0077] The electronic device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0078] in:

[0079] The processor 501 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 502 and calling data stored in the memory 502, it executes various functions of the computer device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0080] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0081] The electronic device also includes a power supply 503 for supplying power to each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 503 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0082] The electronic device may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0083] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 501 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 502 according to the following instructions, and the processor 501 will run the computer program stored in the memory 502, thereby realizing various functions in the aforementioned embodiments of the present application, such as the processor 501 may perform the following steps:

[0084] Acquire a development user interface and a design user interface; use the first branch network in the twin convolutional neural network model to extract features of the development user interface to obtain first interface features; use the second branch network in the twin convolutional neural network model to extract features of the design user interface to obtain second interface features; perform similarity analysis based on the first interface features and the second interface features to obtain the degree of restoration between the development user interface and the design user interface.

[0085] In some embodiments of the present application, the first branch network in the twin convolutional neural network model is used to extract features of the development user interface to obtain a first interface feature, including: using the first branch network to perform convolution and pooling operations on the development user interface to obtain a first same-domain feature; processing the first same-domain feature based on a coordinate attention mechanism to obtain a first coordinate attention feature; performing a full connection operation and activation processing on the first coordinate attention feature to obtain the first interface feature.

[0086] In some embodiments of the present application, the second branch network in the twin convolutional neural network model is used to extract features of the designed user interface to obtain a second interface feature, including: using the second branch network to perform convolution and pooling operations on the designed user interface to obtain a second same-domain feature; processing the second same-domain feature based on a coordinate attention mechanism to obtain a second coordinate attention feature; performing a full connection operation and activation processing on the second coordinate attention feature to obtain the second interface feature.

[0087] In some embodiments of the present application, the first same-domain feature is processed based on the coordinate attention mechanism to obtain the first coordinate attention feature, including: averaging the first same-domain feature in the horizontal direction and the vertical direction to obtain two first pooled features; fusing the two first pooled features to obtain a first fused feature; performing convolution compression on the first fused feature to obtain a first convolution compressed feature; encoding the spatial information in the vertical and horizontal directions of the first convolution compressed feature through batch normalization to obtain two first feature maps; and normalizing and weighting the two first feature maps to obtain the first coordinate attention feature.

[0088] In some embodiments of the present application, the second same-domain feature is processed based on the coordinate attention mechanism to obtain a second coordinate attention feature, including: averaging the second same-domain feature in the horizontal direction and the vertical direction to obtain two second pooled features; fusing the two second pooled features to obtain a second fused feature; performing convolution compression on the second fused feature to obtain a second convolution compressed feature; encoding the spatial information in the vertical and horizontal directions of the second convolution compressed feature through batch normalization to obtain two second feature maps; and normalizing and weighting the two second feature maps to obtain the second coordinate attention feature.

[0089] In some embodiments of the present application, the similarity analysis is performed based on the first interface feature and the second interface feature to obtain the degree of restoration between the development user interface and the design user interface, including: calculating the feature absolute value difference between the first interface feature and the second interface feature; performing a full connection operation on the feature absolute value difference to obtain a fully connected feature; and performing calculations based on the fully connected feature to obtain the similarity between the development user interface and the design user interface as the degree of restoration.

[0090] In some embodiments of the present application, the first coordinate attention feature is fully connected and activated to obtain the first interface feature, including: performing maximum pooling and probability distribution conversion on the first coordinate attention feature to obtain a first conversion feature; performing full connection and activation on the first conversion feature to obtain the first interface feature.

[0091] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0092] To this end, an embodiment of the present application further provides a storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any method provided in the embodiment of the present application.

[0093] The storage medium may be a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0094] Since the computer program stored in the storage medium can execute the steps in any method provided in the embodiments of the present application, the beneficial effects that can be achieved by the method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0095] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0096] It should be understood that the present application is not limited to the embodiments that have been described above and shown in the accompanying drawings, but various modifications and changes may be made without departing from the scope thereof.

Claims

1. An interface analysis method, characterized in that: include: Get development user interface and design user interface; Using the first branch network in the twin convolutional neural network model to extract features of the development user interface to obtain first interface features; Using the second branch network in the twin convolutional neural network model to extract features of the designed user interface to obtain second interface features; A similarity analysis is performed based on the first interface feature and the second interface feature to obtain a degree of restoration between the development user interface and the design user interface.

2. The method according to claim 1, characterized in that The first branch network in the twin convolutional neural network model is used to extract features of the development user interface to obtain first interface features, including: Using the first branch network, performing convolution operations and pooling operations on the development user interface to obtain a first same-domain feature; Processing the first same-domain feature based on the coordinate attention mechanism to obtain a first coordinate attention feature; The first coordinate attention feature is fully connected and activated to obtain the first interface feature.

3. The method according to claim 2, characterized in that The method of using the second branch network in the twin convolutional neural network model to extract features of the designed user interface to obtain second interface features includes: Using the second branch network, performing convolution and pooling operations on the design user interface to obtain a second same-domain feature; Processing the second same-domain feature based on the coordinate attention mechanism to obtain a second coordinate attention feature; The second coordinate attention feature is fully connected and activated to obtain the second interface feature.

4. The method according to claim 2, characterized in that: The processing of the first same-domain feature based on the coordinate attention mechanism to obtain the first coordinate attention feature includes: Performing average pooling on the first same-domain features in the horizontal direction and the vertical direction respectively to obtain two first pooled features; Perform feature fusion on the two first pooled features to obtain a first fused feature; Performing convolution compression processing on the first fused feature to obtain a first convolution compression feature; Encode the spatial information in the vertical direction and the horizontal direction respectively by batch normalization of the first convolutional compression feature to obtain two first feature maps; The two first feature maps are normalized and weighted to obtain the first coordinate attention feature.

5. The method according to claim 3, characterized in that: The processing of the second same-domain feature based on the coordinate attention mechanism to obtain the second coordinate attention feature includes: Performing average pooling on the second same-domain features in the horizontal direction and the vertical direction respectively to obtain two second pooled features; Perform feature fusion on the two second pooled features to obtain a second fused feature; Performing convolution compression processing on the second fused feature to obtain a second convolution compression feature; The second convolutional compression feature is used to encode the spatial information in the vertical direction and the horizontal direction respectively through batch normalization to obtain two second feature maps; The two second feature maps are normalized and weighted to obtain the second coordinate attention feature.

6. The method according to claim 1, characterized in that The performing similarity analysis based on the first interface feature and the second interface feature to obtain the restoration degree between the development user interface and the design user interface includes: Calculating a feature absolute value difference between the first interface feature and the second interface feature; Performing a full connection operation on the absolute value difference of the feature to obtain a fully connected feature; A calculation is performed based on the fully connected features to obtain a similarity between the development user interface and the design user interface as the restoration degree.

7. The method according to claim 2, characterized in that The performing a full connection operation and an activation process on the first coordinate attention feature to obtain the first interface feature includes: Performing maximum pooling processing and probability distribution conversion on the first coordinate attention feature to obtain a first conversion feature; The first conversion feature is fully connected and activated to obtain the first interface feature.

8. An interface analysis device, characterized in that: include: An acquisition module is used to acquire a development user interface and a design user interface; A first extraction module, used for extracting features of the development user interface by using a first branch network in a twin convolutional neural network model to obtain first interface features; A second extraction module is used to extract features of the designed user interface using a second branch network in the twin convolutional neural network model to obtain second interface features; The analysis module is used to perform similarity analysis based on the first interface feature and the second interface feature to obtain the degree of restoration between the development user interface and the design user interface.

9. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: a memory storing a computer program; A processor reads a computer program stored in a memory to execute the method according to any one of claims 1 to 7.

Images