An interface determination method and device, a storage medium and an electronic device

Abstract

This application discloses an interface determination method, device, storage medium, and electronic device, relating to the technical fields of image processing, artificial intelligence, and mobile application interface adaptation. It simulates the user's natural visual scanning process in a right-to-left reading mode by constructing a visual attention energy field. During this process, a directional cognitive flow model is used to process the directional cognitive flow, obtaining the reading path and operation decision path formed under the current interface. Based on the modeling of a preset interface structure model, spatial relationships, and potential paths, the current interface is matched and determined to be consistent with preset interface habits. This allows for the determination of the actual user experience of the current interface in real-world use. Furthermore, this solution does not rely on testers with language and cultural backgrounds for manual experience and repeated verification during the matching and determination of preset language interface habits, thereby improving the efficiency of verifying Arabic interface adaptation.

Description

Technical Field

[0001] This application relates to the technical fields of image processing, artificial intelligence and mobile application interface adaptation, and more specifically, to an interface determination method, apparatus, storage medium and electronic device. Background Technology

[0002] With the rapid proliferation of mobile applications in the Middle East and North Africa (MENA) region, an increasing number of applications (Apps) need to support Arabic interfaces.

[0003] However, Arabic is not only a written language, but also has a series of interface usage norms that are completely different from those of Latin-based languages, such as right-to-left (RTL) layout, specific visual alignment, icon direction semantics, and culturally sensitive interaction logic.

[0004] Traditional Arabic interface adaptation methods typically rely on implementation-level features such as "whether RTL is enabled" and "whether left and right mirroring is performed" for detection. Since the detection focuses on the implementation layer, it fails to assess the actual user experience during real-world use. Furthermore, traditional Arabic interface adaptation methods depend on testers with language and cultural backgrounds conducting manual testing and repeated verification, reducing the efficiency of verifying Arabic interface adaptation.

[0005] Therefore, how to determine the actual user experience of the interface in real-world use, and how to improve the efficiency of verifying Arabic interface adaptation, are the problems that this application urgently needs to solve. Summary of the Invention

[0006] In view of this, this application discloses an interface determination method, apparatus, storage medium and electronic device, which aims to determine the actual user experience of the interface in actual use and improve the efficiency of verifying Arabic interface adaptation.

[0007] To achieve the above objectives, the disclosed technical solution is as follows:

[0008] The first aspect of this application discloses an interface determination method, the method comprising:

[0009] Get the interface image of the current interface after perceptual consistency processing generated when the mobile application is running in a preset language environment;

[0010] The visual region corresponding to the interface image is generated based on a preset visual region generation method;

[0011] Construct the visual attention energy field corresponding to the visual region;

[0012] Based on the visual attention energy field, directional cognitive flow processing is performed through a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface;

[0013] Based on the preset language interface structure model, the reading path, and the operation decision path, the current interface is matched and judged for consistency with preset language interface habits.

[0014] A second aspect of this application discloses an interface determination device, the device comprising:

[0015] The acquisition unit is used to acquire the interface image of the current interface after perceptual consistency processing generated by the mobile application when it runs in a preset language environment.

[0016] The generation unit is used to generate the visual region corresponding to the interface image based on a preset visual region generation method.

[0017] The first construction unit is used to construct the visual attention energy field corresponding to the visual region;

[0018] The first processing unit is used to perform directional cognitive flow processing based on the visual attention energy field through a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface.

[0019] The matching and determination unit is used to perform consistency matching and determination of the current interface with the preset language interface habits based on the preset language interface structure model, the reading path and the operation decision path.

[0020] A third aspect of this application discloses a storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium is located is controlled to perform the interface determination method as described in any one of the first aspects.

[0021] The fourth aspect of this application discloses an electronic device, including a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the interface determination method as described in any one of the first aspects.

[0022] As can be seen from the above technical solutions, this application discloses an interface determination method, device, storage medium, and electronic device. It acquires the interface image of the current interface generated when a mobile application runs in a preset language environment, generates the visual region corresponding to the interface image based on a preset visual region generation method, constructs a visual attention energy field corresponding to the visual region through visual features, and performs directional cognitive flow processing through a pre-constructed directional cognitive flow model based on the visual attention energy field to obtain the reading path and operation decision path formed under the current interface. According to the preset language interface structure model, reading path, and operation decision path, the current interface is matched and determined to be consistent with the preset language interface habits.

[0023] This solution starts by analyzing the underlying pixel distribution, spatial relationships, and visual saliency changes in screenshots of the current real-world interface. It constructs a visual attention energy field by analyzing the visual characteristics of different areas within the current interface in terms of contrast, area proportion, spatial location, and neighborhood complexity. This field simulates the natural visual scanning process of Arabic users in a right-to-left reading mode. During this natural visual scanning process, a directional cognitive flow model is used to process the reading path and operational decision path formed on the current interface. Based on the modeling of the Arabic interface structure, spatial relationships, and potential paths, the solution performs consistency matching and judgment for Arabic interface habits, thereby determining the actual user experience of the current interface in real-world use. Furthermore, this solution does not rely on testers with language and cultural backgrounds for manual experience and repeated verification during the consistency matching and judgment process for Arabic interface habits, thus improving the efficiency of verifying Arabic interface adaptation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating an interface determination method disclosed in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the structure of an interface determination device disclosed in an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of the structure of the electronic device disclosed in the embodiments of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0030] As the background technology indicates, traditional Arabic interface adaptation methods typically detect implementation-level features. Since the detection focuses on the implementation layer, it fails to assess the actual user experience during real-world use. Furthermore, traditional Arabic interface adaptation methods rely on testers with language and cultural backgrounds to manually test and repeatedly verify the interface, reducing the efficiency of verifying Arabic interface adaptation.

[0031] To address the aforementioned issues, this application discloses an interface determination method, apparatus, storage medium, and electronic device. This solution starts by analyzing the underlying pixel distribution, spatial relationships, and visual saliency changes in a screenshot of a real interface. It constructs a visual attention energy field by analyzing the visual characteristics of different regions within the current interface in terms of contrast, area ratio, spatial location, and neighborhood complexity. This visual attention energy field simulates, for example, the natural visual scanning process of an Arabic-speaking user in a right-to-left reading mode. During this natural visual scanning process, a directional cognitive flow model is used to process the reading path and operation decision path formed under the current interface. Based on a preset language interface structure model, spatial relationships, and potential path modeling, the consistency of the current interface with Arabic interface habits is matched and determined. This allows for the assessment of the actual user experience of the current interface during actual use. Furthermore, this solution does not rely on human experience and repeated verification by testers with language and cultural backgrounds during the consistency matching and determination of the current interface with Arabic interface habits, thereby improving the efficiency of verifying Arabic interface adaptation. Specific implementation methods are described in detail in the following embodiments.

[0032] It should be noted that the interface determination method, apparatus, storage medium, and electronic device provided in this application can be used in the fields of image processing, artificial intelligence, and mobile application interface adaptation. The above are merely examples and do not limit the application areas of the interface determination method, apparatus, storage medium, and electronic device provided in this application. Furthermore, this application, through automatic identification and structured analysis of image elements, layout structures, and interactive controls in mobile application interfaces, combined with a preset language interface structure model, such as an Arabic interface structure model, achieves automated detection of whether the current interface conforms to the usage habits of Arabic users. It is applicable to scenarios such as Arabic localization of mobile applications and usability assessment.

[0033] refer to Figure 1 The diagram shown is a flowchart of an interface determination method disclosed in an embodiment of this application. The interface determination method mainly includes the following steps:

[0034] S101: Obtain the perceptually consistent interface image of the current interface generated when the mobile application is running in the preset language environment.

[0035] The system acquires screenshots of the interface generated by the mobile application when it runs in a preset language environment, and performs perceptual consistency processing on the interface image. The perceptual consistency processing includes, but is not limited to, resolution normalization, brightness dynamic range compression, and color channel remapping processing.

[0036] The preset language environment includes, but is not limited to, an Arabic language environment. This solution preferably uses an Arabic language environment as the preset language environment.

[0037] The purpose of performing perceptual consistency processing on interface images is to suppress the impact of differences in device resolution and screen brightness on visual saliency, so that interfaces generated on different terminals have consistency in human visual perception, thus providing a stable foundation for subsequent visual cognition modeling.

[0038] Specifically, based on the logarithmic perception characteristic of human eyes to changes in brightness, a nonlinear perceptual mapping function is constructed, and the original interface image is then subjected to perceptual consistency processing according to this function. The perceptual consistency processing process is as follows:

[0039] Let the original interface image (such as a screenshot) generated when the mobile application runs in an Arabic language environment be represented as a two-dimensional pixel matrix.

[0040] Among them, the two-dimensional pixel matrix is ​​obtained through express, This indicates the coordinate position of a pixel in the interface space.

[0041] To eliminate the differences in brightness, contrast and display characteristics between different terminals, this application constructs a nonlinear perception mapping function based on the logarithmic perception characteristics of human eyes to brightness changes. The calculation formula of the nonlinear perception mapping function is shown in formula (1).

[0042] (1);

[0043] in, This is the sensing adjustment coefficient, used to control the intensity of brightness compression. () represents the interface image after perceptual consistency.

[0044] The nonlinear perceptual mapping function smooths out the perceptual gain in bright areas while enhancing the detail resolution in low-brightness areas, thus ensuring that the results of subsequent visual saliency analysis remain consistent across different devices.

[0045] S102: Generate the visual region corresponding to the interface image based on the preset visual region generation method.

[0046] In S102, the spatial density gradient function of each pixel in the interface image within its local neighborhood is calculated. By analyzing the spatial continuity of the spatial density gradient function and combining it with gradient direction consistency constraints, a set of pixels in the visual region with continuous and consistent gradient changes is determined. This set of pixels with continuous and consistent gradient changes is then aggregated into the same visual region. The visual region is determined through... express.

[0047] The preset visual region generation method is based on pixel density gradient. This method generates visual regions by applying pixel density gradients to the perceptually consistent interface image. The spatial density gradient function of a pixel in its local neighborhood is calculated. By analyzing the spatial continuity of the spatial density gradient function and combining it with the gradient direction consistency constraint, the set of pixels with continuous gradient changes and consistent directions is aggregated into the same visual region. The calculation formula of the spatial density gradient function is shown in formula (2).

[0048] (2);

[0049] in, It is the spatial density gradient function; To perceive the consistent interface image; These are partial derivatives; Let be the partial derivative with respect to x; Let be the partial derivative with respect to y.

[0050] It should be noted that the visual area represents the interface unit that the user can perceive as a whole, rather than relying on any control boundaries or user interface (UI) structure description, thus avoiding analytical biases caused by different implementation methods. Different implementation methods include, but are not limited to, differences in interface structure caused by different terminal platforms (such as iOS and Android), different interface development frameworks (such as native development or cross-platform frameworks), different layout implementation mechanisms (such as constraint layout, fluid layout, or absolute layout), different resolution adaptation strategies, and different rendering mechanisms. This invention analyzes the final visual presentation of the interface, rather than relying on the underlying implementation structure, thereby effectively avoiding the biases caused by the aforementioned different implementation methods.

[0051] Without introducing any control boundary information or parsing the UI description file, the system generates a set of visual regions with potential operational significance by calculating the pixel density change rate, edge continuity, and local texture consistency in the image. This visual region generation process does not attempt to identify specific control types, but rather treats the interface as composed of several visual action blocks to carry subsequent attention analysis and semantic inference.

[0052] S103: Construct the visual attention energy field corresponding to the visual region.

[0053] Specifically, by analyzing the visual characteristics of different regions in the interface in terms of contrast, area ratio, spatial location and neighborhood complexity, a visual attention response model of the interface is constructed for each visual region through a visual attention energy function. Based on the visual attention response model, the visual energy value of each visual region is calculated, and a continuously distributed visual attention energy field corresponding to the visual region is constructed.

[0054] The visual attention energy field is used to simulate the natural visual attention sequence that users may form when there is no explicit task guidance. It is an approximate model of the visual scanning behavior of real users, rather than a rule-based artificial assumption. That is, this solution does not rely on manually preset interface rules or experience judgments, but automatically infers the user's cognitive and operational path through visual data and computing models.

[0055] The visual attention response model is used to simulate the natural visual scanning process of Arabic users in a right-to-left reading mode.

[0056] The visual attention energy function consists of at least weighting coefficients ( ), visual area ( The proportion of space occupied by ) in the visual area ( The degree of difference between the visual region and its immediate neighbors in the perceived color space, and the degree of influence on the formation of cognitive paths. ) and visual area ( The complexity of the surrounding structure determines the visual attention energy function. The formula for calculating the visual attention energy function is shown in equation (3).

[0057] (3);

[0058] in, For visual attention energy function; This refers to the proportion of space occupied by the visual area in the overall interface. This is used to reflect the degree to which the visual area is likely to form a "holistic cognitive unit" at the level of visual perception; , , and All are weighting coefficients; This represents the degree of difference between the visual region and its immediate neighbors in the perceived color space. Used to reflect the region's ability to distinguish within a local visual environment; To describe the degree of influence of visual regions on the formation of RTL cognitive paths within the overall spatial structure of the interface; For visual area The complexity of the surrounding structure reflects how easily a user can quickly identify and make decisions about the area during the cognitive process.

[0059] The parameters in formula (3) above are obtained as follows:

[0060] How to obtain:

[0061] First, the set of visual regions is obtained through the continuity of pixel density gradient. ;

[0062] Then, for any visual region ( Its area is defined as the number of pixels within the visual region. To obtain the total area of ​​the visual region ;in, The k-th visual region is obtained by spatial density gradient segmentation. This region consists of several spatially continuous pixels with similar visual structural features, and is used as the basic unit for subsequent visual energy calculation and cognitive flow analysis.

[0063] Finally, the area parameters of the region are obtained by solving. .in, This represents the total area of ​​the visual region. This represents the number of pixels within the visual area.

[0064] How to obtain:

[0065] First, define the average perceived color of the visual region, and let the set of pixels in the visual region be denoted as . Its perceived color vector mean is ;in, The pixel values ​​are consistent with the preceding perception; This represents the coordinates of the pixel in the interface space.

[0066] Then, define the neighborhood set of the visual region. And calculate the average perceived color of the neighborhood of the visual region. ;in, This is the j-th neighborhood region defined around the current analysis region or pixel. This region consists of a set of pixels spatially adjacent to the target region. The average perceived color of the visual region's neighborhood; The neighborhood set of the visual region; The average perceived color value within the neighborhood is obtained by statistically averaging the color features of all pixels in the neighborhood, and is used to characterize the overall visual features of the neighborhood.

[0067] Finally, through Solve .

[0068] How to obtain:

[0069] Since users' cognitive formation in Arabic interfaces typically follows a pattern of starting from the right-hand high-weight area and gradually expanding horizontally, this invention introduces a directional positional weight function. First, let the interface width be... visual region set The x-coordinate of the geometric center is

[0070] Then, solve The meaning is that the closer to the destination, the better. The larger it is, the further to the left it is. The smaller; among them, for The x-coordinate of the geometric center; This is the width of the interface.

[0071] How to obtain:

[0072] definition Neighborhood region set Solve this problem. ,in, The area of ​​the neighboring region; The distance between the centers of the regions; The i-th neighborhood region is defined around the current analysis region or pixel, and this region consists of a group of pixels that are spatially adjacent to the target region.

[0073] :

[0074] Further discrete Mapped to continuous spatial distribution function To construct a visual attention field for the entire interface. Used to describe the natural shift in visual attention within an interface.

[0075] S104: Based on the visual attention energy field, directional cognitive flow processing is performed through a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface.

[0076] In S104, based on the visual attention energy field, the system performs integral analysis on the energy gradient changes in the horizontal direction of the interface to form a directional cognitive flow model. The spatial inversion calculation of the directional cognitive flow is performed through this model, analyzing the starting region, progression direction, and attenuation mode of the cognitive flow within the interface. The system determines whether the interface forms a reading and comprehension path that conforms to the right-to-left cognitive habit of Arabic in its overall visual structure. This process does not involve "left" or "right" labels, nor does it perform any mirroring judgments; instead, it completes directional inference based on the visual energy transfer relationship in continuous space. This solution, when performing the spatial inversion calculation of the directional cognitive flow, does not rely on preset "left" or "right" direction labels, nor does it employ rule-based processing methods such as interface mirroring. Instead, it performs directional inference based on the visual energy distribution and its transfer relationship in continuous space. Therefore, this solution avoids the subjective bias caused by manually set rules, does not rely on the absolute coordinates of the interface or specific implementation structure, and exhibits good adaptability and consistency under different terminal devices, resolutions, and interface layouts. Meanwhile, this method can handle complex or non-standard layout interfaces and supports the continuous depiction of cognitive direction, thus more realistically reflecting the user's visual reading and comprehension path.

[0077] The system further integrates the occlusion relationships, layer overlap, and accessibility features between interface areas to construct a potential user operation probability graph model. This model is used for visual inference of potential operation decision sequences, inferring the order of operation decisions a user might make after visual attention. The potential user operation probability graph model is a directed weighted graph constructed with visual areas as nodes. Nodes represent potential operation areas, edges represent the user's transition relationship after visual attention, and edge weights characterize the probability of moving from one area to another. These weights are determined based on a combination of regional visual energy, spatial distance, and directional cognitive factors, thus inferring the possible sequence of user operation decisions.

[0078] By analyzing the relative positions of key operation entry points within the visual path and spatial structure, we can determine whether they fall within the priority decision-making channels expected by Arabic users, thereby identifying potential deviations from user interaction habits. Key operation entry points refer to visual area units within the interface that play a decisive role in completing user tasks and have high priority in the potential operation decision sequence.

[0079] The specific process of obtaining the reading path and operation decision path formed under the current interface is shown in A1-A5. Among them, the process of A1-A5 is the spatial inversion calculation process of directional cognitive flow.

[0080] The spatial inversion calculation process of directional cognitive flow is as follows:

[0081] 1. The process of determining the starting region of cognitive flow:

[0082] (1) Construct the energy distribution function in the horizontal direction:

[0083] A1: Integrate the visual attention energy field along the vertical direction to obtain the energy distribution function of the current interface in the horizontal direction.

[0084] Specifically, the visual energy field is first integrated along the vertical direction to obtain the energy distribution function of the current interface in the horizontal direction; where the energy distribution function is used to reflect the horizontal area where the user's vision is most likely to fall first when there is no task guidance; the specific expression of the energy distribution function is shown in formula (4).

[0085] (4);

[0086] in, This is the energy function in the horizontal direction, which is the total visual energy at the position with the horizontal coordinate x. The unit of differentiation when integrating with respect to the variable y; This represents the total height of the interface. For visual attention energy field.

[0087] A2: Construct directional cognitive indicators based on the energy distribution function, and determine the cognitive flow progression direction of the reading path based on these indicators. The cognitive flow progression direction is represented by Θ.

[0088] 2. The process of determining the direction of cognitive flow progression:

[0089] In obtaining Then, calculate First derivative:

[0090] In order to determine the overall trend, a directional cognitive index is constructed. The calculation formula of the directional cognitive index is shown in formula (5).

[0091] (5);

[0092] in, As a directional cognitive indicator; This refers to the width of the interface. It is a differential; for The derivative of .

[0093] like As visual energy decreases from right to left, it migrates to the left along the horizontal axis. >0. >0 indicates that the cognitive flow direction is right to left (RTL).

[0094] like Decreasing from left to right <0. <0 Cognitive flow direction is left to right (LTR).

[0095] 3. Structural analysis of attenuation methods:

[0096] Even if the direction of the cognitive flow is simply correct, it may still be a case of "local highlighting misleading". Therefore, it is necessary to analyze the continuity of attenuation through the visual energy attenuation smoothness index. The expression of the visual energy attenuation smoothness index is shown in formula (6).

[0097] (6);

[0098] in, It is a visual energy decay smoothness index used to characterize the continuity of cognitive flow during spatial propagation.

[0099] like A smaller value indicates a smoother change in visual energy; if... A larger value indicates a violent fluctuation in visual energy. (Attenuation function) This represents the intensity distribution of cognitive flow at a spatial location, with values ​​ranging from [0, 1]. A larger value indicates a higher intensity of cognitive flow at that location, while a smaller value indicates a significant attenuation of the cognitive flow. In practical implementations, this can be achieved by setting a threshold range or based on a global statistical distribution. Classification is carried out when When the value exceeds a preset high threshold or is higher than the overall average, it is determined to be a high-intensity cognitive region; when When the value is less than the preset low threshold or lower than the overall average, it is determined to be a low-intensity cognitive region.

[0100] The natural reading path for Arabic typically includes a right-hand peak, a smooth decrease to the left, and no significant reverse jumps, therefore requiring... ,in, This is the threshold for stable decay.

[0101] 4. Comprehensive judgment mechanism:

[0102] To determine the above-mentioned Arabic natural reading path, a comprehensive cognitive consistency scoring function needs to be constructed. The calculation formula of the comprehensive cognitive consistency scoring function is shown in formula (7).

[0103] (7);

[0104] in: For comprehensive cognitive consistency scoring function; , , All are weighting coefficients, satisfying ; Is the main peak located on the right side? Is the overall progression direction from right to left? To determine whether the decay is smooth.

[0105] like If so, it is determined that the overall visual structure of the current interface forms a reading and comprehension path that conforms to the right-to-left cognitive habit of Arabic. It was determined that there was a directional structural deviation. Among them, The threshold for determining directional consistency. The range of values ​​is .

[0106] A3: Determine the location of the main energy peak based on the energy distribution function, and determine the main visual attention area based on the location of the main energy peak.

[0107] Among them, the main energy peak position is through This indicates that the primary visual attention area is represented by P. ij express.

[0108] (2) Determine the location of the main energy peak:

[0109] The formula for determining the location of the main energy peak is shown in formula (8).

[0110] (8);

[0111] in, In order to make To obtain the horizontal coordinate of the maximum value, if This indicates that the main area of ​​visual attention is on the right side of the interface; This refers to the width of the interface. To determine the independent variable that makes the function reach its maximum value.

[0112] A4: Perform path analysis on the directed graph formed by the main visual attention area to infer the operational decision sequence of the operational decision path.

[0113] In A4, path analysis is performed on the directed graph formed by the main visual attention area to infer the operation decision sequence of the most likely operation decision path that the user will take in order to complete the operation goal.

[0114] Construct the operational probability function between the main visual attention regions, and its expression is shown in formula (9).

[0115] (9);

[0116] in, This is the area of ​​primary visual attention. To represent the visual area Visual energy is used to characterize the degree to which a region attracts the user's attention. Visual energy can be calculated based on factors such as the region's brightness, contrast, salience, and spatial scale, and is used in the operational probability function to characterize the user's tendency to move from the current region to the target region. For the region With the region Spatial distance between them; It is a stabilizing factor used to avoid the denominator being zero.

[0117] This application does not rely on interface layout attributes, control tree structure, or existing RTL mirroring rules for judgment. Instead, it starts from the underlying pixel distribution, spatial relationships, and visual salience changes in the interface screenshot to construct an interface visual cognitive model. Based on this model, it infers the reading order and operational decision path that users may form within the interface. Whether an interface conforms to Arabic usage habits essentially depends on how users visually understand the interface structure, rather than whether the interface meets a certain rule at the implementation level. This elevates the interface adaptation problem from "rule judgment" to "cognitive modeling," thereby achieving more accurate, general, and quantifiable analysis.

[0118] This application is based entirely on in-depth analysis of the pixel, visual energy, and spatial information of the interface screenshot to construct a model that reflects the user's visual scanning and operational decision-making behavior, thereby achieving automatic detection at the "visual semantic level".

[0119] S105: Based on the preset language interface structure model, reading path, and operation decision path, perform consistency matching and judgment on the current interface according to the preset language interface habits.

[0120] In this embodiment, the consistency of Arabic interface habits can be matched and determined based on factors such as the Arabic interface structure model, reading path, and operation decision path.

[0121] The specific process of matching and determining the consistency of Arabic interface habits for the current interface is shown in B1-B4.

[0122] B1: Map the cognitive flow progression direction and operational decision sequence to the Arabic interface structure model (M), and calculate the similarity score between the current interface and the Arabic interface structure model.

[0123] The expression for the similarity score is shown in formula (10).

[0124] (10);

[0125] in, Score the similarity. This represents the progressive direction of cognitive flow; For the operation decision sequence, it represents the evaluation item in the interface habit consistency scoring function, which is used to measure the degree of matching of the interface in the cognitive structure.

[0126] The system maps the reading path model and operation decision sequence formed above to a pre-abstracted Arabic interface structure model. This model is not directly derived from the design specifications, but is a stable spatial relationship pattern summarized from a large number of interface visual structure samples.

[0127] By calculating the similarity between the current interface and the Arabic interface structure model, the system outputs the consistency result of interface interaction habits and locates the specific visual areas that cause the deviation, providing an interpretable basis for interface optimization.

[0128] B2: Compare the similarity score with a preset threshold.

[0129] The preset threshold is set according to the actual situation, and this application does not impose specific limitations.

[0130] B3: If the similarity score is greater than or equal to the preset threshold, it is determined that the current interface does not have any issues with the usage habits of the preset language (such as Arabic) in terms of visual structure and operation guidance.

[0131] B4: If the similarity score is less than the preset threshold, it is determined that the current interface does not conform to the usage habits of the preset language (such as Arabic) in terms of visual structure and operation guidance, and the corresponding visual area is output as the problem location result.

[0132] This solution, without accessing source code, parsing control properties, or reading the semantics of the interface text, determines whether the current interface conforms to the right-to-left cognitive and usage habits of Arabic users in terms of visual guidance, operation sequence, and functional semantic expression, solely based on visual information. Therefore, this solution is no longer positioned as interface specification verification or localization configuration detection, but rather as a technology for judging interface understanding and habit consistency based on visual semantics and interactive cognition.

[0133] The judgment logic proposed in this solution for matching and determining the consistency of Arabic interface habits of the current interface is based on the continuous distribution of visual energy and the directional flow characteristics. It has stronger universality and quantifiable evaluation capabilities, and will not fail due to differences in interface style and display size.

[0134] This solution, based on a continuous visual energy field and a directional cognitive flow model derived from cognitive flow inversion, determines whether an interface conforms to expected usage habits from the perspective of the "user visual perception and operational decision chain." It involves constructing an energy field based on visual saliency, performing cognitive flow inversion on the horizontal visual energy gradient, constructing a potential operational graph, and combining this with an Arabic interface habit model for consistency matching and judgment.

[0135] Starting from the most fundamental principles of visual perception, spatial energy transfer, and cognitive formation, this application achieves automated detection of Arabic interface usage habits without relying on RTL rules, control trees, or text parsing, fully demonstrating the independence and innovation of this invention in its technical implementation path.

[0136] Compared with existing technologies, this solution has the following significant and positive technical effects and application value:

[0137] (1) Improved interface detection capabilities from "implementation correctness" to "use correctness":

[0138] Existing Arabic interface inspection technologies mostly focus on implementation-level checks such as whether RTL attributes are enabled, whether the layout is mirrored, and whether controls are in the prescribed positions. They cannot determine whether the interface will guide users to form operation paths that conform to Arabic cognitive habits during actual use. This solution evaluates the interface structure from the perspective of actual user perception and decision-making through visual attention modeling and cognitive flow inversion, elevating the inspection target from "technical compliance" to "reasonable interactive experience," significantly improving the practical reference value of the inspection results.

[0139] (2) Avoid misjudgment and omission caused by a single rule:

[0140] Existing mainstream solutions rely on fixed RTL rules or design specifications, which can easily lead to numerous misjudgments when faced with complex interfaces, non-standard layouts, or innovative interaction methods. This invention makes judgments based on the visual energy distribution and directional progression in continuous space, without relying on control types or fixed rule thresholds. It can adapt to different interface styles and layouts, and maintains stable detection accuracy even in complex business scenarios.

[0141] (3) Achieve cross-platform and cross-technology stack consistency detection:

[0142] Since this solution is based entirely on the analysis of interface screenshots and does not rely on UI frameworks, layout systems or code implementation details, it can be uniformly applied to iOS, Android and other cross-platform application scenarios, avoiding the problem of inconsistent testing standards due to differences in technology stacks, and significantly improving the international quality control capabilities of enterprises in multi-terminal products.

[0143] (4) Reduce the cost of manual review and improve iteration efficiency:

[0144] In the existing process, Arabic interface adaptation relies heavily on manual testing and repeated communication and modification, which is costly and inefficient. This solution can automatically identify potential deviations from user habits before version release, expose problems in advance, reduce rework, and shorten the delivery cycle of internationalized versions, which is particularly beneficial for large-scale pages and products with high-frequency iterations.

[0145] (5) The test results are interpretable and provide guidance for optimization:

[0146] This solution not only provides a judgment result on whether it conforms to Arabic interface habits, but also can locate the specific visual area that causes the deviation and its position in the cognitive path, providing designers and developers with clear optimization directions, rather than just outputting a simple conclusion on compliance, which helps to continuously improve interface quality.

[0147] (6) It has the potential to be extended to other languages ​​and cultural customs:

[0148] Because this solution adopts a general modeling approach based on visual cognition and spatial relationships, its technical approach can be extended to other language and cultural scenarios that read from right to left or have special reading habits without changing the underlying architecture, by adjusting the cognitive structure model, thus possessing good technical extensibility.

[0149] (7) Interface understanding based on visual attention distribution:

[0150] This solution analyzes the visual characteristics of different areas in the interface, including contrast, area ratio, spatial location, and neighborhood complexity, to construct an interface visual attention response model. This model simulates the natural visual scanning process of Arabic users in a right-to-left reading mode. Unlike mainstream solutions that infer importance based on control type or logical hierarchy, this approach directly inversely determines the user's likely attention order from the intensity of visual stimuli.

[0151] (8) The continuous spatial expression mechanism of directional semantics:

[0152] This solution does not use discrete "left / :right" or "mirror / non-mirror" judgment methods. Instead, it maps the distribution relationship of interface elements in the horizontal direction to a continuous spatial tensor. By analyzing the changes in tensor gradients, it determines whether the interface forms a semantic progression relationship that conforms to the reading order of Arabic, thereby avoiding misjudgments caused by simple flipping rules.

[0153] (9) Visual inference model for operation priority:

[0154] Without relying on control names, event logic, or business semantics, this solution infers the potential priority of each operation entry point in the interface through visual features such as element size, color salience, hierarchical occlusion relationship, and accessibility, and determines whether key operations are located in positions that conform to the decision-making habits of Arabic users.

[0155] (10) Structured computational representation of interface habit rules:

[0156] The "rules" in this solution do not directly reference design specifications or human experience. Instead, they abstract Arabic interface usage habits into a computable relationship between "visual structural features - operational intent - cognitive results," making originally difficult-to-quantify usage habits algorithmically matchable.

[0157] In summary, this invention is significantly superior to existing technologies in terms of detection accuracy, scope of application, degree of automation, and practical application value, and can effectively make up for the shortcomings of existing Arabic interface adaptation detection methods in "real usability assessment".

[0158] The core of this solution lies in its approach: instead of starting with implementation rules, it focuses on the underlying mechanisms of visual perception and interactive cognition to model and infer the underlying structure of interface screenshots. By constructing an interface visual attention energy field, reversing directional cognitive flow, and inferring potential operational decision sequences, it achieves automated determination of whether the interface conforms to Arabic usage habits at the overall visual and interactive level. This effectively addresses the problems of existing technologies, such as their inability to assess the rationality of real-world use, insufficient adaptability, and high detection costs. This solution enables automatic detection of the real usability of Arabic interfaces, rather than merely verifying implementation rules; it is applicable to different platforms and technology stacks without relying on UI frameworks, control trees, or text parsing; it reduces the cost of manual review and improves the objectivity and consistency of interface internationalization quality control; and it provides interpretable visual and interactive structural positioning results for interface optimization.

[0159] From an economic perspective, this solution possesses significant practical application value and scalable deployment potential. In the development of mobile applications for the Arabic-speaking market, interface adaptation issues typically require manual testing and repeated verification by testers with language and cultural backgrounds, the cost of which increases rapidly with the number of pages, version frequency, and product lines. This solution automates the analysis of interface screenshots, quickly identifying interface structure issues that do not conform to Arabic usage habits before release, thus significantly reducing manual testing and rework costs. Furthermore, because this solution is independent of specific operating systems, UI frameworks, or development languages, it can be uniformly applied to iOS, Android, and various cross-platform technology stacks. The same testing capability can be reused across multiple projects and products, helping to establish a standardized Arabic interface quality control process and further reducing the cost per test.

[0160] At the commercial application level, this solution can be integrated into the continuous integration and continuous delivery process as a localized quality inspection or automated testing module. This helps reduce the risk of user churn due to interface incompatibility before products enter the Middle East and North Africa markets, thereby improving user retention and overall product competitiveness, and providing long-term and stable economic benefits.

[0161] In this embodiment, starting from the underlying pixel distribution, spatial relationships, and visual saliency variation patterns of the current real interface screenshot, a visual attention energy field is constructed by analyzing the visual characteristics of different regions in the current interface in terms of contrast, area ratio, spatial position, and neighborhood complexity. This visual attention energy field is used to simulate the natural visual scanning process of Arabic users in a right-to-left reading mode. During this natural visual scanning process, a directional cognitive flow model is used to process the directional cognitive flow to obtain the reading path and operation decision path formed under the current interface. Based on the modeling of the Arabic interface structure model, spatial relationships, and potential paths, the consistency matching and judgment of Arabic interface habits are performed on the current interface. This allows for the determination of the actual user experience of the current interface in actual use. Furthermore, this solution does not rely on testers with language and cultural backgrounds for manual experience and repeated verification during the consistency matching and judgment of Arabic interface habits, thereby improving the efficiency of verifying Arabic interface adaptation.

[0162] Based on the above embodiments Figure 1 In addition to the disclosed interface determination method, this application also discloses an interface determination device, such as... Figure 2 As shown, the interface determination device includes:

[0163] The acquisition unit 201 is used to acquire the interface image of the current interface after perceptual consistency processing generated by the mobile application when it runs in a preset language environment.

[0164] The generation unit 202 is used to generate the visual region corresponding to the interface image based on a preset visual region generation method;

[0165] The first building unit 203 is used to build the visual attention energy field corresponding to the visual region;

[0166] The first processing unit 204 is used to process directional cognitive flow based on the visual attention energy field through a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface.

[0167] The matching and determination unit 205 is used to perform consistency matching and determination of the current interface with the preset language interface habits based on the preset language interface structure model, reading path and operation decision path.

[0168] Furthermore, the interface determination device also includes:

[0169] The second building unit is used to construct the nonlinear sensing mapping function;

[0170] The second processing unit is used to perform perceptual consistency processing on the interface image according to the nonlinear perceptual mapping function.

[0171] Furthermore, the generating unit 202 includes:

[0172] The first calculation module is used to calculate the spatial density gradient function of pixels in the local neighborhood of the interface image.

[0173] The first determining module is used to determine the set of pixels in the visual region whose gradient changes are continuous and whose directions are consistent by analyzing the spatial continuity of the spatial density gradient function and combining it with gradient direction consistency constraints.

[0174] The aggregation module is used to aggregate a set of pixels with continuous gradient changes and consistent directions into the same visual region.

[0175] Furthermore, the first construction unit 203 is specifically used to construct a visual attention energy field for each visual region through a visual attention energy function; wherein, the visual attention energy function is composed of at least a weight coefficient, the spatial proportion of the visual region in the visual region, the degree of difference between the visual region and its direct neighbors in the perceptual color space, the degree of influence of the formation of the cognitive path, and the complexity of the structure around the visual region.

[0176] Furthermore, the first processing unit 204 includes:

[0177] The integration module is used to integrate the visual attention energy field along the vertical direction to obtain the energy distribution function of the current interface in the horizontal direction;

[0178] A determination module is constructed to build directional cognitive indicators based on the energy distribution function, and to determine the cognitive flow progression direction of the reading path based on the directional cognitive indicators.

[0179] The second determining module is used to determine the position of the main energy peak based on the energy distribution function, and to determine the main visual attention area based on the position of the main energy peak.

[0180] The analysis and inference module is used to perform path analysis on the directed graph formed by the main visual attention area and infer the operation decision sequence of the operation decision path.

[0181] Furthermore, the matching determination unit 205 includes:

[0182] The mapping calculation module is used to map the cognitive flow progression direction and operational decision sequence to a preset language interface structure model, and to calculate the similarity score between the current interface and the preset language interface structure model.

[0183] The comparison module is used to compare the similarity score with a preset threshold;

[0184] The third determination module is used to determine that if the similarity score is greater than or equal to a preset threshold, the current interface does not have any issues that do not conform to the preset language usage habits in terms of visual structure and operation guidance.

[0185] The fourth determination module is used to determine if the similarity score is less than a preset threshold, that the current interface does not conform to the preset language usage habits in terms of visual structure and operation guidance, and outputs the corresponding visual area as the problem location result.

[0186] In this embodiment, starting from the underlying pixel distribution, spatial relationships, and visual saliency variation patterns of the current real interface screenshot, a visual attention energy field is constructed by analyzing the visual characteristics of different regions in the current interface in terms of contrast, area ratio, spatial position, and neighborhood complexity. This visual attention energy field is used to simulate the natural visual scanning process of Arabic users in a right-to-left reading mode. During this natural visual scanning process, a directional cognitive flow model is used to process the directional cognitive flow to obtain the reading path and operation decision path formed under the current interface. Based on the modeling of the Arabic interface structure model, spatial relationships, and potential paths, the consistency matching and judgment of Arabic interface habits are performed on the current interface. This allows for the determination of the actual user experience of the current interface in actual use. Furthermore, this solution does not rely on testers with language and cultural backgrounds for manual experience and repeated verification during the consistency matching and judgment of Arabic interface habits, thereby improving the efficiency of verifying Arabic interface adaptation.

[0187] This application embodiment also provides a storage medium, the storage medium including stored instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to execute the interface determination method described above.

[0188] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 3 As shown, it specifically includes a memory 301 and one or more instructions 302, wherein one or more instructions 302 are stored in the memory 301 and are configured to be executed by one or more processors 303 to perform the above-mentioned interface determination method.

[0189] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0190] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0191] The steps in the methods of the various embodiments of this application can be adjusted, combined, or deleted according to actual needs.

[0192] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0193] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0194] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An interface determination method, characterized in that, The method includes: Get the interface image of the current interface after perceptual consistency processing generated when the mobile application is running in a preset language environment; The visual region corresponding to the interface image is generated based on a preset visual region generation method; Construct the visual attention energy field corresponding to the visual region; Based on the aforementioned visual attention energy field, directional cognitive flow processing is performed through a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface; Based on the preset language interface structure model, the reading path, and the operation decision path, the current interface is matched and judged for consistency with preset language interface habits.

2. The method according to claim 1, characterized in that, The process of perceptual consistency processing of interface images includes: Construct a nonlinear sensing mapping function; The interface image is subjected to perceptual consistency processing based on the nonlinear perceptual mapping function.

3. The method according to claim 1, characterized in that, The generation of the visual region corresponding to the interface image based on the preset visual region generation method includes: Calculate the spatial density gradient function of pixels in the local neighborhood of the interface image; By analyzing the spatial continuity of the spatial density gradient function and combining it with gradient direction consistency constraints, the set of pixels in the visual region with continuous gradient changes and consistent directions is determined. The set of pixels with continuous gradient changes and consistent directions is aggregated into the same visual region.

4. The method according to claim 1, characterized in that, The construction of the visual attention energy field corresponding to the visual region includes: For each visual region, a visual attention energy field is constructed using a visual attention energy function; The visual attention energy function is composed of at least the weighting coefficient, the spatial proportion of the visual region within the visual region, the degree of difference between the visual region and its direct neighbors in the perceived color space, the degree of influence of the formation of the cognitive path, and the complexity of the structure surrounding the visual region.

5. The method according to claim 1, characterized in that, Based on the visual attention energy field, directional cognitive flow processing is performed using a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface, including: Integrating the visual attention energy field along the vertical direction yields the energy distribution function of the current interface in the horizontal direction. A directional cognitive index is constructed based on the energy distribution function, and the cognitive flow progression direction of the reading path is determined based on the directional cognitive index. The location of the main energy peak is determined based on the energy distribution function, and the main visual attention area is determined based on the location of the main energy peak. Path analysis is performed on the directed graph formed by the main visual attention area to infer the operation decision sequence of the operation decision path.

6. The method according to claim 5, characterized in that, The step of matching and determining the consistency of the current interface with preset language interface habits based on the preset language interface structure model, the reading path, and the operation decision path includes: The cognitive flow progression direction and the operational decision sequence are mapped to a preset language interface structure model, and the similarity score between the current interface and the preset language interface structure model is calculated. The similarity score is compared with a preset threshold. If the similarity score is greater than or equal to the preset threshold, it is determined that the current interface does not have any issues with the preset language usage habits in terms of visual structure and operation guidance. If the similarity score is less than a preset threshold, it is determined that the current interface does not conform to the preset language usage habits in terms of visual structure and operation guidance, and the corresponding visual area is output as the problem location result.

7. An interface determination device, characterized in that, The device includes: The acquisition unit is used to acquire the interface image of the current interface after perceptual consistency processing generated by the mobile application when it runs in a preset language environment. The generation unit is used to generate the visual region corresponding to the interface image based on a preset visual region generation method. The first construction unit is used to construct the visual attention energy field corresponding to the visual region; The first processing unit is used to perform directional cognitive flow processing based on the visual attention energy field through a pre-constructed directional cognitive flow model to obtain the reading path and operation decision path formed under the current interface. The matching and determination unit is used to perform consistency matching and determination of the current interface with the preset language interface habits based on the preset language interface structure model, the reading path and the operation decision path.

8. The apparatus according to claim 7, characterized in that, Also includes: The second building unit is used to construct the nonlinear sensing mapping function; The second processing unit is used to perform perceptual consistency processing on the interface image according to the nonlinear perceptual mapping function.

9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein when the instructions are executed, the device containing the storage medium is controlled to perform the interface determination method as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 6.

Images