Mobile software test case automatic generation method and system based on multi-modal analysis

Abstract

The invention provides a mobile software test case automatic generation method and system based on multi-modal analysis, and the method comprises the steps: collecting static data and dynamic data of a standard mode application to construct a multi-modal data set, and collecting the related information of a mode suitable for old people; based on four types of meta-level change modes of text / icon change, control position adjustment, event sequence simplification and operation mode reconstruction, mapping analysis is executed on the standard mode and the old-fit mode through a large language model and a rule engine, and a function mapping verification result is generated; key attributes are extracted based on a mapping verification result to construct a structured test case, and the structured test case is converted into an executable script to generate an old-age-suitable mode automatic test case; and executing the test case and outputting a verification report. According to the method and the device, the problems of low generation efficiency, insufficient coverage rate and the like of the elderly-suitable mode test case caused by cross-mode adaptation difference in the prior art are solved, and full-link automatic support is provided for high-quality iteration of the elderly-suitable mobile application.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and artificial intelligence, and specifically to a method and system for automatically generating test cases for mobile software based on multimodal analysis. Background Technology

[0002] The global aging process is accelerating. According to the World Health Organization, the population aged 60 and over is projected to surge from 1 billion in 2020 to 1.4 billion in 2030. Against this backdrop, mobile applications have become indispensable tools in the lives of the elderly, covering diverse scenarios such as social interaction, shopping, and healthcare. However, the elderly commonly face problems such as declining vision and cognitive abilities, making it difficult for them to use conventional mobile applications smoothly due to complex interfaces and lengthy operation processes. To address this, mainstream mobile operating systems (such as iOS and Android) offer basic functions like font magnification and voice assistance in an attempt to lower the barrier to entry. Unfortunately, these general accessibility features are poorly adapted to the actual use of mobile applications, often causing problems such as misaligned interface elements and confusing functional logic, which in turn exacerbates the operational difficulties for the elderly.

[0003] Currently, functional test cases written by developers are the primary means of ensuring the quality of Android applications. Based on their domain knowledge of application logic, developers can write highly targeted test cases that trigger deep application states and expose hidden defects. However, the high cost and low reusability of manually writing test cases are significant factors restricting application development efficiency, especially when test cases are difficult to use across modes: to meet the needs of elderly users, the same Android application often has a standard mode and an age-friendly mode. The differences in functionality and interface design between the two modes make test cases incompatible. Interface design adjustments, the introduction of new features, or the reconstruction of existing features require developers and testers to redesign and maintain test cases for each mode. Despite their highly similar functionality and interface, developers often need to write separate test cases for each application, resulting in a significant waste of development and testing resources.

[0004] While existing recording and playback technologies have performed well in migrating tests within the same device and version, they still have significant shortcomings in test case reuse and adaptation when facing cross-modal testing. Therefore, there is an urgent need to propose a method for automatically generating mobile software test cases to address these issues. Summary of the Invention

[0005] The technical problem to be solved by this invention is that the existing technology suffers from low efficiency and insufficient coverage in generating test cases for age-friendly modes due to cross-mode adaptation differences, making it difficult to automatically generate and test test cases for age-friendly modes.

[0006] This invention solves the above-mentioned technical problems through the following technical means: a method for automatically generating mobile software test cases based on multimodal analysis, comprising: S1. Collect static and dynamic data from standard mode applications to construct a multimodal dataset; collect information related to age-friendly mode and prepare the data required for function mapping; S2. Based on four types of meta-level change patterns—text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction—a large language model and rule engine are used to perform mapping analysis on the standard pattern and the age-friendly pattern, generating functional mapping verification results. S3. Integrate the functional mapping verification results of the four types of meta-level change patterns, extract the key attributes of the age-friendly pattern, and construct structured test cases. S4. Convert the structured test cases into executable scripts to generate age-friendly automated test cases; S5. Execute the automated test cases for the age-friendly mode to verify the functional coverage and operation path consistency of the age-friendly mode, and output a verification report containing test execution logs and interface state comparison results.

[0007] Furthermore, the static data includes GUI layout files, control attributes, and hierarchical relationships, while the dynamic data includes user interaction behavior sequences and page screenshots.

[0008] Furthermore, the specific steps for collecting information related to the age-friendly mode and preparing the data required for function mapping are as follows: Obtain relevant information about the age-friendly mode: Use automated tools to collect screenshots of the current interface of the age-friendly mode and the corresponding XML layout file; Prepare the data required for function mapping: Clean and optimize the collected data of the age-friendly mode, retain the control data related to the core function and its layout coordinates and hierarchical relationship data, and form an age-friendly mode control dataset.

[0009] Furthermore, the text / icon changes include: Text similarity analysis: Using a large language model and rule engine, the text similarity between the control text content in standard mode and age-friendly mode is calculated. If the text similarity exceeds the set threshold, the functions are equivalent. Icon alignment: The rule engine extracts the icon content of the standard mode and the age-friendly mode, and performs visual feature matching on the icon controls of the standard mode and the age-friendly mode based on the image feature matching algorithm.

[0010] Furthermore, the adjustment of the control position includes: Coordinate and layout hierarchy analysis: The rule engine analyzes the coordinates and hierarchy of controls in the age-friendly mode and standard mode. If the position adjustment does not affect the function of the control, the rule engine confirms that the position adjustment does not change the functional logic. Generate position-independent operation paths: After confirming that the position adjustment will not affect the functionality, the rule engine generates operation paths in age-appropriate mode to ensure that the operation logic remains consistent after the control position changes.

[0011] Furthermore, the event sequence simplification includes: The event sequences of the standard mode and the age-friendly mode are analyzed using a rule engine; Based on the aforementioned difference analysis, determine whether event path compression or an increase in the number of event paths is necessary. Event path compression: If it is determined that event path compression is required, generate corresponding equivalent event sequence mapping rules: map multi-step operations in the standard mode to single-step operations in the age-friendly mode; Increased event paths: If it is determined that more event paths are needed, generate corresponding equivalent event sequence mapping rules: In the age-friendly mode, set up additional confirmation or transition operations compared to the standard mode.

[0012] Furthermore, the reconfiguration of the operation mode includes: Interaction Logic Analysis: By combining a rule engine with a large language model, the differences in interaction logic between the standard mode and the age-friendly mode are analyzed, the equivalent operation path of the age-friendly mode is derived, and the operation method of the standard mode is transformed into the operation method of the age-friendly mode. Operation path consistency verification: The rule engine verifies the consistency of operation path logic between the age-friendly mode and the standard mode, checks whether changes in operation methods cause logical errors, and performs reasoning and rule verification for each operation step in the age-friendly mode.

[0013] Furthermore, S3 includes: Based on the functional mapping verification results of the four types of meta-level change patterns, combined with the reasoning ability of the large language model, the interaction logic of the standard pattern and the age-friendly pattern is compared, and the functional mapping and operation steps in the age-friendly pattern are derived. The control ID, operation type and event sequence of the age-friendly pattern are extracted to generate structured test cases for the age-friendly pattern.

[0014] Furthermore, S4 includes: Script Conversion: Based on the control ID, operation type, and operation sequence of the structured test cases in the age-friendly mode, call the automated test script template to generate an age-friendly automated test script containing control location expressions and operation functions; Control positioning optimization: Extract the text content or image features of age-friendly controls and perform dynamic and stable control positioning.

[0015] This invention also provides an automatic test case generation system for mobile software based on multimodal analysis, comprising: The data acquisition module is used to collect static and dynamic data from standard mode applications, construct a multimodal dataset, collect information related to the age-friendly mode, and prepare the data required for function mapping. The mapping analysis module is used to perform mapping analysis on standard patterns and age-friendly patterns based on four types of meta-level change patterns: text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction. It generates functional mapping verification results by using a large language model and rule engine. The test case building module is used to integrate the functional mapping verification results of four types of meta-level change patterns and extract the key attributes of the age-friendly pattern to build structured test cases. The script generation module is used to convert the structured test cases into executable scripts and generate age-friendly automated test cases. The test execution and reporting module is used to execute the automated test cases of the age-friendly mode, verify the functional coverage and operation path consistency of the age-friendly mode, and output a verification report containing test execution logs and interface status comparison results.

[0016] The advantages of this invention are: This invention utilizes multimodal analysis technology to achieve efficient and automated generation of standard mode test cases into age-friendly modes, significantly improving cross-mode adaptation efficiency and functional coverage completeness.

[0017] This invention analyzes the interface differences between the standard mode and the age-friendly mode (such as semantic changes in text / icons, adjustments to control positions, simplification of event sequences, and reconstruction of operation methods), and combines this with multimodal data (GUI layout, page screenshots, control attributes, etc.) for fusion analysis to dynamically generate operation paths adapted to the age-friendly mode. Addressing the issues of low efficiency in cross-modal test case migration and insufficient semantic matching accuracy in existing technologies, this invention utilizes a collaborative mechanism based on rules of four types of meta-level change patterns and a pre-trained large language model to automatically verify functional equivalence (such as the completeness of operational logic), ensuring that the generated test cases accurately cover the core functions of the age-friendly mode.

[0018] This invention addresses existing test cases in standard modes by generating age-friendly test scripts (such as Appium or UI Automator executable scripts) through multimodal feature extraction and rule-guided path derivation. In this process, it avoids relying on depth-first search traversal, instead achieving efficient test case migration based on dynamic mapping of interaction behavior sequences and control properties.

[0019] In the process of test case generation, this invention combines the reasoning ability of a large language model with the constraint mechanism of a rule engine to design targeted guided function mapping, which solves the core problems in the existing technology such as low efficiency of cross-mode test case generation, insufficient accuracy of semantic mapping, and limited coverage of complex scenarios, and provides full-link automated support for high-quality iteration of age-friendly mobile applications. Attached Figure Description

[0020] Figure 1 This is a flowchart of the automatic generation method for mobile software test cases based on multimodal analysis according to Embodiment 1 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 like Figure 1 As shown, the method for automatically generating mobile software test cases based on multimodal analysis includes: S1. Collect static and dynamic data from standard mode applications to construct a multimodal dataset; collect information related to age-friendly mode and prepare the data required for function mapping.

[0023] Specifically, in the standard mode of mobile software, the UI Automator tool is used to automatically traverse the standard mode mobile application. During the execution of test cases (such as sequences of operation events like clicks and swipes), two types of data are collected simultaneously: Static data includes GUI layout files, control properties, and hierarchical relationships.

[0024] Dynamic data: including user interaction sequences and page screenshots.

[0025] Through the above data collection, static data (structured interface information) and dynamic data (operation trajectory, interface feedback, state transition) are associated and integrated to construct a multimodal dataset, which fully represents the operational information of the standard mode application.

[0026] Obtain relevant information about the age-friendly mode: Use automated tools to collect screenshots of the current interface of the age-friendly mode and the corresponding XML layout file.

[0027] Prepare the data required for function mapping: Clean and optimize the collected data of the age-friendly mode, remove irrelevant parts (such as advertisements, non-functional controls, etc.), and retain the controls related to the core functions and their layout coordinates and hierarchical relationship data to form an age-friendly mode control dataset, which will facilitate subsequent operation mapping and function verification.

[0028] S2. Based on four types of meta-level change patterns—text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction—a large language model and rule engine are used to perform mapping analysis on the standard pattern and the age-friendly pattern, generating functional mapping verification results.

[0029] Specifically, the event sequence and runtime information obtained in the standard mode are mapped to the age-friendly mode. After enabling the age-friendly mode in the mobile application, four types of meta-level change patterns are used to map the sequence sequentially to the application in the age-friendly mode.

[0030] Text / Icon Changes: For changes to text or icons in age-friendly mode, semantic similarity of the text is calculated to verify functional consistency, and icon features are matched to ensure functional alignment of controls across modes. This includes: Text similarity analysis: For text controls, extract the text content of the standard mode and the age-friendly mode (such as "shopping cart" and "my basket"). Through a large language model and rule engine, calculate the text similarity between the text content of the control in the standard mode and the age-friendly mode. If the text similarity exceeds a set threshold (such as 0.85), the functions are equivalent.

[0031] Icon alignment: Through the rule engine, the icon content (such as shape, color, size, etc.) of the standard mode and the age-friendly mode is extracted, and visual feature matching is performed on the icon controls of the standard mode and the age-friendly mode based on the image feature matching algorithm.

[0032] Adjusting the position of controls includes: Coordinate and Layout Hierarchy Analysis: The rule engine analyzes the coordinates and hierarchy of controls in the age-friendly and standard modes. If adjusting the position does not affect the functionality of the control, the rule engine confirms that the position adjustment does not change the functional logic. For example, the "click on the top search bar" in the standard mode is mapped to the "click on the bottom search bar" in the age-friendly mode, and the consistency of the search function triggering results is verified.

[0033] Generate position-independent operation paths: After confirming that the position adjustment will not affect the functionality, the rule engine generates operation paths in age-appropriate mode to ensure that the operation logic remains consistent after the control position changes.

[0034] Event sequence simplification includes: The event sequences of the standard mode and the age-friendly mode are analyzed using a rule engine.

[0035] Based on the difference analysis, determine whether event path compression or the number of event paths is needed.

[0036] Event path compression: If event path compression is deemed necessary, a corresponding equivalent event sequence mapping rule is generated: multi-step operations in standard mode are mapped to single-step operations in age-friendly mode. For example, in standard mode, clicking the "My" button followed by the "Personal Information" button is required, while in age-friendly mode, clicking the "Personal Information" button directly is sufficient.

[0037] Increased event paths: If it is determined that more event paths are needed, generate corresponding equivalent event sequence mapping rules: In the age-friendly mode, set up additional confirmation or transition operations compared to the standard mode.

[0038] The operation method refactoring includes: Interaction Logic Analysis: By combining a rule engine with a large language model, the differences in interaction logic between the standard mode and the age-friendly mode are analyzed. The equivalent operation path for the age-friendly mode is derived, and the operation methods of the standard mode are converted into those of the age-friendly mode. This ensures that changes in operation methods (such as clicks and swipes) in the age-friendly mode do not affect the implementation of core functions. For example, a long press operation in the standard mode is converted into a single click operation in the age-friendly mode. The rule engine ensures that the new operation path can achieve the same functionality as the standard mode.

[0039] Operation path consistency verification: The rule engine verifies the consistency of operation path logic between the age-friendly mode and the standard mode, checks whether changes in operation methods cause logical errors, and performs reasoning and rule verification for each operation step in the age-friendly mode.

[0040] S3. Integrate the functional mapping verification results of the four types of meta-level change patterns, extract the key attributes of the age-friendly pattern, and construct structured test cases.

[0041] Specifically, based on the functional mapping verification results of the four types of meta-level change patterns, combined with the reasoning ability of the large language model, the interaction logic between the standard pattern and the age-friendly pattern is compared to deduce the functional mapping and operation steps in the age-friendly pattern; the control ID, operation type and event sequence of the age-friendly pattern are extracted to generate structured test cases for the age-friendly pattern.

[0042] S4. Convert structured test cases into executable scripts to generate age-friendly automated test cases.

[0043] Specifically, the script conversion process involves using the control IDs, operation types, and operation sequences of structured test cases based on the age-friendly model to call an automated test script template and generate an age-friendly automated test script containing control location expressions and operation functions.

[0044] Control positioning optimization: Extract the text content or image features of age-friendly controls and perform dynamic and stable control positioning.

[0045] S5. Execute automated test cases for the age-friendly mode to verify the functional coverage and operation path consistency of the age-friendly mode, and output a verification report including test execution logs and interface state comparison results.

[0046] Specifically, the generated age-friendly mode test cases are executed to verify their functional coverage and consistency with the operation path. Interface anomalies are detected using image difference algorithms, and the page navigation paths of the standard mode and age-friendly mode are compared. A test report containing defect location, quantitative indicators, and repair suggestions is output to drive rapid fixes for compatibility issues.

[0047] In this invention, by analyzing the differences between 221 age-friendly versions and standard versions collected from popular commercial age-friendly mobile applications, it is concluded that the current age-friendly software mainly includes four types of changes: (1) Event sequence simplification: The event path to complete the standard mode in age-friendly mode is compressed or increased. Other automated testing tools cannot efficiently identify the compressed or increased content, wasting automated testing resources; (2) Control position adjustment: The position of GUI interface controls in age-friendly mode is adjusted, placing the core function controls in a more obvious position. Other automated testing tools have difficulty performing global search; (3) Text / icon changes: The names and icons of controls in the GUI interface are updated to text and icons that are more suitable for the elderly to understand. Other automated testing tools have difficulty completing the mapping of the same controls in the two modes; (4) Operation mode reconstruction: There is an equivalent operation path to complete the standard mode function in age-friendly mode. Other automated testing tools have difficulty updating the operation mode, resulting in a low test case generation rate.

[0048] The test cases with updated mappings are run automatically. By performing feature matching analysis on runtime data, the test case results in the age-friendly mode are collected. In this embodiment, the mapping update categories include: text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction.

[0049] By following the steps above, we can automate the testing of age-friendly mobile applications, provide the test results to an automated tool for generating age-friendly mobile applications, correct the generated applications, and ensure that they meet the needs of seniors and provide a good user experience.

[0050] In summary, this invention utilizes multimodal analysis technology to achieve efficient and automated generation of standard mode test cases into age-friendly modes, significantly improving cross-mode adaptation efficiency and functional coverage completeness.

[0051] This invention analyzes the interface differences between the standard mode and the age-friendly mode (such as semantic changes in text / icons, adjustments to control positions, simplification of event sequences, and reconstruction of operation methods), and combines this with multimodal data (GUI layout, page screenshots, control attributes, etc.) for fusion analysis to dynamically generate operation paths adapted to the age-friendly mode. Addressing the issues of low efficiency in cross-modal test case migration and insufficient semantic matching accuracy in existing technologies, this invention utilizes a collaborative mechanism based on rules of four types of meta-level change patterns and a pre-trained large language model to automatically verify functional equivalence (such as the completeness of operational logic), ensuring that the generated test cases accurately cover the core functions of the age-friendly mode.

[0052] This invention addresses existing test cases in the standard mode by generating structured, age-friendly test scripts (such as Appium or UI Automator executable scripts) through multimodal feature extraction and rule-guided path derivation. In this process, it avoids relying on depth-first search traversal, instead achieving efficient test case migration based on dynamic mapping of interaction behavior sequences and control attributes. For example, the standard mode's "multi-step deletion operation" is automatically mapped to the age-friendly mode's "one-click deletion" logic, and the integrity and stability of the operation path are ensured through operation path consistency verification and function trigger result comparison.

[0053] This invention combines the reasoning capabilities of a large language model with the constraint mechanisms of a rule engine during test case generation to design targeted guided function mappings. For example, given the input "Map the standard mode long-press menu operation to the age-friendly mode explicit button click," the model outputs adaptation operation suggestions and verifies the control hierarchy based on the layout file, avoiding manual intervention. Through techniques such as event path compression and control positioning optimization, it solves the adaptation challenges caused by dynamic interface changes (such as control ID updates), generating robust test cases.

[0054] This invention solves the core problems in the prior art, such as low efficiency in generating cross-modal test cases, insufficient accuracy in semantic mapping, and limited coverage of complex scenarios, and provides full-link automated support for high-quality iteration of age-friendly mobile applications.

[0055] Example 2 Based on Example 1, Example 2 further proposes an automatic test case generation system for mobile software based on multimodal analysis, including: The data acquisition module is used to collect static and dynamic data from standard mode applications to build a multimodal dataset; it also collects information related to the age-friendly mode and prepares the data required for function mapping.

[0056] Specifically, static data includes GUI layout files, control properties and hierarchical relationships, while dynamic data includes user interaction sequences and page screenshots.

[0057] The unit for acquiring information related to the age-friendly mode is used to collect screenshots of the current interface of the age-friendly mode and the corresponding XML layout file through automated tools.

[0058] Function mapping data preparation unit: used to clean and optimize the collected data of the age-friendly mode, retain the control data related to the core function and its layout coordinates and hierarchical relationship data, and form an age-friendly mode control dataset.

[0059] The mapping analysis module is used to perform mapping analysis on standard patterns and age-friendly patterns based on four types of meta-level change patterns: text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction. It generates functional mapping verification results by using a large language model and rule engine.

[0060] Specifically, it includes a text similarity analysis unit, which uses a large language model and rule engine to calculate the text similarity between the control text content in standard mode and age-friendly mode. If the text similarity exceeds a set threshold, the functions are equivalent.

[0061] The icon alignment unit is used to extract the icon content of standard mode and age-friendly mode through the rule engine, and perform visual feature matching on the icon controls of standard mode and age-friendly mode based on the image feature matching algorithm.

[0062] The coordinate and layout hierarchy analysis unit is used to analyze the coordinates and hierarchy of controls in the age-friendly and standard modes through the rule engine. If the position adjustment does not affect the function of the control, the rule engine confirms that the position adjustment does not change the functional logic.

[0063] The position-independent operation path generation unit is used by the rule engine to generate operation paths in age-appropriate mode after confirming that the position adjustment will not affect the function, so as to ensure that the operation logic remains consistent after the control position changes.

[0064] The event sequences of the standard mode and the age-friendly mode are analyzed using a rule engine.

[0065] Based on the difference analysis, determine whether event path compression or the number of event paths is needed.

[0066] The event path compression unit is used to generate corresponding equivalent event sequence mapping rules if it is determined that event path compression is required: mapping multi-step operations in the standard mode to single-step operations in the age-friendly mode.

[0067] The event path addition unit is used to generate corresponding equivalent event sequence mapping rules if it is determined that event path addition is required: in the age-friendly mode, add confirmation or transition operations to the operation steps compared to the standard mode.

[0068] The interaction logic parsing unit is used to analyze the differences in interaction logic between the standard mode and the age-friendly mode by combining the rule engine with the large language model, deduce the equivalent operation path of the age-friendly mode, and transform the operation mode of the standard mode into the operation mode of the age-friendly mode.

[0069] The operation path consistency verification unit is used to verify the consistency of operation path logic between the age-friendly mode and the standard mode through the rule engine, check whether changes in operation methods cause logical errors, and perform reasoning and rule verification for each operation step in the age-friendly mode.

[0070] The test case building module is used to integrate the functional mapping verification results of four types of meta-level change patterns and extract the key attributes of the age-friendly pattern to build structured test cases.

[0071] Specifically, based on the functional mapping verification results of the four types of meta-level change patterns, combined with the reasoning ability of the large language model, the interaction logic between the standard pattern and the age-friendly pattern is compared to deduce the functional mapping and operation steps in the age-friendly pattern; the control ID, operation type and event sequence of the age-friendly pattern are extracted to generate structured test cases for the age-friendly pattern.

[0072] The script generation module is used to convert structured test cases into executable scripts and generate age-friendly automated test cases.

[0073] Script Conversion Unit: Based on the control ID, operation type, and operation sequence of structured test cases in the age-friendly mode, it calls the automated test script template to generate an age-friendly automated test script containing control location expressions and operation functions.

[0074] Control positioning optimization unit: used to extract the text content or image features of age-friendly controls and perform dynamic control positioning.

[0075] The test execution and reporting module is used to execute automated test cases for the age-friendly mode, verify the functional coverage and operation path consistency of the age-friendly mode, and output a verification report containing test execution logs and interface status comparison results.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically generating test cases for mobile software based on multimodal analysis, characterized in that, include: S1. Collect static and dynamic data from standard mode applications to construct a multimodal dataset; Collect information related to the age-friendly mode and prepare the data required for function mapping; S2. Based on four types of meta-level change patterns—text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction—a large language model and rule engine are used to perform mapping analysis on the standard pattern and the age-friendly pattern, generating functional mapping verification results. S3. Integrate the functional mapping verification results of the four types of meta-level change patterns, extract the key attributes of the age-friendly pattern, and construct structured test cases. S4. Convert the structured test cases into executable scripts to generate age-friendly automated test cases; S5. Execute the automated test cases for the age-friendly mode to verify the functional coverage and operation path consistency of the age-friendly mode, and output a verification report containing test execution logs and interface state comparison results.

2. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, The static data includes GUI layout files, control properties, and hierarchical relationships, while the dynamic data includes user interaction behavior sequences and page screenshots.

3. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, The specific steps for collecting information related to the age-friendly mode and preparing the data required for function mapping are as follows: Obtain relevant information about the age-friendly mode: Use automated tools to collect screenshots of the current interface of the age-friendly mode and the corresponding XML layout file; Prepare the data required for function mapping: Clean and optimize the collected data of the age-friendly mode, retain the control data related to the core function and its layout coordinates and hierarchical relationship data, and form an age-friendly mode control dataset.

4. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, The text / icon changes include: Text similarity analysis: Using a large language model and rule engine, the text similarity between the control text content in standard mode and age-friendly mode is calculated. If the text similarity exceeds the set threshold, the functions are equivalent. Icon alignment: The rule engine extracts the icon content of the standard mode and the age-friendly mode, and performs visual feature matching on the icon controls of the standard mode and the age-friendly mode based on the image feature matching algorithm.

5. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, The control position adjustment includes: Coordinate and layout hierarchy analysis: The rule engine analyzes the coordinates and hierarchy of controls in the age-friendly mode and standard mode. If the position adjustment does not affect the function of the control, the rule engine confirms that the position adjustment does not change the functional logic. Generate position-independent operation paths: After confirming that the position adjustment will not affect the functionality, the rule engine generates operation paths in age-appropriate mode to ensure that the operation logic remains consistent after the control position changes.

6. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, The event sequence simplification includes: The event sequences of the standard mode and the age-friendly mode are analyzed using a rule engine; Based on the aforementioned difference analysis, determine whether event path compression or an increase in the number of event paths is necessary. Event path compression: If it is determined that event path compression is required, generate corresponding equivalent event sequence mapping rules: map multi-step operations in the standard mode to single-step operations in the age-friendly mode; Increased event paths: If it is determined that more event paths are needed, generate corresponding equivalent event sequence mapping rules: In the age-friendly mode, set up additional confirmation or transition operations compared to the standard mode.

7. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, The reconfiguration of the operation mode includes: Interaction Logic Analysis: By combining a rule engine with a large language model, the differences in interaction logic between the standard mode and the age-friendly mode are analyzed, the equivalent operation path of the age-friendly mode is derived, and the operation method of the standard mode is transformed into the operation method of the age-friendly mode. Operation path consistency verification: The rule engine verifies the consistency of operation path logic between the age-friendly mode and the standard mode, checks whether changes in operation methods cause logical errors, and performs reasoning and rule verification for each operation step in the age-friendly mode.

8. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, S3 includes: Based on the functional mapping verification results of the four types of meta-level change patterns, combined with the reasoning ability of the large language model, the interaction logic of the standard pattern and the age-friendly pattern is compared, and the functional mapping and operation steps in the age-friendly pattern are derived. The control ID, operation type and event sequence of the age-friendly pattern are extracted to generate structured test cases for the age-friendly pattern.

9. The method for automatically generating mobile software test cases based on multimodal analysis according to claim 1, characterized in that, S4 includes: Script Conversion: Based on the control ID, operation type, and operation sequence of the structured test cases in the age-friendly mode, call the automated test script template to generate an age-friendly automated test script containing control location expressions and operation functions; Control positioning optimization: Extract the text content or image features of age-friendly controls and perform dynamic and stable control positioning.

10. A mobile software test case automatic generation system based on multimodal analysis, characterized in that, include: The data acquisition module is used to collect static and dynamic data from standard mode applications to build a multimodal dataset. Collect information related to the age-friendly mode and prepare the data required for function mapping; The mapping analysis module is used to perform mapping analysis on standard patterns and age-friendly patterns based on four types of meta-level change patterns: text / icon changes, control position adjustments, event sequence simplification, and operation method reconstruction. It generates functional mapping verification results by using a large language model and rule engine. The test case building module is used to integrate the functional mapping verification results of four types of meta-level change patterns and extract the key attributes of the age-friendly pattern to build structured test cases. The script generation module is used to convert the structured test cases into executable scripts and generate age-friendly automated test cases. The test execution and reporting module is used to execute the automated test cases of the age-friendly mode, verify the functional coverage and operation path consistency of the age-friendly mode, and output a verification report containing test execution logs and interface status comparison results.

Images