Information processing method, information processing device, and computer program

Abstract

This provides a technique for modifying the data structure of structured data related to screen design. [Solution] The data structure modification device 14 acquires design data as structured data related to the screen design from the design data storage device 12. If the structure of the acquired design data satisfies predetermined modification requirements, the data structure modification device 14 modifies the structure of the design data to a structure suitable for creating or operating an application based on the design data. The data structure modification device 14 outputs the modified design data to the design data storage device 12.

Description

[Technical Field]

[0001] The present disclosure relates to data processing technology, and particularly to an information processing method, an information processing apparatus, and a computer program. [Background Art]

[0002] Technologies have been provided for displaying the layout of a display image on a computer based on structured data (see, for example, Patent Document 1). [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2013-148995 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] In recent years, UI design tools that support the design of UIs (User Interfaces) such as screens have become widespread. Some UI design tools can output structured data related to screen design. It is also conceivable to improve the efficiency of application development by utilizing such structured data in application development.

[0005] However, structured data related to screen design is created with emphasis on the appearance (layout, etc.) of the screen, and the data structure of such structured data may not be suitable for the creation or operation of an application. One object of the present disclosure is to provide a technique for modifying the data structure of structured data related to screen design. [Means for Solving the Problem]

[0006] To solve the above problems, an information processing method in one aspect of the present disclosure involves a computer acquiring structured data relating to a screen design, and if the structure of the structured data satisfies predetermined modification requirements, changing the structure of the structured data to a structure suitable for creating or operating an application based on the structured data, and outputting the modified structured data.

[0007] Another aspect of this disclosure is an information processing device. This device includes an acquisition unit that acquires structured data relating to the design of a screen; a modification unit that, if the structure of the structured data acquired by the acquisition unit satisfies predetermined modification requirements, modifies the structure of the structured data to a structure suitable for creating or operating an application based on the structured data; and an output unit that outputs the modified structured data from the modification unit.

[0008] Furthermore, any combination of the above components, or any conversion of the expressions of this disclosure between systems, computer programs, recording media containing computer programs, etc., are also valid forms of this disclosure. [Effects of the Invention]

[0009] The technology disclosed herein can assist in modifying the data structure of structured data related to screen design. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram shows the application development process using design data. [Figure 2] This figure shows an example of a screen created by a designer. [Figure 3] Figure 3(a) shows an example of design data output from a UI design tool, and Figure 3(b) is a schematic representation of the design data in Figure 3(a) in a tree structure. [Figure 4] This figure shows an example of a screen after modification using the data structure modification device of the embodiment. [Figure 5]Fig. 5(a) is a diagram showing an example of design data after correction by the data structure correction device according to the embodiment, and Fig. 5(b) is a diagram schematically showing the design data of Fig. 5(a) in a tree structure. [Figure 6] It is a diagram showing an example of a screen created by a designer. [Figure 7] It is a diagram showing an example of design data output from a UI design tool. [Figure 8] It is a diagram showing an example of a screen after correction by the data structure correction device according to the embodiment. [Figure 9] It is a diagram showing an example of design data after correction by the data structure correction device according to the embodiment. [Figure 10] It is a diagram showing an example of a screen created by a designer. [Figure 11] It is a diagram showing an example of design data output from a UI design tool. [Figure 12] It is a diagram showing an example of design data after correction by the data structure correction device according to the embodiment. [Figure 13] It is a diagram showing a configuration of an information processing system according to an embodiment. [Figure 14] It is a block diagram showing functional blocks included in the data structure correction device according to the embodiment. [Figure 15] It is a flowchart showing an operation of the data structure correction device according to the embodiment. [Figure 16] It is a flowchart showing details of the tree structuring processing in S11 of Fig. 15. [Figure 17] It is a flowchart showing details of the node deletion processing in S12 of Fig. 15. [Figure 18] It is a flowchart showing details of the node grouping processing in S13 of Fig. 15. DETAILED DESCRIPTION OF THE INVENTION

[0011] The main body of the apparatus or method according to the present disclosure comprises a computer. The functions of the main body of the apparatus or method according to the present disclosure are implemented when the computer executes a program. The computer comprises a processor that operates according to a program as a main hardware configuration. The type of the processor is not limited as long as it can implement functions by executing a program. The processor is constituted of one or a plurality of electronic circuits including a semiconductor integrated circuit (IC) or a large scale integration (LSI). The plurality of electronic circuits may be integrated on a single chip, or may be provided on a plurality of chips. The plurality of chips may be integrated in a single apparatus, or may be provided in a plurality of apparatuses. The program may be recorded in a non-transitory recording medium such as a computer-readable read only memory (ROM), optical disc, or hard disk drive, or may be recorded in a transitory storage medium such as a computer-readable random access memory (RAM). The program may be pre-stored in a recording medium, or may be supplied to the recording medium or storage medium via a wide area communication network including the Internet or the like.

[0012] The structured data according to the embodiment is data shaped into a predetermined structure. For example, the structured data according to the embodiment includes data described in a predetermined data description language (for example, markup languages such as HTML and XML, JSON (JavaScript (registered trademark) Object Notation), etc.). The design data according to the embodiment is data relating to screen design for applications or the like, and includes, for example, data defining a screen layout. The design data according to the embodiment is structured data, specifically, data in JSON format. The design data includes nodes corresponding to elements in the screen (for example, groups, buttons, and the like).

[0013] <Summary Description of Embodiment> An overview of the embodiment will be described. Figure 1 shows the flow of application development using design data. A designer designs the application screen (layout, etc.) using a known UI design tool, and the UI design tool outputs design data related to the screen design by the designer. A conversion device converts the design data into program code based on a predetermined algorithm, for example, into source code written in the object-oriented language Dart. In application development, the developer implements and tests the program code output by the conversion device.

[0014] However, design data is created with an emphasis on the appearance of the screen (layout, etc.), and the data structure of the design data may not be suitable for creating or running an application. As a result, problems such as (1) difficulty in implementing logic into program code based on the design data, (2) failure of application testing after program code implementation, and (3) slow application operation may occur.Therefore, in this embodiment, we propose a data structure modification device that modifies the structure of the design data to a structure suitable for creating or running an application, without changing the design aspect of the design data.Note that the data structure modification technology in this embodiment is not limited to design data and can be applied to various types of structured data.

[0015] Next, we will explain the challenges of the conventional technology and how the data structure modification device of the embodiment solves these problems, using Examples 1 to 3.

[0016] Example 1: Figure 2 shows an example of a screen created by a designer. In screen 100 of Figure 2, group AA is placed inside group A, group AAA is placed inside group AA, and a button element is placed inside group AAA. In other words, in screen 100, descendant nodes are redundantly placed. Such an arrangement can occur, for example, when a designer copies an existing screen with a complex structure and removes unnecessary visible elements.

[0017] Figure 3(a) shows an example of design data (in JSON format) corresponding to screen 100 in Figure 2, output from a UI design tool. Figure 3(b) schematically shows the design data from Figure 3(a) in a tree structure.

[0018] In application development based on such design data, a problem arises where button testing (black-box testing) fails because the button's position (depth) is unknown. Furthermore, the rendering cost is high due to the need to overlay multiple layers.

[0019] Figure 4 shows an example of a screen after modification by the data structure modification device of the embodiment. Figure 5(a) shows an example of design data (JSON format) after modification by the data structure modification device of the embodiment. Figure 5(b) schematically shows the design data of Figure 5(a) in a tree structure. In this modification, groups AA and AAA are deleted, that is, redundant descendant nodes are deleted. In the modified design data, there is no redundant grouping for the buttons, so the problem of button testing failing does not occur. In addition, the number of layers is reduced, and the rendering cost is also reduced. In the following explanation, examples of design data will be shown in a tree structure.

[0020] Example 2: Figure 6 shows an example of a screen created by a designer. In screen 100 of Figure 6, a hexagonal image and a button to make the hexagonal image blink are placed. Figure 7 shows an example of design data corresponding to screen 100 of Figure 6, output from a UI design tool. In screen 100 of Figure 6, the vector image that makes up the hexagon is placed as raw data. Such placement can occur, for example, when a designer imports an existing image and pastes it directly onto the screen.

[0021] In developing applications based on this type of design data, there is a problem in that it is difficult to implement features such as flashing images when a button is pressed because it is unclear how the images are divided. Furthermore, there is also the problem of difficulty in testing whether or not the images flash when a button is pressed.

[0022] Figure 8 shows an example of a screen after modification by the data structure modification device of the embodiment. Figure 9 shows an example of design data after modification by the data structure modification device of the embodiment. In this modification, multiple vector images are grouped into Group A. With the modified design data, it becomes possible to implement and test applications under the category of Group A.

[0023] Example 3: Figure 10 shows an example of a screen created by a designer. In screen 100 of Figure 10, 26 groups (Group A, Group B, Group C, ..., Group Z) are arranged. Each group also has a checkbox and the string "Task n" (where n is a natural number from 1 to 26). Figure 11 shows an example of design data corresponding to screen 100 of Figure 6, output from a UI design tool. In this design data, multiple checkboxes of the same instance are placed on the same screen.

[0024] The multiple checkboxes on screen 100 in Figure 10 are the same instance, and therefore their states are synchronized. This offers advantages during screen design, such as being able to change the color of multiple checkboxes simultaneously, but it causes problems during application development based on the design data. For example, selecting one checkbox selects all of them, which is not the expected behavior and causes problems during implementation and testing.

[0025] Figure 12 shows an example of design data modified by the data structure modification device of the embodiment. In this modification, checkbox nodes are duplicated, and checkbox nodes from different instances are associated with multiple group nodes. In the modified design data, the states of multiple checkboxes are not synchronized, resulting in the expected behavior. This improves implementation and testing in the development work of applications based on the design data.

[0026] <Detailed description of the embodiment> The details of the embodiment will now be described. Figure 13 shows the configuration of the information processing system 10 of the embodiment. The information processing system 10 comprises a design data storage device 12, a data structure modification device 14, a code generation device 16, a program code storage device 18, and a generation AI server 19. Each device constituting the information processing system 10 has a communication function and sends and receives data via a communication network. The communication network may include, for example, a LAN, WAN, or the Internet.

[0027] The design data storage device 12 stores the design data output from the UI design tool used by the designer. The data structure modification device 14 modifies the data structure of the design data and stores the modified design data in the design data storage device 12.

[0028] The code generation device 16 automatically generates program code based on the design code modified by the data structure modification device 14, which is stored in the design data storage device 12. The program code storage device 18 stores the program code generated by the code generation device 16. The generation AI server 19 can also be called a chatbot server and is an information processing device that provides conversational services using the generation AI.

[0029] Figure 14 is a block diagram showing the functional blocks of a data structure modification device 14 according to an embodiment. The various functions shown in Figure 14 are realized by the execution of a program by a computer mounted on the data structure modification device 14. The computer includes, as hardware, a memory for loading programs, one or more processors (CPUs, etc.) for executing the loaded programs, auxiliary storage devices, and other LSIs. The processor is composed of multiple electronic circuits, including semiconductor integrated circuits and LSIs, and these multiple electronic circuits may be mounted on a single chip or on multiple chips. The functional blocks shown in Figure 14 are realized through the cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by hardware alone, software alone, or a combination thereof.

[0030] The data structure modification device 14 comprises a processing unit 20, a storage unit 22, and a communication unit 24. The processing unit 20 performs information processing to modify the data structure of the design data. The storage unit 22 stores data that is referenced or updated by the processing unit 20. The communication unit 24 communicates with external devices according to a predetermined communication protocol. The processing unit 20 sends and receives data to and from the design data storage device 12 and the generation AI server 19 via the communication unit 24.

[0031] The processing unit 20 includes a design data acquisition unit 30, a modification unit 32, and a design data output unit 40. The functions of these multiple functional blocks may be implemented in a computer program. This computer program may be stored on a recording medium and installed on the storage of the data structure modification device 14 via that recording medium. Alternatively, this computer program may be downloaded via a communication network and installed on the storage of the data structure modification device 14. The processor mounted on the data structure modification device 14 may perform the functions of these multiple functional blocks by reading this computer program into main memory and executing it.

[0032] The design data acquisition unit 30 acquires design data from an external device. In this embodiment, the design data acquisition unit 30 acquires design data (for example, in JSON format) stored in the design data storage device 12.

[0033] The modification unit 32 modifies the structure of the design data acquired by the design data acquisition unit 30 to a structure suitable for creating or operating an application based on that design data, if the structure of the design data satisfies predetermined modification requirements. The above modification requirements may include ensuring that the screen design does not change before and after the modification of the design data structure.

[0034] The modification unit 32 includes a tree structuring unit 34, a node deletion unit 36, and a node grouping unit 38. The tree structuring unit 34, as shown in Example 3 above, performs a tree structuring process by duplicating child nodes with multiple parent nodes and associating each of the multiple parent nodes with child nodes of different instances. The node deletion unit 36, as shown in Example 1 above, performs a node deletion process by deleting or merging (integrating) redundant nodes. The node grouping unit 38, as shown in Example 2 above, performs a node grouping process by grouping multiple nodes.

[0035] The design data output unit 40 outputs the modified design data from the modification unit 32 to an external device. In this embodiment, the design data output unit 40 stores the modified design data (for example, in JSON format) from the modification unit 32 in the design data storage device 12.

[0036] Figure 15 is a flowchart showing the operation of the data structure modification device 14 in the embodiment. The operation of the information processing system 10 in the embodiment will be described below with reference to Figure 15.

[0037] The designer designs the application screen using a UI design tool, and the UI design tool stores the design data (layout, etc.) showing the screen design result in the design data storage device 12. The design data acquisition unit 30 of the data structure modification device 14 reads the design data stored in the design data storage device 12 (S10).

[0038] The tree structuring unit 34 of the data structure modification device 14 performs tree structuring processing on the design data (S11). The node deletion unit 36 ​​of the data structure modification device 14 performs node deletion processing on the design data (S12). The node grouping unit 38 of the data structure modification device 14 performs node grouping processing on the design data (S13). The design data output unit 40 of the data structure modification device 14 stores the design data modified in S11-S13 in the design data storage device 12 (S14).

[0039] Figure 16 is a flowchart detailing the tree structuring process in S11 of Figure 15. The tree structuring unit 34 detects target nodes if there is one root node in the acquired design data (Y in S20) and if there are multiple child nodes associated with parent nodes in that design data (also referred to here as "target nodes") (Y in S21).

[0040] The tree structuring unit 34 duplicates the detected target node and associates a different instance of the target node one-to-one with each of the multiple associated parent nodes (S22). If no target node is detected in the design data (N in S21), the process in S22 is skipped. If there are multiple root nodes in the design data (N in S20), the process returns to S15 and terminates.

[0041] Figure 17 is a flowchart detailing the node deletion process in S12 of Figure 15. The design data modification requirements here include node deletion requirements and node merging requirements. The node deletion unit 36 ​​determines whether each node in the design data satisfies the node deletion requirements. If there is a node in the design data that satisfies the node deletion requirements (Y in S30), the node deletion unit 36 ​​deletes the node from the design data (S31). If there is no node in the design data that satisfies the node deletion requirements (N in S30), the process in S31 is skipped.

[0042] The node deletion requirements must include at least one of the following requirements 1-3. Requirement 1 is that the screen design (appearance) does not change before and after deleting the node. Requirement 2 is that it can be determined that deletion is possible based on data other than design data (for example, specifications, memos, etc., also referred to as "design data related information" below). Requirement 3 is that it can be determined that the data within the node (also referred to as "properties") does not have any role related to the screen design (appearance). The node deletion requirements may be any combination of requirements 1-3.

[0043] Requirement 1 will now be explained. Here, the node to be judged is referred to as the target node. The node deletion unit 36 ​​may temporarily delete the data of the target node from the design data. The node deletion unit 36 ​​may input the design data after the target node deletion into the UI design tool and display a screen based on the design data after the target node deletion (hereinafter also referred to as the "modified screen"). The node deletion unit 36 ​​may obtain an image of the modified screen by taking a screenshot of the modified screen.

[0044] The node deletion unit 36 ​​may also input the design data before the target node was deleted (in other words, the original design data) into a UI design tool and display a screen based on the design data before the target node was deleted (hereinafter also referred to as the "pre-change screen"). The node deletion unit 36 ​​may also obtain an image of the pre-change screen by taking a screenshot of the pre-change screen.

[0045] The node deletion unit 36 ​​may compare the image of the modified screen with the image of the screen before the modification, and if they match, it may determine that requirement 1 is met. As previously described, if requirement 1 is met, the node deletion unit 36 ​​may delete the data of the target node from the design data. In other words, the node deletion unit 36 ​​may delete a node from among the multiple nodes that make up the design data, such that its deletion will not cause any change in the screen design.

[0046] Requirement 2 will now be explained. Requirement 2 may be that the generating AI has detected nodes (referred to here as "target nodes") that, if deleted, will not change the screen design, based on the relevant information of the design data. For example, the node deletion unit 36 ​​may send a prompt, design data, and relevant information of the design data to the generating AI server 19. The prompt may be an instruction to analyze the design data and its relevant information to detect target nodes that, if deleted, will not change the screen design. The node deletion unit 36 ​​may delete the data of the target nodes detected by the generating AI, as indicated by the response from the generating AI server 19, from the design data.

[0047] Requirement 3 will now be explained. Requirement 3 may also be the detection of a node that does not have a role related to the screen design (referred to here as the "target node"). For example, the storage unit 22 of the data structure modification device 14 may store predetermined keywords or key phrases of properties that have a role related to the screen design. The node deletion unit 36 ​​may detect as target nodes any nodes included in the design data that do not have properties containing the above keywords or key phrases. The node deletion unit 36 ​​may delete the data of the target nodes from the design data.

[0048] Returning to the explanation of Figure 17, the node deletion unit 36 ​​determines whether each set of nodes in the design data satisfies the node merging requirement. If there is a set of nodes in the design data that satisfies the node merging requirement (Y in S32), the node deletion unit 36 ​​modifies the design data to merge the set of nodes (S33). If there is no set of nodes in the design data that satisfies the node merging requirement (N in S32), the process in S33 is skipped.

[0049] In S33, merging a pair of nodes (in this case, the first node and the second node) can be done, for example, by adding the properties of the first node to the second node and then deleting the first node. In this case, the second node may contain both the original properties and the properties inherited from the first node.

[0050] Node merging requirements must include at least one of the following requirements 1-3: Requirement 1 is that the screen design (appearance) does not change before and after merging nodes. Requirement 2 is that it can be determined that merging is possible based on the relevant information in the design data. Requirement 3 is that it can be determined that properties within a node can be transferred to other nodes (e.g., parent or child nodes). Node merging requirements may be any combination of requirements 1-3.

[0051] Requirement 1 will now be explained. Here, the multiple nodes to be judged are referred to as the target node group. The node deletion unit 36 ​​may update the design data to provisionally merge the data of the target node group. The node deletion unit 36 ​​may input the design data after the target node group merge into the UI design tool and display the modified screen based on the design data after the target node group merge. The node deletion unit 36 ​​may obtain an image of the modified screen by taking a screenshot of the modified screen.

[0052] The node deletion unit 36 ​​may also input the design data before the target node group merge (in other words, the original design data) into a UI design tool and display a screen based on the design data before the target node merge (hereinafter also referred to as the "pre-change screen"). The node deletion unit 36 ​​may also obtain an image of the pre-change screen by taking a screenshot of the pre-change screen.

[0053] The node deletion unit 36 ​​may compare the image of the modified screen with the image of the screen before the modification, and if they match, it may determine that requirement 1 is met. As previously described, the node deletion unit 36 ​​may update the design data to merge the target node group if requirement 1 is met. That is, the node deletion unit 36 ​​may merge multiple nodes from among the multiple nodes that make up the design data, even if merged, so that the screen design does not change.

[0054] Requirement 2 will now be explained. Requirement 2 may be that the generating AI has detected multiple nodes (target node group) that, when merged, will not result in any change to the screen design, based on the relevant information of the design data. The node deletion unit 36 ​​may send a prompt, design data, and relevant information of the design data to the generating AI server 19. The prompt may be an instruction to analyze the design data and its relevant information to detect a target node group that, when merged, will not result in any change to the screen design. The node deletion unit 36 ​​may update the design data to merge the target node group detected by the generating AI, as indicated by the response from the generating AI server 19.

[0055] Requirement 3 will now be explained. Requirement 3 is that, for multiple nodes included in the design data (here referred to as the first node and the second node), even if the role related to the screen design defined by the first node is transferred to a second node that is different from the first node, the screen design should not change. The second node may be, for example, the parent node or child node of the first node.

[0056] The storage unit 22 of the data structure modification device 14 may store predetermined keywords or key phrases of properties that have a role related to the screen design. The node deletion unit 36 ​​may detect multiple nodes as a target node group if both of the parent-child related nodes in the design data contain the above keywords or key phrases. The node deletion unit 36 ​​may update the design data to merge the target node group (for example, merging child nodes into parent nodes).

[0057] Figure 18 is a flowchart detailing the node grouping process in S13 of Figure 15. The node grouping unit 38 determines whether multiple nodes in the design data satisfy the node grouping requirements. If multiple nodes satisfying the node grouping requirements exist in the design data (Y in S40), the node grouping unit 38 groups those nodes and updates the design data (S41). If multiple nodes satisfying the node grouping requirements do not exist in the design data (N in S40), the process in S41 is skipped.

[0058] As explained in relation to Figures 7 and 9, grouping multiple nodes may also mean adding a group node in the design data as a common parent node associated with those multiple nodes. The group node may also correspond to a node that groups multiple screen elements in a UI design tool.

[0059] The node grouping requirements must include at least one of the following requirements 1-4: Requirement 1 is that the screen design (appearance) does not change before and after grouping multiple nodes. Requirement 2 is that the coordinate positions of multiple nodes (i.e., the positions of corresponding elements on the screen) are close together. Requirement 3 is that multiple nodes are of the same type. Requirement 4 is that it can be determined that grouping is possible based on the related information in the design data. The node grouping requirements may be any combination of requirements 1-4.

[0060] Requirement 1 will now be explained. Here, the multiple nodes to be judged are referred to as the target node group. The node grouping unit 38 may update the design data to provisionally group the target node group. The node grouping unit 38 may input the grouped design data into the UI design tool and display the modified screen based on the grouped design data. The node grouping unit 38 may obtain an image of the modified screen by taking a screenshot of the modified screen.

[0061] The node grouping unit 38 may also input the design data of the target node group before grouping (in other words, the original design data) into a UI design tool and display the pre-change screen based on the pre-grouping design data. The node deletion unit 36 ​​may obtain an image of the pre-change screen by taking a screenshot of the pre-change screen.

[0062] The node grouping unit 38 may compare the image of the modified screen with the image of the screen before the modification, and if they match, it may determine that requirement 1 is met. As previously described, if requirement 1 is met, the node grouping unit 38 may group the target node group and update the design data. That is, the node grouping unit 38 may group multiple nodes from among the multiple nodes that make up the design data, such that grouping them will not change the screen design.

[0063] Requirement 2 will now be explained. Requirement 2 may also be that the distance between multiple nodes in the grouping target (target node group) on the screen is less than or equal to a predetermined threshold. Unless otherwise specified, the threshold in this disclosure may be determined based on the developer's knowledge or the results of experiments using the information processing system 10.

[0064] Each of the multiple nodes in the design data includes a property that defines the on-screen position information (coordinate values, etc.) of an element on the screen (e.g., a button or a vector image). The node grouping unit 38 may calculate the on-screen distance of multiple elements on the screen (e.g., the vector images of each side of the hexagon in Figure 6) that correspond to the multiple nodes in the target node group, based on the position information properties contained in the multiple nodes of the target node group. If the on-screen distance of multiple elements on the screen that correspond to the multiple nodes of the target node group is less than or equal to a predetermined threshold, the node grouping unit 38 may update the design data to group the multiple nodes of the target node group.

[0065] Requirement 3 will now be explained. Requirement 3 may also be that the multiple nodes to be grouped are of the same type. Node types include, for example, vector images, buttons, checkboxes, etc. The node grouping unit 38 may determine whether the multiple nodes of the grouping candidates are of the same type based on a property that indicates the type of node contained in each of the multiple nodes of the grouping candidates. If the multiple nodes of the grouping candidates are of the same type, the node grouping unit 38 may update the design data to group those multiple nodes.

[0066] Figure 7 shows an example of grouping related to the design code. The node grouping unit 38 may identify nodes of multiple vector images as grouping candidates because the nodes of multiple vector images are of the same type (satisfying requirement 3) and are located close together on the screen (satisfying requirement 2). The node grouping unit 38 may also group the nodes of multiple vector images and update the design data (Figure 9) because the appearance of the screen before grouping (Figure 6) and the appearance of the screen after grouping (Figure 8) match (satisfying requirement 1).

[0067] Requirement 4 will now be explained. Requirement 4 may be that the generating AI has detected multiple nodes (target node group) that, when grouped, will not result in any change to the screen design, based on the relevant information of the design data. The node grouping unit 38 may send a prompt, design data, and relevant information of the design data to the generating AI server 19. The prompt may be an instruction to analyze the design data and its relevant information to detect a target node group that, when grouped, will not result in any change to the screen design. The node grouping unit 38 may update the design data to group the target node group detected by the generating AI, as indicated by the response from the generating AI server 19.

[0068] Returning to the explanation of the operation of the information processing system 10, the code generation device 16 reads the modified design data stored in the design data storage device 12, which has been modified by the data structure modification device 14. The code generation device 16 generates program code (e.g., in Dart format) based on the modified design data and stores the generated program code in the program code storage device 18. The developer develops an application that displays the screen designed by the designer by implementing and testing the program code stored in the program code storage device 18.

[0069] According to the data structure modification device 14 of the embodiment, the design data, which is structured data related to the screen design, can be modified to a structure suitable for the creation or operation of an application, thereby streamlining the application development work using the design data. Furthermore, according to the data structure modification device 14, the screen design can be maintained while the application development work using the design data can be streamlined by not changing the screen design before and after modifying the data structure of the design data.

[0070] Furthermore, the tree structuring process of the data structure modification device 14 can resolve problems during application development caused by the existence of child nodes associated with multiple parent nodes. Additionally, the node deletion process of the data structure modification device 14 can resolve problems during application development and reduce rendering costs by deleting or merging redundant nodes. Finally, the node grouping process of the data structure modification device 14 can improve the efficiency of application development.

[0071] The present disclosure has been described above based on embodiments. The embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of components or processing processes of the embodiments, and that such modifications are also within the scope of the present disclosure.

[0072] <Example 1> The program code storage device 18 may further include a code generation unit and a code output unit, which are functional blocks corresponding to the code generation device 16. The code generation unit may receive the modified design data from the modification unit 32 and generate program code based on the modified design data. The code output unit may store the program code generated by the code generation unit in the program code storage device 18. The program code may be source code written in Dart.

[0073] For example, the storage unit 22 of the data structure modification device 14 may store data that defines the correspondence between each element of the design data and each element of the program code. The code generation unit of the program code storage device 18 may identify multiple elements of the program code that are associated with multiple elements of the modified design data, and automatically generate program code that includes the identified multiple elements.

[0074] <Modification 2> The requirements for duplicating child nodes in the tree structuring process may include ensuring that the screen design does not change before and after duplicating the child nodes. For example, the tree structuring unit 34 may generate modified design data that associates the duplicated child nodes with each of the multiple parent nodes. The tree structuring unit 34 may input the modified design data into a UI design tool to display a modified screen based on the modified design data. The tree structuring unit 34 may also obtain an image of the modified screen by taking a screenshot of the modified screen.

[0075] Alternatively, the tree structuring unit 34 may input the design data before the change (in other words, the original design data) into a UI design tool and display the pre-change screen based on the pre-change design data. The tree structuring unit 34 may also obtain an image of the pre-change screen by taking a screenshot of the pre-change screen. The tree structuring unit 34 may compare the image of the post-change screen with the image of the pre-change screen, and if they match, it may determine that the requirements for child node duplication are met and finalize the post-change design data.

[0076] Any combination of the embodiments and modifications described above is also useful as an embodiment of the present disclosure. The new embodiments resulting from such combinations will possess the combined effects of the respective embodiments and modifications. Furthermore, it will be understood by those skilled in the art that the functions to be performed by each constituent element described in the claims can be achieved by each component shown in the embodiments and modifications individually or in combination thereof. In addition, where terms such as "first," "second," etc. are used in this specification or in the claims, unless otherwise specified, they do not indicate any order or importance, but are used to distinguish one configuration from another.

[0077] <Note> Based on the above description of embodiments and modifications, the following technology is disclosed. [Technology 1] Computers We obtain structured data related to the screen design, If the structure of the structured data satisfies predetermined modification requirements, the structure of the structured data is modified to a structure suitable for creating or operating an application based on the structured data. Output the modified structured data as described above. Information processing methods. This information processing method allows for the streamlining of subsequent tasks using structured data by modifying structured data related to screen design into a structure suitable for application creation or operation. [Technology 2] The aforementioned change requirement includes ensuring that the design of the screen does not change before and after the structure of the structured data is changed. Information processing method described in Technical 1 This information processing method prevents the screen design from changing before and after modifying the structure of the structured data. [Technology 3] As part of the aforementioned modification, delete any node among the multiple nodes that constitute the structured data that, if deleted, will not change the design of the screen. Information processing method as described in Technology 1 or 2. This information processing method eliminates the problem of test failures by removing redundant nodes, and also reduces rendering costs. [Technology 4] The aforementioned change requirements include the detection by the generating AI of nodes that, based on data other than the structured data, would not result in any change to the screen design if deleted. The information processing method described in Technical 3. This information processing method allows for the efficient and accurate detection of nodes that can be deleted. [Technology 5] The aforementioned change requirements include the detection of nodes that do not have a role in the design of the screen, The information processing method described in Technical 3. This information processing method allows for the efficient and accurate detection of nodes that can be deleted. [Technology 6] As part of the aforementioned modification, among the multiple nodes that constitute the structured data, multiple nodes that, when integrated, do not result in any change to the design of the screen are integrated. Information processing method as described in Technology 1 or 2. This information processing method eliminates the problem of application tests failing by integrating redundant nodes, and also reduces rendering costs. [Technology 7] The aforementioned change requirement includes the detection by the generating AI of multiple nodes, based on data other than the structured data, that, when integrated, would not result in any change to the screen design. Information processing method as described in Technical 6. This information processing method allows for the efficient and accurate detection of nodes that can be integrated. [Technology 8] The aforementioned modification requirement includes ensuring that the design of the screen remains unchanged even if the role related to the design of the screen, as defined in the first node, is transferred to a second node different from the first node. Information processing method as described in Technical 6. This information processing method allows for the efficient and accurate detection of nodes that can be integrated. [Technology 9] As a modification, among the multiple nodes that constitute the structured data, multiple nodes that, when grouped, do not result in any change to the design of the screen are grouped together. Information processing method as described in Technology 1 or 2. This information processing method allows for more efficient application implementation or testing through node grouping, while avoiding changes to the screen design. [Technology 10] The aforementioned modification requirement includes the fact that the distance between the multiple nodes to be grouped on the screen is less than or equal to a predetermined threshold. Information processing method as described in Technical 9. This information processing method allows for the efficient and accurate detection of nodes that can be grouped. [Technology 11] The aforementioned change requirements include the fact that the multiple nodes to be grouped are of the same type. Information processing method as described in Technical 9. This information processing method allows for the efficient and accurate detection of nodes that can be grouped. [Technology 12] The aforementioned change requirement includes the detection by the generating AI of multiple nodes that, when grouped based on data other than the structured data, would not result in any change to the screen design. Information processing method as described in Technical 9. This information processing method allows for the efficient and accurate detection of nodes that can be grouped. [Technology 13] As a modification, if the structured data includes child nodes associated with multiple parent nodes, a different child node is associated with each of the multiple parent nodes. Information processing method as described in Technology 1 or 2. According to this information processing method, if there are child nodes associated with multiple parent nodes, the problem of application tests failing can be resolved by duplicating those child nodes and associating each of the multiple parent nodes with a different child node. [Technology 14] Further outputting program code based on the aforementioned structured data, Information processing method as described in Technology 1 or 2. This information processing method can improve the efficiency of implementing applications based on structured data related to screen design. [Technology 15] An acquisition unit that acquires structured data related to the screen design, If the structure of the structured data acquired by the acquisition unit satisfies predetermined modification requirements, the modification unit modifies the structure of the structured data to a structure suitable for creating or operating an application based on the structured data. An output unit that outputs the structured data after the modification by the modification unit, An information processing device equipped with the following features. According to this information processing device, structured data related to screen design can be modified to a structure suitable for application creation or operation, thereby streamlining subsequent work using said structured data. [Technology 16] On the computer, We obtain structured data related to the screen design, If the structure of the structured data satisfies predetermined modification requirements, the structure of the structured data is modified to a structure suitable for creating or operating an application based on the structured data. Output the modified structured data as described above. A computer program designed to perform a task. According to this computer program, structured data related to screen design can be restructured to a format suitable for application creation or operation, thereby streamlining subsequent tasks using that structured data. [Explanation of Symbols]

[0078] 10 Information processing system, 14 Data structure modification device, 30 Design data acquisition unit, 32 Modification unit, 34 Tree structuring unit, 36 Node deletion unit, 38 Node grouping unit, 40 Design data output unit.

Claims

1. Computers We obtain structured data related to the screen design, If the structure of the structured data satisfies predetermined modification requirements, the structure of the structured data is modified to a structure suitable for creating or operating an application based on the structured data. Output the modified structured data as described above. Information processing methods.

2. The aforementioned change requirement includes ensuring that the design of the screen does not change before and after the structure of the structured data is changed. The information processing method according to claim 1.

3. As part of the aforementioned modification, delete any node among the multiple nodes that constitute the structured data that, if deleted, will not change the design of the screen. The information processing method according to claim 1 or 2.

4. The aforementioned change requirements include the detection by the generating AI of nodes that, based on data other than the structured data, would not result in any change to the screen design if deleted. The information processing method according to claim 3.

5. The aforementioned change requirements include the detection of nodes that do not have a role in the design of the screen, The information processing method according to claim 3.

6. As part of the aforementioned modification, among the multiple nodes that constitute the structured data, multiple nodes that, when integrated, do not result in any change to the design of the screen are integrated. The information processing method according to claim 1 or 2.

7. The aforementioned change requirement includes the detection by the generating AI of multiple nodes that, when integrated, would not result in any change to the screen design, based on data other than the structured data. The information processing method according to claim 6.

8. The aforementioned modification requirement includes ensuring that the screen design remains unchanged even when the role related to the screen design defined in the first node is transferred to a second node different from the first node. The information processing method according to claim 6.

9. As a modification, among the multiple nodes that constitute the structured data, multiple nodes that, when grouped, do not result in any change to the design of the screen are grouped together. The information processing method according to claim 1 or 2.

10. The aforementioned modification requirement includes the fact that the distance between the multiple nodes to be grouped on the screen is less than or equal to a predetermined threshold. The information processing method according to claim 9.

11. The aforementioned change requirements include the fact that the multiple nodes to be grouped are of the same type. The information processing method according to claim 9.

12. The aforementioned change requirement includes the detection by the generating AI of multiple nodes that, when grouped based on data other than the structured data, do not result in any change to the screen design. The information processing method according to claim 9.

13. As a modification, if the structured data includes child nodes associated with multiple parent nodes, a different child node is associated with each of the multiple parent nodes. The information processing method according to claim 1 or 2.

14. Further outputting program code based on the aforementioned structured data, The information processing method according to claim 1 or 2.

15. An acquisition unit that acquires structured data related to the screen design, If the structure of the structured data acquired by the acquisition unit satisfies predetermined modification requirements, the modification unit modifies the structure of the structured data to a structure suitable for creating or operating an application based on the structured data. An output unit that outputs the structured data after the modification by the modification unit, An information processing device equipped with the following features.

16. On the computer, We obtain structured data related to the screen design, If the structure of the structured data satisfies predetermined modification requirements, the structure of the structured data is modified to a structure suitable for creating or operating an application based on the structured data. Output the modified structured data as described above. A computer program designed to perform a task.

Images