Electronic apparatus and managing ab test code method thereof

Abstract

This invention provides a method for managing AB test codes in an electronic device. The AB test code management method may include the following steps: identifying a first AB test code that is the object of code cleanup among one or more AB test codes included in an AB test file, and the identification information of the first AB test code; performing code cleanup by removing the identification information of the first AB test code from the first AB test code; and storing the AB test file in which the code cleanup of the first AB test code has been performed.

Description

[Technical Field]

[0001] This invention relates to an electronic device and a method for managing A / B test code. More specifically, this invention relates to an electronic device and method that automatically performs code cleanup operations on specific codes that require code cleanup upon completion of A / B testing. [Previous Technology]

[0002] A / B testing, a process of comparing control and experimental groups in a digital environment to determine the content that better represents the expected effect, is widely used in product design, development, and marketing to apply validated options. For example, during development, A / B testing can be used to determine whether to update the code to add specific functions or to maintain the existing code. After completing such A / B testing, it is necessary to clean up the code corresponding to either the control or experimental group. This is because failing to clean up code determined to be unused according to A / B testing can lead to increased dependencies between codes, resulting in increased complexity in the development and maintenance processes associated with the code. However, to perform code cleanup, relevant personnel need to verify that each piece of code has been cleaned, and depending on the situation, determine the scope of the code to be cleaned. This process is not directly related to the implementation of the expected function, but it consumes a significant amount of time, and therefore may become an obstacle to performing code cleanup quickly and efficiently. [Summary of the Invention]

[0003] [Problem to be Solved by the Invention] The disclosed embodiments aim to provide an electronic device and a method for providing information therein. More specifically, the technical problem is to provide an electronic device for quickly and efficiently performing code cleanup operations and a method for managing A / B test code therein.

[0004] The technical problem to be solved in this embodiment is not limited to the technical problem described above; other technical problems can be deduced by analogy based on the following embodiments. [Technical Means for Solving the Problem]

[0005] An embodiment of an AB test code management method implemented by an electronic device may include the following steps: identifying the first AB test code, which is the object of code cleanup, and the identification information of the first AB test code among one or more AB test codes included in the AB test file; performing code cleanup by removing the identification information of the first AB test code from the first AB test code; and storing the AB test file in which the code cleanup of the first AB test code has been performed.

[0006] According to one embodiment, one or more AB test codes may include test codes corresponding to group A and test codes corresponding to group B. The first AB test code may be the following code: based on the results of AB testing on the AB test file, the undeployed code is set as the test code corresponding to group A and the test code corresponding to group B.

[0007] According to one embodiment, the identification information of the first AB test code may include AB identification information corresponding to the first AB test code and function flag information corresponding to the first AB test code.

[0008] According to one embodiment, the code cleanup step may further include the following steps: removing the identification information of the first AB test code from each of the one or more AB test codes that includes the identification information of the first AB test code.

[0009] According to one embodiment, the code cleanup steps may include the following steps: setting up an operator corresponding to the code cleanup operation of the first AB test code; providing the identification information of the first AB test code to the terminal of the set operator; and obtaining the AB test file that has been cleaned up from the terminal of the set operator.

[0010] According to one embodiment, the code cleanup operation may include a plurality of operations, including the following operations: a code cleanup operation that performs actions in relation to an application provided by an electronic device, a performance management operation provided by the aforementioned application corresponding to the code cleanup operation, and an operation related to the creation and deployment of code.

[0011] According to one embodiment, the code cleanup steps may include the following steps: inputting the path information of the AB test file into an artificial intelligence-based model; and obtaining from the artificial intelligence-based model an AB test file in which one or more code blocks corresponding to the first AB test code have been removed.

[0012] According to one embodiment, the artificial intelligence-based model can be a machine learning model constructed in the following manner: taking a plurality of A / B test files and the test result information corresponding to each of the plurality of A / B test files as the training input data set, and taking the code block type of each of the plurality of code blocks included in each of the plurality of A / B test files as the training output data set, thereby modeling the correlation between the training input data set and the training output data set.

[0013] According to one embodiment, the step of storing the AB test file of the code cleanup of the first AB test code may include the following steps: confirming that the code cleanup has been completed based on the code cleanup progress status information of the first AB test code; changing the code cleanup progress status information of the first AB test code in a manner corresponding to the storable status based on user input; and storing the AB test file of the code cleanup of the first AB test code based on the changed code cleanup progress status information of the first AB test code.

[0014] According to one embodiment, the step of changing the code cleanup progress status information of the first AB test code in a manner corresponding to the storability state based on user input may further include the following steps: corresponding to user input, confirming whether each of one or more AB test codes includes the identification information of the first AB test code; when the identification information of the first AB test code is removed, confirming whether the AB test file that has undergone code cleanup can be stored; and when the AB test file that has undergone code cleanup can be stored, changing the code cleanup progress status information of the first AB test code in a manner corresponding to the storability state; and when an error related to the storage of the AB test file that has undergone code cleanup is discovered, providing a notification related to the storage error.

[0015] According to one embodiment, the step of changing the code cleanup progress status information of the first AB test code in a manner corresponding to the storable state based on user input may further include the following steps: corresponding to user input, confirming whether each of the more than one AB test code includes the identification information of the first AB test code; and if the identification information of the first AB test code has not been removed, performing code cleanup again.

[0016] According to one embodiment, the progress status information of code cleanup for the first AB test code can be confirmed based on a dashboard, which includes at least one AB test code undergoing code cleanup and the progress status information of code cleanup corresponding to each of the at least one AB test code.

[0017] One embodiment of a non-transitory recording medium may be a non-transitory computer-readable recording medium for recording a program useful for implementing a method carried out by an electronic device.

[0018] An embodiment of the electronic device may include: a transceiver; a memory; and a processor; and the processor is as follows: confirming the first AB test code, which is the object of code cleanup, and the identification information of the first AB test code in one or more AB test codes included in the AB test file; performing code cleanup by removing the identification information of the first AB test code from the first AB test code; and storing the AB test file in which the code cleanup of the first AB test code has been performed.

[0019] Specific details of the embodiments are included in the detailed description and drawings. [Effects of the Invention]

[0020] According to the proposed embodiments, one or more of the following effects can be expected.

[0021] According to the embodiments of this specification, the code that needs to be cleaned up is automatically explored and tracked, and the code cleanup operation is performed and managed in a timely manner, thereby improving the maintainability and scalability of A / B test code.

[0022] Furthermore, it allows operators to easily confirm the progress of code cleanup operations.

[0023] The effects of the invention are not limited to those mentioned above. Those skilled in the art can clearly understand other effects not mentioned based on the description in the claims of the invention.

Implementation Method

[0025] The terminology used in the embodiments is selected as widely used common terms as possible while considering the functions of the present invention. However, it may vary depending on the intentions or precedents of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain situations, there may be terms arbitrarily selected by the applicant. In such cases, their meanings will be described in detail in the corresponding description. Therefore, the terminology used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than simply based on the name of the term.

[0026] Throughout the specification, when a part is referred to as "including" a certain constituent element, it means that other constituent elements may be included, rather than excluded, unless otherwise stated.

[0027] The expression “at least one of a, b, and c” as recorded throughout the specification may include “a alone”, “b alone”, “c alone”, “a and b”, “a and c”, “b and c” or “a, b, and c”.

[0028] The "terminal" mentioned below can be implemented by a computer or a portable terminal, which can be connected to a server or other terminal via a network. Here, a computer includes, for example, a laptop computer, a desktop computer, or a laptop computer equipped with a web browser. A portable terminal is a wireless communication device that ensures portability and mobility, and may include, for example, IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), LTE (Long Term Evolution) and other communication terminals, such as smartphones, tablets, and all types of handheld wireless communication devices.

[0029] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily implement the invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein.

[0030] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0031] Figure 1 is a diagram illustrating an embodiment of a system for managing A / B test code.

[0032] Referring to FIG1, the system 100 for managing AB test codes can be implemented in various devices. For example, the system 100 can be implemented in electronic device 110, AB test management server 120, and operator terminal 130, and the operations of various embodiments of the present invention can be performed based on the system 100. On the other hand, the AB test code management system of various embodiments is not limited to that shown in FIG1, and can also be implemented in more devices and servers.

[0033] The electronic device 110, AB test management server 120, and operator terminal 130 included in system 100 can communicate with each other or with other external devices through the connected network 140. Network 150 may include Local Area Network (LAN), Wide Area Network (WAN), Value Added Network (VAN), Mobile Radio Communication Network, Satellite Communication Network, and combinations thereof. The network is a comprehensive data communication network that enables the constituent entities of the network shown in FIG1 to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile wireless communication network. Wireless communication may include, but is not limited to, wireless LAN (Wi-Fi), Bluetooth, Bluetooth Low Energy, Wi-Fi Direct (WFD), UWB (ultra-wideband), infrared communication (IrDA), and NFC (Near Field Communication).

[0034] Each of the electronic device 110, the AB test management server 120, and the operator terminal 130 may include a transceiver, memory, and a processor. Furthermore, each of the electronic device 110, the AB test management server 120, and the operator terminal 130 refers to a unit that processes at least one function or action, which may be implemented by hardware or software, or by a combination of hardware and software. On the other hand, in all embodiments, each of the electronic device 110, the AB test management server 120, and the operator terminal 130 may be referred to as a separate device or server, but they may be logically divided structures, and at least a portion of them may be implemented by functions separated from a single device or server.

[0035] According to one embodiment, each of the electronic device 110, the AB test management server 120, and the operator terminal 130 may include multiple computer systems or computer software implemented by a network server. For example, at least a portion of the electronic device 110, the AB test management server 120, and the operator terminal 130 may refer to computer systems and computer software that are connected to a lower-level device to accept job execution requests, perform jobs on the requests, and provide execution results. The lower-level device may communicate with other network servers via a computer network such as an intranet or the Internet. In addition, at least a portion of the electronic device 110, the AB test management server 120, and the operator terminal 130 may also be understood as a broad concept including a series of applications that can operate on the network server and various databases built on internal or connected other nodes. For example, at least a portion of the electronic device 110, the AB test management server 120, and the operator terminal 130 may be implemented using network server programs provided in various forms according to operating systems such as DOS, Windows, Linux, UNIX, or MacOS.

[0036] According to one embodiment, the electronic device 110 can identify specific codes that are the targets of code cleanup in the AB test codes included in the AB test file, and the identification information of the specific codes. Code cleanup is performed by removing the identification information from the specific codes, and the AB test file with cleaned codes is stored. During this process, the electronic device 110 can communicate with the AB test management server 120 to identify the specific codes and their information that are the targets of code cleanup. Furthermore, the electronic device 110 can provide progress status information for each code cleanup operation based on the information of each AB test code obtained from the AB test management server 120 and the test files with cleaned codes obtained from the operator terminal 130. Here, a more specific embodiment of the progress status information for each code cleanup operation will be described in detail below with reference to FIGS. 3 and 4.

[0037] On the other hand, the electronic device 110 can provide an application user with an application developed and managed based on the AB test managed by the AB test management server 120 and the various codes included in the AB test. Such an electronic device 110 may be a server, or, according to one embodiment, may include a plurality of servers. According to another embodiment, one or more servers and an external cloud server may also be configured to be connected in a manner that enables electronic communication.

[0038] According to one embodiment, the AB test management server 120 can set up a control group (hereinafter, for convenience, referred to as "Group A") and an experimental group (hereinafter, for convenience, referred to as "Group B") for AB testing, and apply different codes to each group, thereby performing tests after setting up the environment associated with AB testing. The AB test file, which is the object of the test performed therein, may include one or more AB test codes, such as test codes corresponding to Group A and test codes corresponding to Group B.

[0039] Next, the AB test management server 120 can monitor the testing status of the AB test files and collect test results. By analyzing the test results, it can determine whether to change the code associated with a specific function. The AB test management server 120 can provide various AB test information to various communication-connected devices, such as information on each AB test code in all AB test files that are objects of AB testing or may become objects, and information related to the results and status of the tests performed. For example, the AB test management server 120 can provide identification information for AB test codes. The identification information for each AB test code may include AB identification information (AB key) and feature flag information. Here, the AB identification information may be information used to identify AB test files that include the corresponding AB test code, and the feature flag information may be information on flags set for the object code so that the actions of the corresponding AB test code can be controlled or conditionally executed. The AB test code identification information mentioned above may be information included in each AB test code.

[0040] The AB test management server 120 can provide services such as outputting various information as described above in a form that can be recognized by the electronic device 110 (e.g., in the form of an API (Application Programming Interface)). According to embodiments, it may also include one server or a collection of multiple servers. The electronic device 110 can obtain information about each AB test code through the API provided by the AB test management server 120.

[0041] For example, the operator terminal 130 is a terminal used by operators. Operators can use the terminal to obtain information about the code to be cleaned from the electronic device 110 via the network 140, and perform code cleanup operations based on the obtained information. Operators using the operator terminal may include, for example, developers associated with applications provided based on the electronic device 110 and the AB test management server 120, operators who perform quality management of the developed application's actions, and operators who manage the software development platform. For example, the code cleanup operations performed by operators can be performed in a code repository, which is a collection of all code used during the code development process. Operators can use the terminal to manage such a code repository and perform operations based on a system such as Git Hub that reflects changes. Upon completion of the operation, the electronic device 110 can obtain at least one of the operation result information and AB test files from the operator terminal 130.

[0042] Figure 2 is a block diagram showing an electronic device according to an embodiment.

[0043] According to one embodiment, the electronic device 110 may include a processor 210, a memory 220, and a transceiver 230. The electronic device 110 shown in FIG2 only illustrates the components related to this embodiment. Therefore, those skilled in the art related to this embodiment will understand that, in addition to the components shown in FIG2, other common components may be included.

[0044] The processor 210 can control the overall operation of the electronic device 110 and process data and signals. The processor 210 may include at least one hardware unit. Furthermore, the processor 210 can operate by executing one or more software modules generated by executing program code stored in memory 220. The processor 210 may include memory, and the processor 210 can execute program code stored in memory to control the overall operation of the electronic device 110 and process data and signals.

[0045] The processor 210 can be implemented by a computer or similar device, depending on hardware, software, or a combination thereof. In terms of hardware, the processor 210 can be implemented as an electronic circuit that processes electrical signals to perform control functions; in terms of software, it can be implemented as a program that drives the hardware processor 210. On the other hand, unless specifically mentioned in the following description, the operation of the electronic device can be interpreted as being performed by the control of the processor 210. That is, in the case of executing a module implemented in a system for managing AB test code, the module can be interpreted as the processor 210 controlling the electronic device to perform the operations described below.

[0046] Memory 220 can store various types of information. Memory 220 can store data temporarily or semi-permanently. For example, the memory 220 of electronic device 110 can store the operating system (OS) used to drive electronic device 110, data used to manage websites, programs used to generate Braille, and data associated with applications (e.g., website applications). Furthermore, memory 220 can store modules in the form of computer code as described above.

[0047] As an example of memory 220, it may include a hard disk drive (HDD), a solid state drive (SSD), flash memory, read-only memory (ROM), random access memory (RAM), etc. This type of memory 220 can be provided as either built-in or removable.

[0048] The transceiver 230 is a device for performing wired / wireless communication and can communicate with external electronic devices. The external electronic devices can be terminals or servers. The communication technologies used by the transceiver 230 may include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0049] Figure 3 is a flowchart of an AB test code management method according to an embodiment.

[0050] In step S310, the electronic device 110 can confirm the first AB test code, which is the target of code cleanup, and the identification information of the first AB test code among one or more AB test codes included in the AB test file. The first AB test code, which is the target of code cleanup, may be, for example, the following code: based on the results of AB testing on the AB test file, the undeployed code is set as the test code corresponding to group A and the test code corresponding to group B. More specifically, the electronic device 110 can confirm the following code as the first AB test code based on the information of each test code obtained from the AB test management server 120: the code is confirmed as the one whose test result information corresponds to FNL (feature not launched). In addition, the electronic device 110 can also confirm the AB identification information and function flag information, which are the identification information of the test code confirmed as the first AB test code.

[0051] In step S320, the electronic device 110 can perform code cleanup by removing the identification information of the first AB test code from the first AB test code. For example, the electronic device 110 can remove the identification information of the first AB test code from each of the more than one AB test codes that includes the identification information of the first AB test code. The function implemented by the first AB test code may be at least a part of the function to be implemented by other test codes. In this case, the AB identification information and function flag information of the first AB test code may constitute at least a part of the other test codes included in the AB test file. Therefore, the electronic device 110 can not only remove the identification information of the first AB test code from the first AB test code, but also remove the identification information of the first AB test code from each of the other AB test codes that includes the identification information of the first AB test code, thereby performing code cleanup. On the other hand, according to one embodiment, the electronic device 110 can perform code cleaning operations by setting up operators corresponding to the code cleaning operation. According to another embodiment, the code cleaning operation can also be performed based on a trained artificial intelligence-based model. These various embodiments will be described in more detail below in Figures 5 and 6.

[0052] In step S330, the electronic device 110 can store the AB test file containing the code cleanup of the first AB test code. Here, storing the AB test file may refer to the archiving of historical data related to the test results of the AB test file and the execution of the code cleanup operation, which can be archived on the AB test management server 120. This allows for reference when developing new code or designing and implementing AB tests, and can be effectively used for development improvements.

[0053] To store the AB test files that have undergone code cleanup, the electronic device 110 can, for example, confirm the progress status information of the code cleanup for the first AB test code. Specifically, the progress status information of the code cleanup for the first AB test code can be confirmed based on a dashboard, which includes at least one AB test code undergoing code cleanup and the progress status information of the code cleanup corresponding to each of the at least one AB test code. That is, the dashboard can include the progress status information of each code cleanup operation. Hereinafter, a more specific embodiment of the dashboard will be described in detail with reference to FIG7.

[0054] The progress status information for each code cleanup operation may include, for example, status information corresponding to at least one of the following states: the status of confirming the type of code to be cleaned, the status of preparing for the code cleanup operation, the status of performing the code cleanup operation, the status of completing the code cleanup operation and thus completing the cleanup of the target code, the status of preparing to store the cleaned code, and the status of storing the cleaned code. The progress status information for each code cleanup operation is not limited to the above examples. According to the embodiments, the progress status of each code cleanup operation can be intuitively confirmed, or the progress status can be divided into forms more appropriately according to the order.

[0055] The electronic device 110 can confirm whether the code cleanup of the first AB test code is complete based on the code cleanup progress status information of the first AB test code. When it is confirmed that the code cleanup of the first AB test code is complete, the electronic device 110 can change the code cleanup progress status information of the first AB test code in a manner corresponding to a storable state. The change of state as described above can be implemented, for example, based on user input.

[0056] As a more specific example, the electronic device 110 can confirm, based on user input corresponding to the status change, whether each of the more than one AB test codes includes the identification information of the first AB test code. That is, even when the status information indicates that the code cleanup of the first AB test code has been completed, the electronic device 110 can further confirm whether the AB identification information and function flag information of the first AB test code have been completely removed from each of the AB test codes included in the AB test file. If it is confirmed that the identification information of the first AB test code has not been completely removed from each AB test code, the electronic device 110 can perform code cleanup again. That is, steps S320 and S330 can be repeated until the code cleanup operation is completely completed. In this way, the status information of each code cleanup operation provided by the electronic device 110 can display more accurate information, ensuring that the complete code cleanup operation is performed.

[0057] When the identification information of the first AB test code is confirmed by the electronic device 110 to have been completely removed from all AB test codes, the electronic device 110 can determine whether the AB test file that has undergone code cleanup can be stored. In order to minimize data loss and store the AB test file that has undergone code cleanup more securely, it is necessary to terminate the traffic in the AB test management server 120 associated with the AB test file. That is, it is necessary to ensure that the relevant traffic and data requests are completed before the AB test file that has undergone code cleanup is archived, so as not to affect the user experience of the application provided by the electronic device 110. Therefore, the electronic device 110 can determine the error related to the storage of the AB test file that has undergone code cleanup when the traffic has not terminated, and determine that the AB test file that has undergone code cleanup can be stored when the traffic has terminated.

[0058] Accordingly, when it is confirmed that an AB test file with cleaned code can be stored, the electronic device 110 can change the code cleanup progress status information of the first AB test code in a manner that corresponds to the storage status indicating that code cleanup is complete. When it is confirmed that an error related to the storage of the AB test file with cleaned code has been found, the electronic device 110 can provide a notification related to the storage error. The notification related to the storage error may be, for example, a notification indicating the specific content and cause of the storage error. In one example, the code cleanup progress status information of the first AB test code is maintained in a state indicating that code cleanup is complete, but it can also be provided in a form that displays error-related information near the status information. Based on the code cleanup progress status information of the first AB test code changed as described above, the electronic device 110 can perform the following actions when the corresponding test file can be stored: store the AB test file with cleaned code of the first AB test code, or confirm the storage error and resolve the error. The electronic device 110, which stores the AB test files for code cleanup of the first AB test code, can change the progress status of the code cleanup operation corresponding to the status of the stored code, thereby indicating that the corresponding code cleanup operation and the historical storage of the final code cleanup operation data have been completed.

[0059] Figure 4 shows the code cleanup operation process in the code cleanup operation status of an embodiment.

[0060] Referring to Figure 4, symbols 410 to 460 in the figure are examples of the status information of each of the above-mentioned code cleanup operations. The electronic device 110 can display the status information as described above for each code cleanup operation. REC 410 is short for Ready for Eligibility Check, which corresponds to the status of confirming what kind of code the cleanup target code is. RFC 420 is short for Ready for Clean, which corresponds to the status of the code cleanup operation having started preparation. FUC 430 is short for Feature Under Clean, which corresponds to the status of the code cleanup operation being performed. RFD 440 is short for Ready for Deployment, which corresponds to the status of the code cleanup operation ending and the code cleanup of the target code being completed. RFA 450 is short for Ready for Archive, which corresponds to the status of the code cleanup completed and the code ready for storage. ABA 460 is short for AB Archive, which corresponds to the status of the code cleanup completed and the code being stored.

[0061] In REC 410 state, electronic device 110 performs code scanning (step S401). Here, code scanning corresponds to the AB test result FNL, and may be a scanning operation used to identify codes with code annotations or flags as target codes for code cleanup. Code scanning may be performed based on AB test code information provided in the AB test management server 120. Electronic device 110, which identifies target codes through code scanning, can identify the AB identification information and function flag information of the target codes.

[0062] In the RFC 420 state after the code scan is completed, the electronic device 110 can set the code cleanup operator (step S402) or input the path information of the AB test file into the artificial intelligence-based model (step S403). For multiple code cleanup operations, only one step in steps S402 and S403 can be performed, or each code cleanup operation can be set up to correspond to any step in steps S402 or S403, thereby performing them simultaneously or in parallel. In the FUC 430 state, the code cleanup operation can be performed after step S402 and / or step S403 (step S404).

[0063] In RFD 440 state, electronic device 110 can merge the AB test files that have completed the code cleanup operation into the main branch (step S405). The merge operation can be performed by merging the branch corresponding to the AB test files that have undergone code changes into the main branch, so that the system managing the code repository reflects the content of the code cleanup operation in the existing code repository, thereby implementing version management and deployment of updated code for the entire code repository. According to the embodiment, since electronic device 110 can only confirm that the code cleanup operation is substantially completed after the merge operation described above is completed and before the corresponding AB test files are deployed, it can be set in a way that subdivides the progress status information of the code cleanup operation based on whether the merge operation is completed.

[0064] In the RFA 450 state, the electronic device 110 can confirm whether the AB test file of the code cleanup operation is in an archiveable state (step S406). If it is archiveable, after archiving, the progress status of the code cleanup operation is changed to ABA 460. If it is not archiveable, a notification related to the reason for not being archiveable can be provided without performing a status change.

[0065] Figure 5 illustrates an embodiment of the process of obtaining an A / B test file with cleaned code from an operator's terminal.

[0066] In step S510, the electronic device 110 may assign personnel to the code cleanup operation corresponding to the first AB test code. The code cleanup operation for the test code may include a plurality of operations, therefore the electronic device 110 may assign personnel to each of the more than one operation included in the code cleanup operation for the first AB test code. The plurality of operations may include, for example, code cleanup operations performed in relation to an application provided by the electronic device 110, performance management operations provided by the application corresponding to the code cleanup operation, and operations related to code creation and deployment.

[0067] As a more specific example, the code cleanup operation related to the application may be an operation that substantially removes the AB identification information and functional flags of the first AB test code from the codebase. The performance management operation provided by the application corresponding to the code cleanup operation may be an operation that verifies whether the application performs the operation correctly after the code is cleaned. The operation related to the creation and deployment of the code may be an operation that removes or updates the code-related settings cleaned in such a way as correctly handling the creation and deployment process related to the AB test file including the cleaned code after the code is cleaned. According to the example of the application provided by the electronic device 110 based on the iOS operating system, the electronic device 110 may set up iOS operators corresponding to the code cleanup operation related to the application, QA (Quality Assurance) operators corresponding to the performance management operation of the application corresponding to the code cleanup operation, and SDP (Software Development Platform) operators corresponding to the operation related to the creation and deployment of the code to perform code cleanup of the first AB test code.

[0068] In an A / B test file, there may be one or more A / B test codes that are interdependent. Test code that implements a specific function may serve as a logical prerequisite for implementing other functions or may be included in the implementation process. Therefore, mechanically performing code cleanup may lead to significant errors when providing and developing applications. Therefore, code cleanup can be subdivided or specific personnel can be assigned to perform code cleanup for specific code to prevent such problems in advance.

[0069] In step S520, the electronic device 110 can provide the identification information of the first AB test code to the terminal of the designated operator. That is, the electronic device 110 can provide the AB identification information and function flag information of the code that is the target of the code cleanup operation to each operator's terminal. Each operator can use the terminal to perform the corresponding operation based on the provided information.

[0070] In step S530, the electronic device 110 can obtain the AB test file of the code cleanup operation from the terminal of the designated operator. Based on the obtained AB test file, the electronic device 110 can confirm and change the progress status information of the corresponding code cleanup operation.

[0071] Figure 6 illustrates an embodiment of the process of obtaining an A / B test file with cleaned code from an artificial intelligence-based model.

[0072] In step S610, the electronic device 110 can input the path information of the AB test files into the artificial intelligence-based model. Here, the artificial intelligence-based model can be a machine learning model constructed in the following way: using a plurality of AB test files and the test result information corresponding to each of the plurality of AB test files as the training input data set, and using the code block type of each of the plurality of code blocks included in each of the plurality of AB test files as the training output data set, thereby modeling the correlation between the training input data set and the training output data set. Here, the test result information corresponding to each of the plurality of AB test files included in the training input data set may include information related to: whether the test result has been completely completed, and whether the corresponding test result corresponds to True (correct) indicating that the group A has been maintained, or whether it corresponds to False (incorrect) indicating that the group B has been changed. On the other hand, in this invention, a code block can refer to a set of more than one code used to implement one function within an AB test file. The code block type of each of the plurality of code blocks included in the training output dataset can include type A and type B. Here, type A can be a type indicating that each code block corresponds to group A, and type B can be a type indicating that each code block corresponds to group B.

[0073] In this invention, deep learning, as a process of training a neural network model using experience in processing datasets, allows software to independently improve its data processing capabilities. An AI-based model is a model generated by modeling the correlations between datasets, which can be expressed by a plurality of parameters. An AI-based model can derive the correlations between datasets by extracting and analyzing features from its datasets, and can perform deep learning to optimize the parameters of the AI-based model by repeatedly performing this process. Specifically, an AI-based model can learn the correlations between inputs and outputs for datasets with input-output pairs using a deep learning algorithm. Alternatively, an AI-based model can also derive regularities between data and learn their relationships when only input data is available. Here, the deep learning algorithm can be one of deep neural networks, recurrent neural networks, convolutional neural networks, machine learning models for classification-regression analysis, or reinforcement learning models.

[0074] The electronic device 110 can model the correlation between the training input dataset and the training output dataset according to the deep learning algorithm described above, thereby constructing an artificial intelligence model. That is, the electronic device 110 can input the path information of the AB test file including the first AB test code into the artificial intelligence-based model, thereby obtaining the test file output from the model. The obtained test file can be an AB test file in which one or more code blocks corresponding to the first AB test code have been removed. For example, when the first AB test code corresponds to group A and the result obtained from testing the AB test file corresponds to false (error), the electronic device 110 can remove all code blocks of type A in the AB test file including the first AB test code. When the first AB test code corresponds to group B and the result obtained from testing the AB test file corresponds to true (correct), the electronic device 110 can remove all code blocks of type B in the AB test file including the first AB test code.

[0075] On the other hand, the path information of the AB test file may include the path information of one or more sources that use the functions associated with the corresponding AB test file. Therefore, an AI-based model can identify the AB test files in each source that require code cleanup based on the path information, and output the AB test files with the code blocks that need to be removed removed.

[0076] In step S620, the electronic device 110 can obtain from the artificial intelligence-based model an AB test file showing that one or more code blocks corresponding to the first AB test code have been removed. Based on the AB test file obtained from the artificial intelligence-based model, the electronic device 110 can confirm and change the progress status information of the corresponding code cleanup operation.

[0077] Figure 7 shows the progress status information of each code cleanup operation in an embodiment.

[0078] Referring to Figure 7, the progress status information of each code cleanup operation can be confirmed based on the dashboard 710. The dashboard 710 can display the progress status of the code cleanup operations handled by each operator. Therefore, the information of the 28 code cleanup operations handled by operator 1 can be confirmed based on the dashboard 710, wherein operations corresponding to the FUC702 status and operations corresponding to the RFA703 status are respectively categorized and displayed. Furthermore, the electronic device 110 can provide, through the dashboard 710, AB identification information (A / B key) 730, function flag information (FF ID) 740, AB cleanup operation start date 750, and archiving error message 760 corresponding to each code cleanup operation in each status in a form that is easy to recognize at a glance. The archiving error message 760 can refer to information related to the reason for not being able to archive, for example, it can provide information such as errors caused by traffic.

[0079] The various embodiments described above can be implemented in various ways. For example, the various embodiments can be implemented by hardware, firmware, software, or a combination thereof.

[0080] When implemented by hardware, the methods of various embodiments can be implemented by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0081] When implemented using firmware or software, the methods of various embodiments can be implemented in the form of modules, processes, or functions that perform the functions or actions described above. For example, software code can be stored in memory and driven by a processor. The memory can be located inside or outside the processor and can send and receive data with the processor in various known ways.

[0082] The electronic device of the above embodiments may include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with external devices, and a user interface device such as a touch panel, key, or button. Computer-readable code or program commands that can be executed in the processor, implemented by a software module or algorithm, can be stored in a computer-readable recording medium. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.). The computer-readable recording medium is distributed across a network-connected computer system, thereby enabling distributed storage and execution of computer-readable code. The medium can be read by a computer, stored in memory, and executed in a processor.

[0083] This embodiment can be represented by functional blocks and various processing steps. These functional blocks can be implemented by different numbers of hardware and / or software components performing specific functions. For example, the embodiment can employ integrated circuits capable of performing various functions via the control of one or more microprocessors or other control devices, such as memory, processing, logic, lookup tables, etc. The constituent elements can be executed by software programming or software components. Similarly, this embodiment includes algorithms implemented by combinations of data structures, programs, conventions, or other programming structures, and therefore can be implemented using programming or scripting languages ​​such as C, C++, Java, and assemblers. Functionally, it can be implemented by algorithms executed in one or more processors. Furthermore, this embodiment can employ prior art for electronic environment setup, signal processing, and / or data processing. Terms such as "mechanism," "component," "mechanism," and "structure" are used broadly and are not limited to mechanical and physical structures. The above terms may be associated with processors and include the meaning of a series of software processes.

[0084] The above embodiment is only an example, and other embodiments can be implemented within the scope of the technical solutions described below. [Simplified Explanation of the Diagram]

[0024] Figure 1 is a diagram illustrating a system for managing A / B test code according to an embodiment. Figure 2 is a block diagram showing an electronic device according to an embodiment. Figure 3 is a flowchart showing an A / B test code management method according to an embodiment. Figure 4 shows the code cleanup operation execution process corresponding to the code cleanup operation status according to an embodiment. Figure 5 shows the process of obtaining A / B test files with cleaned code from the operator's terminal according to an embodiment. Figure 6 shows the process of obtaining A / B test files with cleaned code from an artificial intelligence-based model according to an embodiment. Figure 7 shows the progress status information of each code cleanup operation according to an embodiment.

Claims

1. An A / B test code management method, implemented via an electronic device, comprising the following steps: identifying a first A / B test code, which is the object of code cleanup, among one or more A / B test codes included in an A / B test file, and the identification information of the first A / B test code; performing code cleanup by removing the identification information of the first A / B test code from the first A / B test code; and storing the A / B test file for which the code cleanup of the first A / B test code has been performed, wherein the step of performing the code cleanup further includes: Remove the identification information of the first AB test code from each of the above-mentioned AB test codes that includes the identification information of the first AB test code.

2. As in the AB test code management method of request item 1, the above-mentioned one or more AB test codes include test codes corresponding to group A and test codes corresponding to group B. The above-mentioned first AB test code is the following code: Based on the results obtained from performing AB tests on the above-mentioned AB test files, the undeployed code is set as the test code corresponding to group A and the test code corresponding to group B.

3. The AB test code management method as described in Request 1, wherein the identification information of the first AB test code includes AB identification information corresponding to the first AB test code and function flag information corresponding to the first AB test code.

4. The AB test code management method as described in Request 1, wherein the steps for performing the above code cleanup include the following steps: setting up an operator corresponding to the code cleanup operation of the first AB test code; providing the identification information of the first AB test code to the terminal of the set operator; and obtaining the AB test file that has undergone the above code cleanup from the terminal of the set operator.

5. The AB test code management method as described in Request 4, wherein the code cleanup operation includes a plurality of operations, the plurality of operations including the following: code cleanup operations that perform actions related to the application provided by the electronic device, performance management operations provided by the application corresponding to the code cleanup operations, and operations related to the creation and deployment of code.

6. The AB test code management method as requested in item 1, wherein the steps for performing the above code cleanup include the following steps: inputting the path information of the above AB test file into the artificial intelligence-based model; and obtaining from the above artificial intelligence-based model the AB test file in which one or more code blocks corresponding to the above first AB test code have been removed.

7. As in Request 6, the A / B test code management method, wherein the aforementioned artificial intelligence-based model is a machine learning model constructed in the following manner: using a plurality of A / B test files and the test result information corresponding to each of the plurality of A / B test files as the training input data set, and using the code block type of each of the plurality of code blocks included in each of the plurality of A / B test files as the training output data set, thereby modeling the correlation between the aforementioned training input data set and the aforementioned training output data set.

8. The AB test code management method of request item 1, wherein the step of storing the AB test file for which the code cleanup of the first AB test code has been performed includes the following steps: Based on the progress status information of the code cleanup of the first AB test code, confirm that the code cleanup has been completed; Based on user input, change the progress status information of the code cleanup of the first AB test code in a manner corresponding to the storable status; and Based on the changed progress status information of the code cleanup of the first AB test code, store the AB test file for which the code cleanup of the first AB test code has been performed.

9. The AB test code management method of request item 8, wherein the step of changing the code cleanup progress status information of the first AB test code in a manner corresponding to the storability state based on the above user input further includes the following steps: Based on the above user input, confirming whether each of the above one or more AB test codes includes the identification information of the first AB test code; If the identification information of the first AB test code is removed, confirming whether the AB test file that has undergone the above code cleanup can be stored; and if the AB test file that has undergone the above code cleanup can be stored, changing the code cleanup progress status information of the first AB test code in a manner corresponding to the storability state; and if an error related to the storage of the AB test file that has undergone the above code cleanup is discovered, providing a notification related to the storage error.

10. The AB test code management method of request item 9 further includes the following steps: If the identification information of the first AB test code is not removed, perform the above code cleanup again.

11. The AB test code management method as described in request item 8, wherein the code cleanup status information of the first AB test code is confirmed based on a dashboard, which includes at least one AB test code undergoing code cleanup and the code cleanup status information corresponding to each of the at least one AB test code.

12. A non-transitory computer-readable recording medium recording a program for performing the method as described in claim 1 on a computer.

13. An electronic device comprising: transceiver; The memory; and the processor; wherein the processor is configured to: identify the first AB test code, which is the object of code cleanup, and the identification information of the first AB test code in one or more AB test codes included in the AB test file; perform code cleanup by removing the identification information of the first AB test code from the first AB test code; and store the AB test file in which the code cleanup of the first AB test code has been performed, wherein the processor performs code cleanup by removing the identification information of the first AB test code from each of the one or more AB test codes that includes the identification information of the first AB test code.

Images