Method for conducting test across domains and electronic apparatus therefor

Abstract

This invention discloses a method for performing tests across a plurality of domains and an electronic device for using the method. One embodiment of the method may include the following steps: setting a target feature to be tested; when a user terminal enters a page of one of the plurality of domains, determining whether the page meets the prerequisites for applying the target feature; if the page meets the prerequisites, providing the user terminal with a page including the target feature, wherein the target feature is one of a plurality of options applicable to the target feature; and managing a log corresponding to input made by the user terminal on the page.

Description

Technical Field

[0001] This invention relates to a technique for performing tests across multiple domains, such as multiple pages providing different functions across an e-commerce platform. Prior Technology

[0002] A / B testing is an operation that compares two or more versions to evaluate which version performs better. It is used to analyze user reactions in various services to find the best option. As service developers, it is natural to conduct A / B testing before launching a service to pre-design services with higher user preferences. This is even more important in e-commerce services because understanding which interface designs users prefer and which activity statements elicit positive responses and lead to purchases directly affects the service's profitability.

[0003] However, this type of service is not only provided through various electronic devices, but even within a single device, a large number of service-related pages are transformed based on user input. Nevertheless, existing A / B testing is still limited to a single electronic device, such as a "single domain" of a single page. Therefore, there are problems such as: it is difficult to determine which version provides a superior user experience throughout the entire service, and how the user's service utilization path changes with each version.

[0004] For related information, see previous references such as KR 2024-0143984 A and KR 2653375 B1. Summary of the Invention

[0005] [The problem the invention aims to solve] The present invention is proposed to solve the above-mentioned problems and aims to provide a methodology that allows meaningful test results to be extracted when testing (especially A / B testing) is performed across multiple domains, despite the differences between the domains.

[0006] The technical problem to be achieved by the present invention is not limited to the technical problem described above, and other technical problems can be deduced from the following embodiments. [Technical means to solve the problem]

[0007] According to the present invention, a method for an electronic device to perform inter-domain testing may include the following steps: setting a target feature to be tested; when a user terminal enters a page of one of the multiple domains, determining whether the page meets the prerequisites for applying the target feature; when the page meets the prerequisites, providing the page including the target feature to the user terminal, wherein the target feature is an option among a plurality of options applicable to the target feature; and managing logs corresponding to the inputs made by the user terminal on the page.

[0008] According to one embodiment, the pages of each of the aforementioned plurality of domains, as pages jointly provided by an item sales service application, may include at least a portion of a Search Result Page (SRP), a Single Detail Page (SDP), a Cart Page, and a Product List Page (PLP).

[0009] According to one embodiment, the aforementioned prerequisites may include at least a portion of the following: user attributes, which distinguish whether a user account connected to the user terminal is a member; the operating system of the user terminal; and item attributes, which distinguish items to which the aforementioned target features can be applied.

[0010] According to one embodiment, the aforementioned prerequisites and the aforementioned option applied to the aforementioned target features can be set identically for all domain pages accessed by the user terminal during the testing process.

[0011] According to one embodiment, the same log identifier can be assigned to the logs recorded in all domain pages accessed by the user terminal during the testing process.

[0012] On the other hand, in the process of managing the above-mentioned logs, not only can the above-mentioned electronic device generate logs, but the above-mentioned electronic device can also collect the generated logs from external devices.

[0013] According to one embodiment, the steps for managing the aforementioned logs may include the following steps: generating real-time logs corresponding to the interaction between the aforementioned page and the aforementioned user terminal. Furthermore, it may include the following steps: receiving auxiliary logs from the aforementioned user terminal to confirm the validity of the aforementioned real-time logs; and deleting real-time logs from the generated real-time logs that were not confirmed as valid by the auxiliary logs. Relatedly, the auxiliary logs may be generated by the aforementioned user terminal when the aforementioned target feature is displayed on the screen of the aforementioned user terminal as the user terminal scrolls through the aforementioned page. Furthermore, the auxiliary logs may also include: identifiers assigned to each test group to which the user account connected to the aforementioned user terminal belongs; and identifiers for the aforementioned user accounts.

[0014] According to another embodiment, the steps of managing the above-mentioned logs may include the following steps: when the above-mentioned target feature is displayed on the screen of the user terminal as the user terminal scrolls on the above-mentioned page, receive real-time logs from the user terminal corresponding to the interaction between the page and the user terminal.

[0015] According to another embodiment, the step of managing the log may include the following step: generating a web page log when a user account connected to the user terminal selects an item with the target characteristics on the page. In this case, the web page log may include at least a portion of the identifier of the item selected by the user account and the identifier of the user account.

[0016] According to one embodiment, the steps of providing the above-mentioned page may include the following steps: assigning the user terminal to one of the test groups classified according to the plurality of options; and applying the option corresponding to the group to which the user terminal is assigned to the target feature. Specifically, the assignment step may include the following steps: obtaining a hash value of the identifier of the user account connected to the user terminal based on a hash function; and assigning the user terminal based on the hash value. More specifically, the step of assigning the user terminal based on the hash value may also include the following steps: determining the order of the user accounts among a specific plurality of user accounts based on the order corresponding to the hash value; and assigning the user terminal to the group in the test group corresponding to the specific order.

[0017] According to one embodiment, the above-described method for conducting tests may further include the following steps: providing a dashboard representing the results of the above tests to a user terminal based on the above-described logs; and the dashboard may include: the purchase conversion rate before importing the above-described target feature; the purchase conversion rate after importing the above-described target feature; and the rate of change of the purchase conversion rate before and after importing the above-described target feature.

[0018] According to the present invention, an electronic device for performing tests between a plurality of domains may include a memory for storing commands and a processor, wherein the processor is connected to the memory to set a target feature to be tested; when a user terminal enters a page of one of the plurality of domains, it determines whether the page meets the prerequisites for applying the target feature; when the page meets the prerequisites, it provides the page including the target feature to the user terminal, wherein the target feature is one of a plurality of options applicable to the target feature, and manages a log corresponding to the input made by the user terminal on the page.

[0019] Specific details of other embodiments are included in the detailed description and drawings. [Effects of the Invention]

[0020] According to the present invention, when a user terminal enters a new page during a domain transition, it controls whether a target feature can be applied to the new page, which options can be applied to the target feature, and manages logs corresponding to the user terminal's input on the page. In this way, the impact of changes in target features on users can be understood losslessly across multiple different service domains.

[0021] Furthermore, according to the present invention, it is confirmed whether the target feature has been displayed on the screen of the user terminal in the page entered by the user terminal, and the log is managed accordingly. In this way, the situation where the target feature is not displayed can be considered when analyzing the test results.

[0022] Furthermore, according to the present invention, test groups are divided according to multiple options to be applied to the target features, and multiple user terminals are equally allocated to each group, thereby adjusting the number of the parent group to be tested so as not to favor a certain option.

[0023] The effects of the invention are not limited to those mentioned above. Those skilled in the art can clearly understand other effects not mentioned based on the description in the patent application. Simple Explanation of the Diagram

[0024] Figure 1 is a diagram of a system for implementing testing according to an embodiment. Figure 2 is a block diagram illustrating an embodiment of an electronic device. Figure 3 is an illustrative diagram showing the process of an electronic device testing an application. Figure 4 is an illustrative diagram illustrating the process of determining whether to display a target feature on the screen of a user terminal. Figure 5 is an illustrative diagram showing the dashboard screen displaying the test results. Figure 6 is a flowchart illustrating a method for conducting a test according to an embodiment. Implementation

[0025] The specific embodiments will now be described with reference to the drawings. The detailed description below is provided to aid in a comprehensive understanding of the methods, apparatus, and / or systems described in this specification. However, these are merely examples, and the disclosed embodiments are not limited thereto.

[0026] In describing the embodiments, detailed descriptions of relevant well-known technologies will be omitted where it is determined that such detailed descriptions might unnecessarily obscure the essence of the disclosed embodiments. Furthermore, the terminology used below is defined in consideration of the functionality in the disclosed embodiments, and such terminology may vary depending on the user's, operator's, or customary practices. Therefore, its definition should be based on the overall content of this specification. The terminology used in the detailed description is for the purpose of describing embodiments only and is not restrictive. Unless otherwise expressly stated, singular expressions include the meaning of plural expressions. In this specification, expressions such as "comprising" or "possessing" are used to refer to certain features, numbers, steps, actions, elements, or a portion or combination thereof, and should not be construed as excluding the presence or possibility of one or more other features, numbers, steps, actions, elements, or a portion or combination thereof other than those described.

[0027] The terminology used in the embodiments is selected to be as widely used as possible while considering the functions of this invention, but may vary depending on the intent of those skilled in the art, precedents, or the emergence of new technologies. Furthermore, in certain situations, there may be terms arbitrarily chosen by the applicant; in such cases, their meanings will be described in detail in the corresponding explanatory section. Therefore, the terminology used in this invention should be defined based on its meaning and the overall content of the invention, rather than simply based on the name of the term.

[0028] Throughout this specification, when a part is mentioned as "including" a certain component, it means that, unless otherwise stated, other components may also be included, and other components are not excluded. Furthermore, terms such as "...part" and "...module" used in the specification refer to a unit that processes at least one function or action. This unit may be implemented by hardware or software, or by a combination of hardware and software. Unlike the example shown, it may not explicitly distinguish specific actions.

[0029] The expression "at least one of a, b, and c" as used throughout the specification may include "a alone", "b alone", "c alone", "a and b", "a and c", "b and c", or "all three of a, b, and c".

[0030] The term "terminal" as used below can be implemented as a computer or a portable terminal, which can connect to a server or other terminal via a network. Here, a computer may include, for example, a laptop, desktop computer, or laptop computer equipped with a web browser. A portable terminal is a wireless communication device that ensures portability and mobility, and may include communication terminals such as IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), and LTE (Long Term Evolution), as well as all types of handheld wireless communication devices such as smartphones and tablets.

[0031] In the following description, the terms “transmission,” “communication,” “send,” “receive,” and other terms with similar meanings for signal or information include not only the direct transmission of signal or information from one component to another, but also the transmission via another component.

[0032] In particular, "transmitting" or "sending" a signal or information to a constituent element indicates the final destination of the signal or information, not its direct destination. The same applies when "receiving" a signal or information. Furthermore, in this specification, "related" to two or more pieces of data or information means that if one piece of data (or information) is obtained, at least a portion of the other data (or information) can be obtained based on that data (or information).

[0033] Furthermore, terms such as "first" and "second" can be used to describe various constituent elements, but the constituent elements themselves should not be limited by these terms. These terms can be used to distinguish one constituent element from other constituent elements.

[0034] For example, as long as it does not depart from the scope of the invention application patent, the first constituent element can be named the second constituent element, and similarly, the second constituent element can be named the first constituent element.

[0035] In describing the embodiments, descriptions of technical content well-known in the technical field to which this invention pertains and not directly related to this invention are omitted. This is because omitting redundant descriptions more clearly conveys the essence of the invention and avoids obscuring its meaning.

[0036] For the same reasons, some constituent elements are exaggerated, omitted, or approximated in the accompanying drawings. Furthermore, the dimensions of each constituent element do not perfectly reflect their actual dimensions. In each drawing, the same or corresponding constituent elements are assigned the same reference symbols.

[0037] The advantages and features of the present invention, as well as the methods for achieving such advantages and features, will become clear from the accompanying drawings and the embodiments described in detail below. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various different forms. These embodiments are provided only to fully disclose the invention and to fully inform those skilled in the art of the scope of the invention. The invention is defined only by the scope of the technical solution. Throughout this specification, the same reference numerals refer to the same constituent elements.

[0038] It is understood that the blocks of a flowchart and the combination of the flowchart can be implemented by computer program instructions. These computer program instructions can be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment. Therefore, these instructions, executed by the processor of a computer or other programmable data processing equipment, produce methods for implementing the functions described in the flowchart blocks. To implement the functions in a specific manner, these computer program instructions can also be stored in computer-usable or computer-readable memory that is accessible to a computer or other programmable data processing equipment. Therefore, the instructions stored in such computer-usable or computer-readable memory can also produce manufactured items containing instruction methods that implement the functions described in the flowchart blocks. The computer program instructions can also be loaded into a computer or other programmable data processing equipment. Therefore, a series of action steps are performed in the computer or other programmable data processing equipment to produce a flow executed by the computer. Thus, the instructions executed by the computer or other programmable data processing equipment can also provide steps for executing the functions described in the flowchart blocks.

[0039] Furthermore, each block can represent a module, segment, or portion of code comprising one or more executable instructions used to perform a specific logical function. It should also be noted that in some alternative execution examples, the functions mentioned in the blocks may be executed out of order. For example, two blocks shown successively may actually be executed simultaneously, or occasionally in reverse order depending on their respective functions.

[0040] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily implement the invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. The methods shown in Figures 2 and 3 can be implemented, for example, by using the electronic device 110 described with reference to Figure 1. In the present invention, the electronic device 110 is an exemplary device for identifying codes, and may be a single device, but according to embodiments, it may also be a system comprising a plurality of detailed devices communicating with each other via wired or wireless networks.

[0041] Figure 1 is a diagram of a system for implementing testing according to an embodiment.

[0042] Referring to FIG1, the system may include at least a portion of electronic device 110, user terminal 120, and network 130. The system shown in FIG1 only illustrates the components relevant to this embodiment. Therefore, those skilled in the art related to this embodiment will understand that, in addition to the components shown in FIG1, other general components may be included.

[0043] Electronic device 110 is a device for performing tasks required by the testing process and can communicate wirelessly and wiredly with one or more user terminals 120. When electronic device 110 is connected to other storage, electronic device 110 may only include a storage device with a small storage capacity for storing temporary data. However, according to embodiments, electronic device 110 may also include a large storage device, such as a cloud device, that comprehensively stores data generated during the testing process.

[0044] However, the tasks required for the testing process are not entirely controlled by the electronic device 110. According to the embodiment, some tasks can be performed by the test management system of the user terminal 120. That is, according to the embodiment, an emulator for testing the application can be implemented in the form of software programs in both the electronic device 110 and the user terminal 120.

[0045] User terminal 120 can communicate with electronic device 110 to connect to a service for testing via a web application or a dedicated application. The user of user terminal 120 can input commands to utilize the service through an input interface and confirm information provided by the service through an output interface. Relatedly, the input interface may include at least one of the following: a touchscreen, keyboard, mouse, stylus, microphone, gesture sensor, gyroscope, GPS (Global Positioning System) sensor, and light sensor of user terminal 120. The output interface may include at least one of the following: a display and a speaker of user terminal 120. However, the types of input and output interfaces are not limited to the above. Any mechanism capable of recognizing user input can be used as the input interface of this invention; any mechanism capable of providing information to the user through visual, auditory, or tactile means can be used as the output interface of this invention.

[0046] Electronic device 110 and user terminal 120 can send and receive information via network 130. Network 130 can include Local Area Network (LAN), Wide Area Network (WAN), Value Added Network (VAN), Mobile Radio Communication Network, Satellite Communication Network, and combinations thereof. This network is a comprehensive data communication network that enables the constituent entities of the networks shown in Figure 1 to communicate smoothly with each other. It can include wired internet, wireless internet, and mobile wireless communication networks. Wireless communication may include, for example, Wi-Fi, Bluetooth, Bluetooth Low Energy, Wi-Fi Direct (WFD), UWB (ultra-wideband), Infrared Data Association (IrDA), and Near Field Communication (NFC), but is not limited to these.

[0047] On the other hand, only one user terminal 120 is shown in Figure 1, but this is for illustrative purposes. According to the embodiment, there may be multiple user terminals 120 communicating with electronic device 110. Furthermore, throughout the embodiment, electronic device 110 and user terminal 120 are referred to as separate devices and servers, but they can be logically separated and implemented by functions separated from a single device or server. For example, at least a portion of the multiple user terminals 120 may be included in or integrated with electronic device 110.

[0048] Figure 2 is a block diagram showing an electronic device 110 according to one embodiment.

[0049] According to one embodiment, the electronic device 110 may include a memory 111 and a processor 113, and may further include an input / output interface (not shown). The electronic device 110 shown in FIG2 only illustrates the components relevant to this embodiment. Therefore, those skilled in the art related to this embodiment will understand that other common components may be included in addition to those shown in FIG2. According to the embodiment, the electronic device 110 can communicate with external electronic devices via the input / output interface. The external electronic device may serve as a user terminal 120 or other server. Furthermore, the communication technologies used by the electronic device 110 may include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0050] The processor 113 can control the overall operation of the electronic device 110 and process data and signals. In an embodiment, the processor 113 may be included in a controller. The processor 113 may include at least one hardware unit. Furthermore, the processor 113 can operate by executing one or more software modules generated by executing program code stored in memory 111. The processor 113 may include memory, and the processor 113 can execute program code stored in memory to control the overall operation of the electronic device 110 and process data and signals.

[0051] The processor 113 can be connected to the memory 111 to set the target feature to be tested. When the user terminal 120 enters a page of one of the plurality of fields, it determines whether the entered page meets the prerequisites for applying the target feature. If the page meets the prerequisites, the page including the target feature is provided to the user terminal 120. The target feature is one of the plurality of options that can be applied to the target feature. The processor 113 manages the logs corresponding to the input made by the user terminal 120 on the page.

[0052] The electronic device described in the above embodiments may include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with external devices, and a user interface device such as a touch panel, key, or button. Methods implemented using software modules or algorithms can store computer-readable code or program commands executable in the processor in a computer-readable recording medium. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.). The computer-readable recording media is distributed across a network-connected computer system, enabling distributed storage and execution of computer-readable code. The media can be read by a computer, stored in memory, and executed in a processor.

[0053] The following will describe the process of testing the electronic device 110 and various embodiments with reference to other drawings.

[0054] Figure 3 is an illustrative diagram 300 showing the process of electronic device 110 testing an application.

[0055] Referring to Figure 3, the electronic device 110 can test existing versions of applications installed in the user terminal 120, or test new versions of applications before their official release. The embodiment shown in Figure 3 illustrates the following process: when there are pages corresponding to four fields (Search Result Page (SRP), Single Detail Page (SDP), Cart Page (CART), and Product List Page (PLP)), and four options (A, B, C, D) exist for a target feature, the electronic device 110 controls the user terminal 120 by communicating with it, thereby testing the target feature.

[0056] First, the electronic device 110 can set target features to be tested across multiple domains within the application. The setting of target features can be achieved through external input or automatically based on the test order or priority set internally by the electronic device 110. For example, the electronic device 110 can refer to statistics related to the ranking of interfaces that influence the purchase conversion rate of users of the product sales service application, and thereby sequentially set the top N interfaces (where N is a natural number) as target features.

[0057] On the other hand, the multiple domains to be tested can be distinguished by the application's pages, with each domain corresponding to at least one page. For example, there may be three types of pages corresponding to the first domain, one type of page corresponding to the second domain, and two types of pages corresponding to the third domain. This domain-page matching relationship can be predefined, and all pages can be provided by a single application, but according to embodiments, they can also be provided by different applications. Of course, two pages corresponding to the same domain may also be provided by different applications. In one embodiment, the pages of each of the multiple domains are pages jointly provided by the product sales service application, and may also include at least a portion of SRP, SDP, CART, and PLP.

[0058] On the other hand, when user terminal 120 enters a page of one of the plurality of domains, electronic device 110 can determine whether the page entered by user terminal 120 meets the prerequisites for applying the target feature. Considering the driving environment of user terminal 120, the status of user account, and the attributes of the page entered by user terminal 120, the page may be in a state where the target feature cannot be applied. Therefore, electronic device 110 can first determine whether the prerequisites are met and provide a page with the target feature applied, or interrupt the test process. For example, in Figure 3, if when user terminal 120 enters SRP, the "discount banner" as the target feature is currently in a state where it cannot be applied to SRP, then the test cannot be performed normally, so electronic device 110 can skip the subsequent process.

[0059] In one embodiment, the electronic device 110 may determine at least a portion of the prerequisites, including: user attributes that distinguish whether a user account connected to the user terminal 120 is a member; the operating system of the user terminal 120; and item attributes that distinguish items to which the target feature can be applied. For example, regarding the target feature provided to members of an application service, the electronic device 110 may provide a page with the target feature to the user terminal 120 only if the user account is a member according to the user attributes that distinguish whether the user account is a member. As another example, the electronic device 110 may provide a page with the target feature to the user terminal 120 only if the operating system information of the user terminal 120 currently connected to the application supports the target feature. As yet another example, the electronic device 110 may provide a page with the target feature to the user terminal 120 only if the attributes of the items to which the target feature can be applied support the target feature.

[0060] On the other hand, when the page accessed by user terminal 120 meets the preconditions, electronic device 110 can provide the user terminal 120 with a page including the target feature, which is an option among a plurality of options applicable to the target feature. Relatedly, the electronic device 110 can determine the preconditions and the option applied to the target feature in the same way for all pages accessed by user terminal 120 during the testing process.

[0061] In one embodiment, electronic device 110 may assign user terminal 120 to one of a test group categorized according to a plurality of options, and apply the options corresponding to the group to which user terminal 120 is assigned to a target feature. Specifically, electronic device 110 may obtain a hash value of the identifier of the user account connected to user terminal 120 based on a hash function, assign user terminal 120 to one of the test groups based on the hash value, and apply the options corresponding to the group to which user terminal 120 is assigned to a target feature. More specifically, if electronic device 110 obtains a hash value, it may, based on the order corresponding to the hash value, specify the order of user accounts among a plurality of user accounts, and after assigning user terminal 120 to the test group corresponding to the specific order, apply the options corresponding to the group to which user terminal 120 is assigned to a target feature. The reason for assigning user terminal 120 to the group of options to which the target feature is applied based on the hash value is as follows. To conduct high-quality testing, multiple user terminals need to be equally distributed among groups that have applied their respective options to the target feature. For example, the target feature in the SRP of Figure 3 can be applied with options A, B, C, and D. This is because if most user terminals test the target feature with option A, it is difficult to analyze the differences and effects of other options. Therefore, user accounts are distinguished by hash values, and user terminals 120 are assigned to groups with each option applied in order. This allows multiple users to test the target feature with various options to the maximum extent. In Figure 3, 25% is shown for each option to represent the ideal state where multiple user terminals are equally distributed among the options. However, whenever a page of each domain (SRP, SDP, CART, PLP) is accessed, some user terminals may interrupt the test depending on whether the preconditions are met. In Figure 3, it is assumed that the preconditions are met for each page.

[0062] On the other hand, the electronic device 110 can manage logs corresponding to inputs made by the user terminal 120 on the pages accessed by the user terminal 120. In this invention, managing logs is used to mean all of the following: the electronic device 110 directly generates logs, receives (collects) logs from external devices, processes logs, and stores logs in a storage device connected to the electronic device 110.

[0063] In one embodiment, the same log identifier can be assigned to the logs recorded in all domain pages accessed by the user terminal 120 during the test. This allows the logs generated by one user terminal 120 to be distinguished from those of other user terminals when the logs are subsequently processed or used as statistical information on a dashboard, based on the test process performed by that user terminal 120.

[0064] In one embodiment, as part of the management log, when a user account connected to user terminal 120 selects an item with target characteristics on a page accessed by user terminal 120, electronic device 110 can generate a web page log. A web page log is a type of data that records user activity. Electronic device 110 can generate a web page log and store it as a log file in electronic device 110 or external storage, triggered by the selection of an item with target characteristics. Furthermore, web page logs can be generated periodically or at the moment of item selection. On the other hand, web page logs may include at least a portion of (1) the identifier of the item selected by the user account and (2) the identifier of the user account. Furthermore, web page logs may include at least a portion of HTTP request / response information, IP address, or browser information.

[0065] In another embodiment, as part of the management log, the electronic device 110 can generate a real-time log corresponding to the interaction between the page accessed by the user terminal 120 and the user terminal 120. The real-time log is generated in real time and can be generated and updated in a streaming manner. On the other hand, the electronic device 110 can also receive auxiliary logs from the user terminal 120 to confirm the validity of the real-time logs, and delete real-time logs whose validity has not been confirmed by the auxiliary logs from the generated real-time logs. In this case, the auxiliary logs can be generated by the user terminal when the target feature is displayed on the screen of the user terminal 120 as the user terminal 120 scrolls on the page accessed by the user terminal 120. On the other hand, the auxiliary logs can also include: (1) identifiers assigned to each test group to which the user account connected to the user terminal 120 belongs; and (2) user account identifiers. Therefore, when the target features are not displayed on the screen of the user terminal 120, the corresponding real-time log is not a normal test log. Therefore, filtering is performed before use, thereby reducing the time and resources consumed in storing, processing and using the logs.

[0066] In another embodiment, as part of the management log, when a target feature is displayed on the screen of the user terminal 120 as the user terminal 120 scrolls through the page it has accessed, the electronic device 110 can receive a real-time log corresponding to the interaction between the page and the user terminal 120. To limit the collection conditions of the real-time log itself, the user terminal 120 may generate real-time logs only when the target feature is displayed on the screen of the user terminal 120, or it may generate real-time logs indiscriminately based on the interaction between the page and the user terminal 120, but only transmit the real-time logs to the electronic device 110 when the target feature is displayed on the screen of the user terminal 120.

[0067] The storage media for storing real-time logs and web page logs may also be different, but according to the embodiments, they can also be logically distinguished and stored in the storage space within the electronic device 110 or in the storage device associated with the electronic device 110.

[0068] Figure 4 is an illustrative diagram 400 used to illustrate the process of determining whether to display the target feature on the screen of the user terminal 120. In Figure 4, for ease of explanation, a screen showing a list of items provided in an item sales service is assumed.

[0069] Referring to Figure 4, a maximum of six items can be displayed at a time on the screen of the user terminal 120, which has a limited size. That is, when the user account has entered the item list page, the top six items (items 1 to 6) of the list can be displayed on the screen of the user terminal 120, while the six items below them (items 7 to 12) are hidden. Subsequently, if the user scrolls the screen of the user terminal 120 using gestures, touch input, or other input methods, the area displayed on the screen can move down to display items 7 to 12.

[0070] In this situation, if we assume the target feature has been applied to item 3, then when the user account enters the item list page, item 3 with the applied target feature will be immediately displayed on the screen of user terminal 120. Therefore, the user can confirm the target feature and perform testing, thus enabling normal testing. However, if we assume the target feature has been applied to item 9, item 9 will not be displayed on the initial screen of the item list page. Therefore, the target feature cannot be confirmed unless the user scrolls. Since the selection made by the user without confirming the target feature cannot be used as data to judge the effect of the target feature, it is crucial to determine whether the target feature is displayed on the screen of user terminal 120. The criteria for determining this can be set in various ways, and the following examples can be used as examples for reference.

[0071] When the position of a specific pixel on the screen changes by a preset value due to scrolling, the electronic device 110 or user terminal 120 can determine that the target feature is displayed on the screen of the user terminal 120. For example, the electronic device 110 or user terminal 120 can identify the position of an item with the target feature and calculate the required screen displacement to display the item based on the position of the item. When the position of a specific pixel on the screen changes to a value greater than or equal to that displacement, it can be determined that the target feature is displayed on the screen of the user terminal 120. Alternatively, the electronic device 110 or user terminal 120 can calculate the scrolling length itself. When the calculated scrolling length is greater than or equal to that displacement, it can be determined that the target feature is displayed on the screen of the user terminal 120.

[0072] Alternatively, electronic device 110 or user terminal 120 may also determine that the target feature is displayed on the screen of user terminal 120 only when the user confirms through the interface of the item list page (e.g., check the box).

[0073] Figure 5 is an illustrative diagram of a dashboard display showing the test results.

[0074] Electronic device 110 can manage logs corresponding to inputs made by user terminal 120 on pages accessed by user terminal 120, and based on these logs, provide a dashboard representing the test results to user terminal 120 or other external devices. In particular, in tests related to product sales service applications, the dashboard may include: purchase conversion rate before importing target features, purchase conversion rate after importing target features, and the rate of change of purchase conversion rate before and after importing target features.

[0075] In Figure 5, Option A represents the existing options applied to the target feature, while Option B represents the new option applied to the target feature. Based on the dashboard shown in Figure 5, the benchmark (measurement benchmark) for judging the merits of Option A and Option B is the "purchase conversion rate." This provides the purchase conversion rate when Option A is applied, the purchase conversion rate when Option B is applied, and the rate of change in purchase conversion rate when switching from Option A to Option B. In addition, it provides the p-value (statistically significant difference), the confidence interval (CI) representing the probability that the actual average of the population exists within a specific interval, or the minimum detectable effect (MDE) that can be detected by testing.

[0076] In particular, the dashboard provides the results of an analysis of the differences between all groups with option B and option A in each application ("Purchase Conversion Rate - Segment Combination Calculation"). In addition, according to an embodiment, the results of an analysis of the differences between detailed groups (segment groups 1 to 4) with option B and option A in each application can also be provided. Segment groups can be distinguished by domain, by pages constituting a domain, or by attributes of user terminal 120 or user account. For example, segment group 1 may represent the analysis results in SRP, segment group 2 may represent the analysis results in SDP, segment group 3 may represent the analysis results in CART, and segment group 4 may represent the analysis results in PLP. As another example, segment group 1 can represent the analysis results of a user account that has joined a membership but has not yet withdrawn or canceled its membership; segment group 2 can represent the analysis results of a user account that has withdrawn or canceled its membership and then rejoined and is currently a member; segment group 3 can represent the analysis results of a user account that has joined a membership but has withdrawn or canceled its membership and is currently a non-member; and segment group 4 can represent the analysis results of a user account that has not yet joined a membership and is currently a non-member.

[0077] Figure 6 is a flowchart illustrating a method for conducting a test according to an embodiment.

[0078] The method shown in Figure 6 will be described below as being implemented by the electronic device 110 described above. However, this is just an example. The method shown in Figure 6 can also be implemented by other devices, or by a combination of electronic device 110 and other devices.

[0079] Electronic device 110 can set the target feature to be tested (S610). Next, when user terminal 120 enters a page of one of a plurality of fields, electronic device 110 can determine whether the entered page meets the prerequisites for applying the target feature (S620). Subsequently, if the page meets the prerequisites, electronic device 110 can provide the page including the target feature to user terminal 120, wherein the target feature has one of a plurality of options that can be applied to the target feature (S630). Subsequently, electronic device 110 can manage the log corresponding to the input made by user terminal 120 on the page (S640).

[0080] This embodiment can be represented by functional blocks and various processing steps. These functional blocks can be implemented by different numbers of hardware and / or software components performing specific functions. For example, the embodiment can employ integrated circuits capable of performing various functions via the control of one or more microprocessors or other control devices, such as memory, processing, logic, lookup tables, etc. The components can be executed by software programs or software elements. Similarly, this embodiment includes various algorithms implemented as combinations of data structures, programs, routines, or other programs, and therefore can be implemented using programming or scripting languages ​​such as C, C++, Java, and assemblers. Functionally, it can be implemented by algorithms executed in one or more processors. Furthermore, this embodiment can employ prior art for electronic environment setup, signal processing, and / or data processing. Terms such as "mechanism," "component," "mechanism," and "structure" are used broadly and are not limited to mechanical and physical structures. The above terms may be associated with processors and include the meaning of a series of software processes.

[0081] The above embodiment is only an example, and other embodiments can be implemented within the scope of the invention application described below.

[0082] 110: Electronic devices 111: Memory 113: Processor 120: User Terminal 130: Internet 300: Figure 400: Figure 500: Image S610: Steps S620: Steps S630: Steps S640: Steps

Claims

1. A test implementation method for an electronic device performing tests across a plurality of domains, comprising the following steps: setting a target feature to be tested on a page of the plurality of domains accessed by a user terminal during the test; when the user terminal accesses a page of one of the pages of the plurality of domains, determining whether the page satisfies a prerequisite for applying the target feature; when the page satisfies the prerequisite, providing the page including the target feature to the user terminal, wherein the target feature is equipped with one of a plurality of options applicable to the target feature; and managing a log corresponding to inputs made by the user terminal on the page, wherein the option applied to the target feature is set identically for the pages of the plurality of domains.

2. The test implementation method as described in Request 1, wherein the pages of the plurality of domains mentioned above are pages commonly provided by the product sales service application, including at least a portion of search results pages, individual detail pages, shopping cart pages, and list pages.

3. The test implementation method as described in Request 1, wherein the aforementioned prerequisites for applying the target features are set identically for the pages of the plurality of domains, and wherein the aforementioned prerequisites include at least a portion of the following: user attributes that distinguish whether a user account connected to the user terminal is a member; the operating system of the user terminal; and item attributes that distinguish items to which the target features can be applied.

4. The test implementation method as described in Request 1, comprising: when another user terminal enters the above page, determining whether the above page satisfies another precondition for applying the above target feature; when the above page satisfies the other precondition, providing the above page including the above target feature to the other user terminal, the target feature having another option among the above plurality of options, wherein the other option applied to the above target feature is set identically for the above plurality of fields when the other user terminal enters the pages.

5. The test implementation method as described in Request 1, wherein the logs recorded on the pages of the plurality of domains mentioned above are assigned the same log identifier.

6. The test implementation method as described in Request 1, wherein the steps for managing the above logs include the following steps: generating real-time logs corresponding to the interactions between the above page and the above user terminal.

7. The test implementation method as described in Request 6, wherein the steps of managing the above-mentioned logs further include the following steps: receiving auxiliary logs from the above-mentioned user terminal to confirm the validity of the above-mentioned real-time logs; and deleting real-time logs from the generated real-time logs that have not been confirmed as valid by the above-mentioned auxiliary logs.

8. The test implementation method as described in Request 7, wherein the auxiliary log is generated by the user terminal when the target feature is displayed on the screen of the user terminal as the user terminal scrolls the page.

9. As per the test implementation method in request item 7, the aforementioned auxiliary logs include: Identifiers assigned to each test group to which the user account connected to the aforementioned user terminal belongs; and identifiers for the aforementioned user accounts.

10. The test implementation method as described in Request 1, wherein the step of managing the above logs includes the following steps: when the above target feature is displayed on the screen of the user terminal as the user terminal performs scrolling behavior on the above page, receive real-time logs from the user terminal corresponding to the interaction between the above page and the user terminal.

11. The test implementation method as described in Request 1, wherein the steps for managing the above logs include the following steps: generating a web page log when a user account connected to the above user terminal selects an item with the above target characteristics on the above page.

12. The test implementation method as described in Request 11, wherein the aforementioned web page logs include at least a portion of the identifiers of the items selected by the aforementioned user account and the identifiers of the aforementioned user account.

13. The test implementation method as described in Request 1, wherein the steps of providing the above page include the following steps: assigning the above user terminal to one of the test groups classified according to the above plurality of options; and applying the options corresponding to the group to which the above user terminal is assigned to the above target feature.

14. The test implementation method as described in Request 13, wherein the above allocation step includes the following steps: obtaining a hash value of the identifier of the user account connected to the above user terminal based on a hash function; and allocating the above user terminal based on the hash value.

15. The test implementation method as described in Request 14, wherein the step of allocating the user terminal based on the hash value includes the following steps: determining the order of the user accounts among a specific plurality of user accounts based on the order corresponding to the hash value; and allocating the user terminal to the group in the test group corresponding to the specific order.

16. The test implementation method as described in Request 1 further includes the following steps: providing a dashboard representing the results of the above test to a user terminal based on the aforementioned logs; and the dashboard includes: Purchase conversion rate before importing the above target features; purchase conversion rate after importing the above target features; and the rate of change of purchase conversion rate before and after importing the above target features.

17. A non-transitory computer-readable recording medium recording a program for performing the method as described in claim 1 on a computer.

18. An electronic device for performing tests across a plurality of domains, comprising a memory storing commands and a processor, the processor being configured to: be connected to the memory to set a target feature to be tested on a page of the plurality of domains accessed by a user terminal during testing; determine, when the user terminal accesses a page of one of the pages of the plurality of domains, whether the page satisfies a prerequisite for applying the target feature; when the page satisfies the prerequisite, provide the page including the target feature to the user terminal, the target feature being an option among a plurality of options applicable to the target feature; and manage a log corresponding to inputs made by the user terminal on the page, wherein the option applied to the target feature is set identically for the pages of the plurality of domains.

Images