Manage test resources to perform reliability testing of memory components

Abstract

Filter information associated with a test to be performed on one or more memory components is determined. A set of memory components matching the filter information may be retained for testing. Test execution information defining a set of test procedures for the test is determined. A connection to a first test procedure may be established and used to receive feedback information associated with execution of the test procedure. Based on the feedback information, a failure of the first test procedure may be identified.

Description

Technical Field

[0001] The present disclosure relates generally to a memory subsystem, and more particularly to testing of memory components for a memory subsystem. Background Art

[0002] The memory subsystem may be a storage system, such as a solid-state drive (SSD) or a hard disk drive (HDD). The memory subsystem may be a memory module, such as a dual in-line memory module (DIMM), a small outline DIMM (SO-DIMM), or a non-volatile dual in-line memory module (NVDIMM). The memory subsystem may include one or more memory components that store data. The memory components may be, for example, non-volatile memory components and volatile memory components. In general, the host system may utilize the memory subsystem to store data at the memory components and retrieve data from the memory components. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present disclosure will be more fully understood through the detailed description given below and the accompanying drawings of various implementations of the present disclosure.

[0004] Figure 1 An exemplary environment for allocating test resources to perform memory component testing according to some embodiments of the present disclosure is shown.

[0005] Figure 2 A flow chart illustrating an exemplary method of allocating test resources to perform memory component testing according to some embodiments of the present disclosure.

[0006] Figure 3 An exemplary interface associated with configuring a test of a memory component according to some embodiments of the present disclosure is shown.

[0007] Figure 4 An exemplary test platform for performing memory component testing according to some embodiments of the present disclosure is shown.

[0008] Figure 5 is a flow chart of an exemplary method of replicating a failed test of a memory component according to some embodiments.

[0009] Figure 6 is a flow chart of an exemplary method of extending test samples in a usage space associated with memory component testing.

[0010] Figures 7A-7C is an exemplary representation produced in execution of a multi-dimensional usage space test of a memory component according to some embodiments.

[0011] Figure 8 is a block diagram of an exemplary computer system in which implementations of the present disclosure may operate. DETAILED DESCRIPTION

[0012] Aspects of the present disclosure relate to multi-dimensional usage space testing of memory components. A memory subsystem is also referred to as a "memory device" hereinafter. An example of a memory subsystem is a storage device coupled to a central processing unit (CPU) via a peripheral interconnect (e.g., an input / output bus, a storage area network). Examples of storage devices include solid-state drives (SSDs), flash drives (SSDs), universal serial bus (USB) flash drives, and hard disk drives (HDDs). Another example of a memory subsystem is a memory module coupled to a CPU via a memory bus. Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), non-volatile dual in-line memory modules (NVDIMMs), etc. In some embodiments, a memory subsystem may be a hybrid memory / storage subsystem. Typically, a host system may utilize a memory subsystem that includes one or more memory components. The host system may provide data to be stored at a memory subsystem, and may request to retrieve data from the memory subsystem.

[0013] The memory components used in the memory subsystem can be tested before being used in the memory subsystem. In a conventional test process, the memory components can be placed in a chamber (i.e., an oven) that tests the memory components under various conditions (e.g., temperature conditions). For example, a single chamber can be used to test multiple memory components at a specific temperature at one time. The test process can instruct to perform various operations at the memory component at a specific temperature. Such operations may include, but are not limited to, read operations, write operations, erase operations, fast raw bit error rate measurements, conditional threshold voltage (Vt) distribution collection, thermal loop control, asynchronous power loss control, open non-flash interface (ONFI) testing, etc. When the test process is performed, the performance and behavior of the memory component can be observed. For example, performance characteristics (e.g., read or write delays) and the reliability of data stored at the memory component can be measured and recorded during the test process. However, since the chamber can only apply a limited set of test conditions (e.g., a single temperature) to the memory component at any particular time, it takes a lot of time to test the memory component under a variety of different test conditions (e.g., different temperatures) because the test process will need to be performed for each of one or more required test conditions. In addition, the chamber can only perform one test process at a time. Thus, if the test process of the memory component requires a variety of different conditions, performing multiple different tests on the memory component under different operating conditions (e.g., different temperatures) will take a lot of time.

[0014] Aspects of the present disclosure perform the test process by allocating test components to solve the above and other defects. A distributed test platform may include multiple test resources. Each test component (e.g., test resource) may be a test socket including a memory component. The test socket may further include a temperature control component for applying a specific temperature condition to the memory component included in the test socket. The test platform may further include multiple test boards, each of which includes one or more test sockets, so that tests can be performed under various conditions using multiple geographically dispersed test resource locations. The test boards may be organized into groups or racks, and multiple racks may be located at specific locations or places. In this way, multiple locations may include a test board rack including multiple test components (e.g., test sockets and memory components). The racks of the test platform may be coupled to a centralized resource allocator (e.g., one or more servers or other such computing devices) via a network, which is used to control, manage and facilitate the execution of tests on different test boards, test racks and locations or positions at each test component of the test platform.

[0015] The test components or test resources of the test platform can be used to perform multiple different test processes on the test platform at the same time. For example, a first subgroup test socket can be used to perform a first test, and a second subgroup test socket of the test platform can be used to perform a second test at the same time. Therefore, the resource allocator can allocate or assign the test sockets to be used in the new test of the memory component according to the filter information. In some embodiments, the filter information may include one or more elements of feature information and usage information. Exemplary feature information may include one or more of the product type, probe version, category version, version information (i.e., design or manufacturing revision) of the memory component. Exemplary usage information associated with the required test component may include a minimum block cycle count, a maximum block cycle count, the number or range of required blocks, etc.

[0016] The new test can specify the specific features or types of memory components to be used in the test, and the temperature control component will be applied to the specific temperature of the memory component at the test socket. These features may include, but are not limited to, the type (e.g., design version) of the memory component and the usage characteristics of the memory component. Then, the resource allocator can assign the available test sockets of the test platform to the new test based on the features specified by the new test and the additional information of the test platform. For example, the available test sockets can be further assigned based on the positions of the test sockets on the various test racks or locations of the test platform, and the temperature control component can be used to apply the temperature specified by the test. Therefore, the test platform can be a group of test sockets distributed in different locations or locations. Based on the specific features of the test sockets specified by different tests, the test sockets of different groups of the test platform can be assigned to different tests. In some embodiments, the test components or test resources of the test platform can be used to perform the same test process simultaneously under different test conditions.

[0017] In one embodiment, the one or more test processes may identify one or more failures, including a failure in the operation of a memory component (e.g., a problem with the operation of a memory component), a failure in the operation of a memory component that causes a corresponding software layer to respond to the failure (e.g., throwing an exception), a failure generated by a test program due to the operation of a memory component (e.g., an exception), or a software, hardware, or firmware bug or error in the test process itself.

[0018] Advantages of the present disclosure include, but are not limited to, a reduced amount of time for a test platform to perform memory component testing. In addition, multiple different tests can be performed at multiple distributed test resources to test multiple different conditions (e.g., different temperatures). In addition, a distributed system can be used to perform multiple different sequences of operations to achieve more robust testing and improved reliability and performance of memory components. In this way, since multiple different tests of memory components can be performed more quickly, the reliability of the memory components can also be increased because any potential defects or flaws can be identified and later addressed when designing or manufacturing memory components to be included in the memory subsystem.

[0019] Figure 1 An exemplary environment for allocating test resources to perform memory component testing according to some embodiments of the present disclosure is shown. Test resources 100 may include one or more racks 110A, 110B, and 110N. Each rack 110A, 110B, and 110N may include multiple test boards 120, wherein each test board 120 includes one or more test sockets (i.e., test resources). Test resources 100 may include any number of racks or test sockets.

[0020] As shown, the test board 120 may include one or more test sockets. For example, the test board 120 may include a first test socket 121, a second test socket 122, and a third test socket 123. Although three test sockets are shown, the test board 120 may include any number of test sockets. Each test socket may include a memory assembly that has been embedded in the corresponding test socket. In addition, each test socket may include a temperature control assembly for applying temperature conditions to the embedded memory assembly. In some embodiments, the temperature control assembly may be a dual Peltier device (e.g., two Peltier devices), which utilizes the Peltier effect to apply a heating or cooling effect at the surface of the dual Peltier device coupled to the embedded memory assembly. In the same or alternative embodiments, the temperature control assembly may be placed on the top of the memory assembly in the corresponding test socket.

[0021] As shown, each test rack 110A, 110B and 110N may include a plurality of test boards 120. Each test board 120 of a particular test rack may be coupled to a local test assembly. For example, each test rack 110A, 110B and 110N may include local test assemblies 111A, 111B and 111N, respectively. Each of the local test assemblies 111A, 111B and 111N may receive instructions to perform a test or a portion of a test to be performed at a test socket of the corresponding test rack. For example, a resource allocator component 130 may receive (e.g., from a user) conditions of a test to be performed, and the resource allocator component 130 may determine a specific test socket on a different test board 120 at one or more of the test racks 110A, 110B and 110N that the test may use. In some embodiments, the resource allocator component 130 may be provided by a server 131. In some embodiments, server 131 is a computing device or system coupled to local test components 111A, 111B, and 111N via a network.

[0022] The temperature control assembly of each test socket 121, 122, and 123 of each test board 120 can be used to apply different temperature conditions to the corresponding embedded memory components. In addition, each test socket 121, 122, and 123 can be used to perform different operations at the embedded memory components.

[0023] Resource allocator component 130 may receive test input from a user. In some embodiments, the test input (or test request) may be received via, for example, Figure 3Interfaces such as the exemplary interface 300 shown are received from a user. Test inputs may specify conditions (also referred to as "test conditions") for tests to be performed with one or more memory components. For example, a test may specify specific temperature conditions to be applied to the memory components and a sequence of operations to be performed at the memory components under the specific conditions. The resource allocator 130 may retrieve a data structure identifying available test sockets on the test platform 100 and characteristics of the available test sockets. Subsequently, the resource allocator component 130 may assign test sockets at the test platform 100 that include embedded memory components that match or satisfy the test conditions. In some embodiments, the resource allocator component 130 may identify test sockets that match the test conditions by analyzing feature information and usage information associated with the available test sockets. In one embodiment, with respect to a rack (e.g., Figure 1 The characteristic information of each test socket of the racks 110A, 110B, 110N) can be stored in one or more databases (referred to as "characteristic databases"). In one embodiment, the usage information about each test socket of the rack can be stored in one or more databases (referred to as "usage databases"). The resource allocator component 130 can then send instructions to the local test components of the test rack, which include the test sockets identified as being used in the test based on the associated characteristics and usage information. Further details about the resource allocator component 130 are described below.

[0024] Figure 2 An exemplary method 200 for allocating test resources to perform memory component testing according to some embodiments of the present disclosure is shown. The method 200 may be performed by a processing logic that may include hardware (e.g., a processing device, a circuit, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 200 is performed by Figure 1 The resource allocator component 130 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the process can be modified. Therefore, the embodiments shown should be understood as examples only, and the processes shown can be performed in different orders, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not every embodiment requires all processes. Other process flows are also possible.

[0025] As shown, at operation 210, processing logic receives a request to perform a test on one or more memory components. For example, a user may provide a request to run a memory component test by a test platform. The test may specify an operation to be performed on the memory component. Such operations may include, but are not limited to, a read operation, a write operation, and an erase operation. In some embodiments, the test specifies a sequence of multiple operations. In this way, a user may provide information specifying conditions for performing a memory component test at a test platform.

[0026] The test may specify an operation or test condition under which an operation is performed at a particular memory component. The test condition may be a temperature condition of the memory component when the operation is performed at the memory component. In some embodiments, the test request may specify a test condition or may be used to identify a test condition corresponding to desired characteristics (e.g., component type) and usage (e.g., historical usage data) of a memory component to be allocated and used for testing, as described below.

[0027] At operation 220, processing logic determines filter information (e.g., one or more elements of characteristic information and usage information) associated with the memory components to be assigned for testing. In some embodiments, the test conditions can be used as filter information for the filter, and the memory components with corresponding characteristic information and / or usage information are identified. In some embodiments, each test platform (e.g., a rack of boards) can be associated with one or more databases (e.g., Figure 4 The test components are associated with a feature database 416-1 and 416-N), which is configured to store feature information related to multiple associated test components.

[0028] In some embodiments, the usage information of the memory component may include, but is not limited to, a number of operations that have been performed by the memory component. For example, the test conditions may be used to identify the required usage information corresponding to a memory component having a more or less specific number of program-erase operations or cycles and a specific number of read operations. In some embodiments, the required usage information may include previous temperature conditions that have been applied to the memory component. For example, the usage information may include a previous temperature at which the operations performed at the memory component for the previous test (i.e., the temperature profile of the memory component) may be specified.

[0029] The test request may specify test conditions for the test, or may be used to identify test conditions corresponding to required filter information for a memory component to be used in the test (e.g., a test component that matches a standard associated with the identified characteristic information and usage information). In some embodiments, the available test sockets may be test sockets that are not used by another test and that match the required characteristics and / or usage information of the memory component specified by the test. Thus, a first subset of test sockets (i.e., test resources) of the test platform may be available test sockets that match the conditions specified by the test, and a second subset of test sockets may be test sockets that are unavailable and / or do not match the conditions specified by the test. In combination Figure 3 Describes further details about matching with the required characteristics.

[0030] At operation 230, based on the identified characteristic information and usage information associated with the test, the processing logic retains one or more test sockets. In some embodiments, the processing logic determines the available test sockets of one or more test platforms (e.g., racks). For example, a test socket that is not currently used to perform another test or operation and is not arranged to perform another test at the test board of the test platform can be identified as available. For example, the test platform may include multiple test sockets, and a subgroup of test sockets can currently be used by another test of the memory component, while the second subgroup of test sockets is not currently used by another test of the memory component. Therefore, the second subgroup of test sockets can be considered as available test sockets. In some embodiments, the test can specify a time period in which the test is to be completed. In this case, the available test sockets can be test sockets that can be used for testing within the time period. In this way, if a specific test socket is currently being used by a test and will be arranged to be idle or available later, and the test can perform an operation at a specific test socket within a specified time period, the specific test socket can be considered as an available test socket.

[0031] The processing logic may identify a set of one or more test sockets available and match the characteristic information and usage information associated with the test. In some embodiments, the processing logic filters the available test resources (e.g., available boards or available test sockets) based on the characteristic information and usage information. For example, each rack at each location may maintain a local usage database (e.g., Figure 4 of the usage databases 414-1 and 414-N) and local feature databases (e.g., Figure 4 The feature databases 416-1 and 416-N) are used to store usage and feature information about various test resources for filtering available test resources according to test conditions associated with the test.

[0032] At operation 240, the processing logic configures and executes the test of one or more memory components at the reserved available test sockets based on the feature information and usage information of the one or more memory components. For example, the test can be performed at a test socket that is not currently used by another test and has a feature that matches the required feature specified by the test. The test can be performed at different test sockets, which are included in different test racks of the entire test platform. The execution of the test can apply different temperature conditions at different test sockets, while performing different or identical operation sequences at the test sockets. For example, a first socket can run an operation sequence at a first temperature, while a second test socket can run the same operation sequence at a different second temperature. In some embodiments, a first test socket can run a first operation sequence at a certain temperature, while a second test socket can run a different second operation sequence at the same temperature. In the same or alternative embodiments, the temperature conditions applied to a specific test socket can change during the execution of the operation sequence.

[0033] In some embodiments, to configure a test for execution, processing logic creates a list of one or more source code dependencies associated with the test. In some embodiments, processing logic identifies one or more file locations associated with the source code dependencies (e.g., corresponding folders or uniform resource locators (URLs) associated with a code repository). In some embodiments, processing logic identifies one or more command lines (i.e., run properties) to run in conjunction with the code associated with the test. In some embodiments, processing logic may identify a relative folder location at which to run the one or more command lines. In some embodiments, processing logic may identify a relative folder location at which to generate output associated with the test (herein, "test result output").

[0034] According to some embodiments, method 200 may be performed using a communication coupled to Figure 1 In some embodiments, the user can request, start and manage the resource allocator 130 by providing Figure 2 The operation uses the information to create a test request (for example, based on Figure 2 Request for operation 210).

[0035] Figure 3 An exemplary interface 300 is shown that is configured to collect various information related to a test request from a user. For example, after identifying and retaining a plurality of memory component boards in conjunction with a test, the interface 300 (e.g., a web-based interface such as Figure 4The web interface 422 of the test program allows starting a test by identifying the location of the source code (e.g., in a local folder or code repository) and the command line to run it. In some embodiments, one or more additional parameters may be identified in conjunction with the execution of the test, such as, for example, the relative location of the folder where the main function is located, the command line to run the main function, and the relative output folder location.

[0036] In some embodiments, the interface 300 may provide filters for identifying characteristic information associated with the desired test component (such as, for example, product type, probe version, and class version). The interface 300 may also provide filters for identifying usage information associated with the desired test component (such as, for example, minimum block cycle count, maximum block cycle count, number or range of required blocks, etc.).

[0037] In some embodiments, interface 300 may also be used to enable a user to configure and launch a test by identifying the location of the source code and the command line to run it. Figure 3 As shown, additional parameters associated with the test may be identified, such as, for example, the location of the folder where the main function is located (eg, the run folder), the command line for running the main function, and the relative output folder location.

[0038] In some embodiments, a client device (e.g., Figure 4 The client 420) interface 300 and the resource allocator (eg, Figure 4 ) to copy code to one or more racks (e.g., Figure 4 In some embodiments, the execution of the test identification code results in one or more test processes (e.g., Figure 4 As the tests complete (e.g., whether they complete successfully or fail), the results are copied to a network drive from which the test results can be collected. In some embodiments, test resources reserved and used in conjunction with the completed tests can be released after the tests are completed. In some embodiments, as the test result data is collected, the processing logic can store the data in a folder (e.g., in Figure 3 The " / client_output_name" folder identified in the "Find data at:" field of interface 300) or other storage location accessible to the user.

[0039] In some embodiments, during the testing process, the client device may not be aware of the location of the physical test components (e.g., physical boards, sockets, or memory components). In some embodiments, code associated with the testing is run on one or more geographically dispersed workstations (e.g., Linux workstations) associated with the board that includes the retained test components. In some embodiments, the resource allocator can optimize the selection of the location of the test components to manage (e.g., reduce) data transfer time execution.

[0040] Figure 4 An exemplary environment according to an embodiment of the present disclosure is shown, including a management module 480 that is communicatively connected to a client 420 requesting to perform a test. In some embodiments, the management module 480 coordinates with the resource allocator 130 to perform a test using test components (e.g., test components 411-A and 411-N) that are physically located in one or more geographically dispersed component racks (e.g., rack A 110A, rack B 110B, rack C 110C). As described above with respect to method 200, the client 420 may submit a test request to the resource allocator 130 via a network interface 422 of the client 420. The resource allocator 130 identifies and reserves test components for executing a test process. In some embodiments, the reserved test components 411 may be located in geographically dispersed racks 110. For example, the requested test may include executing a set of test processes, including process 1, process 2...process N and process X, process Y...and process Z. In Figure 4 In the example shown, the resource allocator 130 may identify the test component 411-A of the rack A 110A to perform process 1, process 2, ... process N. In addition, the resource allocator 130 may identify the test component 411-N of the rack N 110N to perform process X, process Y, ... process Z. In this example, the rack A 110A and the rack N 110N are located in different geographical locations. In some embodiments, the resource allocator 130 may analyze the usage information stored in the usage database 440 and the feature information stored in the communication connection feature database 450 to identify the one or more test components that meet the test conditions associated with the requested test.

[0041] In some embodiments, resource allocator 130 communicates with a resource program (e.g., resource program 412-A, 412-N), such as a daemon, to pass information associated with the requested test (e.g., identifying source code, command line, etc.). Figure 4As shown, each resource program of each rack workstation may be associated with a usage database (e.g., usage database 414-A, 414-N) and a feature database (e.g., feature database 416-A, 416-N) for storing usage and feature information associated with the test components 411-A, 411-N of the local rack. For example, usage database 414-A stores usage information associated with test component 411-A of rack A 110A, and feature database 416-A stores feature information associated with test component 411-A of rack A 110A.

[0042] The test information (e.g., test instructions) received by the resource programs 412-A, 412-N is provided to the local test managers 410-A, 410-N to facilitate the execution of the one or more test processes (e.g., process 1, process 2, ... process N of rack A 110A) by the identified and retained test components 411A, 411N. In one embodiment, the test component 411 may include one or more sockets, the one or more sockets including one or more memory components, such as Figure 1 For example, the resource allocator 130 may be configured according to Figure 2 The method 200 performs operations to coordinate the execution of tests using various plurality of test components 411A, 411N of a plurality of geographically dispersed racks 110A, 110B, 110C ... 110N.

[0043] In some embodiments, the management module 480 is communicatively connected to the client 420 and is configured to execute a container process 482 to manage the execution of one or more test processes by multiple racks (also referred to as a rack cluster). The management module 480 can be used to implement feedback communication from each individual test process (e.g., process 1, process 2...process N, process X, process Y...process Z). In some embodiments, the execution of the container process 482 enables communication between the management module 480 and the client 420 via a client API (e.g., a Python API) to coordinate the reservation of test components by the resource allocator 130 based on filters (e.g., usage and feature filters), as described below with reference to Figure 5Detailed description. In some embodiments, the management module 480 establishes a feedback communication link with the individual test processes by providing unique connection information (e.g., IP address 1, IP address 2, IP address 3, IP address X, IP address Y, and IP address Z) to each test process. In some embodiments, unique connection information (e.g., a unique IP address or other such network identification information) is provided to each individual process to allow each individual process to communicate directly with the management module 480. For example, the first test process (e.g., process 1 executed on rack A 110A) can receive the connection information assigned by the management module 480 and establish a connection to provide feedback information to the management module 480. For example, the feedback information may include the results of the corresponding test process, conditions associated with the test (e.g., temperature conditions, feature information, usage information, etc.), and a list of the one or more events or operations performed by the test process (e.g., events or operations performed before and / or including a failure associated with the operation of the memory component (e.g., an actual operating error or an anomaly caused by noise or some other factor) or a failure associated with the test process (e.g., a portion or part of the test process fails, indicating an execution problem). In some embodiments, the management module 480 can collect feedback information from multiple different processes via their own unique connections and store the feedback information in the test management database 484.

[0044] Figure 5Flowchart of an exemplary method 500 (also referred to as a "container process") for collecting feedback information from a plurality of test processes that perform tests associated with a plurality of distributed test components (e.g., a test socket that includes one or more memory components) and replicating a failed test process to identify a failure as an error (e.g., an operational error associated with one or more memory components) or an anomaly (e.g., a failure caused by noise). In some embodiments, having identified a failure of a test process associated with one or more memory components (i.e., a first memory component), processing logic may replicate the failed test process on one or more different memory components (also referred to as a "replicated component"). In some embodiments, processing logic uses the feedback information received from the failed test process to recreate the condition of the failed test process. In some embodiments, if the replication of the test process produces a failure with respect to the one or more replicated components (i.e., a memory component having the same characteristics and usage information as the first memory component), the processing logic identifies the failure associated with the first memory component as an actual error (e.g., an error in verification or validation related to the operation of the memory component). If the replication of the failure test process produces a successful result (i.e., no failure is detected), the processing logic identifies the failure associated with the first memory component as an anomaly (e.g., due to noise). The method 500 may be performed by processing logic that may include hardware (e.g., a processing device, a circuit, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 500 is performed at least in part by Figure 4 The management module 480 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the process can be modified. Therefore, the embodiments shown should be understood as examples only, and the processes shown can be performed in different orders, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not every embodiment requires all processes. Other process flows are also possible.

[0045] As shown, at operation 510, the processing logic of management module 480 (e.g., the processing logic of container process 482) retains one or more test components according to the filter information associated with the test. In some embodiments, management module 480 communicates with client 420 via an API (e.g., client API 424 of client 420). Figure 4 The resource allocator 130 communicates with the memory component to identify the test and corresponding filter information (e.g., characteristic information and usage information). (For example, the resource allocator receives an instruction to perform a test on the memory component.)

[0046] At operation 520, processing logic generates test execution information defining one or more test processes for the test. In some embodiments, the test execution information includes, but is not limited to, a list of code dependencies associated with the test, run properties for each board corresponding to each reserved test component, and unique connection information (e.g., a unique IP address) for each individual test process associated with the test. In one example, the connection information can be provided by the management module as a run type input in a corresponding command line. In some embodiments, the management module transmits a list of each reserved host name (e.g., rack workstation) and each host to the client (e.g., Figure 4 The client can then use the test execution information to trigger or start one or more test processes on the reserved board and corresponding test components.

[0047] At operation 530, processing logic receives feedback information from each of the plurality of test processes via a corresponding connection established using the connection information provided for each test process. For example, the feedback information may include the number of program and erase cycles that the failed memory component has received, the temperature at which the failure of the memory component occurred, one or more temperatures at which data was written on the memory component, the state of the memory component (e.g., suspended), the time since the last memory operation, the state of the memory block (e.g., partially or fully programmed), etc.

[0048] In some embodiments, the feedback information may include information associated with the corresponding test process, for example, information identifying a failure (e.g., an exception or error) associated with the test process performed on one or more memory components, a list of one or more events that occurred in association with the failure of the test process, associated characteristic information and usage information of one or more memory components, and a set of conditions associated with the execution of the test process. Exemplary conditions associated with the test may include temperature conditions of the one or more memory components being tested, temperature as a function of time (e.g., temperature every second of the test), Vcc values, identification of pause and resume instances, etc. The management module may collect and aggregate the feedback information received from multiple test processes and store the collected feedback information in one or more databases (e.g., Figure 4 In some embodiments, the test process can identify the occurrence of an error and generate a list of events that caused or contributed to the error. The test process can set a flag to indicate the error and store the flag and event list in a file that is provided to the management module via the unique IP address in the form of feedback information.

[0049] At operation 540, the processing logic determines the failure of the first test process and the corresponding first test information. In some embodiments, based on the feedback information, the processing logic determines the failure of the test process and the information associated with the test process (e.g., the test information). For example, the management module may check the first test process. Figure 4 The management module may determine that process 1 failed during execution. The management module may further identify the feedback information, including the first test information associated with process 1 received via IP address 1. In some embodiments, the first test information may include one or more of an identification of command line information associated with process 1, associated feature information, associated usage information, a list of the one or more events or operations performed in conjunction with process 1, a set of one or more conditions associated with the execution of process 1, and any other information associated with the execution of process 1.

[0050] At operation 550, the management module can replicate the execution of the first process on one or more copy test components. The copy test component can be a selected test component that matches the test component used during the failed test process. For example, the copy test component can have features and usage information that match the test component of the failed test process. The management module can use the same or substantially similar test conditions used during the failure execution of the identification of the test process to replicate the first test process. In this regard, the management module can copy all variables (e.g., test conditions, test parameters, component parameters, etc.) to perform the failed test process in a controlled and monitored environment, thereby identifying the cause or source of the failure. In some embodiments, the copy test component can be located on different boards of the same rack or on different racks at different locations. In some embodiments, the management module can use one or more boards dedicated to acting as a copy test component. The set of events identified in the feedback information can be used to replicate errors using a dedicated copy test board.

[0051] Advantageously, failures associated with one or more distributed test processes can be identified using feedback information returned by the individual test processes via a direct connection established with the management module. The management module can replicate or recreate specific test conditions associated with the failed test process to determine whether the failed test process is an anomaly (e.g., due to noise) or an error that requires remediation. In some embodiments, if the replication of the failed test process produces a failure with respect to the one or more replicated components, the processing logic can identify the failure associated with the first memory component (e.g., one or more memory components involved in the first failed test process) as an actual error associated with the memory component. If the replication of the failed test process produces a successful result (i.e., no failure is detected), the processing logic identifies the failure associated with the first memory component as an anomaly (e.g., the initial failure is due to noise), rather than an error or failure of the first memory component. In some embodiments, the management module can identify that the failed process is due to an error, and can identify the source or cause of the error. In some embodiments, the management module can coordinate the release of the test component when the test is completed.

[0052] As described above, conventional quality and reliability testing of memory components is limited to testing a limited number of usage parameters (e.g., intrinsic charge loss (ICL), read disturb loss, program disturb loss, etc.) in a single chamber under a single set of conditions (e.g., temperature conditions). As described above, aspects of the present disclosure address these deficiencies by employing a distributed test platform that is configured to distribute multiple test components to perform tests and collect feedback information covering a wider set of conditions in the usage space associated with the memory components, thereby reducing gaps in traditional qualification testing.

[0053] In some embodiments, the test platform of the present disclosure can perform testing of usage space associated with a memory component, including test operations (e.g., erase, program, and read operations) to generate multiple different test samples using usage parameters of multiple different test dimensions.

[0054] The test methodology provides multi-dimensional usage space testing to force normally low probability events to occur at a higher frequency. The test methodology applies analytical techniques to process large amounts of test data. Advantageously, the test platform generates a set of different usage conditions for testing memory components.

[0055] In some embodiments, the test platform identifies and extracts failures that may occur in a memory component that can be observed during a write or read operation. For example, a write failure can be identified as an abnormal deviation in a failed state or program time. In another instance, a read failure can be identified as a margin loss, an unexpectedly high error rate, or unrecoverable data. The test platform can further identify the boundaries or limits of the usage space in which the memory component remains operable. For example, the test platform can determine whether the memory component is operable in a multidimensional usage space, such as under a combination of the following usage conditions: half of the total cycle, full crossover temperature, and half of the maximum allowed read. Note that any suitable combination of conditions of the usage space can be combined to produce a multidimensional space (e.g., a set of usage conditions) to test the memory component. In some embodiments, the test platform can develop a model to predict the optimal media management recovery flow for a given set of usage conditions.

[0056] In some embodiments, the test system utilizes multiple different sockets for multiple different tests of the usage space of the associated memory components. Figure 1-5 As described in detail, the test system uses connections to multiple test processes of multiple tests to collect test results, identify failures and determine the occurrence of errors. In one embodiment, the dimensions of the usage space associated with the memory component can be identified and classified as vectors, path variables or media management attributes. In one embodiment, the vector can represent an attribute describing the end state of the sample or constant use, such as, for example, total writes during the life of the memory component, retention baking at the end of the life of the memory component, etc. In some embodiments, the usage parameters that can be defined as path variables include attributes or conditions that change continuously during the operation of the memory component. In one embodiment, the path variable can be an attribute that changes during the life of the memory component. In one embodiment, the media management attribute can be an attribute that represents or describes how to read or recover data from the test sample (e.g., default read, read entry, automatic read calibration, soft read, cRLC, etc.).

[0057] In some embodiments, multiple different tests are associated with a common goal or target vector associated with a usage parameter (e.g., number of writes) of a memory component. For example, 100 memory components may be tested to generate test samples based on a defined goal (or target vector) of performing 10,000 write operations (i.e., usage parameters). The steps and conditions (i.e., path variables) associated with each test sample for reaching 10,000 write operations may be different. In some embodiments, different test conditions (i.e., a set of path variables) are applied to each corresponding test to generate a distributed sampling of test samples corresponding to the target vector.

[0058] Figure 66 is a flow chart of an exemplary method 600 (also referred to as a "container process") for expanding test samples and reducing test gaps in a usage space associated with memory component testing. The method 600 may be performed by processing logic that may include hardware (e.g., a processing device, a circuit, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 600 is performed at least in part by Figure 4 The management module 480 of the embodiment of the present invention is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the process can be modified. Therefore, the embodiments shown should be understood as examples only, and the processes shown can be performed in different orders, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not every embodiment requires all processes. Other process flows are also possible.

[0059] In operation 610, processing logic generates a target vector (e.g., a vector representation) corresponding to the usage parameters of the memory component test. In some embodiments, one or more first usage parameters of the usage space of the memory component can be established as a target vector. In some embodiments, the target vector represents the final goal or result that the test is configured to test or achieve. For example, the test can be configured to have 100 memory components perform 10,000 write operations. In this example, the target vector represents the endpoints of each memory component used in the test.

[0060] Exemplary usage parameters that may be represented as target vectors include write count, read count, block type, partial programming rate, interrupt type, time between writes, time between reads, block erase type, read page fraction, interrupt rate, time between writes and reads, block type rate, and interrupt time. In some embodiments, a target vector is defined as an attribute that describes an end state or constant usage of a test sample.

[0061] In operation 620, processing logic assigns the test samples to the target vector. In some embodiments, the test samples refer to one or more memory blocks of the memory component that experience the same set of test conditions during their lifetime. In some embodiments, processing logic assigns each test sample in a set of multiple test samples to the target vector. In some embodiments, Figure 1 and Figure 4 The resource allocator 130 may coordinate the assignment or allocation of test samples to various memory components using corresponding target vector information.

[0062] In operation 630, the processing logic generates a set of path variables to associate with the test samples. In some embodiments, the set of path variables are randomly selected by the processing logic so that each test sample follows a random path to the target vector. In some embodiments, one or more usage parameters of the usage space are selected as path variables. In one embodiment, the path variable can be a property that changes continuously during the life of the memory component. In one embodiment, the test samples assigned to the target vector follow the random path of the path variables to the target vector by reading using media management techniques. Exemplary usage parameters that can be selected as path variables include write temperature, written Vcc, Vcc noise, read temperature, write temperature, read Vcc, and concurrency of write and read operations.

[0063] In some embodiments, each test pattern (e.g., a set of memory component blocks) is assigned a separate target vector and follows a random path to the target vector (e.g., a set of randomly selected path variables). In some embodiments, processing logic establishes multiple test patterns corresponding to the usage space of the memory component to identify execution results at a multi-dimensional level of the usage space associated with the memory component. Fig. 7A An example of two three-dimensional target vectors (e.g., representing usage parameters for retention, program-erase (P / E) cycles, and reads) and three associated random paths leading to the corresponding target vectors is shown. For example, the target vector may specify usage parameters for 10,000 write operations to be performed at each memory component assigned to the test. Each memory component may perform 10,000 write operations corresponding to the usage parameters, but for each memory component used in the test, the path or execution of the additional operation based on the path variable may be different. For example, the first memory component may perform 10,000 write operations and perform an intermediate read operation after every 100 write operations. The second memory component may perform 10,000 write operations, but perform an intermediate erase operation after every 120 write operations. Therefore, each memory component used in the test may perform the same 10,000 write operations, but during the execution of the 10,000 write operations, the intermediate operations or other such parameters may be different. For example, each memory component may perform 10,000 write operations, but under different temperature conditions that vary at different rates and times.

[0064] In some embodiments, processing logic generates a plurality of test sample points corresponding to the usage space by generating a plurality of sets of path variables associated with a plurality of different test samples. Figure 7BAn exemplary generation of 10,000 test sample points of a two-dimensional sequence is shown. In some embodiments, the test sample points are generated using a methodology of a sequencing method (e.g., a low-variance method such as Sobol sequencing, etc.). In some embodiments, a set of target vector samples are generated based on the test sample points. Figure 7C A set of 1,000 target vector samples are shown, including vector type labels generated by the processing logic of the present disclosure. In some embodiments, the target vector samples are generated using low-discrepancy sequences, such as the Sobol sequencing method.

[0065] In block 640, processing logic performs a test using the test sample according to the set of path variables to generate a test result. In some embodiments, the set of path variables includes one or more random path variables. In some embodiments, having established a target vector corresponding to the test sample and the set of path variables, processing logic can identify and retain a memory component to be tested. In some embodiments, processing logic (e.g., Figure 1 and 4 The processing logic of the resource allocator 130 of the embodiment of the present invention identifies and reserves test resources corresponding to the set of path variables (eg, test sockets including memory components matching the usage parameters of the set of path variables) for use as test samples.

[0066] In operation 650, processing logic analyzes the test results (e.g., the first test result associated with the first test sample and other test results corresponding to the respective test samples) to identify a failure. In some embodiments, one or more failures associated with the one or more test samples may be identified. In some embodiments, dimensional data from the randomized tests is stored in a file and parsed into a smaller representation (e.g., a table). In one embodiment, failure detection and the associated determination of whether the failure is a true error or an anomaly (e.g., due to noise) may be performed by combining the above Figure 4 and 5 In some embodiments, the test results associated with each test sample are provided as feedback information to the management module (e.g., Figure 4 management module 480).

[0067] In one example, one or more functional failures associated with erase, program, and read operations are visible and constitute a "hit" to the test blocks involved in the test process. In one example, a read raw bit error rate (RBER) threshold can be applied to trigger an error handling flow configured to recover read operations (e.g., by selecting various fine-tuning (e.g., commands that enable an operating system to notify a memory component (e.g., with a non-flash solid state drive (SSD)) which data blocks can be erased because they are no longer used)), an adaptive replacement cache (ARC), a Vt scan for segmenting failures due to errors associated with the memory component (e.g., a 'hit'). In some embodiments, after a threshold number of hits is reached (e.g., after three hits), the block can exit the test or be removed from the test. In some embodiments, anomalies can also be classified as statistical outliers, unstable behavior, or other pathological behavior in an analysis (e.g., RBER analysis, typical page program time (tProg), tErase response, etc.). In some embodiments, anomaly detection analysis can be performed in conjunction with the above parsing and / or during post-processing of the output table.

[0068] Figure 8 An exemplary machine of a computer system 800 is shown in which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein. In some embodiments, the computer system 800 may correspond to a host or server system that includes, is coupled to, or utilizes a test platform that includes Figure 4 The resource allocator 130 and Figure 4 One or more of the management modules 480 (e.g., to execute corresponding Figure 2 , 5 In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine may operate in the capacity of a server or a client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0069] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the machine. Furthermore, while a single machine is shown, the term "machine" should also be understood to include any collection of machines that individually or collectively execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0070] The exemplary computer system 800 includes a processing device 802, a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or random access DRAM (RDRAM)), etc.), a static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 818, which communicate with each other via a bus 830.

[0071] Processing device 802 represents one or more general processing devices, such as microprocessors, central processing units, etc. More specifically, the processing device can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. Processing device 802 can also be one or more special processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. Processing device 802 is configured to execute instructions 826 for performing the operations and steps discussed herein. Computer system 800 may further include a network interface device 808 that communicates via a network 820.

[0072] The data storage system 818 may include a machine-readable storage medium 824 (also referred to as a computer-readable medium) on which is stored one or more sets of instructions 826 or software implementing any one or more of the methodologies or functions described herein. The instructions 826 may also reside completely or at least partially in the main memory 804 and / or the processing device 802 during execution by the computer system 800, the main memory 804 and the processing device 802 also constituting machine-readable storage media. The machine-readable storage medium 824, the data storage system 818, and / or the main memory 804 may correspond to a memory subsystem.

[0073] In one embodiment, instructions 826 include implementing a test process corresponding to that described herein (e.g., Figure 2 , 5 and 6 processes 200, 500 and 600), Figure 4 The resource allocator 130 and Figure 4 The machine-readable storage medium 824 is shown as a single medium in the exemplary embodiment, but the term "machine-readable storage medium" should be understood to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be understood to include any medium that can store or encode a set of instructions executed by a machine and cause the machine to perform any one or more methodologies of the present disclosure. Therefore, the term "machine-readable storage medium" should be understood to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0074] Some parts of the above detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate their work content to other persons skilled in the art. Here, an algorithm is generally considered to be a self-consistent sequence of operations that leads to a desired result. These operations are those that require physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. It has proven to be convenient, primarily for general reasons, to sometimes refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc.

[0075] It should be remembered, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities in the computer system's registers and memories into other data similarly represented as physical quantities in the computer system's memories or registers or other such information storage systems.

[0076] The present disclosure also relates to an apparatus for performing the operations herein. The apparatus may be specially constructed for the intended purpose, or it may include a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk, including a floppy disk, an optical disk, a CD-ROM and a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card or an optical card, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0077] The algorithms and displays presented herein are not inherently related to any particular computer or other device. According to the teachings herein, various general purpose systems can be used together with the program, or it may prove convenient to construct more specialized equipment to perform the method. The structures of various these systems will be set forth in the following description. In addition, the present disclosure is not described with reference to any particular programming language. It should be understood that various programming languages ​​can be used to implement the teachings of the present disclosure described herein.

[0078] The present disclosure may be provided as a computer program product or software, which may include a machine-readable medium having instructions stored thereon, which may be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a machine (e.g., computer) readable form. In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a disk storage medium, an optical storage medium, a flash memory component, etc.

[0079] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments thereof. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of embodiments of the present disclosure as set forth in the following claims. Therefore, the description and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. A method for memory testing, comprising: Determining, by a processing device, filter information associated with a test to be performed on one or more memory components, wherein the filter information includes: a feature information, usage information, or both, and wherein the characteristic information and the usage information are associated with at least a portion of the one or more memory components; selecting, based on the filter information, a first test component of a first group of memory components including the one or more memory components and a second test component of a second group of memory components including the one or more memory components; generating test execution information defining a set of test procedures for the test; distributing a first portion of the test execution information corresponding to a first test process to the first test component including the first group of memory components; distributing a second portion of the test execution information corresponding to a second test process to the second test component including the second set of memory components, wherein the second test process is different from the first test process; receiving first feedback information via a first connection with the first test process in the set of test processes; as well as A failure of the first testing process is determined based on the first feedback information.

2. The method according to claim 1, further comprising: receiving second feedback information via a second connection with the second test process in the set of test processes; and Successful execution of the second testing procedure is determined based on the second feedback information.

3. The method of claim 1, further comprising replicating the first test process on a replicated test component using the first feedback information.

4. The method of claim 3, further comprising determining, based on the replication of the first test process that produced the failure, that the failure represents an error associated with a memory component in the first set of memory components.

5. The method of claim 3, further comprising determining, based on the copy of the first test process that produced a successful result, that the failure represents an anomaly associated with a memory component in the first set of memory components.

6. The method of claim 1, further comprising providing unique connection information to the first test process to establish the first connection to transmit the first feedback information.

7. A system for memory testing, comprising: a set of memory components; and a processing device operatively coupled to the set of memory components to: receiving a request to perform a test of the set of memory components; determining filter information corresponding to one or more conditions of the test; retaining a first test component comprising a first portion of the set of memory components, wherein the first portion comprises: first characteristic information, first usage information, or both, and wherein the first characteristic information and the first usage information match the filter information; retaining a second test component comprising a second portion of the set of memory components, wherein the second portion includes: second characteristic information, second usage information, or both, and wherein the second characteristic information and the second usage information match the filter information; performing a first sequence of operations of the test using the first portion of the set of memory components; performing a second sequence of operations of the test using the second portion of the set of memory components, wherein the first sequence is different from the second sequence; receiving, via a connection with a first test procedure of the test, feedback information indicating a first failed test result; and The first test process is replicated on a replicated test component using the feedback information.

8. The system of claim 7, wherein the first test assembly is located in a first position and the second test assembly is located in a second position.

9. The system of claim 7, wherein the first test component includes an available test socket that includes the first portion of the set of memory components to be tested.

10. The system according to claim 7, wherein the processing device is further configured to: Test execution information defining the first sequence of operations for the test and the second sequence of operations for the test is generated.

11. The system according to claim 10, wherein the processing device is further configured to: Additional feedback information is received indicating successful performance of the second sequence of operations.

12. The system of claim 11, wherein the first portion of the test execution information includes connection information assigned to the first test procedure.

13. The system according to claim 12, wherein the processing device is further configured to: Based on the first test process replicated on the replicated test component that produced a second failing test result, it is determined that the first failing test result represents an error associated with a memory component in the first portion of the set of memory components.

14. The system according to claim 13, wherein the processing device is further configured to: The first failed test result is determined to represent an anomaly associated with a memory component in the first portion of the set of memory components based on the first test procedure replicated on the replicated test component that produced a successful test result.

15. A non-transitory computer readable medium comprising instructions which, when executed by a processing device, cause the processing device to perform operations comprising: determining filter information associated with a test to be performed on one or more memory components, wherein the filter information includes: a feature information, usage information, or both, and wherein the characteristic information and the usage information are associated with at least a portion of the one or more memory components; selecting, based on the filter information, a first test component of a first group of memory components including the one or more memory components and a second test component of a second group of memory components including the one or more memory components; distributing a first portion of test execution information corresponding to a first test process to the first test component including the first set of memory components; distributing a second portion of the test execution information corresponding to a second test process to the second test component including the second set of memory components, wherein the second test process is different from the first test process; receiving first feedback information via a first connection with the first test process in a set of test processes; Determine a first failure test result of the first test process based on the first feedback information; as well as The first test process is replicated on a first replicated test component using the first feedback information.

16. The non-transitory computer readable medium of claim 15, wherein replicating the first test process on the first replica test component produces a second failing test result.

17. The non-transitory computer readable medium of claim 16, the operations further comprising: Based on the second failing test result, it is determined that the first failing test result indicates an error associated with a memory component in the first group of memory components.

18. The non-transitory computer readable medium of claim 15, wherein replicating the first test process on the first replica test component produces a successful test result.

19. The non-transitory computer readable medium of claim 18, the operations further comprising: Based on the successful test result, it is determined that the first failed test result represents an anomaly associated with a memory component in the first group of memory components.

20. The non-transitory computer readable medium of claim 15, wherein each test process in the set of test processes is assigned a unique IP address for establishing a connection to provide feedback information associated with the corresponding test process.

Images