Sample serialization method and apparatus
A serialization and sample technology, applied in the field of machine training, can solve the problems of long serialization time and insufficient loading of single-machine memory
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0043] refer to figure 1 , which shows a flow chart of the steps of an embodiment of a sample serialization method of the present application, which may specifically include the following steps:
[0044] Step 110, obtaining each character string in the sample to be serialized;
[0045] In the embodiment of this application, the serialization server first receives the sample data to be serialized. In a preferred embodiment, before step 110, it also includes:
[0046] Step S100, obtaining each sample data to be serialized;
[0047] This application embodiment can have one or more serialization server slaves. Each serialization server can obtain a batch of sample data processed by the serialization server according to the notification of the scheduling server coordinator.
[0048] In the embodiment of this application, each serialization server, each management server, and scheduling server can form a training cluster for machine training.
[0049] In another preferred embodi...
Embodiment 2
[0090] refer to figure 2 , which shows a flow chart of the steps of an embodiment of a sample serialization method of the present application, which may specifically include the following steps:
[0091] Step 210, receiving a character string; the character string is sent by the serialization server according to the correspondence between the character string and each management server; the character string is obtained from the sample data by the sequence server;
[0092] In this embodiment of the application, each management server receives the character string sent by one or several serialization servers.
[0093] In this embodiment of the application, on the serialization server side, for the sample data to be serialized, a character string can be extracted from it, and then the management server can be determined according to the correspondence between the character string and each management server, and then the character string can be sent to The management server.
...
Embodiment 3
[0130] refer to image 3 , shows a flow chart of steps of a preferred embodiment of a sample serialization method in the present application.
[0131] In order to describe the serialization method more clearly in this embodiment, the description is made from the perspective of the overall architecture of the scheduling server, the serialization server, and the management server.
[0132] In the embodiment of the present application, the scheduling server and the serialization server may be combined to create a mapping table for each management server. Such as step S30-step S38.
[0133] In step S32, the scheduling server distributes all sample data equally, and notifies each serialization server to acquire a batch of sample data belonging to each serialization server according to the distribution result.
[0134] Before the entire training starts, after the scheduling server obtains the identification information of all sample data, it can evenly distribute all sample data. ...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com