Data management and control method, system and device and medium
A data control and data technology, applied in multi-programming devices, electrical digital data processing, version control, etc., can solve problems such as the reduction of data processing efficiency, and achieve the effect of reducing storage and improving storage utilization.
Pending Publication Date: 2021-10-22
江苏云从曦和人工智能有限公司
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] The present invention aims to solve the above technical problems, that is, to solve the problem of how to effectively reduce the storage space of these data sets for different versions of data sets with a part of repeated data, and the data processing caused by external devices such as GPUs reading multiple data sets at the same time The problem of reduced efficiency
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0071] Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention, and are not intended to limit the protection scope of the present invention.
[0072] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of both. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor can be implemented in software, hardware or a combination of both. The non-transitory computer readabl...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to the technical field of data management and control, in particular to a data management and control method, system and device and a medium, and aims at solving the problem of how to effectively reduce the storage space of data sets and solving the problem of low data processing efficiency caused by the fact that external equipment reads multiple data sets. In order to achieve the purpose, the data management and control method comprises the steps: after a data query request sent by a data demand platform is received, determining a version directory of an image-text data set file of each version in a version dependence path according to the request, and further combining the version directories to generate a new version directory, so that the data demand platform reads the corresponding image-text data set file; wherein the image-text data set file of each version in the version dependence path is created through data increment updating according to the image-text data set file of the previous version, and it is ensured that the data size of the image-text data set meets the requirement of the data demand platform; the bottom storage space is effectively utilized, and the data processing efficiency of the data demand platform is improved.
Description
technical field [0001] The present invention relates to the technical field of data management and control, and specifically provides a data management and control method, system, device and medium. Background technique [0002] The training platform provides functions including dataset data management, data labeling and model training. Among them, the data set contains a large number of data files, including pictures, videos, and text files. At the same time, due to the needs of model training, there will be different versions of the data set to achieve different purposes of model training and training improvement functions. There is an inheritance relationship between data sets of different versions, for example, the data set of the V2 version is derived from the data set of the V1 version. Therefore, there will be a lot of duplicate data between different versions of the data set, and there will also be some unique differences. [0003] However, a large amount of dupli...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50G06F8/71
CPCG06F9/5027G06F8/71
Inventor 余晓峰
Owner 江苏云从曦和人工智能有限公司