A data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device, electronic equipment and storage medium. The method comprises: in response to a data storage request, obtaining initial to-be-stored data; determining a first heat type of the initial to-be-stored data, and determining an initial heat area in a first storage area in a data lake according to the first heat type; determining a first sensitivity degree of the initial to-be-stored data, and determining an initial sensitivity area in the initial heat area according to the first sensitivity degree; and storing the initial to-be-stored data into the initial sensitivity area. The technical solution of the embodiments of the present application can realize efficient division of area storage data, thereby facilitating the management and use of data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and particularly relate to a data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the development of informationization and digitization of various industries, it is very important to build a complete and efficient data system.

[0003] At present, the amount of data is increasing, and various types of data are mixed together. There is a lack of efficient data processing method to divide the data, which is not conducive to the management and use of data, and needs to be improved. SUMMARY

[0004] Embodiments of the present application provide a data processing method and device, electronic equipment and storage medium to realize efficient division of regional data storage, thereby facilitating the management and use of data.

[0005] According to an aspect of the present application, a data processing method can include:

[0006] In response to a data storage request, obtaining initial data to be stored;

[0007] Determining a first heat type of the initial data to be stored, and determining an initial heat area in a first storage area in the data lake according to the first heat type;

[0008] Determining a first sensitivity degree of the initial data to be stored, and determining an initial sensitivity area in the initial heat area according to the first sensitivity degree;

[0009] Storing the initial data to be stored into the initial sensitivity area.

[0010] According to another aspect of the present application, a data processing device can include:

[0011] A data to be stored obtaining module for obtaining initial data to be stored in response to a data storage request;

[0012] An initial heat area determining module for determining a first heat type of the initial data to be stored, and determining an initial heat area in a first storage area in the data lake according to the first heat type;

[0013] An initial sensitivity area determining module for determining a first sensitivity degree of the initial data to be stored, and determining an initial sensitivity area in the initial heat area according to the first sensitivity degree;

[0014] An initial data to be stored storing module for storing the initial data to be stored into the initial sensitivity area.

[0015] According to another aspect of the present application, there is provided an electronic device, which can comprise:

[0016] at least one processor; and

[0017] a memory connected with the at least one processor in communication; wherein,

[0018] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to cause the at least one processor to implement the data processing method provided by any of the embodiments of the present application when executed.

[0019] According to another aspect of the present application, there is provided a computer readable storage medium having stored thereon computer instructions for causing a processor to implement the data processing method provided by any of the embodiments of the present application when executed.

[0020] The technical solution of the embodiments of the present application can efficiently divide data in a data processing manner, and facilitate the management and use of data. The above technical solution stores data in a storage area divided according to multiple dimensions such as data processing stage, heat type and sensitivity, thereby realizing efficient division of the storage area and storage of data, and facilitating the management and use of data.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0023] Figure 1 is a flowchart of a data processing method provided in the first embodiment of the present application;

[0024] Figure 2 is a flowchart of a data processing method provided in the second embodiment of the present application;

[0025] Figure 3 is a region division structure diagram corresponding to a data storage mode using multiple dimensions in a data lake provided in Embodiment Two of the present application;

[0026] Figure 4 is a flowchart of a data processing method provided in Embodiment Three of the present application;

[0027] Figure 5 is a flowchart of an optional example in a data processing method provided in Embodiment Three of the present application;

[0028] Figure 6 is a structural block diagram of a data processing device provided in Embodiment Four of the present application;

[0029] Figure 7 is a structural schematic diagram of an electronic device implementing the data processing method of the embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The case of "target", "original" and the like is similar, and will not be described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment One

[0033] Figure 1is a flowchart of a data processing method provided in Embodiment One of the present application. The present embodiment can be applied to the case of processing data, specifically, the case of processing data on the basis of a data lake. The method can be executed by a data processing apparatus provided in the present application, which can be implemented in the form of software and / or hardware, and can be integrated on an electronic device, which can be a data lake platform, various user terminals or servers.

[0034] Referring to Figure 1 The method of the present embodiment specifically includes the following steps:

[0035] S110, in response to a data storage request, obtaining initial to-be-stored data.

[0036] The data storage request can be understood as a request indicating storage of data. The initial to-be-stored data can be understood as data to be stored into the data lake.

[0037] It should be understood that the data lake is a centralized repository, which is a database that stores various large raw data sets in native format, and the data lake allows storage of all structured or unstructured data at any scale.

[0038] In the present embodiment, the initial to-be-stored data can also be converted into the form of a string after being obtained, so as to ensure the integrity of the initial to-be-stored data.

[0039] S120, determining a first hotness type of the initial to-be-stored data, and determining an initial hotness area in a first storage area in the data lake according to the first hotness type.

[0040] The first hotness type is a hotness type of the initial to-be-stored data. The hotness type can be understood as a data type capable of reflecting the access frequency of the data. The hotness type of the data can be determined according to the data state of the data and the access frequency within a preset time length. The access frequency can be determined according to a data access auditing mechanism, for example, and the method for determining the access frequency is not limited in the embodiment of the present application. The data state can be understood as a working period in which the data is located, and the data state can include a start period, a working period, an end period, a post-service period, and the like, for example. The hotness type can include cold data, warm data, or hot data. The cold data can be understood as a data type with a low access frequency, for example, data with a low access frequency within a preset time length and located in an end period is cold data. The hot data can be understood as a data type with a high access frequency, for example, data with a high access frequency within a preset time length and located in a working period is hot data. The warm data can be understood as a data type with a medium access frequency, for example, data with a general access frequency within a preset time length and located in a working period is warm data. The hotness type of the data can be determined according to a preset threshold range of the access frequency within the preset time length, for example, if the access frequency of the data within the preset time length is less than the minimum value in the preset threshold range, the data is cold data; if the access frequency of the data within the preset time length is within the preset threshold range, the data is warm data; and if the access frequency of the data within the preset time length is greater than the maximum value in the preset threshold range, the data is hot data. The initial hotness region can be understood as a hotness region required to be stored by the initial to-be-stored data when the data is stored in the data lake. The data lake is an operation of storing the data in the data lake. The hotness region can be understood as a storage region of the stored data corresponding to the hotness type, for example, in the case where the hotness type can include cold data, warm data, or hot data, the hotness region can include a cold data region corresponding to the cold data, a warm data region corresponding to the cold warm data, or a hot data region corresponding to the hot data.

[0041] It should be noted that there can be a case where the data access frequency is high and the current data state of the data just reaches the end period. In this case, although the data has a high access frequency within a preset time, the work of the data has been completed, and the data can not be accessed frequently thereafter. Therefore, the hotness type of the data can be cold data. There can also be a case where the data access frequency is low and the current data state of the data is the end period. However, there can be a post-service period after the end period. In this case, although the data is located in the end period and has a low access frequency within a preset time, the data can be accessed moderately in the post-service period. Therefore, the hotness type of the data can be warm data.

[0042] It should be noted that in the embodiments of the present application, a plurality of dimensions of data storage mode are adopted in the data lake, that is, a first storage area can be set for the initial storage data into the lake stage in the data lake according to the processing stage of the data, and the first storage area is the area set for the initial storage data into the lake in the data lake, for example, the first storage area can be a raw area, and the set raw area can retain the original appearance of the initial storage data, without any processing of the initial storage data, and strictly controls the access permission, which provides guarantee for subsequent data backtracking and verification. In the first storage area, the area can be further divided according to the heat type of the data, for example, the first storage area can be divided into a cold data area, a hot data area and a warm data area, so that the initial storage data is stored in the corresponding initial heat area according to the first heat type, thereby facilitating effective data division storage when subsequent data storage is implemented.

[0043] In the embodiments of the present application, the first heat type of the initial storage data can be determined, and the initial heat area in the first storage area in the data lake is determined according to the first heat type, for example, the first heat type is cold data, and the determined initial heat area is a cold data area.

[0044] It can be understood that if the initial storage data carries a related field or label of the heat type, or the data state of the initial storage data and the access frequency within a preset time length can be determined, the first heat type can be determined according to the related field or label of the heat type, or the data state of the initial storage data and the access frequency within a preset time length. However, there can be a case that the initial storage data does not carry a related field or label of the heat type, and the data state of the initial storage data and the access frequency within a preset time length cannot be determined, that is, the first heat type cannot be determined, and the default heat type preset according to the nature of the first storage area can be used as the first heat type to prevent the case that the first heat type cannot be determined. The default heat type is a heat type preset according to the nature of the first storage area, for example, if the first storage area is a raw area, the data in the raw area is the original data stored for subsequent data backtracking and verification, so that the access frequency of most data in this area is very low, that is, most of the data stored in the raw area is cold data, based on this nature, the raw area can be preset with a heat type of cold data.

[0045] In the embodiments of the present application, different types of storage media or data storage methods are also set for different types of hotness regions. For example, since the data stored in the hot data region has a high access frequency, and the solid state disk (SSD) has the characteristics of flash memory, the SSD can be set as the storage medium for the hot data region. Since the data stored in the cold data region has a low access frequency, and the hard disk drive (HDD) has a relatively large storage space and a relatively low cost, the HDD can be set as the storage medium for the cold data region. That is, one partitioned region of the data lake can correspond to multiple different types of storage media. For another example, since the data stored in the hot data region has a high access frequency, a memory database suitable for a large amount of access data can be set as the storage method for the hot data region. Since the data stored in the cold data region has a low access frequency, a relational database suitable for storing a large amount of data can be set as the storage method for the cold data region.

[0046] In the embodiments of the present application, the data lake can also be partitioned according to the data processing stage, for example, the first storage region. According to the storage region divided according to the data processing stage, the space size occupied by different hotness regions in the storage region can be determined based on the property of the storage region. For example, the data lake includes an original region. Since most of the data in the original region is cold data, the cold data region in the original region can occupy more storage space.

[0047] In the embodiments of the present application, according to the storage region divided according to the data processing stage, in the case that only one storage medium can be used in one storage region, the corresponding storage medium of the storage region can also be determined based on the property of the storage region. For example, most of the data in the original region has a low storage frequency, and the cold data is more, so the HDD can be used as the storage medium of the original region.

[0048] S130, determine the first sensitivity degree of the initial to-be-stored data, and determine the initial sensitivity degree region in the initial hotness region according to the first sensitivity degree.

[0049] The first sensitivity degree is the sensitivity degree of the initial to-be-stored data. The sensitivity degree can reflect the confidentiality of the data. The sensitivity degree of the data can be determined by machine learning or clustering algorithm, for example, can be determined according to a pre-trained sensitivity determination model; can also be determined by keywords related to the sensitivity degree in the data, for example, if there are keywords such as 'name', 'gender' or 'phone' in the data, the sensitivity degree is confidential data; can also be determined according to the sensitivity label carried by the initial to-be-stored data itself. The sensitivity degree can include public data, internal data or confidential data. The public data can be understood as data that does not involve relatively confidential data, which can be publicly visible or publicly disclosed by the data holder. The internal data can be understood as data that is generally visible to authorized users, or data that is visible after being applied by the data access party and authorized by the holder; the internal data can be the default sensitivity degree when the initial to-be-stored data is entered into the lake. The initial sensitivity degree area can be understood as the sensitivity degree area required for the initial to-be-stored data to be stored when entering the lake. The sensitivity degree area can be understood as a storage area of the stored data corresponding to the sensitivity degree, for example, in the case where the sensitivity degree can include public data, internal data or confidential data, the hotness area can include a public data area corresponding to the public data, an internal data area corresponding to the internal data, or a confidential data area corresponding to the confidential data.

[0050] It should be noted that in the embodiments of the present application, since the data storage method of multiple dimensions is adopted in the data lake, the hotness area can be further divided according to the sensitivity degree of the data, for example, the hotness area can be divided into a public data area, an internal data area and a confidential data area, so that the initial to-be-stored data is stored in the corresponding sensitivity degree area according to the first sensitivity degree, thereby facilitating further effective data division and storage when storing data subsequently.

[0051] S140, store the initial to-be-stored data into the initial sensitivity degree area.

[0052] In the embodiments of the present application, the initial to-be-stored data can be stored in the initial sensitivity degree area in the initial hotness area in the data lake, thereby realizing the storage of the initial to-be-stored data in the data lake divided into multiple dimensions, and realizing efficient division and storage of data.

[0053] In the embodiments of the present application, after storing the initial to-be-stored data into the initial sensitivity degree area, dynamic data hotness area adjustment can be performed based on the Least Recently Used (LRU) algorithm, for example, dynamic data hotness area adjustment can be performed based on the LRU-3 algorithm with 3 times of recent use.

[0054] The technical scheme of the embodiment of the present application, in response to a data storage request, obtains initial to-be-stored data; determines a first heat type of the initial to-be-stored data, and determines an initial heat area in a first storage area in a data lake according to the first heat type; determines a first sensitive degree of the initial to-be-stored data, and determines an initial sensitive degree area in the initial heat area according to the first sensitive degree; and stores the initial to-be-stored data into the initial sensitive degree area. The technical scheme of the embodiment of the present application can divide data in an efficient data processing manner, facilitating the management and use of data. The above technical scheme stores data into a storage area divided according to multiple dimensions such as data processing stage, heat type and sensitive degree, thereby realizing efficient division of the storage area and storage of data, and facilitating the management and use of data.

[0055] An optional technical scheme, determining the first sensitive degree of the initial to-be-stored data, comprises: inputting the initial to-be-stored data into a pre-trained sensitive degree determination model, and determining the first sensitive degree of the initial to-be-stored data according to an output result of the sensitive degree determination model.

[0056] In the embodiment of the present application, the sensitive degree determination model can be trained in advance according to existing or self-set training samples. After obtaining the initial to-be-stored data in response to the data storage request, the initial to-be-stored data is input into the pre-trained sensitive degree determination model, and the first sensitive degree of the initial to-be-stored data is determined according to the output result of the sensitive degree determination model, so as to improve the accuracy of the determined first sensitive degree.

[0057] It should be noted that after the first sensitive degree is determined, a corresponding sensitive degree label can be established for the initial to-be-stored data, so that when the sensitive degree of the data is required to be known, the sensitive degree of the data can be known only according to the sensitive degree label, without the need to determine the sensitive degree again through the sensitive degree determination model.

[0058] Embodiment two

[0059] Figure 2is a flowchart of a data processing method provided in Embodiment Two of the present application. The present embodiment is optimized based on the above technical solutions. In the present embodiment, optionally, the data lake further comprises a second storage area; and the above data processing method further comprises: in response to a first processing instruction, determining first to-be-stored data from the first storage area, and determining a first hotness area corresponding to the first to-be-stored data in the first storage area and a first sensitivity area corresponding to the first to-be-stored data in the first hotness area; determining a second hotness area corresponding to the first hotness area from the second storage area, and determining a second sensitivity area corresponding to the first sensitivity area from the second hotness area; and determining structured data of the first to-be-stored data, and storing the structured data into the second sensitivity area. Wherein, the same or corresponding terms in the above embodiments are not repeated here.

[0060] Referring to Figure 2 The method of the present embodiment can specifically include the following steps:

[0061] S210, in response to a data storage request, obtaining initial to-be-stored data.

[0062] S220, determining a first hotness type of the initial to-be-stored data, and determining an initial hotness area in a first storage area in a data lake according to the first hotness type, wherein the data lake further comprises a second storage area.

[0063] It should be noted that in the present embodiment, since a multi-dimensional data storage mode is adopted in the data lake, i.e., in addition to setting the first storage area for the to-be-stored data lake stage, a second storage area can also be set for the data aggregation stage, which is the area for storing the first to-be-stored data aggregated as structured data in the data lake. For example, the second storage area can be an aggregation area, which can perform necessary metadata analysis and extraction in combination with the metadata center, and convert semi-structured data into structured data. The second storage area can be further divided into hotness areas according to the hotness type of the data, and further divided into sensitivity areas according to the sensitivity degree of the data. The multi-dimensional division of the second storage area is the same as the division of the first storage area, which will not be described in detail here. Wherein, the first to-be-stored data can be understood as data that needs to be stored in the first storage area of the second storage area.

[0064] S230, determining a first sensitivity degree of the initial to-be-stored data, and determining an initial sensitivity area in the initial hotness area according to the first sensitivity degree.

[0065] S240, storing the initial to-be-stored data into the initial sensitivity area.

[0066] S250, in response to the first processing instruction, determining the first to-be-stored data from the first storage area, and determining a first hotness area corresponding to the first to-be-stored data in the first storage area and a first sensitivity area corresponding to the first to-be-stored data in the first hotness area.

[0067] The first processing instruction can be understood as an instruction indicating that the first to-be-stored data is determined from the first storage area and stored in the second storage area. The first hotness area can be understood as a hotness area in which the first to-be-stored data is located in the first storage area. The first sensitivity area can be understood as a sensitivity area in which the first to-be-stored data is located in the first hotness area.

[0068] S260, determining a second hotness area corresponding to the first hotness area from the second storage area, and determining a second sensitivity area corresponding to the first sensitivity area from the second hotness area.

[0069] The second hotness area can be understood as a hotness area in which the first to-be-stored data needs to be stored in the second storage area. The second sensitivity area can be understood as a sensitivity area in which the first to-be-stored data needs to be stored in the second hotness area.

[0070] It can be understood that since the first hotness area can represent the hotness type of the first to-be-stored data, and when the first to-be-stored data is stored in the second storage area, it also needs to be stored in the hotness area corresponding to its hotness type, therefore, the hotness area in the second storage area with the same hotness type as the first hotness area is the second hotness area in which the first to-be-stored data needs to be stored. Since the first sensitivity area can represent the sensitivity type of the first to-be-stored data, and when the first to-be-stored data is stored in the first hotness area, it also needs to be stored in the sensitivity area corresponding to its sensitivity type, therefore, the sensitivity area in the second hotness area with the same sensitivity type as the first sensitivity area is the second sensitivity area in which the first to-be-stored data needs to be stored.

[0071] S270, determining the structured data of the first to-be-stored data, and storing the structured data in the second sensitivity area.

[0072] It can be understood that the data stored in the second storage area is structured data, therefore, when the first to-be-stored data is unstructured data, the structured data corresponding to the first to-be-stored data can be determined; when the first to-be-stored data is structured data, the first to-be-stored data is taken as structured data. Then the structured data is stored in the second sensitivity area.

[0073] The technical scheme of the embodiment of the present application, the data lake further comprises a second storage area; in response to a first processing instruction, first to-be-stored data is determined from the first storage area, and a first hotness area corresponding to the first to-be-stored data in the first storage area and a first sensitivity area are determined, wherein the first to-be-stored data is stored in the first sensitivity area in the first hotness area; a second hotness area corresponding to the first hotness area is determined from the second storage area, and a second sensitivity area corresponding to the first sensitivity area is determined from the second hotness area; structured data of the first to-be-stored data is determined, and the structured data is stored in the second sensitivity area. The above technical scheme further divides the area in the data lake according to the data processing stage, and then stores the data in the storage area divided according to the hotness type and the sensitivity and other dimensions, thereby further realizing efficient division of the area for storing the data, and facilitating management and use of the data.

[0074] An optional technical scheme, before the structured data is stored in the second sensitivity area, further comprising: in a case where a second sensitivity degree of the first to-be-stored data corresponding to the structured data is a preset degree, performing desensitization processing on the structured data.

[0075] It can be understood that part of the relatively sensitive data may cover confidential content, and such data needs to be desensitized to ensure the security of the data. Therefore, a preset degree can be preset, and the preset degree is a sensitivity degree that needs to be desensitized due to the existence of sensitive content, for example, confidential data; in a case where a second sensitivity degree of the first to-be-stored data corresponding to the structured data is a preset degree, the structured data is desensitized to ensure the security of the data and avoid leakage of sensitive content.

[0076] Another optional technical scheme, the data lake further comprises a third storage area; after the structured data is stored in the second sensitivity area, further comprising: in response to a second processing instruction, second to-be-stored data is determined from the second storage area, and a third hotness area corresponding to the second to-be-stored data in the second storage area and a third sensitivity area are determined, wherein the second to-be-stored data is stored in the third sensitivity area in the third hotness area; a fourth hotness area corresponding to the third hotness area is determined from the third storage area, and a fourth sensitivity area corresponding to the third sensitivity area is determined from the fourth hotness area; the second to-be-stored data is stored in the fourth sensitivity area.

[0077] It should be noted that in the embodiment of the present application, since the multi-dimensional data storage mode is adopted in the data lake, that is, in the data lake, the third storage area can be set for the data usage in addition to the first storage area and the second storage area set for the data lake stage of the to-be-stored data, the third storage area is the area for storing the second to-be-stored data when the demand for using the second to-be-stored data is set in the data lake, for example, the third storage area can be a work area or a service area, the work area can be accompanied by a large number of data access operations, for example, a large number of data access operations using the second to-be-stored data for data warehouse hierarchical modeling access and other core work in the work area; the service area can provide data services to the outside, and efficient data access, permission management and data sharing are the core of this layer, so the service area also needs to use the second to-be-stored data through a large number of data access operations. The third storage area can be further divided into hotness areas according to the heat type of the data, and the sensitivity areas are divided according to the sensitivity of the data in the hotness areas. The multi-dimensional division of the third storage area is the same as the division of the first storage area and the second storage area, and will not be described in detail here. The second to-be-stored data can be understood as data required to be stored in the second storage area of the third storage area.

[0078] Figure 3 is a region division structure diagram corresponding to the multi-dimensional data storage mode adopted in the data lake in the second embodiment of the present application. Referring to Figure 3 , the data lake can be divided into four storage areas: original area, convergence area, work area and service area. Since the original area has more cold data, HDD can be used as its storage medium; since the convergence area has more cold data, HDD can be used as the storage medium; since the work area has more hot data, SSD can be used as its storage medium; since the service area has more hot data, SSD can be used as its storage medium; each storage area is further divided into hot data area, warm data area and cold data area; each hotness area is further divided into public data area, internal data area and confidential data area.

[0079] The second processing instruction can be understood as an instruction for determining that the second to-be-stored data is stored in the third storage area from the second storage area. The third hotness area can be understood as the hotness area where the second to-be-stored data is located in the second storage area. The third sensitivity area can be understood as the sensitivity area where the second to-be-stored data is located in the third hotness area. The fourth hotness area can be understood as the hotness area in the third storage area where the second to-be-stored data needs to be stored. The fourth sensitivity area can be understood as the sensitivity area in the fourth hotness area where the second to-be-stored data needs to be stored.

[0080] It can be understood that, since the third hot degree area can represent the hot degree type of the second to-be-stored data, and the second to-be-stored data needs to be stored in the hot degree area corresponding to the hot degree type when the second to-be-stored data is stored in the third storage area, the hot degree area in the third storage area corresponding to the same hot degree type as the third hot degree area is the fourth hot degree area in which the second to-be-stored data needs to be stored. Since the third sensitivity area can represent the sensitivity type of the second to-be-stored data, and the second to-be-stored data needs to be stored in the sensitivity area corresponding to the sensitivity type when the second to-be-stored data is stored in the third hot degree area, the sensitivity area in the fourth hot degree area corresponding to the same sensitivity type as the third sensitivity area is the fourth sensitivity area in which the second to-be-stored data needs to be stored.

[0081] In the embodiment of the present application, the data lake further comprises a third storage area; in response to a second processing instruction, second to-be-stored data is determined from the second storage area, and a third hot degree area and a third sensitivity area corresponding to the second to-be-stored data in the second storage area are determined, wherein the second to-be-stored data is stored in the third sensitivity area in the third hot degree area; a fourth hot degree area corresponding to the third hot degree area is determined from the third storage area, and a fourth sensitivity area corresponding to the third sensitivity area is determined from the fourth hot degree area; and the second to-be-stored data is stored in the fourth sensitivity area. The above technical solution further divides the area in the data lake according to the data processing stage, and further stores the data in the storage area divided according to the hot degree type and the sensitivity and other dimensions, thereby further realizing efficient division of the area for storing data, and facilitating management and use of the data.

[0082] Embodiment three

[0083] Figure 4 is a flowchart of a data processing method provided in the third embodiment of the present application. The present embodiment is optimized on the basis of the above technical solutions. In the present embodiment, the above data processing method further comprises: in response to a third processing instruction, determining a third sensitivity degree of each lake data, and determining a second hot degree type of each lake data according to the data state of each lake data stored in the data lake and the access frequency within a preset time length; determining a target hot degree area of each lake data in the original storage area according to the second hot degree type of each lake data, and determining a target sensitivity area of each lake data in the target hot degree area according to the third sensitivity degree of each lake data; and storing each lake data from an original sensitivity area in an original hot degree area in the original storage area into a target sensitivity area in a target hot degree area in the original storage area. The explanations of the same or corresponding terms as in the above embodiments are not repeated here.

[0084] Reference Figure 4The method of the embodiment can specifically include the following steps:

[0085] S310, in response to the data storage request, obtaining initial to-be-stored data.

[0086] S320, determining a first heat type of the initial to-be-stored data, and determining an initial heat area in a first storage area in the data lake according to the first heat type.

[0087] S330, determining a first sensitivity degree of the initial to-be-stored data, and determining an initial sensitivity area in the initial heat area according to the first sensitivity degree.

[0088] S340, storing the initial to-be-stored data into the initial sensitivity area.

[0089] S350, in response to a third processing instruction, determining a third sensitivity degree of each lake data stored in the data lake, and determining a second heat type of each lake data according to a data state of each lake data and an access frequency within a preset time length.

[0090] The third processing instruction can be understood as an instruction indicating to update a lake data storage area. The lake data can be understood as data stored in the data lake. The third sensitivity degree can be understood as a sensitivity degree of the lake data. The second heat type is a heat type of the lake data.

[0091] It should be noted that the data state, the access frequency within the preset time length and the sensitivity degree of each lake data in the data lake can change at any time, and therefore, the third processing instruction can be generated periodically to achieve timely storage of the lake data in a storage area corresponding to the heat type and the sensitivity degree of the lake data when the heat type and the sensitivity degree of the lake data change. The third processing instruction can also be generated manually.

[0092] In the embodiment of the application, in response to the third processing instruction, the third sensitivity degree of each lake data stored in the data lake can be determined by inputting the lake data into a pre-trained sensitivity degree determination model. When the sensitivity degree of the data changes, the sensitivity degree label of the data also changes, and therefore, the sensitivity degree label of the lake data can be directly read as the third sensitivity degree. The second heat type of each lake data is determined according to the data state of each lake data and the access frequency within the preset time length, and the determination manner of the second heat type is the same as that of the first heat type, which will not be described in detail herein.

[0093] S360, determine a target hotness region of each lake data in the original storage region according to the second hotness type of each lake data, and determine a target sensitivity region of each lake data in the target hotness region according to the third sensitivity degree of each lake data.

[0094] The original storage region can be understood as a storage region according to a data processing stage in which the lake data is located before responding to the third processing instruction, and the original storage region can be the first storage region, the second storage region or the third storage region. The target hotness region can be understood as a hotness region in which the lake data needs to be stored. The target sensitivity region can be understood as a sensitivity region in which the lake data needs to be stored.

[0095] S370, store each lake data from an original sensitivity region in an original hotness region in the original storage region to a target sensitivity region in a target hotness region in the original storage region.

[0096] It should be noted that the determined second hotness type and third sensitivity degree can only determine the update change of the hotness region and the sensitivity region of the lake data, so the lake data is still stored in the original storage region, and only the hotness region and the sensitivity region in which the lake data is stored may be changed.

[0097] The technical scheme of the embodiment of the application, in response to the third processing instruction, determines the third sensitivity degree of each lake data, and determines the second hotness type of each lake data according to the data state of each lake data stored in the data lake and the access frequency in a preset time length; determines a target hotness region of each lake data in the original storage region according to the second hotness type of each lake data, and determines a target sensitivity region of each lake data in the target hotness region according to the third sensitivity degree of each lake data; stores each lake data from an original sensitivity region in an original hotness region in the original storage region to a target sensitivity region in a target hotness region in the original storage region. The above technical scheme realizes the timely storage of the storage region corresponding to the hotness type and the sensitivity degree of the lake data when the hotness type and the sensitivity degree of the lake data change.

[0098] An optional technical scheme, the above data processing method further comprises: in response to a change instruction of the metadata of the lake data, extracting the changed metadata of the lake data; updating the directory corresponding to the data lake according to the changed metadata.

[0099] The change instruction of the metadata of the lake data can be understood as an instruction indicating that the metadata of the lake data has changed.

[0100] In the embodiment of the present application, the metadata change instruction of the lake data generated in response to the monitoring of the metadata change of the lake data can be, for example, a metadata change instruction of the lake data generated in response to the monitoring of the metadata change of the lake data through a hook mechanism. The metadata of the changed lake data is extracted; since the metadata can obtain the storage location, historical data, resource search, and file record of the lake data, the directory corresponding to the data lake can be updated according to the changed metadata.

[0101] In the embodiment of the present application, the metadata of the lake data stored in the data lake can also be extracted in advance, and the directory of the metadata management center is constructed at the level of the data lake platform according to the extracted metadata of the lake data stored in the data lake, so as to update the directory when the metadata of the lake data changes subsequently.

[0102] In the embodiment of the present application, the metadata of the changed lake data is extracted in response to the metadata change instruction of the lake data; and the directory corresponding to the data lake is updated according to the changed metadata, so as to realize low-cost construction and update of the directory.

[0103] On the basis of the above-mentioned scheme, another optional technical scheme, the above-mentioned data processing method further comprises: in response to a viewing request, determining a target entry object in the directory; according to a sixth sensitivity area stored by the lake data corresponding to the target entry object, an identifier of the viewer, and a preset viewing policy, verifying whether the viewer has viewing permission; and in the case that the viewer has viewing permission, displaying the lake data corresponding to the target entry object.

[0104] The viewing request can be understood as a request for viewing the lake data corresponding to the target entry object. The viewing request can include a request for the viewer to select the target entry object to indicate viewing of the corresponding lake data; it can also include a request for viewing the directory and a request for the viewer to select the target entry object in the directory to indicate viewing of the corresponding lake data. The target entry object can be understood as an entry object in the directory corresponding to the lake data that the viewer wants to view. The sixth sensitivity area can be understood as a sensitivity area stored by the lake data corresponding to the target entry object. The identifier of the viewer can be understood as an identifier representing the identity of the viewer, which can be determined through an office automation (OA) system. The preset viewing policy can be understood as a policy preset to determine whether the viewer has viewing permission, for example, the preset viewing policy can include that if the viewer identifier is an internal staff and the sixth sensitivity area is a confidential data area, the viewer does not have viewing permission.

[0105] In the embodiment of the present application, in response to the viewing request, the target entry object in the directory can be determined, and the lake data corresponding to the target entry object is the lake data required to be viewed by the viewer; according to the sixth sensitivity area stored in the lake data corresponding to the target entry object, the identity of the viewer and the preset viewing strategy, it can be verified whether the viewer has the viewing permission, and the viewer has the viewing permission, which means that the viewer has the qualification to view the lake data corresponding to the target entry object; in the case that the viewer has the viewing permission, the lake data corresponding to the target entry object is displayed.

[0106] In the embodiment of the present application, in the case that the viewer does not have the viewing permission, the related information that the viewer does not have the viewing permission can be displayed.

[0107] In the embodiment of the present application, in response to the viewing request, the target entry object in the directory is determined; according to the sixth sensitivity area stored in the lake data corresponding to the target entry object, the identity of the viewer and the preset viewing strategy, it can be verified whether the viewer has the viewing permission; in the case that the viewer has the viewing permission, the lake data corresponding to the target entry object is displayed, which realizes that the viewer can be provided with display content according to the identity of the viewer, and improves the security of lake data access.

[0108] Figure 5 is a flowchart of an optional example in the data processing method provided in the third embodiment of the present application. In order to better understand the technical scheme of the above-mentioned embodiment of the present application, an optional example is provided here. For example, referring to Figure 5 , in response to the data storage request, the initial to-be-stored data is obtained; the heat type of the initial to-be-stored data is determined, and the initial heat area in the first storage area in the data lake is determined according to the heat type; the sensitivity degree of the initial to-be-stored data is determined, and the initial sensitivity area in the initial heat area is determined according to the sensitivity degree; the initial to-be-stored data is stored into the initial sensitivity area.

[0109] In response to the first processing instruction, the first to-be-stored data is determined from the first storage area, and the heat type and the sensitivity degree of the first to-be-stored data are determined; the first to-be-stored data is converted into structured data, and it is judged whether the structured data is confidential data, and in the case that the structured data is confidential data, the structured data is desensitized and updated as structured data; according to the heat type and the sensitivity degree of the first to-be-stored data, the second heat area and the second sensitivity area are determined; the structured data is stored into the second sensitivity area in the second heat area.

[0110] In response to the third processing instruction, the sensitivity degree and the heat type of each lake data are determined, and it is judged whether the sensitivity degree and the heat type of each lake data have changed compared with the previous original heat type and original sensitivity degree; if the change has occurred, the lake data which has changed is stored from the original sensitivity degree area in the original heat area in the original storage area into the target sensitivity degree area in the target heat area in the original storage area corresponding to the changed sensitivity degree and heat type.

[0111] In response to the metadata change instruction of the lake data, the changed metadata of the lake data is extracted, and the directory corresponding to the data lake is updated according to the changed metadata.

[0112] Embodiment four

[0113] Figure 6 is the structural block diagram of the data processing device provided by the fourth embodiment of the application, which is used to execute the data processing method provided by any of the above embodiments. The device and the data processing method of each embodiment belong to the same inventive concept, and the details not described in the embodiment of the data processing device can be referred to the embodiment of the data processing method. Referring to Figure 6 , the device can specifically include: a to-be-stored data acquisition module 410, an initial heat area determination module 420, an initial sensitivity degree determination module 430, and an initial to-be-stored data storage module 440.

[0114] The to-be-stored data acquisition module 410 is configured to acquire initial to-be-stored data in response to a data storage request.

[0115] The initial heat area determination module 420 is configured to determine a first heat type of the initial to-be-stored data, and determine an initial heat area in a first storage area in a data lake according to the first heat type.

[0116] The initial sensitivity degree determination module 430 is configured to determine a first sensitivity degree of the initial to-be-stored data, and determine an initial sensitivity degree area in the initial heat area according to the first sensitivity degree.

[0117] The initial to-be-stored data storage module 440 is configured to store the initial to-be-stored data into the initial sensitivity degree area.

[0118] Optionally, the initial sensitivity degree determination module 430 includes:

[0119] The sensitivity degree determination unit is configured to input the initial to-be-stored data into a pre-trained sensitivity degree determination model, and determine the first sensitivity degree of the initial to-be-stored data according to an output result of the sensitivity degree determination model.

[0120] Optionally, the data lake further includes a second storage area; and the data processing device further includes:

[0121] The first sensitivity area determining module is configured to determine the first to-be-stored data from the first storage area and determine a first hotness area and a first sensitivity area corresponding to the first to-be-stored data in the first storage area in response to the first processing instruction, wherein the first to-be-stored data is stored in the first sensitivity area in the first hotness area.

[0122] The second sensitivity area determining module is configured to determine a second hotness area corresponding to the first hotness area from the second storage area and determine a second sensitivity area corresponding to the first sensitivity area from the second hotness area.

[0123] The structured data storing module is configured to determine structured data of the first to-be-stored data and store the structured data in the second sensitivity area.

[0124] On the basis of the above scheme, the data processing apparatus can further include:

[0125] The structured data desensitizing module is configured to desensitize the structured data in a case where a second sensitivity degree of the first to-be-stored data corresponding to the structured data is a preset degree before storing the structured data in the second sensitivity area.

[0126] On the basis of the above scheme, the data lake can further include a third storage area, and the data processing apparatus can further include:

[0127] The third sensitivity area determining module is configured to determine the second to-be-stored data from the second storage area and determine a third hotness area and a third sensitivity area corresponding to the second to-be-stored data in the second storage area in response to a second processing instruction after storing the structured data in the second sensitivity area, wherein the second to-be-stored data is stored in the third sensitivity area in the third hotness area.

[0128] The fourth sensitivity area determining module is configured to determine a fourth hotness area corresponding to the third hotness area from the third storage area and determine a fourth sensitivity area corresponding to the third sensitivity area from the fourth hotness area.

[0129] The to-be-stored data storing module is configured to store the second to-be-stored data in the fourth sensitivity area.

[0130] On the basis of the above scheme, the data processing apparatus can further include:

[0131] The second hotness type determining module is configured to determine a third sensitivity degree of each lake data stored in the data lake in response to a third processing instruction and determine a second hotness type of each lake data according to a data state of each lake data and an access frequency within a preset time length.

[0132] a target sensitivity area determination module, configured to determine a target hotness area of each lake data in the original storage area according to a second hotness type of each lake data, and determine a target sensitivity area of each lake data in the target hotness area according to a third sensitivity degree of each lake data;

[0133] a lake data storage module, configured to store each lake data from an original sensitivity area in an original hotness area in the original storage area into a target sensitivity area in a target hotness area in the original storage area.

[0134] On the basis of the above-mentioned scheme, optionally, the data processing apparatus further comprises:

[0135] a metadata extraction module, configured to extract the changed metadata of the lake data in response to a change instruction of the metadata of the lake data;

[0136] a directory updating module, configured to update a directory corresponding to the data lake according to the changed metadata.

[0137] On the basis of the above-mentioned scheme, optionally, the data processing apparatus further comprises:

[0138] a target entry object determination module, configured to determine a target entry object in the directory in response to a viewing request;

[0139] a viewing permission verification module, configured to verify whether a viewing party has viewing permission according to a sixth sensitivity area in which lake data corresponding to the target entry object is stored, an identifier of the viewing party, and a preset viewing strategy;

[0140] a lake data display module, configured to display the lake data corresponding to the target entry object in the case where the viewing party has the viewing permission.

[0141] The data processing apparatus provided in Embodiment Four of the present application, through the to-be-stored data acquisition module, in response to a data storage request, acquires initial to-be-stored data; through the initial hotness area determination module, determines a first hotness type of the initial to-be-stored data, and determines an initial hotness area in a first storage area in a data lake according to the first hotness type; through the initial sensitivity area determination module, determines a first sensitivity degree of the initial to-be-stored data, and determines an initial sensitivity area in the initial hotness area according to the first sensitivity degree; through the initial to-be-stored data storage module, stores the initial to-be-stored data into the initial sensitivity area. The above-mentioned apparatus stores data in a storage area divided according to multiple dimensions such as data processing stage, hotness type and sensitivity, thereby realizing efficient division area storage of data, and facilitating management and use of data.

[0142] The data processing apparatus provided by the embodiments of the present application can execute the data processing method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0143] It is worth noting that in the embodiments of the above data processing apparatus, each unit and module included is only divided according to the functional logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for the convenience of mutual differentiation, and does not serve to limit the protection scope of the present application.

[0144] Embodiment five

[0145] Figure 7 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0146] As shown in Figure 7 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is in communication with the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0147] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0148] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as a data processing method.

[0149] In some embodiments, the data processing method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the data processing method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the data processing method by any other suitable means, such as by means of firmware.

[0150] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0151] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0152] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0154] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0155] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak data state expansibility in traditional physical host and VPS service.

[0156] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0157] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized by, The method comprises: in response to a data storage request, obtaining initial to-be-stored data; determining a first heat type of the initial to-be-stored data, and determining an initial heat area in a first storage area in a data lake according to the first heat type; determining a first sensitivity degree of the initial to-be-stored data, and determining an initial sensitivity area in the initial heat area according to the first sensitivity degree; storing the initial to-be-stored data into the initial sensitivity area; The data lake further comprises a second storage area; The method further comprises: in response to a first processing instruction, determining first to-be-stored data from the first storage area, and determining a corresponding first heat area and a first sensitivity area of the first to-be-stored data in the first storage area, wherein the first to-be-stored data is stored in the first sensitivity area in the first heat area; determining a second heat area corresponding to the first heat area from the second storage area, and determining a second sensitivity area corresponding to the first sensitivity area from the second heat area; determining structured data of the first to-be-stored data, and storing the structured data into the second sensitivity area; The data lake further comprises a third storage area; After storing the structured data into the second sensitivity area, the method further comprises: in response to a second processing instruction, determining second to-be-stored data from the second storage area, and determining a corresponding third heat area and a third sensitivity area of the second to-be-stored data in the second storage area, wherein the second to-be-stored data is stored in the third sensitivity area in the third heat area; determining a fourth heat area corresponding to the third heat area from the third storage area, and determining a fourth sensitivity area corresponding to the third sensitivity area from the fourth heat area; storing the second to-be-stored data into the fourth sensitivity area.

2. The method of claim 1, wherein, The determination of the first sensitivity degree of the initial to-be-stored data comprises: inputting the initial to-be-stored data into a pre-trained sensitivity determination model, and determining the first sensitivity degree of the initial to-be-stored data according to the output result of the sensitivity determination model.

3. The method of claim 1, wherein, Before storing the structured data into the second sensitivity area, the method further comprises: in a case where a second sensitivity degree of the first to-be-stored data corresponding to the structured data is a preset degree, performing desensitization processing on the structured data.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: in response to a third processing instruction, determining a third sensitivity degree of each lake data stored in the data lake, and determining a second heat type of each lake data according to a data state of each lake data and an access frequency within a preset time length; determining a target heat area of each lake data in an original storage area according to the second heat type of each lake data, and determining a target sensitivity area of each lake data in the target heat area according to the third sensitivity degree of each lake data; The each lake data is stored from a original sensitivity area in a original hot degree area in the original storage area into a target sensitivity area in the target hot degree area in the original storage area.

5. The method of claim 4, wherein, The method further comprises: in response to a metadata change instruction of the lake data, extracting changed metadata of the lake data; updating a directory corresponding to the data lake according to the changed metadata.

6. The method of claim 5, wherein, The method further comprises: in response to a viewing request, determining a target entry object in the directory; according to a sixth sensitivity area in which lake data corresponding to the target entry object is stored, an identifier of a viewer, and a preset viewing policy, verifying whether the viewer has a viewing right; in a case where the viewer has the viewing right, displaying the lake data corresponding to the target entry object.

7. A data processing apparatus, characterized by, comprise: a to-be-stored data obtaining module configured to obtain initial to-be-stored data in response to a data storage request; an initial hot degree area determining module configured to determine a first hot degree type of the initial to-be-stored data, and determine an initial hot degree area in a first storage area in a data lake according to the first hot degree type; an initial sensitivity area determining module configured to determine a first sensitivity degree of the initial to-be-stored data, and determine an initial sensitivity area in the initial hot degree area according to the first sensitivity degree; an initial to-be-stored data storing module configured to store the initial to-be-stored data into the initial sensitivity area; the data lake further comprises a second storage area; and the data processing apparatus further comprises: a first sensitivity area determining module configured to determine first to-be-stored data from the first storage area, and determine a first hot degree area and a first sensitivity area corresponding to the first to-be-stored data in the first storage area in response to a first processing instruction, wherein the first to-be-stored data is stored in the first sensitivity area in the first hot degree area; a second sensitivity area determining module configured to determine a second hot degree area corresponding to the first hot degree area from the second storage area, and determine a second sensitivity area corresponding to the first sensitivity area from the second hot degree area; a structured data storing module configured to determine structured data of the first to-be-stored data, and store the structured data into the second sensitivity area; the data lake further comprises a third storage area; and the data processing apparatus further comprises: a third sensitivity area determining module configured to determine second to-be-stored data from the second storage area, and determine a third hot degree area and a third sensitivity area corresponding to the second to-be-stored data in the second storage area in response to a second processing instruction after the structured data is stored into the second sensitivity area, wherein the second to-be-stored data is stored in the third sensitivity area in the third hot degree area; a fourth sensitivity area determining module configured to determine a fourth hot degree area corresponding to the third hot degree area from the third storage area, and determine a fourth sensitivity area corresponding to the third sensitivity area from the fourth hot degree area; The second to-be-stored data is stored into the fourth sensitivity area by a to-be-stored data storage module.

8. An electronic device, comprising: The method comprises the steps of: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data processing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to implement the data processing method according to any one of claims 1-6 when executed.

Images