Computer equipment based on big data processing

Abstract

The invention discloses computer equipment based on big data processing. The computer equipment comprises a data processing and storage module which is responsible for receiving, processing, analyzing and storing data; the cache management module is responsible for managing cache data and reducing direct access to a database; the data verification module is responsible for performing multiple verification before data deletion; the access control and authority management module is responsible for managing user access authority; the user interaction module is responsible for providing a user interface and interacting with a user; the security module is responsible for remote monitoring and management and data backup and recovery; according to the method, a multi-check mechanism is added before data deletion, the accuracy and importance of the data are ensured, meanwhile, the system inquires whether a user agrees before data deletion through sufficient user interaction in the decision making process, and data information and deletion reason explanation are provided; the deletion reason interpretation comprises an event sequence and an association mode related to the data to be deleted, and the conditions of misjudgment, mistaken deletion and the like are avoided.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a computer device based on big data processing. Background Art

[0002] With the rapid development and popularization of information technology, computers have become an indispensable tool in modern society and are widely used in various fields such as education, entertainment, and work. With the improvement of video resolution (such as 4K and 8K videos) and the popularity of streaming services, users tend to download more high-quality videos for offline viewing. These video files occupy a huge amount of storage space. Computer devices based on big data processing technology have emerged. These devices integrate advanced storage technology, optimize data processing algorithms, and build efficient data management systems to achieve effective management and efficient use of massive video data.

[0003] After searching, the invention patent with Chinese patent number CN115510272A discloses a computer data processing system based on big data analysis, which belongs to the field of computer technology. The processing system includes a storage space analysis module, a target data selection module and a status information analysis module. The storage space analysis module obtains the video download data stored in the current computer as analysis data. If the storage space occupied by the analysis data is greater than the storage space threshold, the target data selection module is ordered to analyze the analysis data. If there is a video download data in the analysis data, the time interval between the last time it was triggered and the current time is greater than the time threshold, then the video download data is the target data. The status information analysis module analyzes the status information of the target data to determine whether to delete the target data.

[0004] Compared with the prior art, the invention patent with Chinese patent number CN115510272A analyzes videos that have not been watched for a long time to determine the probability of subsequent users watching the video. If the probability is low, the video is directly deleted, thereby reducing the computer storage space occupied by idle video data, ensuring the normal operation of the computer, and improving the computer's operating efficiency.

[0005] However, during the use of the above device, although the system has designed complex analysis logic to determine which data should be deleted, there is still a possibility of misjudgment. Some users may watch certain videos occasionally, but these videos are still very important to them and should not be deleted. Therefore, a computer device based on big data processing is proposed. Summary of the invention

[0006] The purpose of the present invention is to solve the shortcomings in the prior art that, although the system is designed with complex analysis logic to determine which data should be deleted, there is still a possibility of misjudgment. Some users may only watch certain videos occasionally, but these videos are still very important to them and should not be deleted. A computer device based on big data processing is proposed to solve the problem.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: A computer device based on big data processing, comprising: Data processing and storage module: responsible for data reception, processing, analysis and storage; Cache management module: responsible for managing cache data and reducing direct access to the database; Data verification module: responsible for multiple verifications before data deletion to ensure the accuracy and importance of the data; Access control and authority management module: responsible for managing user access rights and ensuring data security; User interaction module: responsible for providing the user interface and interacting with users; Security module: responsible for remote monitoring and management as well as data backup and recovery; The data processing and storage module is the core. The cache management module receives the data to be cached from the data processing and storage module. The data verification module receives the data to be deleted and its corresponding information from the data processing and storage module, returns the verification result to the data processing and storage module, and decides whether to continue the deletion operation. The access control and permission management module receives the access request and user identity information from the user interaction module, and returns the access result (allow or deny) to the user interaction module.

[0008] The above technical solution further includes: Furthermore, the data processing and storage module includes a storage space analysis submodule, a target data selection submodule, a status information analysis submodule and an event correlation analysis submodule. The storage space analysis submodule triggers the data optimization process by monitoring and analyzing the storage space usage, and sends a request to the target data selection submodule. The target data selection submodule filters the target data according to preset conditions and passes the screening results to the status information analysis submodule. The status information analysis submodule processes (sorting, demarcation, set division, set analysis, deletion control) the target data and determines the data processing strategy (such as deletion, retention, archiving, etc.).

[0009] Furthermore, the storage space analysis submodule includes a monitoring unit, an analysis unit and a trigger unit. The monitoring unit continuously monitors the storage space occupancy of the video download data and collects real-time data of the storage space. The analysis unit receives the data from the monitoring unit, analyzes the usage trend and remaining space of the storage space, and evaluates whether to trigger the data optimization process. According to the results of the analysis unit, when the storage space reaches a preset threshold or meets other optimization conditions, the trigger unit triggers the data optimization process and sends a request to the target data selection submodule.

[0010] Furthermore, the target data selection submodule includes a screening unit and a condition configuration unit. The condition configuration unit allows a user or system administrator to configure conditions for screening target data, such as time interval, file size, data type, etc. The screening unit receives a trigger request from the storage space analysis submodule and a condition configuration of the condition configuration unit, and screens out target data from the video download data according to the trigger request of the storage space analysis submodule and the condition configuration of the condition configuration unit. The screening result will be passed to the status information analysis submodule.

[0011] Furthermore, the status information analysis submodule includes a sorting unit, a demarcation unit, a set division unit, a set analysis unit and a deletion control unit. The sorting unit receives the target data from the target data selection submodule, sorts the data according to preset sorting rules (such as time, size), and the sorting result will be passed to the demarcation unit. The demarcation unit analyzes the sorted data and defines the data boundaries according to business needs or data characteristics, such as dividing the data into multiple batches or paragraphs. The set division unit further divides the demarcated data into different sets or subsets for more detailed analysis or processing. The set analysis unit analyzes each set or subset to evaluate its importance, value or processing priority. The evaluation result will be used to guide the data processing strategy. The deletion control unit decides which data needs to be deleted or retained based on the result of the set analysis unit. If the data needs to be deleted, a request is sent to the data verification module for multiple verifications, and finally the deletion operation is performed. If the data needs to be retained, it is archived or moved to other storage locations.

[0012] Further, the event correlation analysis submodule includes an event collection unit, an event standardization unit, an event storage unit, a correlation rule engine, a correlation analysis unit, and an explanation generation unit; When any data-related events occur within the system (such as data expiration notifications, illegal access records, data migration logs, etc.), the event collection unit captures these events and passes them to the event standardization unit; The event standardization unit processes the collected event data to make it conform to a unified format and standard; The processed event data is stored in the event storage unit for subsequent analysis and query; When the user initiates a deletion request, the system triggers the association analysis process, and the association analysis unit retrieves event data related to the data to be deleted from the event storage unit; The association analysis unit uses an association rule engine to perform association analysis on the retrieved event data, and identifies event sequences and association patterns related to the data to be deleted by analyzing attributes such as event type, time, source, and content; The explanation generation unit generates deletion reason explanations based on the results of the association analysis. These explanations may include the specific date of data expiration, detailed information on illegal access, reasons and processes for data migration, etc. The deletion reason explanations are sent to the user interaction module.

[0013] Furthermore, the cache management module includes a cache layer integration unit, a cache policy unit and a cache synchronization unit. The cache layer integration unit is responsible for integrating cache systems such as Redis and Memcached, configuring cache connections, parameters, etc., to ensure that the cache system can work normally. The cache layer integration unit communicates with the cache system (such as Redis, Memcached), sends cache read and write requests, receives responses from the cache system, and processes cache data. The cache policy unit is responsible for implementing cache elimination strategies such as LRU (least recently used) and LFU (least frequently used). The cache policy unit monitors cache usage, including cache hit rate, cache size, etc. The cache policy unit determines which cache items need to be eliminated based on the cache policy, and sends an elimination request to the cache layer integration unit. The cache policy unit receives feedback from the cache layer integration unit on the results of the elimination operation. The cache synchronization unit is responsible for ensuring the consistency of cache data with database data, and handling cache invalidation and update issues.

[0014] Furthermore, the data verification module provides a manual review channel, which can add additional verification steps before high-risk operations to further improve data security; The data verification module constructs a data dependency graph, checks the data dependency through a graph theory algorithm, and can identify and process the data dependency before deleting the data, including the following steps: Define data entities and dependencies: Identify data entities: Identify the data entities (such as database tables, files, objects, etc.) that are included in the dependency graph; Define dependencies: determine the dependency types (such as direct dependency, indirect dependency, circular dependency, etc.) and rules between data entities; Build the dependency graph: Create a graph structure: Use the graph structure in graph theory to represent data entities and the dependencies between them. The graph consists of nodes and edges, where nodes represent data entities and edges represent dependencies. Add node: add a corresponding node in the graph for each data entity; Adding edges: According to the dependency relationship between data entities, corresponding edges are added to the graph. The direction of the edge usually indicates the direction of dependency (such as from the relying party to the dependent party). Implement a graph theory algorithm to check dependencies: Traversal algorithm: Use depth-first search to check dependencies in the graph; Dependency analysis: Direct dependency: Checks whether there is a direct edge connecting two nodes in the graph, indicating that there is a direct dependency between them; Indirect dependencies: Find all possible dependency paths through depth-first search and identify indirect dependencies; Circular Dependency: Checks whether there is a cycle in the graph. The existence of a cycle indicates the existence of a circular dependency, which is usually undesirable because it may lead to increased complexity when deleting or updating data. Dependency conflict detection: When planning to delete a data entity, use depth-first search to check whether the operation will violate any defined dependency rules or cause other data entities to become invalid or unavailable; Handling dependencies: Dependency resolution strategy: Based on the inspection results, formulate a corresponding dependency resolution strategy. For example, if deleting a data entity will cause other data entities to become invalid, you may need to update or delete these dependent data entities first, or cancel the deletion operation. Automated processing: Automates the process of checking and handling dependencies.

[0015] Furthermore, the access control and authority management module includes a role management unit, a authority allocation unit, a user management unit, an access control decision unit, a sensitive data protection unit and an audit log unit. The role management unit is responsible for defining different roles in the system and assigning a set of authorities to each role. The role management unit interacts with the authority allocation unit and the user management unit to ensure the correct association between roles, authorities and users. The authority allocation unit is responsible for allocating access rights to system resources (such as data, functions, etc.) according to the roles defined by the role management unit. The authority allocation unit interacts with the role management unit, the user management unit and the access control decision unit to ensure the accurate allocation and execution of authorities. The user management unit is responsible for managing user registration, authentication, authorization and other information, and associating users with roles. The user management unit interacts with the role management unit, the authority allocation unit and the access control decision unit to ensure that users can access corresponding The access control decision unit is responsible for deciding whether to allow access based on the user's role, authority and other attributes (such as time, location, etc.) when the user attempts to access system resources. The access control decision unit interacts with the user management unit, the authority allocation unit, the sensitive data protection unit and the audit log unit to determine the legitimacy of the access request in real time and record relevant information. The sensitive data protection unit is responsible for encrypting the storage and transmission of sensitive data and implementing access control policies to limit direct access to sensitive data. The sensitive data protection unit interacts with the access control decision unit and the storage system to ensure the security of sensitive data during storage and transmission. The audit log unit is responsible for recording all users' data access and operation behaviors, including login, access to resources, modification of data, etc., for subsequent tracking and auditing. The audit log unit interacts with the user management unit, the access control decision unit and the storage system to collect and store audit log information.

[0016] Furthermore, the user interaction module includes an interface presentation unit, a user operation unit, a user interaction logic unit, a deletion confirmation unit and an option customization unit. The interface presentation unit is responsible for rendering and displaying a graphical user interface (GUI) or a command line interface (CLI) to enable a user to interact with the system. The interface presentation unit receives user input from the user operation unit and displays system feedback to the user. The user operation unit is responsible for capturing user operation instructions, such as clicks, inputs, etc., and converting them into instructions recognizable by the system. The user operation unit interacts with the user interaction logic unit, transmits user operation instructions, and receives feedback from the interface presentation unit and displays it to the user. The user interaction logic unit is responsible for processing the instructions transmitted by the user operation unit, executing corresponding logical judgments, such as authority verification, data verification, etc., and calling other units. The function of the element, the user interaction logic unit interacts with the user operation unit, the deletion confirmation unit, the option customization unit and the data management module to coordinate the execution of user operations. Before deleting data, the deletion confirmation unit asks the user whether to agree through pop-up windows, email notifications, etc., and provides data information and explanations of deletion reasons. The explanations of deletion reasons are provided by the event association analysis submodule. The deletion confirmation unit receives the deletion request from the user interaction logic unit, displays the deletion confirmation information to the user, and sends instructions to the option customization unit according to user feedback (agree / reject). The option customization unit allows the user to choose whether to delete immediately, delay deletion or mark it for review, thereby increasing the flexibility of user operations. The option customization unit receives the user's selection from the deletion confirmation unit, and sends corresponding deletion or marking instructions to the data management module according to the user's selection.

[0017] The present invention has the following beneficial effects: In the present invention, the cache management module uses a cache layer to cache the results of frequent queries, reducing the number of direct accesses to the database, improving system performance, and avoiding users having to wait for a long time to see the results.

[0018] In the present invention, multiple verification mechanisms are added before data is deleted to ensure the accuracy and importance of the data. At the same time, the system asks the user whether they agree before deleting the data through sufficient user interaction during the decision-making process, and provides data information and an explanation of the reason for deletion. The explanation of the reason for deletion includes the event sequence and association pattern related to the data to be deleted to avoid misjudgment and accidental deletion. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A system block diagram of a computer device based on big data processing proposed by the present invention; Figure 2 It is a system block diagram of the data processing and storage module in the present invention; Figure 3It is a system block diagram of the event association analysis submodule in the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1-Figure 2 As shown, the present invention is a computer device based on big data processing, comprising: Data processing and storage module: responsible for data reception, processing, analysis and storage; Cache management module: responsible for managing cache data and reducing direct access to the database; Data verification module: responsible for multiple verifications before data deletion to ensure the accuracy and importance of the data; Access control and authority management module: responsible for managing user access rights and ensuring data security; User interaction module: responsible for providing the user interface and interacting with users; Security module: responsible for remote monitoring and management as well as data backup and recovery; The data processing and storage module is the core. The cache management module receives the data to be cached from the data processing and storage module. The data verification module receives the data to be deleted and its corresponding information from the data processing and storage module, returns the verification result to the data processing and storage module, and decides whether to continue the deletion operation. The access control and permission management module receives the access request and user identity information from the user interaction module, and returns the access result (allowed or denied) to the user interaction module. When the user initiates a data query request in the user interaction module, the required data is first obtained from the cache management module. If the required data exists in the cache management module, it is directly returned to the user. If not, it will be returned to the user from the data processing and storage module.

[0022] The working principle of a computer device based on big data processing proposed by the present invention is that an external data source (such as a network or other system) transmits data to a data processing and storage module, which receives, analyzes and preliminarily processes the data, and the processed data is stored or transmitted to other modules according to demand; The cache management module receives the data to be cached from the data processing and storage module, stores the data in a cache system such as Redis, and sets appropriate cache strategies (such as LRU, LFU). When there is a data access request, it first tries to obtain the data from the cache. If the cache misses, it obtains the data from the database or other storage systems. The data processing and storage module identifies the data to be deleted and passes the relevant information to the data verification module. The data verification module performs multiple verification logics, including data comparison, integrity verification, dependency check, etc. According to the verification results, the data verification module returns an instruction to the data processing and storage module on whether to continue the deletion operation. If the verification passes, the data processing and storage module executes the deletion operation. When executing the deletion operation, it provides more detailed explanations and options and asks the user whether to agree. If the verification fails, the deletion operation is not performed or other processing is performed; The user initiates an access request through the user interaction module and provides user identity information. The access control and permission management module receives the access request and user identity information, performs identity authentication and permission checks, and returns the access result (allowed or denied) to the user interaction module based on the check results. If access is allowed, the user interaction module displays the corresponding data or performs the corresponding operation; if access is denied, the denial information is displayed to the user. When the user initiates a data query request in the user interaction module, the system will first try to obtain the required data from the cache management module. If the required data exists in the cache, it is directly returned to the user, reducing the number of database accesses and improving the response speed. Users interact with the system through a graphical user interface or command line interface. Users can initiate operation requests such as data query, data deletion, and permission application. The system performs corresponding operations based on the user's request and displays the operation results or prompt information to the user through the user interaction module.

[0023] In one embodiment, for the above-mentioned data-free processing and storage module, the data processing and storage module includes a storage space analysis submodule, a target data selection submodule, a status information analysis submodule and an event correlation analysis submodule. The storage space analysis submodule triggers the data optimization process by monitoring and analyzing the storage space usage, and sends a request to the target data selection submodule. The target data selection submodule filters the target data according to preset conditions and passes the screening results to the status information analysis submodule. The status information analysis submodule processes (sorting, demarcation, set division, set analysis, deletion control) the target data and determines the data processing strategy (such as deletion, retention, archiving, etc.).

[0024] In one embodiment, for the above-mentioned storage space analysis submodule, the storage space analysis submodule includes a monitoring unit, an analysis unit and a triggering unit. The monitoring unit continuously monitors the storage space occupancy of the video download data and collects real-time data of the storage space. The analysis unit receives the data of the monitoring unit, analyzes the usage trend and the remaining space of the storage space, and evaluates whether to trigger the data optimization process. According to the results of the analysis unit, when the storage space reaches a preset threshold or meets other optimization conditions, the triggering unit triggers the data optimization process and sends a request to the target data selection submodule.

[0025] In one embodiment, for the above-mentioned target data selection submodule, the target data selection submodule includes a screening unit and a condition configuration unit. The condition configuration unit allows the user or system administrator to configure the conditions for screening the target data, such as time interval, file size, data type, etc. The screening unit receives the trigger request from the storage space analysis submodule and the condition configuration of the condition configuration unit, and screens out the target data from the video download data according to the trigger request of the storage space analysis submodule and the condition configuration of the condition configuration unit. The screening result will be passed to the status information analysis submodule.

[0026] In one embodiment, for the above-mentioned status information analysis submodule, the status information analysis submodule includes a sorting unit, a demarcation unit, a set division unit, a set analysis unit and a deletion control unit. The sorting unit receives the target data from the target data selection submodule, sorts the data according to a preset sorting rule (such as time, size), and the sorting result will be passed to the demarcation unit. The demarcation unit analyzes the sorted data and defines the data boundary according to business needs or data characteristics, such as dividing the data into multiple batches or paragraphs. The set division unit further divides the demarcated data into different sets or subsets for more detailed analysis or processing. The set analysis unit analyzes each set or subset to evaluate its importance, value or processing priority. The evaluation result will be used to guide the data processing strategy. The deletion control unit decides which data needs to be deleted or retained based on the result of the set analysis unit. If the data needs to be deleted, a request is sent to the data verification module for multiple verifications, and finally the deletion operation is performed. If the data needs to be retained, it is archived or moved to other storage locations.

[0027] In one embodiment, for the above-mentioned event correlation analysis submodule, the event correlation analysis submodule includes an event collection unit, an event standardization unit, an event storage unit, a correlation rule engine, a correlation analysis unit, and an explanation generation unit; When any data-related events occur within the system (such as data expiration notifications, illegal access records, data migration logs, etc.), the event collection unit captures these events and passes them to the event standardization unit; The event standardization unit processes the collected event data to make it conform to a unified format and standard; The processed event data is stored in the event storage unit for subsequent analysis and query; When a user initiates a deletion request, the system triggers the association analysis process, and the association analysis unit retrieves event data related to the data to be deleted from the event storage unit; The association analysis unit uses the association rule engine to perform association analysis on the retrieved event data, and identifies the event sequence and association pattern related to the data to be deleted by analyzing the type, time, source, content and other attributes of the event; The explanation generation unit generates deletion reason explanations based on the results of the association analysis. These explanations may include the specific date of data expiration, detailed information on illegal access, reasons and processes for data migration, etc. The deletion reason explanations are sent to the user interaction module.

[0028] In one embodiment, for the above-mentioned cache management module, the cache management module includes a cache layer integration unit, a cache policy unit and a cache synchronization unit. The cache layer integration unit is responsible for integrating cache systems such as Redis and Memcached, configuring cache connections, parameters, etc., to ensure that the cache system can work normally. The cache layer integration unit communicates with the cache system (such as Redis, Memcached), sends cache read and write requests, receives responses from the cache system, and processes cache data. The cache policy unit is responsible for implementing cache elimination strategies such as LRU (least recently used) and LFU (least frequently used). The cache policy unit monitors cache usage, including cache hit rate, cache size, etc. The cache policy unit determines which cache items need to be eliminated based on the cache policy, and sends elimination requests to the cache layer integration unit. The cache policy unit receives feedback from the cache layer integration unit on the results of the elimination operation. The cache synchronization unit is responsible for ensuring the consistency of cache data with database data and handling cache invalidation and update issues.

[0029] In one embodiment, for the above data verification module, the data verification module provides a manual review channel, which can add additional verification steps before high-risk operations to further improve data security; The data verification module builds a data dependency graph and uses graph theory algorithms to check data dependencies. It can identify and process data dependencies before deleting the data, including the following steps: Define data entities and dependencies: Identify data entities and specify the data entities (such as database tables, files, objects, etc.) to be included in the dependency graph; define dependencies and determine the dependency types (such as direct dependency, indirect dependency, circular dependency, etc.) and rules between data entities; Build a dependency graph: Create a graph structure, using the graph structure in graph theory to represent data entities and the dependency relationships between them. The graph consists of nodes and edges, where nodes represent data entities and edges represent dependency relationships. Add nodes, add a corresponding node to the graph for each data entity; add edges, add corresponding edges to the graph according to the dependency relationships between data entities, and the direction of the edge usually indicates the direction of dependency (such as from the dependent party to the dependent party); Implement graph theory algorithms to check dependencies: traversal algorithm, use depth-first search to check dependencies in the graph; dependency analysis: direct dependency, check whether there is a direct edge connecting two nodes in the graph, indicating that there is a direct dependency between them; indirect dependency, find all possible dependency paths through depth-first search and identify indirect dependencies; cyclic dependency, check whether there is a cycle in the graph. The existence of a cycle indicates that there is a cyclic dependency, which is usually an undesirable situation because it may lead to increased complexity when deleting or updating data; dependency conflict detection, when planning to delete a data entity, use depth-first search to check whether the operation will violate any defined dependency rules or cause other data entities to become invalid or unavailable; Handling dependencies: Dependency resolution strategy: formulate corresponding dependency resolution strategy based on the inspection results. For example, if deleting a data entity will cause other data entities to become invalid, you may need to update or delete these dependent data entities first, or cancel the deletion operation; automated processing: automate the dependency inspection and processing process.

[0030] In one embodiment, for the above-mentioned access control and permission management module, the access control and permission management module includes a role management unit, a permission allocation unit, a user management unit, an access control decision unit, a sensitive data protection unit and an audit log unit. The role management unit is responsible for defining different roles in the system and assigning a set of permissions to each role. The role management unit interacts with the permission allocation unit and the user management unit to ensure the correct association between roles, permissions and users. The permission allocation unit is responsible for allocating access rights to system resources (such as data, functions, etc.) according to the roles defined by the role management unit. The permission allocation unit interacts with the role management unit, the user management unit and the access control decision unit to ensure accurate allocation and execution of permissions. The user management unit is responsible for managing user registration, authentication, authorization and other information, and associating users with roles. The user management unit interacts with the role management unit, the permission allocation unit and the access control decision unit to ensure that users can The access control decision unit is responsible for deciding whether to allow access based on the user's role, authority and other attributes (such as time, location, etc.) when the user tries to access system resources. The access control decision unit interacts with the user management unit, authority allocation unit, sensitive data protection unit and audit log unit to determine the legitimacy of access requests in real time and record relevant information. The sensitive data protection unit is responsible for encrypting, storing and transmitting sensitive data and implementing access control policies to limit direct access to sensitive data. The sensitive data protection unit interacts with the access control decision unit and the storage system to ensure the security of sensitive data during storage and transmission. The audit log unit is responsible for recording all users' data access and operation behaviors, including login, access to resources, modification of data, etc., for subsequent tracking and auditing. The audit log unit interacts with the user management unit, access control decision unit and storage system to collect and store audit log information.

[0031] In one embodiment, for the above-mentioned user interaction module package, the user interaction module includes an interface presentation unit, a user operation unit, a user interaction logic unit, a deletion confirmation unit and an option customization unit. The interface presentation unit is responsible for rendering and displaying a graphical user interface (GUI) or a command line interface (CLI) so that the user can interact with the system. The interface presentation unit receives user input from the user operation unit and displays system feedback to the user. The user operation unit is responsible for capturing user operation instructions, such as clicks, inputs, etc., and converting them into instructions recognizable by the system. The user operation unit interacts with the user interaction logic unit, transmits user operation instructions, and receives feedback from the interface presentation unit and displays it to the user. The user interaction logic unit is responsible for processing the instructions transmitted by the user operation unit and performing corresponding logical judgments, such as authority verification and data verification. Etc., and call the functions of other units. The user interaction logic unit interacts with the user operation unit, the deletion confirmation unit, the option customization unit and the data management module to coordinate the execution of user operations. Before deleting data, the deletion confirmation unit asks the user whether he agrees through pop-up windows, email notifications, etc., and provides data information and explanations of deletion reasons. The explanations of deletion reasons are provided by the event association analysis submodule. The deletion confirmation unit receives the deletion request from the user interaction logic unit, displays the deletion confirmation information to the user, and sends instructions to the option customization unit based on user feedback (agree / reject). The option customization unit allows the user to choose whether to delete immediately, delay deletion or mark it for review, thereby increasing the flexibility of user operations. The option customization unit receives the user's selection from the deletion confirmation unit, and sends corresponding deletion or marking instructions to the data management module based on the user's selection.

[0032] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A computer device based on big data processing, characterized in that: include: Data processing and storage module, used for receiving, processing, analyzing and storing data; Cache management module, used to manage cache data and reduce direct access to the database; Data verification module, used to perform multiple verifications before data deletion; Access control and permission management module, used to manage user access rights; A user interaction module is used to provide a user interface and interact with users; Security module for remote monitoring and management as well as data backup and recovery; The data processing and storage module is the core, the cache management module receives the data to be cached from the data processing and storage module, the data verification module receives the data to be deleted and its corresponding information from the data processing and storage module, and returns the verification result to the data processing and storage module, and the access control and permission management module receives the access request and user identity information from the user interaction module, and returns the access result to the user interaction module.

2. A computer device based on big data processing according to claim 1, characterized in that: The data processing and storage module includes a storage space analysis submodule, a target data selection submodule, a status information analysis submodule and an event association analysis submodule. The storage space analysis submodule triggers the data optimization process by monitoring and analyzing the storage space usage, and sends a request to the target data selection submodule. The target data selection submodule filters the target data according to preset conditions and passes the screening results to the status information analysis submodule. The status information analysis submodule processes the target data and determines the data processing strategy.

3. A computer device based on big data processing according to claim 2, characterized in that: The storage space analysis submodule includes a monitoring unit, an analysis unit and a trigger unit. The monitoring unit continuously monitors the storage space occupancy of video download data and collects real-time data of the storage space. The analysis unit receives the data from the monitoring unit, analyzes the usage trend and remaining space of the storage space, and evaluates whether to trigger the data optimization process. According to the results of the analysis unit, when the storage space reaches a preset threshold or meets other optimization conditions, the trigger unit triggers the data optimization process and sends a request to the target data selection submodule.

4. A computer device based on big data processing according to claim 2, characterized in that: The target data selection submodule includes a screening unit and a condition configuration unit. The condition configuration unit allows a user or system administrator to configure conditions for screening target data. The screening unit receives a trigger request from the storage space analysis submodule and a condition configuration of the condition configuration unit, and screens out target data from the video download data according to the trigger request of the storage space analysis submodule and the condition configuration of the condition configuration unit. The screening result will be passed to the status information analysis submodule.

5. The computer device based on big data processing according to claim 2, characterized in that: The status information analysis submodule includes a sorting unit, a demarcation unit, a set division unit, a set analysis unit and a deletion control unit. The sorting unit receives the target data from the target data selection submodule, sorts the data according to a preset sorting rule, and the sorting result is passed to the demarcation unit. The demarcation unit analyzes the sorted data and defines the data boundary according to business requirements or data characteristics. The set division unit further divides the demarcated data into different sets or subsets. The set analysis unit analyzes each set or subset to evaluate its importance, value or processing priority. The evaluation result will be used to guide the data processing strategy. The deletion control unit decides which data needs to be deleted or retained based on the result of the set analysis unit. If the data needs to be deleted, a request is sent to the data verification module for multiple verifications, and finally the deletion operation is performed. If the data needs to be retained, it is archived or moved to other storage locations.

6. A computer device based on big data processing according to claim 2, characterized in that: The event correlation analysis submodule includes an event collection unit, an event standardization unit, an event storage unit, a correlation rule engine, a correlation analysis unit, and an explanation generation unit; When any data-related events occur within the system, the event collection unit captures these events and passes them to the event standardization unit; The event standardization unit processes the collected event data; The processed event data is stored in the event storage unit; When the user initiates a deletion request, the system triggers the association analysis process, and the association analysis unit retrieves event data related to the data to be deleted from the event storage unit; The association analysis unit uses an association rule engine to perform association analysis on the retrieved event data, and identifies event sequences and association patterns related to the data to be deleted by analyzing the attributes of the events; The explanation generating unit generates a deletion reason explanation according to the result of the association analysis, and the deletion reason explanation is sent to the user interaction module.

7. The computer device based on big data processing according to claim 1, characterized in that: The cache management module includes a cache layer integration unit, a cache policy unit and a cache synchronization unit. The cache layer integration unit is responsible for integrating the cache system. The cache layer integration unit communicates with the cache system, sends cache read and write requests, receives responses from the cache system, and processes cache data. The cache policy unit is responsible for implementing the cache elimination strategy. The cache policy unit monitors cache usage. The cache policy unit determines which cache items need to be eliminated based on the cache strategy and sends an elimination request to the cache layer integration unit. The cache policy unit receives feedback from the cache layer integration unit on the results of the elimination operation. The cache synchronization unit is responsible for ensuring the consistency of cache data with database data and handling cache invalidation and update issues.

8. The computer device based on big data processing according to claim 1, characterized in that: The data verification module provides a manual review channel; The data verification module constructs a data dependency graph, checks the data dependency through a graph theory algorithm, and can identify and process the data dependency before deleting the data, including the following steps: Define data entities and dependencies: Identify data entities and specify the data entities to be included in the dependency graph; define dependencies and determine the dependency types and rules between data entities; Build a dependency graph: Create a graph structure, using the graph structure in graph theory to represent data entities and the dependency relationships between them. The graph consists of nodes and edges, where nodes represent data entities and edges represent dependency relationships. Add nodes, add a corresponding node to the graph for each data entity. Add edges, add corresponding edges to the graph according to the dependency relationships between data entities, and the direction of the edge usually indicates the direction of dependency. Implement graph theory algorithms to check dependencies: traversal algorithm, use depth-first search to check dependencies in the graph; dependency analysis: direct dependency, check whether there is a direct edge connecting two nodes in the graph, indicating that there is a direct dependency between them; indirect dependency, find all possible dependency paths through depth-first search and identify indirect dependencies; circular dependency, check whether there is a loop in the graph, the existence of the loop indicates the existence of a circular dependency; dependency conflict detection, when planning to delete a data entity, use depth-first search to check whether the operation will violate any defined dependency rules or cause other data entities to become invalid or unavailable; Handling dependencies: Dependency resolution strategy: formulate corresponding dependency resolution strategy based on the inspection results; automated processing: automate the dependency inspection and processing process.

9. The computer device based on big data processing according to claim 1, characterized in that: The access control and authority management module includes a role management unit, an authority allocation unit, a user management unit, an access control decision unit, a sensitive data protection unit and an audit log unit. The role management unit is responsible for defining different roles in the system and assigning a set of authorities to each role. The role management unit interacts with the authority allocation unit and the user management unit. The authority allocation unit is responsible for allocating access rights to system resources according to the roles defined by the role management unit. The authority allocation unit interacts with the role management unit, the user management unit and the access control decision unit to ensure accurate authority allocation and execution. The user management unit is responsible for managing user information and associating users with roles. The user management unit interacts with the role management unit , the permission allocation unit interacts with the access control decision unit. The access control decision unit is responsible for deciding whether to allow access based on the user's role, permission and other attributes when the user attempts to access system resources. The access control decision unit interacts with the user management unit, the permission allocation unit, the sensitive data protection unit and the audit log unit. The sensitive data protection unit is responsible for encrypting the storage and transmission of sensitive data and implementing access control policies to limit direct access to sensitive data. The sensitive data protection unit interacts with the access control decision unit and the storage system. The audit log unit is responsible for recording the data access and operation behaviors of all users. The audit log unit interacts with the user management unit, the access control decision unit and the storage system.

10. The computer device based on big data processing according to claim 2, characterized in that: The user interaction module includes an interface presentation unit, a user operation unit, a user interaction logic unit, a deletion confirmation unit and an option customization unit. The interface presentation unit is responsible for rendering and displaying a graphical user interface or a command line interface, so that the user can interact with the system. The interface presentation unit receives user input from the user operation unit and displays system feedback to the user. The user operation unit is responsible for capturing the user's operation instructions and converting them into instructions recognizable by the system. The user operation unit interacts with the user interaction logic unit, transmits the user operation instructions, and receives feedback from the interface presentation unit and displays it to the user. The user interaction logic unit is responsible for processing the instructions transmitted by the user operation unit. The unit interacts with the user operation unit, the deletion confirmation unit, the option customization unit and the data management module to coordinate the execution of user operations. Before deleting data, the deletion confirmation unit asks the user whether he agrees and provides data information and an explanation of the deletion reason. The deletion reason explanation is provided by the event association analysis submodule. The deletion confirmation unit receives the deletion request from the user interaction logic unit, displays the deletion confirmation information to the user, and sends instructions to the option customization unit based on user feedback. The option customization unit allows the user to choose whether to delete immediately, delay deletion or mark it for review. The option customization unit receives the user's selection from the deletion confirmation unit and sends corresponding deletion or marking instructions to the data management module based on the user's selection.

Images