Data mining method and device and storage medium

Abstract

The invention relates to a data mining method and device and a storage medium, and the method comprises the steps: obtaining real-time data; performing classification processing on the registered modular algorithm models through an algorithm bin management platform, assembling the modular algorithm models under each classification into independent card units, and dynamically sending each independent card unit to an application platform for configuration; detecting a first operation instruction for the independent card unit through the application platform; in response to a detected first operation instruction, determining target algorithm models in the modular algorithm model, and performing logic configuration on the plurality of target algorithm models through the algorithm bin management platform to obtain a target workflow; and calling the target workflow to perform data mining on the real-time data to generate a data mining result. Through the data mining method and device, the problem of low accuracy of data mining in a complex application scene is solved.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to data mining methods, devices, and storage media. Background Art

[0002] Data mining technology generally refers to the use of algorithms to extract hidden information from large amounts of data. Data mining technology is widely used in many industries, such as finance, communications, and transportation. However, data mining platforms designed for processing massive amounts of visual data are often designed by professional developers. Non-professional users struggle to quickly master the complex algorithm selection and configuration process, hindering the widespread adoption of big data mining and resulting in low accuracy in complex application scenarios.

[0003] Currently, no effective solution has been proposed to address the problem of low accuracy of data mining in complex application scenarios in related technologies. Summary of the Invention

[0004] The embodiments of the present application provide a data mining method, device, and storage medium to at least solve the problem of low accuracy of data mining in complex application scenarios in related technologies.

[0005] In a first aspect, an embodiment of the present application provides a data mining method, the method comprising:

[0006] Get real-time data;

[0007] Through the algorithm warehouse management platform, the registered modular algorithm models are classified and processed, the modular algorithm models under each category are assembled into independent card units, and each independent card unit is dynamically sent to the application platform for configuration;

[0008] Detecting, via the application platform, a first operation instruction for the independent card unit; determining, in response to the detected first operation instruction, a target algorithm model in the modular algorithm model, and logically configuring a plurality of the target algorithm models via the algorithm warehouse management platform to obtain a target workflow;

[0009] The target workflow is called to perform data mining on the real-time data to generate data mining results.

[0010] In some embodiments, in response to the detected first operation instruction, determining a target algorithm model in the modular algorithm model, and logically configuring multiple target algorithm models via the algorithm warehouse management platform to obtain a target workflow includes:

[0011] parsing the first operation instruction to determine the operated card unit indicated by the first operation instruction in the independent card unit;

[0012] Determine the target algorithm model corresponding to the operated card unit from each of the modular algorithm models, configure the target algorithm model according to the reference model parameters in the operated card unit, and obtain the target workflow through the algorithm warehouse management platform.

[0013] In some embodiments, configuring the target algorithm model according to the reference model parameters in the operated card unit and obtaining the target workflow via the algorithm warehouse management platform includes:

[0014] Detecting, via the algorithm warehouse management platform, a parameter adjustment instruction received by the application platform;

[0015] The reference model parameters are adjusted based on the parameter adjustment instruction to obtain target model parameters, and the target algorithm model is configured according to the target model parameters to obtain the target workflow.

[0016] In some embodiments, the real-time data is multimodal data obtained from a multimodal data source.

[0017] In some embodiments, calling the target workflow, performing data mining processing on the real-time data, and generating data mining results include:

[0018] Calling the target workflow, performing data mining processing on the real-time data, and generating a first data mining result;

[0019] In response to the first data mining result, constructing a business logic rule based on a second operation instruction detected by the application platform;

[0020] Determine the logical judgment condition corresponding to the business logic rule based on the attribute feature information of the multimodal data, and generate dynamic logic configuration information;

[0021] Based on the dynamic logic configuration information, a detection event is triggered; in response to the triggered detection event, a second data mining result is generated; the data mining result includes the first data mining result and the second data mining result.

[0022] In some embodiments, determining the logical judgment condition corresponding to the business logic rule based on the attribute feature information of the multimodal data and generating dynamic logic configuration information includes:

[0023] Get the preset alarm rules;

[0024] detecting a rule adjustment instruction of the application platform; adjusting the alarm rule in response to the received rule adjustment instruction to obtain an alarm triggering condition;

[0025] The logic judgment condition is determined based on the attribute feature information of the multimodal data, and the dynamic logic configuration information is generated according to the logic judgment condition and the alarm triggering condition.

[0026] In some embodiments, after generating the second data mining result, the method further includes:

[0027] In response to the second data mining result, acquiring target labeled data via the application platform;

[0028] Based on the target marking data, an optimization configuration scheme is generated, and based on the optimization configuration scheme, the target algorithm model is adjusted to obtain a new algorithm model.

[0029] In some embodiments, obtaining real-time data includes:

[0030] In the case where the data collection task is a multi-concurrent task, the task information of each concurrent task is sent to the corresponding algorithm service node via the algorithm warehouse management platform, and each algorithm service node is called to execute the concurrent tasks in parallel to obtain the real-time data.

[0031] In a second aspect, an embodiment of the present application provides a data mining device, comprising:

[0032] An acquisition unit, used to acquire real-time data;

[0033] An independent card configuration unit is used to classify the registered modular algorithm models through the algorithm warehouse management platform, assemble the modular algorithm models under each category into independent card units, and dynamically send each independent card unit to the application platform for configuration;

[0034] A workflow generation unit is configured to detect, via the application platform, a first operation instruction for the independent card unit; determine a target algorithm model in the modular algorithm model in response to the detected first operation instruction, and logically configure a plurality of the target algorithm models to obtain a target workflow;

[0035] The mining unit is used to call the target workflow to perform data mining on the real-time data and generate data mining results.

[0036] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the program is executed by a processor, the data mining method described in the first aspect above is implemented.

[0037] Compared with the related art, the data mining method, device and storage medium provided in the embodiments of the present application obtain real-time data; classify and process the registered modular algorithm models through the algorithm warehouse management platform, assemble the modular algorithm models under each category into independent card units, and dynamically send each independent card unit to the application platform for configuration; detect the first operation instruction for the independent card unit through the application platform; determine the target algorithm model in the modular algorithm model in response to the detected first operation instruction, and logically configure multiple target algorithm models through the algorithm warehouse management platform to obtain a target workflow; call the target workflow to perform data mining on the real-time data to generate data mining results.

[0038] Through the above method, the algorithms are standardized and classified and packaged into cards through the algorithm warehouse management platform, which significantly improves the reusability and maintainability of algorithm components. At the same time, the dynamic card push mechanism enables front-end applications to quickly adapt to new algorithms, reducing the cost of technical docking; combined with the visual orchestration capabilities of the application platform, users can independently build workflows through drag-and-drop operations on the low-code interface, and convert professional algorithm capabilities into business-understandable configuration items, thereby building an end-to-end adaptive analysis system, which not only meets personalized business needs, but also reduces manual intervention through automated configuration, while improving development efficiency. It enhances the real-time and accuracy of data processing and analysis, effectively solving the problem of low accuracy of data mining in complex application scenarios.

[0039] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a hardware structure block diagram of a terminal for a data mining method according to an embodiment of the present application;

[0042] Figure 2 is a flow chart of a data mining method according to an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a graphical card interface according to an embodiment of the present application;

[0044] Figure 4 is a structural diagram of a data mining method according to an embodiment of the present application;

[0045] Figure 5This is a structural block diagram of a data mining device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means and should not be understood as the contents disclosed in the present application being insufficient.

[0047] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0048] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote limitations on quantity and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means greater than or equal to two. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The terms "first", "second", "third" and the like involved in this application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0049] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 FIG. 1 is a block diagram of the hardware structure of a terminal according to a data mining method according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0050] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data mining method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0051] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0052] As mentioned in the background technology, although the existing data mining platforms can realize basic data storage and management functions when processing massive visual big data, they still have the following shortcomings when dealing with complex and diverse application scenarios: 1. Low flexibility: lack of modular design, unable to quickly integrate and replace the latest target detection, anomaly detection and other algorithm models, and difficult to adapt to the needs of rapid iteration of artificial intelligence algorithms; 2. High user threshold: existing platforms are mostly designed for professional developers, and it is difficult for non-professional users to quickly master the complex algorithm selection and configuration process, affecting the popularity of big data mining; 3. Lack of deep visualization and analysis capabilities: existing platforms only provide basic detection result display, and cannot intuitively and dynamically present data features and analysis results, limiting users' in-depth understanding and decision-making capabilities of anomaly causes, data distribution and optimization directions.

[0053] Based on this, this embodiment provides a data mining method. Figure 2 is a flow chart of a data mining method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0054] Step S210: Acquire real-time data.

[0055] In this step, sensors, cameras, and other devices deployed in locations such as high-speed rail gates collect multi-source data in real time. For example, cameras near the gates can feed real-time video data into the data mining platform via a network video stream, such as the Real-Time Streaming Protocol (RTSP). The data acquisition and preprocessing module in this data mining platform employs efficient video decoding technology, utilizing hardware decoding (e.g., GPUs) to decode the video stream and break it down into consecutive frames. To reduce processing redundancy, the module also incorporates a keyframe extraction mechanism, extracting every five frames as a keyframe. This reduces the data volume while preserving the continuity of the frame sequence, facilitating subsequent analysis.

[0056] After decoding, the keyframes enter the data preprocessing module, where they undergo standardization, including format conversion, resizing, and pixel normalization. For example, the original video frame is resized from 1920×1080 to 640×480 to meet the input requirements of the object detection model. The module also supports data augmentation strategies such as random cropping, color jittering, and brightness adjustment to enhance the model's robustness under varying lighting conditions. Furthermore, the module utilizes a graphical interface to allow users to configure preprocessing parameters, such as selecting the frame interval and adjusting image resolution, to meet the specific needs of different gate environments.

[0057] In step S220, the registered modular algorithm models are classified through the algorithm warehouse management platform, the modular algorithm models under each category are assembled into independent card units, and each independent card unit is dynamically sent to the application platform for configuration.

[0058] It should be noted that in this application, an algorithm warehouse management platform is used to uniformly manage models, process orchestration, and computing resources. Specifically, the application platform or business front-end submits the algorithm model process configuration to the algorithm warehouse management platform; the algorithm warehouse management platform is responsible for scheduling computing resources and attaching the algorithm warehouse platform address in the startup command; after each algorithm service is started, it will register with the platform and connect to the algorithm warehouse; the algorithm warehouse management platform assigns or creates specific video analysis tasks, and the parameter information includes: task ID, video channel address, business algorithm ID, algorithm version number, model version number, etc. If the algorithm ID does not exist or the version number is expired, the model list, download address, and update will be automatically obtained; when all dependencies are correctly loaded, the analysis task will be created and started; after the task is successfully created, it will start running and report the results. During this process, the algorithm service node continues to report events according to the front-end requirements or detection rules; if the detection results meet the preset threshold conditions, subsequent business logic or alarms will be triggered.

[0059] More specifically, the algorithm warehouse management platform categorizes registered modular algorithm models by their functional type, including object detection, anomaly detection, and re-identification algorithms. Within each category, related models are organized into independent card units. For example, the object detection category includes models such as Faster Region-based Convolutional Neural Network (Faster R-CNN) and YOLOv3 (You Only Look Once version 3) to YOLOv5 (You Only Look Once version 5). These models are distributed to the application platform via a dynamic push mechanism. Each card is dynamically generated using a graphical card interface configured by the application platform and includes interactive features such as hovering to view details or clicking to select. Cards can be configured declaratively, such as using JSONSchema (a JSON-based specification) to define input feature dimensions. These cards support plug-and-play and cross-node collaboration, providing a standardized component library for low-code configuration.

[0060] The algorithm warehouse management platform mentioned above mainly includes: a model warehouse, which stores a variety of packaged algorithm models (models 1, 2, 3, 4, etc.) and provides version management and metadata management capabilities; algorithm orchestration, which combines or orchestrates models in a process-based manner to form reusable business algorithms (business algorithms 1, 2, 3, etc.) to meet the needs of different scenarios; computing power management / scheduling, which dynamically allocates computing power resources such as GPUs and CPUs to each algorithm service node (nodes 1, 2, 3, etc.), and can perform real-time scheduling and scaling according to load conditions.

[0061] With the help of the above architectural design, the present invention effectively solves the shortcomings of traditional platforms in model management and computing power scheduling, allowing front-end businesses to quickly call and deploy models through dragging, selection, configuration, etc.

[0062] Step S230, detect the first operation instruction for the independent card unit through the application platform; in response to the detected first operation instruction, determine the target algorithm model in the modular algorithm model, and logically configure multiple target algorithm models through the algorithm warehouse management platform to obtain the target workflow.

[0063] Specifically, the application platform can capture the user's first operation instructions on independent card units (such as drag and drop, click, hover, etc.) in real time through a graphical card interface. After analyzing the operation intention, the selected target algorithm model is connected in series according to the business logic through the orchestration engine of the algorithm warehouse, and finally an executable target workflow is formed. Among them, the workflow supports complex structures such as sequential execution, branch routing (such as condition-based judgment), loop iteration, and inherits automation nodes. The above-mentioned graphical card interface will be dynamically updated according to the results of the model selection, and the currently selected model and its parameter settings will be displayed. In the case of multiple model combinations, it supports the simultaneous selection of multiple models and the setting of parameters separately to meet the logical requirements of complex businesses.

[0064] In addition, the application platform can interact with the algorithm warehouse management platform through a graphical card interface. The application platform (business layer) allows for drag-and-drop configuration of algorithms in a low-code environment, such as selecting a pedestrian detection algorithm or anomaly detection algorithm. The algorithm warehouse management platform centrally manages algorithm services, allowing multiple algorithm service nodes (ARS) to access the platform and perform different analysis tasks. During execution, algorithm service nodes maintain communication with the platform, allowing the platform to monitor each node's task status and load in real time, ensuring real-time processing capabilities under high concurrency.

[0065] Step S240: calling the target workflow to perform data mining on the real-time data and generate data mining results.

[0066] Call the generated target workflow, input the real-time data obtained in step S210 into the target workflow to perform distributed computing, and generate structured mining results (such as the probability value of tailing events) through pipeline reasoning (such as target detection to spatiotemporal feature extraction to behavior classification); the results are written to the database in real time and trigger an alarm, while recording performance logs for closed-loop optimization.

[0067] Through the above steps S210 to S240, the algorithms are standardized and classified and packaged into cards through the algorithm warehouse management platform, which significantly improves the reusability and maintainability of the algorithm components. At the same time, the dynamic card push mechanism enables the front-end application to quickly adapt to the new algorithm, reducing the cost of technical docking; combined with the visual orchestration capabilities of the application platform, users can independently build workflows through drag-and-drop operations on the low-code interface, and convert professional algorithm capabilities into business-understandable configuration items, thereby building an end-to-end adaptive analysis system, which not only meets personalized business needs, but also reduces manual intervention through automated configuration, while improving development efficiency. It enhances the real-time and accuracy of data processing and analysis, effectively solving the problem of low accuracy of data mining in complex application scenarios.

[0068] In some embodiments, the above-mentioned steps of determining a target algorithm model in the modular algorithm model in response to the detected first operation instruction, and logically configuring multiple target algorithm models via the algorithm warehouse management platform to obtain a target workflow may also include the following steps:

[0069] Parse the first operation instruction and determine the operated card unit indicated by the first operation instruction in the independent card unit; determine the target algorithm model corresponding to the operated card unit from each modular algorithm model, and configure the target algorithm model according to the reference model parameters in the operated card unit, and obtain the target workflow through the algorithm warehouse management platform.

[0070] In this embodiment, precise configuration and process orchestration of the algorithm model are achieved by parsing user interaction instructions: when the application platform detects the first operation instruction for an independent card unit (such as drag-and-drop, parameter adjustment, or connection relationship change), the system first parses the operation type and target object in the instruction to locate the specific card unit being operated; then, through the mapping relationship between the card metadata and the algorithm warehouse registration information, the target algorithm model corresponding to the card is matched from the modular algorithm model library. For example, the "face recognition card" dragged by the user corresponds to the pre-trained face recognition network (Face Recognition Network, FaceNet); then, the target algorithm model is initialized and configured according to the reference model parameters embedded in the card unit (such as confidence threshold and detection area coordinates) to ensure that the model input and output interfaces are aligned with business requirements; finally, the orchestration engine of the algorithm warehouse management platform combines the configured target algorithm models according to the logical relationship implicit in the first operation instruction (such as sequential execution and conditional branching) to generate a target workflow that includes data flow, model call order, and exception handling mechanism. After version verification and performance estimation, the workflow can be deployed and executed.

[0071] Specifically, see Figure 3 , which is used to display a graphical card interface. On the right side of the interface, a menu display bar of all components, including atomic capabilities, business processing, and other components, is displayed. Users can drag components from the display bar to the center of the interface to assemble them to form the target workflow. For example, when the user connects the "License Plate Recognition Card" and the "Trajectory Analysis Card" in series through the visual interface, the system automatically configures the parameters of the two models, such as the Region of Interest (ROI) area for license plate detection and the time window for trajectory analysis, and generates a series of workflows including image preprocessing, license plate positioning, character recognition, and trajectory modeling, realizing a complete analysis link from video stream input to illegal vehicle tracking.

[0072] Through the above embodiment, the operated card unit is accurately located by parsing the user's first operation instruction, and the target algorithm model can be quickly matched in combination with the mapping relationship between the card metadata and the algorithm model. This mechanism transforms traditional code-level development into visual configuration, significantly lowering the technical threshold and allowing business personnel to directly participate in algorithm process design; secondly, dynamic configuration is performed based on the reference model parameters embedded in the card unit. This parameterized configuration method achieves precise adaptation of algorithm capabilities and business needs while keeping the core logic of the model unchanged.

[0073] In some embodiments, the above-mentioned configuration of the target algorithm model according to the reference model parameters in the operated card unit and obtaining the target workflow via the algorithm warehouse management platform may further include the following steps:

[0074] The algorithm warehouse management platform detects the parameter adjustment instructions received by the application platform; adjusts the reference model parameters based on the parameter adjustment instructions to obtain the target model parameters, and configures the target algorithm model according to the target model parameters to obtain the target workflow.

[0075] Specifically, when a user initiates a parameter adjustment operation on a configured algorithm card on the application platform (such as modifying detection sensitivity, adjusting classification thresholds, or specifying input data sources), the platform first identifies the parameter type and adjustment range through its built-in instruction parsing module and generates a structured parameter adjustment instruction. After receiving the instruction, the algorithm warehouse management platform extracts the reference model parameters corresponding to the operated card (such as pre-trained weight files and hyperparameter configuration tables) from the centralized parameter library based on the mapping relationship between card metadata and model parameters. It then uses the parameter verification engine to ensure that the adjustment values ​​meet the model constraints (for example, the confidence threshold must be in the range of 0-1). Subsequently, the platform adopts an incremental update strategy, modifying only the parameters affected by the instruction and retaining other default parameters to generate target model parameters that match the specific business scenario. Next, the algorithm warehouse management platform injects the target parameters into the corresponding target algorithm model through the model service interface, triggering the model's hot reload mechanism to ensure that the parameter changes take effect without restarting the service. Finally, the platform re-evaluates the dependencies between each node in the workflow based on the updated model parameters, optimizes the execution order through a dynamic programming algorithm, and integrates exception handling logic (such as a rollback strategy for illegal parameters) to generate a target workflow that both meets new requirements and maintains system stability. For example, when a user adjusts the "pedestrian detection confidence" in the video analysis card from 0.8 to 0.95, the system automatically updates the classification threshold parameters of the corresponding YOLOv8 model and re-arranges the workflow including preprocessing, detection, and post-processing to ensure high-precision detection while maintaining real-time processing performance.

[0076] Through the above embodiment, by real-time detection of parameter adjustment instructions of the application platform, it is possible to quickly respond to users' refined demands for model performance, thereby improving workflow execution efficiency while ensuring system stability.

[0077] In some embodiments, the real-time data is multimodal data acquired from multimodal data sources. Specifically, the data mining platform supports dynamic data acquisition from multimodal data sources (such as time series data and geographic location information), including target detection results in real-time video streams, positioning data in geographic information systems (GIS), and time series signals collected by sensors. After data acquisition, it undergoes preprocessing, including format conversion, data normalization, and timestamp alignment, to ensure the consistency of the integration of data from different modalities.

[0078] Based on this, the above-mentioned calling target workflow, performing data mining processing on real-time data, and generating data mining results may also include the following steps:

[0079] Call the target workflow, perform data mining processing on the real-time data, and generate a first data mining result; in response to the first data mining result, build a business logic rule based on the second operation instruction detected by the application platform; based on the attribute feature information of the multimodal data, determine the logical judgment condition corresponding to the business logic rule, and generate dynamic logic configuration information; based on the dynamic logic configuration information, trigger a detection event; in response to the triggered detection event, generate a second data mining result; the data mining result includes the first data mining result and the second data mining result.

[0080] In this embodiment, the business logic support module in the above-mentioned data mining platform implements dynamic logic judgment and rule configuration based on multimodal data through a visual rule definition tool; the method includes the following steps:

[0081] First, a pre-configured target workflow is invoked to perform initial mining on the real-time data stream, generating a first data mining result containing basic analytical conclusions (such as device status indicators, user behavior tags, or target detection coordinates). Subsequently, based on the first operational instructions issued by the user on the application platform (such as setting anomaly thresholds or defining association rules), the first result is converted into executable business logic rules. Regarding the provision of rule definition tools: the tool utilizes a graphical card interface, allowing users to construct business logic rules by dragging and dropping components or selecting pre-set templates. Interface components include, but are not limited to, conditional nodes, logical operators (such as "and" and "or"), and action nodes, enabling users to intuitively define complex logical relationships.

[0082] Next, multimodal data access and analysis is performed. Dynamic data acquisition from multimodal data sources (such as time series data and geographic location information) is supported, including target detection results in real-time video streams, positioning data in geographic information systems (GIS), and time series signals collected by sensors. After access, the data undergoes preprocessing, including format conversion, data normalization, and timestamp alignment, to ensure the consistency of the fusion of data from different modalities.

[0083] When real-time data meets dynamically configured detection conditions, the system automatically triggers a pre-set detection event (such as initiating a secondary verification algorithm or invoking an emergency response process). It then generates a second data mining result (such as root cause analysis or predictive maintenance recommendations) by re-executing the target workflow or invoking a specialized algorithm model. Ultimately, the two analysis results are combined and output, forming a complete closed loop from real-time monitoring to rule-driven development and dynamic response. This mechanism enables the system to adaptively adjust business logic based on initial analysis results and deepen the value of data through secondary mining, shortening decision cycles while improving analysis accuracy and scenario adaptability.

[0084] In some embodiments, the above-mentioned process of determining the logical judgment conditions corresponding to the business logic rules based on the attribute feature information of the multimodal data and generating the dynamic logic configuration information may further include the following steps:

[0085] Obtain preset alarm rules; detect rule adjustment instructions from the application platform; adjust the alarm rules in response to the received rule adjustment instructions to obtain alarm trigger conditions; determine logical judgment conditions based on the attribute feature information of multimodal data, and generate dynamic logical configuration information based on the logical judgment conditions and alarm trigger conditions.

[0086] By building a collaborative adjustment mechanism between alarm rules and dynamic logic, intelligent processing from pre-set rules to scenario-based adaptation is achieved. Specifically, pre-configured alarm rules (such as triggering an alert when the device temperature exceeds 80°C) are first obtained as the basic judgment framework. Then, rule adjustment instructions from the application platform are detected in real time. Users can dynamically adjust alarm rules through rule definition tools. For example, in a real-time monitoring scenario, alarm trigger conditions can be modified based on the number and distribution of detected personnel. A rule engine (such as Drools) parses and executes the set rules in real time to ensure the accuracy and timeliness of logical judgments.

[0087] On this basis, combined with the attribute characteristics of multimodal data (such as target detection results in video streams and timing signals collected by sensors) (such as data timing correlation, spatial distribution range, and type differences), the logical judgment conditions are further refined (for example, defining the correlation between temperature data and equipment load, and setting the effective range of personnel detection areas); finally, the adjusted alarm trigger conditions are integrated with the logical judgment conditions determined based on multimodal data to generate dynamic logical configuration information containing conditional expressions, threshold parameters, and associated actions, so that the system can automatically match the optimal alarm logic according to real-time data characteristics, while ensuring the rigor of the rules and improving scene adaptability.

[0088] In some embodiments, after generating the second data mining result, the data mining method may further include the following steps:

[0089] In response to the second data mining result, target label data is obtained via the application platform; based on the target label data, an optimization configuration scheme is generated, and the target algorithm model is adjusted based on the optimization configuration scheme to obtain a new algorithm model.

[0090] Specifically, the platform will feed back the results of rule execution (i.e., the second data mining results mentioned above) to users through visual charts or alarm prompts, such as highlighting the abnormal area corresponding to the trigger condition on the interface; users can adjust rule parameters or add new conditions based on the execution results to optimize business logic.

[0091] More specifically, the above method supports users to mark missed detections and false detections in the model detection results through a graphical card interface, and generates a targeted optimization plan based on the user-marked data. The method includes the following steps:

[0092] The model's detection results are displayed through a graphical card interface, specifically including: target detection tasks, which display the location and confidence of the detected target in the form of a bounding box; anomaly detection tasks, which display the detection results in the form of a heatmap or abnormal area overlay, mark areas where anomalies may exist, and provide anomaly scores or probabilities; pedestrian re-identification tasks, which provide a list of matching results and corresponding confidence scores; users can carefully view every detail of the detection results through zooming, panning, and clicking interactive functions.

[0093] Second, the graphical card interface provides a marking tool that allows users to manually select undetected targets (missed detections) or incorrectly detected targets (false detections);

[0094] For missed targets, users can add markers by drawing bounding boxes or directly clicking on the abnormal area. For falsely detected targets, users can directly click on the corresponding detection results to mark them and select the marking type (such as "area error" or "abnormal miss").

[0095] The data mining platform stores user-labeled data in a dedicated labeling database, recording the labeling location, category, and user notes. After statistical analysis of the labeled data, an error distribution report is generated, including the missed detection rate, false detection rate, and the distribution of error types. Ultimately, targeted optimization solutions are automatically generated based on the labeled data, including but not limited to: hyperparameter adjustment suggestions, such as adjusting the threshold settings in anomaly detection tasks or the scoring range of model outputs; data enhancement strategy recommendations, such as simulating more abnormal samples or enhancing the characteristics of abnormal areas based on the distribution characteristics and background of abnormal areas in the labeled data; and constraint definition, such as defining constraints based on the labeled error type, such as designating specific areas as key detection areas or increasing the detection weight for certain types of anomalies.

[0096] These optimization solutions can be directly applied to model retraining or parameter adjustment. A graphical card interface allows users to manually modify or regenerate optimization solutions to ensure their applicability and flexibility. After applying the optimization solution, the data mining platform re-evaluates the model and generates a performance improvement report comparing the performance before and after optimization. Users can re-label the optimization results, forming a feedback loop to further improve model performance.

[0097] In some embodiments, the above-mentioned acquisition of real-time data may further include the following steps:

[0098] When the data collection task is a multi-concurrent task, the task information of each concurrent task is sent to the corresponding algorithm service node through the algorithm warehouse management platform, and each algorithm service node is called to execute the concurrent tasks in parallel to obtain real-time data.

[0099] Under high-concurrency conditions, the platform uses a task scheduling mechanism to distribute video streams from multiple cameras to different nodes for parallel processing to ensure real-time performance. At this time, the algorithm warehouse management platform will send the corresponding analysis task (including information such as the video channel address and algorithm ID) to the corresponding algorithm service node.

[0100] During peak gate hours, the simultaneous video streams captured by multiple cameras can increase system processing pressure. This invention achieves high concurrency support through a resource optimization algorithm. The task scheduling system dynamically allocates video stream processing tasks based on the CPU and GPU load of each node, ensuring that computing pressure is evenly shared across all nodes. Furthermore, a pipelined processing architecture is employed to seamlessly integrate and execute video decoding, data preprocessing, model inference, and alarm logic in separate steps to minimize latency.

[0101] More specifically, the data mining platform supports real-time processing of input data from multiple signal sources under high concurrency conditions, and the method includes the following steps:

[0102] Access and management of multiple signal sources: The platform supports access to multiple signal sources, including local video files, network video streams (such as RTSP streams), and real-time camera input. It uses a unified data access interface to standardize data sources of different formats, frame rates, and resolutions, ensuring the system's compatibility with multiple signal types. It manages concurrent connections to signal sources through connection pooling technology, dynamically allocating network bandwidth and decoding resources to ensure stable data access.

[0103] Concurrent scheduling of real-time data processing: Under high-concurrency conditions, a distributed task scheduling system is used to distribute signal source input data in blocks to multiple processing nodes. Advanced queue management mechanisms are used to group input data, giving priority to high-priority data streams while avoiding blockages caused by resource competition.

[0104] Hardware acceleration: The platform integrates hardware acceleration modules (such as GPUs or dedicated hardware decoders) to enable rapid execution of video decoding and data preprocessing. It also supports batch decoding at the hardware level, optimizing the decoding of high-frame-rate data streams and reducing the processing time per frame.

[0105] Application of resource optimization algorithms: The platform uses a dynamic resource allocation algorithm to adjust computing resource allocation in real time during task execution based on the current CPU and GPU load status. It also utilizes a load balancing strategy to evenly distribute high-computing-demand tasks across multiple nodes to avoid single-point performance bottlenecks. A resource recycling mechanism is introduced for low-priority tasks to reallocate underutilized resources, improving overall resource utilization.

[0106] Real-time performance guarantee mechanism: The platform uses real-time optimization algorithms to reduce data processing delays, including network latency during data transmission and queue time during task scheduling. It also uses a pipeline processing architecture to perform data access, decoding, preprocessing, and inference tasks in separate steps and seamlessly connect them, shortening the processing cycle.

[0107] Performance monitoring and feedback: Equipped with a performance monitoring module, it monitors the utilization of key resources such as CPU, GPU, memory, and network bandwidth in real time. It provides users with feedback on resource usage status through data visualization tools, facilitating manual adjustment of resource allocation strategies when needed.

[0108] Through the above-mentioned embodiments, multiple concurrent tasks are split up and executed in parallel across multiple algorithm service nodes, fully utilizing computing resources and significantly reducing task processing time. Furthermore, using the algorithm warehouse management platform as a unified scheduling center allows for real-time monitoring of node loads and dynamic task allocation, ensuring maximum resource utilization.

[0109] In some embodiments, the above-mentioned acquisition of real-time data may further include the following steps:

[0110] Obtain continuous video frames; set frame interval parameters according to preset task requirement information; extract corresponding key frames from continuous video frames based on the frame interval parameters; perform integrity check on the key frames, and obtain real-time data based on the key frames if the integrity check passes.

[0111] Specifically, the data acquisition and preprocessing module in the above-mentioned data mining platform uses a video key frame extraction method based on a fixed frame interval to extract key frames from a video stream or a stored video file to reduce computational redundancy and improve data processing efficiency. The method includes the following steps:

[0112] First, the video frame sequence is read. Consecutive video frames are parsed from the input video stream through hardware or software decoding to form a frame sequence. The frame interval is set. Specifically, the frame interval parameter N is set according to the specific task requirements, defining the interval between key frames. Next, data redundancy control is implemented to reduce the total number of frames to be processed, effectively reducing the data processing workload in video analysis tasks. This is particularly suitable for optimizing real-time stream processing and large-scale video data storage. Frame information verification is then performed to verify the integrity of the extracted key frames to ensure that the frame data is intact and undamaged. If necessary, pixel normalization and color space conversion are performed on the key frames to meet subsequent processing requirements. Finally, frame buffer management is performed to store the key frames in an efficient cache for subsequent model inference or further data preprocessing operations such as object detection and image segmentation.

[0113] The following is a detailed description of this application with reference to specific embodiments. In the high-speed railway station gate tailgating detection scenario, multiple modules work together to provide an efficient, real-time and intelligent solution from video stream data collection to behavior judgment and alarm; please refer to Figure 4 The data mining platform includes a data management module, a container deployment module for Docker (a containerized platform), a user interface module, a model integration module, a business logic module for spatiotemporal business logic constraints, and a standard platform function module. The standard platform function module supports platform management, product management, user management, low-code application development, resource allocation, and standard continuous integration / continuous deployment (CI / CD). The following is the implementation process of each component of the data mining platform:

[0114] The data management module includes a data acquisition submodule, a data storage submodule, and a data preprocessing submodule. This data management module automatically manages the type, format, and source of stored data through metadata services, supporting access to diverse data sources, including IoT sensor data, social media platform data, e-commerce transaction data, medical imaging data, traffic surveillance video data, and government public sector data. The module employs data sharding and distributed storage strategies, dividing large-scale data into multiple shards and distributing them across multiple nodes to improve data access speed and system fault tolerance. The module supports dynamic scalability based on a distributed storage architecture, enabling online expansion of storage capacity based on business needs to accommodate data storage requirements ranging from terabytes (TB) to petabytes (PB). The module provides data access interfaces based on multiple access protocols, such as SQL and the Hadoop Distributed File System API (HDFS API), enabling users to efficiently retrieve and process stored data. Indexing technology is also used to further optimize retrieval performance. The module uses data compression and deduplication technologies to reduce storage space requirements while ensuring data integrity, making it suitable for efficient management in large-scale data scenarios.

[0115] The model integration module integrates various algorithmic models, including abnormal behavior detection models, target detection models, and re-identification models. It provides a modular design and standardized interfaces, supporting flexible configuration of model parameters and business logic. For example, the model integration module loads the YOLOv5 target detection model, which is specifically designed to identify pedestrian targets within the gate area. Users select a model through a graphical card interface and intuitively view the model introduction, applicable scenarios, and default parameters on the card. To adapt to the tailgating detection scenario, users adjust the confidence threshold to 0.5 and set the coordinate range of the gate area in the interface. The module loads models through a standardized interface and supports hardware-accelerated inference to improve real-time detection performance. The inference results include the coordinate information and confidence score of the pedestrian bounding box, providing spatial distribution data of pedestrians in the gate area.

[0116] The user interface module provides a library of drag-and-drop UI components, visualizations, and statistics. For example, users select the algorithm by clicking the card for the desired model. The system provides default parameter settings for each model, such as the confidence threshold for object detection or the region size for segmentation algorithms. Users can adjust model parameters based on the actual application scenario by clicking the parameter field and entering a new value or using the slider.

[0117] The business logic module provides flexible logic rule definition for tailgating detection. It supports the definition of complex business logic through the rule engine, and implements logical judgment and event triggering of multimodal data. Users use the visual rule definition tool to build business logic, for example:

[0118] Rule 1: If two or more pedestrians are detected in the gate area at the same time and the distance between their bounding boxes is less than 30 pixels, it is considered a tailgating behavior;

[0119] Rule 2: If the bounding boxes of the detected pedestrian targets partially overlap and exist for more than 3 consecutive frames, an alarm signal is issued.

[0120] Users can drag and drop components in the interface to set logical nodes and dynamically adjust rule parameters (such as distance threshold and frame rate). Rules are parsed into event trigger conditions and executed in real time by the Drools rule engine.

[0121] When tailgating is detected, the business logic support module triggers an alarm. The model evaluation module highlights the relevant image frames for the triggering condition and displays details of the abnormal behavior through a visual interface, including the location of the bounding box, the number of pedestrians detected, and the confidence score. Simultaneously, the alarm information is sent to the security personnel's mobile device in text format, including the gate number, event time, and anomaly type. The platform also supports storing alarm records in the operation log for subsequent tracking and analysis.

[0122] In addition, the above-mentioned data mining platform may also include a model evaluation module and a user rights management module.

[0123] The model evaluation module allows users to annotate missed detections and false detections. For example, users can mark incorrectly identified tailing behaviors as missed detections, or misidentified behaviors as false detections. Statistical analysis of the annotated data generates an error distribution report, including missed detection rates, false detection rates, and major error types. Based on the annotated data, the platform automatically generates optimization solutions, such as: recommending adjustments to the model's confidence threshold to reduce missed detections; generating data enhancement samples that simulate tailing behaviors to enrich the model training set; and defining more refined detection area weights to improve detection accuracy. The optimization solution can be directly used for model retraining or parameter fine-tuning to achieve continuous performance improvement.

[0124] To ensure the security of system operations, the user rights management module establishes hierarchical permissions for different user roles. For example, ordinary security personnel can only view alarm logs and image data, while administrators can modify business logic rules and model parameters. Users must undergo two-factor authentication (e.g., SMS verification code and password) when logging into the system. A unique audit record is generated for each operation, including the time, module, and specific details of the operation. The platform also supports abnormal behavior detection, such as triggering account lockout after repeated login failures, further enhancing system security.

[0125] More specifically, the user's operation authority is controlled through a hierarchical management and dynamic authorization mechanism, and the method includes the following steps:

[0126] User identity authentication: Verify user identity through various authentication methods, including but not limited to username and password authentication, two-factor authentication, and biometric authentication (such as fingerprint or facial recognition). When a user logs in, the system generates a unique session identifier (Session ID) and stores it in encrypted form to ensure session security.

[0127] Permission classification and role definition: The system pre-defines multiple user roles, such as administrator, developer, general user, and guest, each with a different scope of permissions. It also supports customizing roles based on business needs, allowing users to assign specific permissions to new roles through the interface.

[0128] Dynamic permission management: Provides dynamic authorization based on user groups or individuals, allowing administrators to adjust user access rights based on real-time needs; supports fine-grained permission control, such as setting access rights for specific datasets, functional modules, or task operations (such as model training, parameter setting, or result export);

[0129] Operation log recording and auditing: The system records all user operations in real time, including login time, modules accessed, modified parameters, and operations performed. Operation logs are stored in encrypted form and support retrieval based on time period, user, or operation type, facilitating security audits and problem tracing.

[0130] Access control strategy: A strategy combining role-based access control (RBAC) and attribute-based access control (ABAC) is adopted to ensure that different users' access to resources meets security and business requirements; the system has a built-in permission conflict detection mechanism to prevent excessive authorization caused by overlapping permissions of multiple roles.

[0131] Through the above embodiments, a low-code data mining platform for massive visual big data is provided. This platform utilizes hardware acceleration and a distributed task scheduling mechanism to achieve real-time data processing under high concurrency conditions, meeting the requirements of multiple signal source inputs and being suitable for intelligent traffic monitoring and rapid response scenarios. The model call and integration module, algorithm warehouse management platform, and other components utilize modular and standardized interfaces to support the rapid integration and dynamic replacement of multiple algorithm models, which can be flexibly configured within a front-end visual interface. Through a visual rule definition tool, users can construct complex logical rules to achieve fusion analysis of multimodal data and dynamic event triggering. The rule engine supports real-time parsing and adjustment, enhancing the system's flexibility and intelligence. A graphical user interface and drag-and-drop operation enable even non-professional users to easily complete complex algorithm configuration, logic definition, and data mining tasks, significantly lowering the barrier to entry. The platform supports the annotation and analysis of false positives and missed detections, and combines optimization solutions (such as parameter adjustment and data augmentation strategies) to form a feedback loop to continuously improve model performance and stability. The platform also manages algorithm and model versions and dependencies in a unified manner, automating download, update, and registration to ensure consistency and correctness when calling models from the front end, improving system maintainability and iteration efficiency. In addition, the technical solution can be applied to multiple scenarios such as intelligent traffic monitoring, medical image analysis, and industrial anomaly detection.

[0132] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0133] This embodiment also provides a data mining device, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated. As used below, the terms "module," "unit," "subunit," etc. may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0134] Figure 5 is a structural block diagram of a data mining device according to an embodiment of the present application, such as Figure 5 As shown, the device includes:

[0135] The acquisition unit 51 is used to acquire real-time data; the independent card configuration unit 52 is used to classify the registered modular algorithm models through the algorithm warehouse management platform, assemble the modular algorithm models under each category into independent card units, and dynamically send each independent card unit to the application platform for configuration; the workflow generation unit 53 is used to detect the first operation instruction for the independent card unit through the application platform; in response to the detected first operation instruction, determine the target algorithm model in the modular algorithm model, and logically configure multiple target algorithm models to obtain a target workflow; the mining unit 54 is used to call the target workflow to perform data mining on real-time data and generate data mining results.

[0136] In some embodiments, the workflow generation unit 53 is also used to parse the first operation instruction, determine the operated card unit indicated by the first operation instruction in the independent card unit; determine the target algorithm model corresponding to the operated card unit from each modular algorithm model, and configure the target algorithm model according to the reference model parameters in the operated card unit, and obtain the target workflow through the algorithm warehouse management platform.

[0137] In some embodiments, the workflow generation unit 53 is also used to detect parameter adjustment instructions received by the application platform via the algorithm warehouse management platform; adjust the reference model parameters based on the parameter adjustment instructions to obtain the target model parameters, and configure the target algorithm model according to the target model parameters to obtain the target workflow.

[0138] In some embodiments, the mining unit 54 is further used to call the target workflow, perform data mining processing on real-time data, and generate a first data mining result; in response to the first data mining result, construct a business logic rule based on the second operation instruction detected by the application platform; based on the attribute feature information of the multimodal data, determine the logical judgment condition corresponding to the business logic rule, and generate dynamic logic configuration information; based on the dynamic logic configuration information, trigger a detection event; in response to the triggered detection event, generate a second data mining result; the data mining result includes the first data mining result and the second data mining result.

[0139] In some embodiments, the mining unit 54 is also used to obtain preset alarm rules; detect rule adjustment instructions from the application platform; adjust the alarm rules in response to the received rule adjustment instructions to obtain alarm triggering conditions; determine logical judgment conditions based on the attribute feature information of the multimodal data, and generate dynamic logical configuration information based on the logical judgment conditions and the alarm triggering conditions.

[0140] In some embodiments, the above-mentioned data mining device also includes an optimization unit; the optimization unit is used to obtain target label data via the application platform in response to the second data mining result; generate an optimization configuration plan based on the target label data, and adjust the target algorithm model based on the optimization configuration plan to obtain a new algorithm model.

[0141] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination. For specific examples in this embodiment, reference can be made to the examples described in the above embodiment and optional implementations, and will not be repeated in this embodiment.

[0142] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0143] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0144] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0145] S1, obtain real-time data.

[0146] S2, through the algorithm warehouse management platform, classify the registered modular algorithm models, assemble the modular algorithm models under each category into independent card units, and dynamically send each independent card unit to the application platform for configuration.

[0147] S3, through the application platform, detect the first operation instruction for the independent card unit; in response to the detected first operation instruction, determine the target algorithm model in the modular algorithm model, and logically configure multiple target algorithm models through the algorithm warehouse management platform to obtain the target workflow.

[0148] S4, calling the target workflow to perform data mining on real-time data and generate data mining results.

[0149] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0150] In addition, in conjunction with the data mining method in the above embodiments, the present application embodiment may provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the data mining methods in the above embodiments is implemented.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0152] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0153] Those skilled in the art should understand that the various technical features of the above-described embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A data mining method, characterized in that: The method comprises: Get real-time data; Through the algorithm warehouse management platform, the registered modular algorithm models are classified and processed, the modular algorithm models under each category are assembled into independent card units, and each independent card unit is dynamically sent to the application platform for configuration; Detecting, via the application platform, a first operation instruction for the independent card unit; determining, in response to the detected first operation instruction, a target algorithm model in the modular algorithm model, and logically configuring a plurality of the target algorithm models via the algorithm warehouse management platform to obtain a target workflow; The target workflow is called to perform data mining on the real-time data to generate data mining results.

2. The data mining method according to claim 1, characterized in that: In response to the detected first operation instruction, determining a target algorithm model in the modular algorithm model, and logically configuring multiple target algorithm models via the algorithm warehouse management platform to obtain a target workflow, including: parsing the first operation instruction to determine the operated card unit indicated by the first operation instruction in the independent card unit; Determine the target algorithm model corresponding to the operated card unit from each of the modular algorithm models, configure the target algorithm model according to the reference model parameters in the operated card unit, and obtain the target workflow through the algorithm warehouse management platform.

3. The data mining method according to claim 2, characterized in that: The configuring the target algorithm model according to the reference model parameters in the operated card unit and obtaining the target workflow via the algorithm warehouse management platform includes: Detecting, via the algorithm warehouse management platform, a parameter adjustment instruction received by the application platform; The reference model parameters are adjusted based on the parameter adjustment instruction to obtain target model parameters, and the target algorithm model is configured according to the target model parameters to obtain the target workflow.

4. The data mining method according to claim 1, wherein: The real-time data is multimodal data obtained from a multimodal data source; the calling of the target workflow to perform data mining processing on the real-time data to generate data mining results includes: Calling the target workflow, performing data mining processing on the real-time data, and generating a first data mining result; In response to the first data mining result, constructing a business logic rule based on a second operation instruction detected by the application platform; Determine the logical judgment condition corresponding to the business logic rule based on the attribute feature information of the multimodal data, and generate dynamic logic configuration information; Based on the dynamic logic configuration information, a detection event is triggered; in response to the triggered detection event, a second data mining result is generated; the data mining result includes the first data mining result and the second data mining result.

5. The data mining method according to claim 4, characterized in that: The determining of the logical judgment condition corresponding to the business logic rule based on the attribute feature information of the multimodal data and generating dynamic logic configuration information includes: Get the preset alarm rules; detecting a rule adjustment instruction of the application platform; adjusting the alarm rule in response to the received rule adjustment instruction to obtain an alarm triggering condition; The logic judgment condition is determined based on the attribute feature information of the multimodal data, and the dynamic logic configuration information is generated according to the logic judgment condition and the alarm triggering condition.

6. The data mining method according to claim 4, characterized in that: After generating the second data mining result, the method further includes: In response to the second data mining result, acquiring target labeled data via the application platform; Based on the target marking data, an optimization configuration scheme is generated, and based on the optimization configuration scheme, the target algorithm model is adjusted to obtain a new algorithm model.

7. The data mining method according to claim 1, characterized in that: The obtaining of real-time data includes: In the case where the data collection task is a multi-concurrent task, the task information of each concurrent task is sent to the corresponding algorithm service node via the algorithm warehouse management platform, and each algorithm service node is called to execute the concurrent tasks in parallel to obtain the real-time data.

8. The data mining method according to any one of claims 1 to 7, characterized in that: The obtaining of real-time data includes: Get continuous video frames; Setting a frame interval parameter according to preset task requirement information; extracting corresponding key frames from the continuous video frames based on the frame interval parameter; An integrity check is performed on the key frame, and if the integrity check passes, real-time data is acquired based on the key frame.

9. A data mining device, characterized in that: include: An acquisition unit, used to acquire real-time data; An independent card configuration unit is used to classify the registered modular algorithm models through the algorithm warehouse management platform, assemble the modular algorithm models under each category into independent card units, and dynamically send each independent card unit to the application platform for configuration; A workflow generation unit, configured to detect, via the application platform, a first operation instruction for the independent card unit; In response to the detected first operation instruction, determining a target algorithm model in the modular algorithm model, and logically configuring a plurality of the target algorithm models to obtain a target workflow; The mining unit is used to call the target workflow to perform data mining on the real-time data and generate data mining results.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the data mining method according to any one of claims 1 to 8 when running.

Images