Data auditing method and system in sandbox environment, electronic device and medium
By clustering and splitting field types of user operation datasets in a sandbox environment, and building a gateway verification program, the problems of excessive computational pressure and low intelligence caused by direct connection between the user system and the backend case library are solved, achieving more efficient and secure data auditing.
Patent Information
- Application Number
- CN202011120714.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-19
- Publication Date
- 2026-02-13
- Estimated Expiration
- 2040-10-19
AI Technical Summary
In existing technologies, when the user system is directly connected to the backend case library, the computational pressure on the backend case library is too high and its intelligence level is low, which can easily lead to crashes.
In the sandbox environment, user operation datasets are obtained from the user system, clustered to obtain a verification dataset and a reference case set, the dataset is split using a pre-trained field type splitting model, a gateway verification program is built to perform field verification, and the case set is indexed from the backend case library according to the case configuration file.
This reduces the computational burden on the backend case library, improves the intelligence and security of data auditing, prevents computational pressure caused by directly analyzing user operation datasets, and ensures the legality and security of data distribution.
Smart Images

Figure CN112182562B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of data processing, in particular to a data auditing method and system in a sandbox environment, an electronic device and a computer readable storage medium. BACKGROUND
[0002] A user system refers to a service system developed in accordance with user usage habits, and a backend case library corresponding to the user system includes a database supporting data interaction of the user system. Currently, the user system and the backend case library generally adopt a direct connection method, that is, when the user system needs to call backend data, the user system directly sends a calling instruction to the backend case library, the backend responds to the calling instruction, and feeds back corresponding data after verifying the security of the user system.
[0003] Although this direct connection method can improve data transmission efficiency, when the backend case library is directly connected with the user system, the backend case library needs to perform security verification and data analysis before returning data to the user system, and the backend case library is not intelligent enough for data analysis, which brings great computing pressure to the backend case library and is prone to cause the backend case library to crash. SUMMARY
[0004] The present application provides a data auditing method and system in a sandbox environment, an electronic device and a computer readable storage medium, which mainly aims to solve the problems of excessive computing pressure and low intelligence during data auditing.
[0005] To achieve the above-mentioned purpose, the present application provides a data auditing method in a sandbox environment, which comprises:
[0006] Obtaining user operation data set from the user system, clustering the user operation data set to obtain verification data set and reference case set;
[0007] Using a pre-trained field type splitting model to split the verification data set to obtain field data set of different field types;
[0008] According to the field data set, a corresponding gateway verification program is constructed, and the corresponding field data set is verified by using the gateway verification program;
[0009] When all field data sets pass the verification, a case configuration file is generated according to the reference case set, and a case set is indexed from a pre-constructed backend case library according to the case configuration file;
[0010] When the case set meets a preset index relationship, the case set is distributed to the user system.
[0011] Optionally, the constructing of the corresponding gateway verification program according to the field data set comprises:
[0012] receiving a script corresponding to the field data set;
[0013] generating a to-be-compiled verification program by using the script and the field data set;
[0014] compiling the to-be-compiled verification program to obtain the gateway verification program.
[0015] Optionally, the generating a to-be-compiled verification program by using the script and the field data set comprises:
[0016] extracting an input parameter from the script;
[0017] splitting the field data set according to the input parameter to obtain a field parameter set;
[0018] combining the field parameter set and the script to generate the to-be-compiled verification program.
[0019] Optionally, the pre-training of the field type splitting model comprises:
[0020] Step A: obtaining a field training data set and a field real label corresponding to the field training data set;
[0021] Step B: inputting the field training data set into the field type splitting model to perform field type extraction to obtain a field predicted label;
[0022] Step C: performing loss value calculation on the field predicted label and the field real label by using a preset loss function to obtain a loss value;
[0023] Step D: when the loss value is greater than or equal to a preset loss threshold, adjusting parameters of the field type splitting model, and returning to Step B to perform field type prediction again;
[0024] Step E: when the loss value is less than the loss threshold, obtaining the field type splitting model.
[0025] Optionally, the field type comprises a string type, an integer type, a long integer type, a double precision type, an array type and a Boolean type.
[0026] Optionally, the clustering of the user operation data set to obtain a verification data set and a reference case set comprises:
[0027] randomly selecting K operation data from the user operation data set to obtain K clusters;
[0028] calculating distances between each user operation data in the user operation data set and each cluster in the K clusters to obtain a distance value set;
[0029] Adjust positions of the K clusters by using the distance value set until each distance value in the distance value set meets a preset threshold value, to obtain an optimal distance value set;
[0030] Divide the user operation data set by using the optimal distance value set, to obtain the verification data set and the reference case set.
[0031] To solve the above problems, the application further provides a sandbox system, which comprises:
[0032] A classification module is configured to receive a user operation data set sent by a user system and cluster the user operation data set to obtain a verification data set and a reference case set;
[0033] A splitting module is configured to split the verification data set into field data sets of different field types;
[0034] A gateway verification module is configured to verify the field data sets;
[0035] A case searching module is configured to, if the verification is passed, generate a corresponding case configuration file according to the reference case set, and index a case set from a backend case library by using the case configuration file;
[0036] An indexing judging module is configured to judge whether the case set meets a preset indexing relationship, and if the case set meets the indexing relationship, distribute the case set to the user system.
[0037] Optionally, the sandbox system further comprises:
[0038] An error reporting module is configured to send an error reporting instruction to a management developer of the sandbox system if the verification is not passed or the case set does not meet the indexing relationship.
[0039] To solve the above problems, the application further provides an electronic device, which comprises:
[0040] at least one processor; and
[0041] a memory connected to the at least one processor in communication; wherein
[0042] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data auditing method in the sandbox environment.
[0043] To solve the above problems, the application further provides a computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the data auditing method in the sandbox environment.
[0044] The embodiment of the present application first acquires a user operation data set from a user system, and sequentially performs clustering on the user operation data set to obtain a verification data set and a reference case set. Compared with traditional manual classification or classification based on machine learning, clustering does not require manual intervention, and the classification is faster. The embodiment of the present application uses a pre-trained field type splitting model to split the verification data set to obtain a field data set. Compared with splitting the field type by using a simple distance judgment formula, the splitting accuracy is higher. In addition, in the embodiment of the present application, the user operation data set is first classified, split, and the like, rather than being directly input into a backend case library, so as to prevent the user system from being directly connected to the backend case library. When the backend case library directly analyzes the user operation data set, the backend case library is subjected to great calculation pressure. At the same time, the present application verifies the legality of the verification data through verification operation, checks whether the case set indexed can be directly sent to the user system through the judgment of the index relationship, and improves the security in the entire data auditing process. Therefore, the data auditing method, system and computer readable storage medium in the sandbox environment provided by the present application can solve the problems of excessive calculation pressure and low intelligentization degree in data auditing. BRIEF DESCRIPTION OF DRAWINGS
[0045] Figure 1 A data interaction diagram of a sandbox system provided by an embodiment of the present application;
[0046] Figure 2 A flowchart of data distribution provided by an embodiment of the present application;
[0047] Figure 3 A flowchart of a data auditing method in a sandbox environment provided by an embodiment of the present application;
[0048] Figure 4 A flowchart of acquiring a user operation data set from a user system in a data auditing method in a sandbox environment provided by an embodiment of the present application;
[0049] Figure 5 An internal structure diagram of an electronic device for implementing a data auditing method in a sandbox environment provided by an embodiment of the present application;
[0050] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION
[0051] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.
[0052] Embodiments of the present application relate to a sandbox environment data auditing method and a sandbox system. The execution subject of the sandbox environment data auditing method provided by the embodiments of the present application includes but is not limited to at least one of the electronic devices such as a server and a terminal which can be configured to execute the method provided by the embodiments of the present application. In other words, the sandbox environment data auditing method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc.
[0053] The core of the present embodiment is to use the constructed sandbox system to obtain a verification data set and a reference case set from an interface, use the verification data set to verify security, and distribute a case set to the user system according to the index relationship of the reference case set, thereby solving the problems of large computing pressure of the back end and low security analysis degree.
[0054] The following will specifically describe the implementation details of the sandbox environment data auditing in the present embodiment, and the following content is only provided for the implementation details for easy understanding, and is not necessary for implementing the present solution.
[0055] As shown in Figure 1 The data interaction process among the sandbox system, the user system and the back-end case library bound to the user system in the embodiments of the present application is shown, which in detail includes:
[0056] 1. The user system sends a user operation data set to the sandbox system.
[0057] 2. The sandbox system performs verification according to the user operation data set, and if the verification fails, the sandbox system re-receives the user operation data set sent by the user system.
[0058] 3. If the verification passes, the sandbox system sends a case configuration file to the back-end case library bound to the user system.
[0059] 4. The back-end case library generates a case set according to the case configuration file and returns the case set to the sandbox system.
[0060] 5. The sandbox system indexes the case set, and if the index passes, the sandbox system sends the case set to the user system.
[0061] The user system includes an API connected to the sandbox system, a database, etc., such as an API of a bank financial product webpage, a database required for model training, etc., such as a user selecting a commodity of the bank financial product webpage and clicking a recommendation button according to the bank financial product webpage, hoping to obtain other recommended commodities corresponding to the commodity from a backend case library corresponding to the API of the bank financial product webpage, wherein the operation data generated by the user selecting the commodity and clicking the recommendation button is the user operation data set;
[0062] The sandbox system is a technical service tool, which can be applied in the field of bank, etc., such as a sandbox system of a bank, which can assist a bank developer to perform interface function joint debugging, help the developer to quickly perform data auditing and distribution. The sandbox system can respond, process and distribute corresponding data according to the user operation data set sent by the user system. Figure 1 As shown in the figure, when the sandbox system receives the user operation data set, if the sandbox system satisfies the verification and the index judgment, the case set is received from the backend case library to the user system.
[0063] In detail, as shown in the figure, Figure 2 The sandbox system includes a classification module, a splitting module, a gateway verification module, a case searching module and an index judgment module. When the sandbox system receives the user operation data set, the data interaction process between the modules includes:
[0064] 1. The classification module receives the user operation data set and classifies the user operation data set into a verification data set and a reference case set.
[0065] Further, the user operation data set obtained from the user system includes: receiving a data sending instruction sent by the user system; opening a data receiving thread according to the data sending instruction; and obtaining the user operation data set according to the data receiving thread.
[0066] Since the method can simultaneously interface with multiple user systems, preferably, the application interfaces with each user system by a multithreading method, which can maximize the data distribution demand of each user system.
[0067] In the embodiment of the application, the classification is different according to different user systems, such as the user operation data set generated by the API of the bank financial product webpage including operation data generated by the user selecting a satisfied financial product and clicking a recommendation button, wherein the financial product is the reference case set, and the webpage operation data generated by selecting the commodity and clicking the recommendation button is the verification data set.
[0068] In another embodiment of the present application, the classification can adopt an unsupervised classification method, such as the currently disclosed K-Means algorithm, and the distance formula corresponding to the unsupervised classification method can adopt a square Euclidean distance calculation method.
[0069] Further, the field types of the verification data set include string type, integer type, long integer type, double precision type, array type, Boolean type, etc., such as the web operation data generated by clicking a recommendation button in the verification data set, wherein the button is a string, and after clicking the button, whether the connection to the background database is successful is a Boolean type.
[0070] Therefore, according to the different field types, the verification data set can be divided into a field data set of string type, a data set of double precision type, etc.
[0071] In a preferred embodiment of the present application, a pre-trained field type splitting model can be used to split the verification data set into field data sets of different field types. The pre-trained field type splitting model can adopt a support vector machine model, a decision tree model, etc.
[0072] 3. The gateway verification module uses a gateway verification program to verify the corresponding field data set respectively, and if the verification fails, the result of the failed verification is fed back to the user system, and the user operation data set of the user system is continuously received;
[0073] In an embodiment of the present application, according to the field data set corresponding to different field types, a corresponding gateway verification program is constructed, and in an embodiment of the present application, according to the field data set corresponding to different field types, a corresponding gateway verification program is constructed, in detail, the generation process of the gateway verification program includes: receiving a script input by a user according to the field data set; generating a to-be-compiled verification program by using the script and the field data set; compiling the to-be-compiled verification program to obtain the gateway verification program.
[0074] Further, the generation of the to-be-compiled verification program by using the script and the field data set includes: extracting the input parameters of each function from the script; splitting the field data set according to the input parameters to obtain a field parameter set; combining the field parameter set to the script to generate the to-be-compiled verification program.
[0075] Specifically, for example, for a string type field data set, a corresponding string parameter length check program is constructed, such as a gateway check program for constructing a button string parameter length not greater than 20, for a long integer field data set, a gateway check program is constructed for taking a value not greater than a threshold value, and the like. In the embodiment of the application, each of the field data sets is taken as an input of a corresponding gateway strong check program, and the gateway strong check program is run to perform a check operation on each of the field data sets.
[0076] If the check fails, it indicates that the check data set generated by the user system is sent abnormally, which may be due to that the user system is maliciously tampered with, the check data set is stolen in the process of being transmitted to the sandbox system, and the like, and therefore the result of the check failure needs to be fed back to the user system and a development and maintenance personnel is notified to detect.
[0077] 4. If the check passes, a case configuration file corresponding to the reference case set is generated according to the reference case set, and the case configuration file is used to index a case set from a back-end case library.
[0078] In the preferred embodiment of the application, the case configuration file includes a storage location of each reference case in the back-end case library, a historical call record of each reference case, a category of each reference case, and an access manner of accessing the back-end case library, and the like. For example, in a family member insurance business transaction, the case configuration file further contains user system request body data information, a gateway call interface name, a trigger field in the back-end case library, and a trigger field value.
[0079] Further, when the database, the sandbox system and the user system are bound, a binding test is performed, in the binding test stage, the developer manually generates a case configuration file, and once the binding test passes, the developer imports a pre-constructed case configuration file generation script to automatically generate the case configuration file.
[0080] For example, a user operates an APP developed by a bank and hopes that the APP can present a current financial product being promoted by the bank, and the sandbox system of the application automatically generates a case configuration file related to the current financial product after the check passes, so as to index a case set from a back-end case library according to the case configuration file.
[0081] 5. The indexing judgment module judges whether the case set satisfies a preset indexing relationship, if the case set does not satisfy the indexing relationship, a result of not satisfying the requirement of indexing is fed back to the user system, and the user operation data set of the user system is continuously received, and if the case set satisfies the indexing relationship, the case set is distributed to the user system.
[0082] In the preferred embodiment of the present application, the index relationship can be preset by the user, such as setting that the number of cases in the case set cannot be greater than a preset number threshold. When the number of cases in the case set is greater than the preset number threshold, it indicates that the number of cases matched with the reference case set is too large, and the case set can be sent to the management personnel of the interface data analysis system for further elimination until the case set meets the requirements and is distributed to the user system.
[0083] In another embodiment of the present application, the sandbox system further includes an error reporting module in addition to the classification module, the splitting module, the gateway verification module, the case searching module and the index judgment module.
[0084] When the gateway verification module fails or the index judgment module fails, the error reporting module sends an error reporting instruction to the management and development personnel of the interface data distribution system to remind them to check and correct.
[0085] Referring to Figure 3 FIG. 1 is a flowchart of a data auditing method in a sandbox environment according to an embodiment of the present application. In this embodiment, the data auditing method in the sandbox environment includes the following steps.
[0086] S1, obtaining a user operation data set from a user system, clustering the user operation data set to obtain a verification data set and a reference case set.
[0087] Further, referring to Figure 4 FIG. 2, the step of obtaining a user operation data set from a user system includes the following steps.
[0088] S11, receiving a data sending instruction sent by the user system;
[0089] S12, opening a data receiving thread according to the data sending instruction;
[0090] S13, obtaining the user operation data set by using the data receiving thread.
[0091] Since the method of the present application can interface with multiple user systems at the same time, preferably, the present application uses a multi-threaded method to interface with each user system to maximize the response to the data distribution needs of each user system.
[0092] In detail, the classification is different according to different user systems. For example, the user operation data set generated by a bank financial product webpage API includes the operation data generated by the user selecting a product and clicking a recommendation button. In this case, the product is the reference case set, and the webpage operation data generated by selecting the product and clicking the recommendation button is the verification data set.
[0093] In another embodiment of the present application, the clustering of the user operation data set to obtain the verification data set and the reference case set comprises: randomly selecting K operation data from the user operation data set to obtain K clusters; calculating the distance between each user operation data in the user operation data set and each cluster in the K clusters to obtain a distance value set; adjusting the position of the K clusters by using the distance value set until each distance value in the distance value set meets a preset threshold to obtain an optimal distance value set, and dividing the user operation data set by using the optimal distance value set to obtain the verification data set and the reference case set.
[0094] In detail, K=2 user operation data are selected from the user operation data set as clusters, and the user operation data set is divided into the verification data set and the reference case set by using the K=2 clusters.
[0095] Further, the calculation formula of the distance value set can adopt the squared Euclidean distance method.
[0096]
[0097] wherein d(x,y) 2 is the distance of the user operation data not selected to the K cluster, x is the coordinate point of the K cluster, y is the coordinate point of the user operation data not selected, and j is the number of coordinate points of the user operation data not selected.
[0098] S2, using the pre-trained field type splitting model to split the verification data set to obtain field data sets of different field types.
[0099] Further, the field types of the verification data set include string type, integer type, long integer type, double precision type, array type, Boolean type, etc. For example, the verification data set includes web operation data generated by clicking a recommended button, wherein the button is a string, and after the button is clicked, whether the database in the background is connected successfully is a Boolean type.
[0100] Therefore, according to the different field types, the verification data set can be divided into a field data set of string type and a data set of double precision type.
[0101] In the preferred embodiment of the present application, the pre-trained field type splitting model can be used to split the verification data set into field data sets of different field types. In detail, the pre-training of the field type splitting model comprises:
[0102] Step A: obtaining a field training data set and a field true label corresponding to the field training data set;
[0103] Step B: inputting the field training data set into the field type splitting model for field type extraction to obtain a field prediction label;
[0104] Step C: calculating a loss value of the field prediction label and the field real label by using a preset loss function to obtain the loss value;
[0105] Step D: when the loss value is greater than or equal to a preset loss threshold, adjusting parameters of the field type splitting model, and returning to Step B to re-perform field type prediction;
[0106] Step E: when the loss value is less than the loss threshold, obtaining the field type splitting model.
[0107] The calculation method of the loss value is as follows:
[0108]
[0109] wherein, Y is the field real label, and a represents an error factor, and is a preset constant.
[0110] In the embodiment of the present application, the field type splitting model is trained to extract the field type of the verification data, and the field type splitting model can identify different field types of data in the verification data set, thereby improving the analysis efficiency, and the model training can improve the accuracy of data extraction, and avoid errors when manually extracting data.
[0111] Preferably, the gradient descent algorithm can be used to adjust the parameters of the field type extraction model, and the gradient descent algorithm is the most commonly used parameter adjustment method in machine learning, and the loss value obtained from the preset field training data set, the field real label and the field prediction label can be used to calculate the parameters of the field type extraction model. In the embodiment of the present application, the gradient descent algorithm includes a stochastic gradient descent algorithm, a batch gradient descent algorithm, a small batch gradient descent algorithm and the like.
[0112] S3, according to the field data set, a corresponding gateway verification program is constructed, and the corresponding gateway verification program is used to verify the corresponding field data set respectively.
[0113] In the embodiment of the present application, according to the field data set corresponding to different field types, a corresponding gateway verification program is constructed, in detail, the S3 includes: receiving a script input by a user according to the field data set; generating a to-be-compiled verification program by using the script and the field data set; and compiling the to-be-compiled verification program to obtain the gateway verification program.
[0114] Further, the generating the to-be-compiled verification program by using the script and the field data set comprises: extracting input parameters of each function from the script; splitting the field data set according to the input parameters to obtain a field parameter set; and combining the field parameter set to the script to generate the to-be-compiled verification program.
[0115] For example, for a field data set of a string type, a corresponding string parameter length verification program is constructed, such as a gateway verification program for constructing a button string parameter length not greater than 20, for a field data set of a long integer type, a gateway verification program for constructing a value not greater than a threshold value, and the like. In the embodiment of the application, each of the field data sets is taken as an input of a corresponding gateway strong verification program, and the gateway strong verification program is run to perform a verification operation on each of the field data sets.
[0116] If the verification fails, it indicates that a verification data set generated by a user system is sent abnormally, which may be due to that the user system is maliciously tampered with, the verification data set is stolen in the process of being transmitted to the sandbox system, and the like, and therefore, a result of the verification failing needs to be fed back to the user system, and a development and maintenance personnel is notified to detect.
[0117] S4, judging whether the verification passes or not, if the verification fails, re-receiving a user operation data set.
[0118] S5, if the verification passes, generating a corresponding case configuration file according to the reference case set, and indexing a case set from a back-end case library according to the case configuration file.
[0119] In the preferred embodiment of the application, the case configuration file comprises a storage location of each reference case in the back-end case library, a historical calling record of each reference case, a category of each reference case, and an access manner of accessing the back-end case library, and the like. For example, in a family member insurance business transaction, the case configuration file further comprises user system request body data information, a gateway calling interface name, a trigger field in the back-end case library, and a trigger field value.
[0120] Further, when the back-end case library, the sandbox system and the user system are bound, a binding test is performed by a developer, in the binding test stage, the developer manually generates a case configuration file, and once the binding test passes, the developer imports a pre-constructed case configuration file generation script to automatically generate the case configuration file.
[0121] For example, a user operates an APP developed by a bank and hopes that the APP can present a current financial product being promoted by the bank, and the sandbox system in the application generates a case configuration file related to the current financial product after the verification passes, so as to index a case set from a back-end case library according to the case configuration file.
[0122] S6, judging whether the case set meets a preset index relationship, if the case set does not meet the index relationship, re-receiving a user operation data set.
[0123] S7, if the case set meets the index relationship, distributing the case set to the user system.
[0124] In the preferred embodiment of the present application, the index relationship can be preset by the user, for example, the number of cases in the case set cannot be greater than a preset number threshold. When the number of cases in the case set is greater than the preset number threshold, it means that the number of cases matched with the reference case set is too large, and the case set can be sent to the management personnel of the interface data analysis system for further elimination until the case set meets the requirements and is distributed to the user system.
[0125] In the embodiment of the present application, the user operation data set is first obtained from the user system, and the user operation data set is sequentially classified and split to obtain a field data set and a reference case set. The corresponding gateway verification program is constructed according to the field data set for verification. If the verification is passed, the case configuration file is generated through the reference case set, the case set is indexed from the back-end case library according to the case configuration file, and if the case set meets the requirements of the index relationship, the case set is sent to the user system. It can be seen that in the implementation steps of the present application, classification, splitting and indexing operations are added in the user system and the back-end case library to prevent the user system from being directly connected to the back-end case library. When the back-end case library directly analyzes the user operation data set, it causes great calculation pressure to the back-end case library. At the same time, the present application verifies the legality of the verification data through the verification operation, checks whether the indexed case set can be directly sent to the user system through the judgment of the index relationship, and improves the security in the entire data distribution process. Therefore, the data auditing method, system and computer readable storage medium in the sandbox environment proposed by the present application can solve the problems of too large calculation pressure and low distribution security during data distribution.
[0126] As shown in FIG. 1, it is a structural schematic diagram of an electronic device for realizing the data auditing method in the sandbox environment. Figure 5
[0127] The electronic device 1 can include a processor 10, a memory 11 and a bus, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a data auditing method program 12 in the sandbox environment.
[0128] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used to store application software and various data installed in the electronic device 1, such as the code of the data auditing method program 12 in the sandbox environment, and can also be used to temporarily store data that has been output or will be output.
[0129] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 10 is the control unit of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as the data auditing method program in the sandbox environment), and calls data stored in the memory 11, to execute various functions and process data of the electronic device 1.
[0130] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.
[0131] Figure 5 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 5The illustrated structure does not constitute a limitation on the electronic device 1, and can include fewer or more components than illustrated, or combine certain components, or arrange different components.
[0132] For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.
[0133] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.
[0134] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visual user interface.
[0135] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.
[0136] The data auditing method program 12 stored in the memory 11 in the electronic device 1 in a sandbox environment is a combination of multiple instructions, which, when executed in the processor 10, can implement:
[0137] Obtain a user operation data set from a user system, classify the user operation data set to obtain a verification data set and a reference case set;
[0138] According to different field types in the verification data set, the verification data set is split to obtain field data sets of different field types;
[0139] According to the field data set, a corresponding gateway verification program is constructed respectively, and the corresponding field data set is verified by using the gateway verification program respectively;
[0140] When all the field data sets pass the verification, a case configuration file is generated according to the reference case set, and a case set is indexed from a pre-constructed backend case library according to the case configuration file;
[0141] When the case set meets a preset index relationship, the case set is distributed to the user system.
[0142] Further, the modules / units integrated in the electronic device 1 are stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).
[0143] Further, the computer readable storage medium can mainly include a storage program area and a storage data area, wherein the storage program area can store an operating system, at least one application required by a function, etc.; and the storage data area can store data created according to the use of the blockchain node, etc.
[0144] In several embodiments provided by the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the above-described system embodiments are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, another division manner can be adopted.
[0145] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. According to actual needs, some or all of the modules can be selected to achieve the purpose of the embodiment.
[0146] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function module.
[0147] It will be obvious to a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments but can be implemented in other embodiments without departing from the scope of the application.
[0148] The embodiments should therefore be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the description given above, so that all changes coming within the meaning and equivalency range of the claims are intended to be embraced therein. Any annexed drawings are not to be considered limiting as to the claims.
[0149] Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A single processor or other unit can fulfil the functions of several units or entities recited in the above description. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures can not be used to advantage.
[0150] Finally, it should be noted that the above-described embodiments are merely intended to illustrate the technical solutions of the present application, rather than limit the scope of the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalent replaced without departing from the spirit and scope of the present application.
Claims
1. A data auditing method in a sandbox environment, characterized in that, The method includes: The user operation dataset is obtained from the user system. The user operation dataset is clustered to obtain a verification dataset and a reference case set. The verification dataset is the behavioral operation data in the user operation dataset, and the reference case set is the operation object corresponding to the behavioral operation data. The validation dataset is split using a pre-trained field type splitting model to obtain field datasets of different field types. Based on the field dataset, construct corresponding gateway verification programs and use the gateway verification programs to verify the corresponding field datasets. When all field datasets pass the validation, a case configuration file is generated based on the reference case set. The case configuration file includes the storage location, historical call records, category, and access method of each reference case in the reference case set in the backend case library. Based on the case configuration file, the case set is indexed from the pre-built backend case library. When the case set satisfies a preset index relationship, the case set is distributed to the user system. The index relationship is that the number of cases in the case set is not greater than a preset threshold. If the case set does not satisfy the index relationship, the case set is removed until the case set satisfies the index relationship, and then the case set is distributed to the user system.
2. The data auditing method in a sandbox environment as described in claim 1, characterized in that, The step of constructing corresponding gateway verification programs based on the field dataset includes: Receive the script corresponding to the field dataset; Using the script and the field dataset, generate a verification program to be compiled; The verification program to be compiled is compiled to obtain the gateway verification program.
3. The data auditing method in a sandbox environment as described in claim 2, characterized in that, The step of generating a verification program to be compiled using the script and the field dataset includes: Extract input parameters from the script; The field dataset is split according to the input parameters to obtain the field parameter set; The set of field parameters is combined with the script to generate the verification program to be compiled.
4. The data auditing method in a sandbox environment as described in claim 1, characterized in that, The pre-training of the field type splitting model includes: Step A: Obtain the field training dataset and the corresponding field real labels; Step B: Input the field training dataset into the field type splitting model to extract the field type and obtain the field prediction label; Step C: Calculate the loss value by comparing the predicted label and the true label of the field using a preset loss function; Step D: When the loss value is greater than or equal to the preset loss threshold, adjust the parameters of the field type splitting model and return to step B to re-predict the field type. Step E: When the loss value is less than the loss threshold, the field type splitting model is obtained.
5. The data auditing method in a sandbox environment as described in claim 1, characterized in that, The field types include string, integer, long integer, double, array, and boolean.
6. The data auditing method in a sandbox environment as described in claim 1, characterized in that, The process of clustering the user operation dataset to obtain a verification dataset and a reference case set includes: K operation data points are randomly selected from the user operation dataset to obtain K clusters; Calculate the distance between each user operation data in the user operation dataset and each of the K clusters to obtain a set of distance values; The position of the K cluster is adjusted using the distance value set until each distance value in the distance value set meets a preset threshold, thus obtaining the optimal distance value set; The user operation dataset is divided using the optimal distance value set to obtain the verification dataset and the reference case set.
7. A sandbox system, characterized in that, The system includes: The classification module is used to receive user operation datasets sent by the user system, and cluster the user operation datasets to obtain a verification dataset and a reference case set. The verification dataset is the behavioral operation data in the user operation dataset, and the reference case set is the operation object corresponding to the behavioral operation data. The splitting module is used to split the validation dataset into field datasets of different field types; The gateway verification module is used to verify the field dataset; The case search module is used to generate a corresponding case configuration file based on the reference case set if the verification passes. The case configuration file includes the storage location, historical call records, category, and access method of each reference case in the reference case set in the backend case library. The case configuration file is used to index the case set from the backend case library. An index determination module is used to determine whether the case set satisfies a preset index relationship. If the case set satisfies the index relationship, the case set is distributed to the user system. The index relationship is that the number of cases in the case set is not greater than a preset number threshold. If the case set does not satisfy the index relationship, the case set is removed until the case set satisfies the index relationship, and then the case set is distributed to the user system.
8. A sandbox system as described in claim 7, characterized in that, The system also includes: The error reporting module is used to send an error reporting instruction to the management and development personnel of the sandbox system when the verification fails or the case set does not satisfy the index relationship.
9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the data auditing method in a sandbox environment as described in any one of claims 1 to 6.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the data auditing method in a sandbox environment as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Data quality checking method and device, storage medium and electronic device
CN108647358A
Field label obtaining method and device, storage medium and electronic device
CN110442568A