A database-in inference system suitable for object detection algorithms

Abstract

The application discloses a database reasoning system suitable for a target detection algorithm.The system comprises a database client and a database server;the database client is used for transmitting input operations and commands of a user, receiving a calling request of a reasoning function of the user and data parameters to be measured;the database server is internally provided with a database with a system table and used for storing model files and reasoning functions of the model files, has a local file system, a model memory for loading the model files and running the model files is created in the memory, and the database client is in communication connection with the database server.The application can significantly simplify the deployment difficulty of a deep learning model, reduce the burden of a developer, lower a use threshold, simultaneously reduce transmission loss of data between a database and other applications, and thus improve running efficiency.

Description

Technical Field

[0001] This invention relates to database processing systems in the fields of databases, deep learning, neural networks, and object detection, and particularly to an in-database inference system and method applicable to object detection algorithms in the field of combining databases and artificial intelligence. Background Technology

[0002] In recent years, the industrial sector has made significant progress in improving the level of production automation. However, quality inspection still relies heavily on manual operation, which leads to low efficiency, inconsistent quality of inspection results, and high labor costs. In particular, the industrial sector urgently needs a technological solution that can automate the detection and location of defects in product surfaces.

[0003] Automating the detection of surface defects in products involves two core issues: first, the deployment of artificial intelligence (AI) models. The high knowledge threshold and complex deployment and management processes pose challenges to achieving automated detection; second, the interaction between collected data and the AI ​​model, as well as the processing of results. This requires retrieving content from a database, sending it to the AI ​​application for inference, obtaining the results, and then performing subsequent operations based on those results. This process not only necessitates repeated data transfers between applications but also requires developers to customize operations based on specific business needs each time, making it cumbersome and inefficient. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes an in-database inference system and method for object detection algorithms. This invention significantly reduces the difficulty of deploying AI models and provides inference and data processing functions directly through SQL, thereby greatly improving the efficiency of automated quality inspection and achieving automation and intelligence in quality inspection.

[0005] The purpose of this invention is to provide a simple and efficient solution for industrial automated quality inspection, as follows:

[0006] The system includes a database client and a database server;

[0007] It includes a database client for transmitting user input and commands, and for receiving user requests to call inference functions and parameters of the data to be tested;

[0008] It includes a database server with built-in system tables for storing model files and their inference functions, a local file system, a model storage in memory for loading and running model files, and a communication connection with the database client.

[0009] The database client is typically a local computer or other device, while the database server is typically a cloud server or other device.

[0010] The database server includes:

[0011] The model management module is used for uploading and updating model files; it stores model files by name based on the file system, creates a system table to manage the query and storage of model information, and provides an interface to the outside world through structured query language, and uses message digest algorithm to verify model files to ensure consistency;

[0012] The model invocation module is used for loading and inference of model files. It creates a global model memory in memory, which mainly uses a dictionary as the underlying structure to store the mapping from model names to models. It also creates data preprocessing functions and data postprocessing functions for data processing.

[0013] The model file refers to the file of a computer big data processing model such as a neural network or BP network used for object detection.

[0014] The model management module creates a system table to store the metadata of model files and stores it in the database.

[0015] The model management module processes data in the following manner:

[0016] First, the user specifies the model name and corresponding model file to be uploaded using Structured Query Language. Then, the database client uses a message digest algorithm to calculate the hash value of the model file as the first hash value. At the same time, the database client uses the large object transfer function to transfer the model file to the database server and obtains the large object identifier of the model file from the database server. The database client then sends the model name, the first hash value, and the large object identifier to the database server.

[0017] The database server locates the stored model file in the database based on the obtained large object identifier, exports the model file to the local file system, and simultaneously calculates the hash value of the exported model file again using the message digest algorithm as a second hash value. It then compares the second hash value with the input first hash value to see if they are the same.

[0018] If they are the same, the relevant information of the model file will be recorded in the created system table, and a response will be sent to the database client indicating that the upload was successful.

[0019] If they are different, a message will be sent indicating that the database client upload failed, and the current database client operation will be rolled back, deleting the model file with the large object identifier.

[0020] The model invocation module integrates the deep learning framework as a third-party library in the database beforehand; the model invocation module creates a global model storage in its own application memory, which uses HashMap (hash table) as the main underlying data structure to manage all loaded model files.

[0021] The model invocation module is processed in the following manner:

[0022] On the database client, the system receives the inference function call request from the user and the test data parameters input by the user, and sends them to the database server.

[0023] On the database server side, upon receiving a call to the inference function, it processes it as follows:

[0024] S1. Search the model storage for a model object corresponding to the model name based on the model name:

[0025] If it exists, directly obtain the model object corresponding to the model name;

[0026] If it does not exist, the system will query the corresponding model file path in the system table based on the model name, calculate the hash value of the model file using the message digest algorithm, and compare it with the second hash value in the system table. If the comparison result is inconsistent, the system will respond to the database client that the model call has failed and end the call.

[0027] S2. Call the deep learning framework, pass in the model file and load the model file, and store the model object in the model storage;

[0028] S3. Based on the user's test data parameters, extract the corresponding row of data from the system table; using this row of data as parameters, call the preprocessing function to process the test data parameters and obtain the preprocessing result;

[0029] If the model file has a registration data preprocessing function, then the obtained row of data is used as a parameter to call the preprocessing function, and then the preprocessing result is obtained.

[0030] The data preprocessing function refers to the processing operations performed before model inference to standardize the data input into the model object, such as image size and color conversion.

[0031] S4. Convert the preprocessing results into vector data types in deep learning and use them as parameters to call the inference function of the model object to process and obtain the inference results;

[0032] If the user inputs a large number of test data parameters, repeat steps S3-S4 above until all test data parameters have been processed.

[0033] S5. After obtaining the inference result, use the inference result as a parameter to call the post-processing function to process and obtain the post-processing result, and then return the final post-processing result to the database client.

[0034] If the model has a registered post-processing function, then all inference results are used as parameters to call the post-processing function for processing, and then the post-processing result is obtained.

[0035] The data post-processing function refers to the operation of extracting, transforming, and filtering the unified inference results after model inference, such as data filtering and format conversion, to meet certain information required to achieve the desired function and requirements.

[0036] The database client is equipped with a detection result extraction module, which is used to extract data from the target detection output results. The extracted data includes the target category, confidence level, and target coordinates.

[0037] A new detection data type is added to the database through the model call module. The detection data type includes three fields: detection category, result confidence, and detection location. At the same time, three data extraction functions are added: category extraction function, confidence extraction function, and location extraction function.

[0038] After receiving the post-processing results from the model calling module on the database server, the detection result extraction module creates the same number of detection data type objects according to the number of post-processing results, and fills the post-processing results into the fields of the corresponding detection data type objects. Then, it calls the data extraction function of the detection data type used for target detection to extract the extraction fields from the detection data type objects, converts them into strings, and outputs them to the database client.

[0039] Furthermore, the detection result extraction module calls the extraction function based on the user's input request to convert the string back into a detection data type object. Then, it calls the corresponding data extraction function based on the extraction function name entered by the user. After the data extraction function extracts the field from the detection object, it calls the corresponding output function built into the database based on the field's corresponding basic type to convert it into a string and output it on the database client. Alternatively, the extracted data can be used as a data source to participate in the next step of database calculation.

[0040] The aforementioned detection data type object specifically refers to a new internal database type created within the database server for the output structure of the target detection algorithm, used to receive the inference results of the target detection algorithm model. This type serves as the object carrier for the model's inference results within the database server.

[0041] This invention, based on database and deep learning technologies, is a method for deploying and executing object detection algorithms within a database. It uses a structured query language as the interaction interface, supports the uploading and deployment of deep learning models, and the database comprehensively manages all models. It also includes a built-in prediction function that, based on the input model name and the internal data to be inferred, finds the corresponding deployment model and data, and then calls the model within the database to perform inference on the data. Furthermore, it provides additional data extraction functions specifically designed to extract specific information from the inference results.

[0042] The innovation of this invention lies in placing the model inference function within the database settings, processing / operation. The model is invoked through database calls, enabling the execution and inference of large model files within the server, resulting in performance improvements, reducing the difficulty of using the model, and allowing the inference results of the target detection algorithm to participate in the next step of database in-process calculations. This facilitates the rapid transformation of database users into intelligent software developers, bringing benefits / advantages to the software development speed of industrial automation quality inspection.

[0043] This invention enables model inference of object detection algorithms within a database and adapts to complex output formats. It utilizes a built-in deep learning framework to provide inference capabilities, employs metadata management technology to track and manage the model, and provides an interactive interface through a structured query language.

[0044] The beneficial effects of this invention are as follows:

[0045] This invention primarily addresses two problems: First, data needs to be exported from a database, sent to a deep learning application for inference, and then the results brought back to the database for subsequent operations—a complex and inefficient process. Second, there are difficulties in deploying and managing deep learning models.

[0046] This invention addresses target detection algorithms applicable to industrial quality inspection. Based on a database and a deep learning framework, it implements internal database reasoning and provides a hybrid query function combining model reasoning and relational algebra. This significantly reduces the threshold for deploying deep learning models and the difficulty of developing related applications, while also improving query efficiency. Attached Figure Description

[0047] Figure 1 This is a modular framework diagram of the present invention.

[0048] Figure 2This is an example diagram of the hybrid query of database inference and relational algebra in this invention.

[0049] Figure 3 This is a timing diagram of the upload model of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0051] The embodiments of the present invention are as follows:

[0052] The specific system implemented includes a database client and a database server;

[0053] like Figure 1 As shown in the diagram, the modular framework of the entire database inference system includes the following:

[0054] The application tool layer contains the interfaces and supporting application tools that provide services to users from the database. It mainly includes the interface module that calls the database server through the program, the module that controls the database server, and the database client module that users can use directly on their computers. All of these modules involve the use of the structured query language module at the underlying level.

[0055] Modify the database's built-in Structured Query Language module, enhance its lexical functionality, and add keywords used in the model management function to enable the module to correctly identify them.

[0056] The core module layer and the underlying capability layer constitute the main content of the database server. In the underlying module, a deep learning framework is compiled and added to enable the database to have model inference capabilities. Secondly, based on the deep learning framework, model management, invocation, and object detection result processing modules are added to the core module. At the same time, existing database system tables are modified, and a model information metadata table is added to support subsequent use. The parsing system is modified, and its syntax parsing and semantic parsing functions are enhanced for model-related modules, so that the input on the database client can correctly match the operations on the database server.

[0057] like Figure 2 As shown, a relational algebra graph is presented that uses in-database reasoning and further performs in-database operations on the reasoning result.

[0058] S1. Extract data from database table R1, input it into the model for inference, and obtain the inference result.

[0059] S2. Extract fields from the inference results based on the extraction function used by the user.

[0060] S3. Merge database tables R2 and R3 into an intermediate table by using an equi-join according to the conditions entered by the user.

[0061] S4. Combine the extracted results and intermediate tables using equi-joins according to the conditions input by the user to generate the final result.

[0062] It includes a database client for transmitting user input and commands, and for receiving user requests to call inference functions and parameters of the data to be tested;

[0063] It includes a database server with built-in system tables for storing model files and their inference functions, a local file system, a model storage in memory for loading and running model files, and a communication connection with the database client.

[0064] The database server includes:

[0065] The model management module is used for uploading, updating, and deleting model files, as well as managing model metadata, providing a foundation for subsequent inference using the model. The model management module stores model files by name based on the file system, uses metadata management technology to create a dedicated system table to manage the query and storage of model information, and provides an interface through structured query language and uses message digest algorithm to verify model files to ensure consistency.

[0066] The model invocation module is used for loading and inference of model files. It creates a global model memory in memory, which mainly uses a dictionary as the underlying structure to store the mapping from model names to models. It also creates data preprocessing functions and data postprocessing functions for data processing.

[0067] Since model loading is time-consuming, this invention stores all loaded models in the model memory of the model calling module, achieving the effect of loading each model only once and improving efficiency. Furthermore, the model memory also stores preprocessing and post-processing functions for the data, allowing users to define and process relevant functions, thus increasing flexibility.

[0068] The model management module creates a system table to store the metadata of model files and stores it in the database.

[0069] The system table includes fields such as model name, model path, creation time, update time, message summary, uploader, and description.

[0070] like Figure 3 As shown, the model management module processes data in the following manner:

[0071] First, the user specifies the name of the model to be uploaded and the corresponding local model file through Structured Query Language. Then, the database client uses the message digest algorithm to calculate the hash value of the model file as the first hash value. At the same time, the database client uses the large object transfer function to transfer the model file to the database server and obtains the large object identifier of the model file from the database server. The database client sends the model name, the obtained first hash value and the large object identifier together to the database server.

[0072] The database server locates the stored large object (model file) in the database based on the obtained large object identifier, exports the model file to the local file system, and places it in the data directory folder of the current database. Simultaneously, the database server uses a message digest algorithm to calculate the hash value of the exported model file again as a second hash value, and compares the second hash value with the input first hash value.

[0073] If they are the same, the relevant information of the model file, including the second hash value, will be recorded in the system table that was created, and a response will be sent to the database client indicating that the upload was successful.

[0074] If they are different, a message will be sent indicating that the database client upload failed, and the current database client operation will be rolled back, deleting the model file with the large object identifier.

[0075] The model invocation module is responsible for loading models and performing model inference, and is the core module for implementing in-database inference. Based on the input model name and the data to be inferred, the model invocation module finds or loads the corresponding model and infers the result.

[0076] The model invocation module integrates the deep learning framework as an underlying third-party library within the database, giving the database the ability to reason. The model invocation module creates a global model storage in its own application memory. The model storage uses a HashMap (hash table) as the main underlying data structure to manage all loaded model objects.

[0077] Beforehand, users register data preprocessing functions for a specified model using an extended approach, and these data preprocessing functions are registered in the model memory.

[0078] Similarly, users can use an extended approach to register data post-processing functions for a specified model.

[0079] Provide a specific inference function within the database, the name of the model to be used, and the data to be inferred as parameters of the function.

[0080] In the model invocation module, it is handled in the following way:

[0081] On the database client, the system receives the inference function call request from the user and the test data parameters input by the user, and sends them to the database server.

[0082] On the database server side, upon receiving a call to the inference function, it processes it as follows:

[0083] S1. Search the model storage for a model object that corresponds to the model name:

[0084] If it exists, directly retrieve the model object corresponding to the model name;

[0085] If it does not exist, the system will query the corresponding model path in the system table based on the model name, use the message digest algorithm to calculate the hash value of the model file, compare it with the second hash value in the system table to ensure that the file has not been modified, and then locate and find the model file through the file system.

[0086] S2. Call the deep learning framework, pass in the model file and load the model file, and store the model file in the model storage.

[0087] S3. Based on the user's test data parameters, extract the corresponding row of data from the system table; using this row of data as parameters, call the preprocessing function to process the test data parameters and obtain the preprocessing result;

[0088] The specific data preprocessing function used is the PreProcessCallback function.

[0089] S4. Convert the preprocessing result into a vector data type in deep learning and use it as a parameter. Call the inference function of the model file to perform the conversion and obtain the inference vector. Then, input the inference vector result into the model file for processing to obtain the inference result.

[0090] The inference function implemented specifically uses the predict function.

[0091] If the user inputs a large number of test data parameters, such as many photos, repeat steps S3-S4 above until all test data parameters have been processed.

[0092] S5. After obtaining the inference result, use the inference result as a parameter to call the post-processing function to process and obtain the post-processing result, and then return the final post-processing result to the database client.

[0093] The specific data post-processing function used is the OutputProcessCallback function.

[0094] The database client includes a detection result extraction module, which is used to extract data from the target detection output. The extracted data includes the target category, confidence level, and target coordinates. The extracted data can be further processed in subsequent relational algebra processing, such as being used as a filtering condition to meet the needs of mixed queries.

[0095] The detection result extraction module can further extract the result data, and the extracted data can be used in the next step of relational algebra operations.

[0096] A new detection data type is added to the database through the model call module. The detection data type includes three fields: detection category, result confidence, and detection location, so as to match the inference results output of the target detection algorithm. At the same time, three data extraction functions are added: category extraction function, confidence extraction function, and location extraction function.

[0097] After receiving the post-processing results from the model calling module on the database server, the detection result extraction module creates the same number of detection data type objects based on the number of post-processing results, and fills the post-processing results into the fields of the corresponding detection data type objects. Then, it calls the data extraction function of the detection data type used for target detection to extract the extraction fields from the detection data type objects, converts them into strings, and outputs them to the database client.

[0098] Furthermore, the detection result extraction module calls the extraction function based on the user's input request to convert the string back into a detection data type object. Then, it calls the corresponding data extraction function based on the extraction function name entered by the user. After the data extraction function extracts the field from the detection object, it calls the corresponding output function built into the database based on the field's corresponding basic type to convert it into a string and output it on the database client. Alternatively, the extracted data can be used as a data source to participate in the next step of database calculation.

[0099] Since these field types are all basic types within the database, they directly meet the requirements of built-in relational algebra operations and can participate in the next step of the calculation.

[0100] The data type detection object specifically adopts the structure pattern, combining the existing basic types in the database server to form a new composite type.

[0101] Thus, the database-in-inference system of this invention significantly simplifies the deployment of deep learning models, reduces the burden on developers, and lowers the barrier to entry. Simultaneously, it reduces data transmission losses between the database and other applications, thereby improving operational efficiency.

Claims

1. A database-based inference system suitable for object detection algorithms, characterized in that: The system includes a database client and a database server; It includes a database client for transmitting user input and commands, and for receiving user requests to call inference functions and parameters of the data to be tested; It includes a database server with a built-in database containing system tables for storing model files and their inference functions, a local file system, a model storage in memory for loading and running model files, and a communication connection with the database client. The database server includes: The model management module is used for uploading and updating model files; it stores model files by name based on the file system, creates a system table to manage the query and storage of model information, and provides an interface to the outside world through structured query language, and uses message digest algorithm to verify model files to ensure consistency; The model invocation module is used for loading and inference of model files. It creates a global model memory in memory, which mainly uses a dictionary as the underlying structure to store the mapping from model names to models. It also creates data preprocessing functions and data postprocessing functions for data processing. The model management module processes data in the following manner: First, the user specifies the model name and corresponding model file to be uploaded using Structured Query Language. Then, the database client uses a message digest algorithm to calculate the hash value of the model file as the first hash value. At the same time, the database client uses the large object transfer function to transfer the model file to the database server and obtains the large object identifier of the model file from the database server. The database client then sends the model name, the first hash value, and the large object identifier to the database server. The database server locates the stored model file in the database based on the obtained large object identifier, exports the model file to the local file system, and simultaneously calculates the hash value of the exported model file again using the message digest algorithm as a second hash value. It then compares the second hash value with the input first hash value to see if they are the same. If they are the same, record the relevant information of the model file in the created system table and send a response to the database client that the upload was successful; A new detection data type is added to the database through the model call module. The detection data type includes three fields: detection category, result confidence, and detection location. At the same time, three data extraction functions are added: category extraction function, confidence extraction function, and location extraction function. After receiving the post-processing results from the model calling module on the database server, the detection result extraction module creates the same number of detection data type objects according to the number of post-processing results, and fills the post-processing results into the fields of the corresponding detection data type objects. Then, it calls the data extraction function of the detection data type used for target detection to extract the extraction fields from the detection data type objects, converts them into strings, and outputs them to the database client. Furthermore, the detection result extraction module calls the extraction function based on the user's input request to convert the string back into a detection data type object. Then, it calls the corresponding data extraction function based on the extraction function name entered by the user. After the data extraction function extracts the field from the detection object, it calls the corresponding output function built into the database based on the field's corresponding basic type to convert it into a string and output it on the database client. Alternatively, the extracted data can be used as a data source to participate in the next step of database calculation.

2. The database inference system for target detection algorithms according to claim 1, characterized in that: The model management module creates a system table to store the metadata of model files and stores it in the database.

3. The database inference system for target detection algorithms according to claim 1, characterized in that: The model invocation module integrates the deep learning framework as a third-party library within the database in advance; the model invocation module creates a global model storage in its own application memory, which uses a hash table (HashMap) as the main underlying data structure to manage all loaded model files.

4. The database inference system for target detection algorithms according to claim 1, characterized in that: The model invocation module is processed in the following manner: On the database client, the system receives the inference function call request from the user and the test data parameters input by the user, and sends them to the database server. On the database server side, upon receiving a call to the inference function, it processes it as follows: S1. Search the model storage for a model object corresponding to the model name based on the model name: If it exists, directly obtain the model object corresponding to the model name; If it does not exist, the system will query the corresponding model file path in the system table based on the model name, calculate the hash value of the model file using the message digest algorithm, and compare it with the second hash value in the system table. If the comparison result is inconsistent, the system will respond to the database client that the model call has failed and end the call. S2. Call the deep learning framework, pass in the model file and load the model file, and store the model object in the model storage; S3. Based on the user's test data parameters, extract the corresponding row of data from the system table; Using a row of data as a parameter, the preprocessing function is called to process the parameters of the data to be tested and obtain the preprocessing result; S4. Convert the preprocessing results into vector data types in deep learning and use them as parameters to call the inference function of the model object to process and obtain the inference results; S5. After obtaining the inference result, use the inference result as a parameter to call the post-processing function to process and obtain the post-processing result, and then return the final post-processing result to the database client.

5. A database inference system suitable for target detection algorithms according to claim 1, characterized in that: The database client is equipped with a detection result extraction module, which is used to extract data from the target detection output results. The extracted data includes the target category, confidence level, and target coordinates.

Images