An industrial intelligent vision algorithm cloud service system and method

Abstract

The present invention discloses an industrial intelligent vision algorithm cloud service system and method, the system includes an industrial intelligent vision infrastructure layer, an industrial intelligent vision algorithm support layer and an industrial intelligent vision algorithm application layer; the industrial vision infrastructure layer is used to provide the basic capabilities and guarantees for model operation, algorithm storage and data processing in the industrial intelligent vision algorithm support layer and the industrial intelligent vision algorithm application layer; the industrial vision algorithm support layer is constructed to provide an algorithm basis for the industrial intelligent vision algorithm application layer; the industrial intelligent vision algorithm application layer is used to select appropriate data processing algorithms and industrial vision models based on the service requirements of application scenario data processing obtained from users, and generate solutions for application scenarios for users to download and use. By constructing the industrial vision algorithm support layer, integrating various machine learning and deep learning models, as well as different visual algorithm models, it is effectively guaranteed that the system can meet various visual business requirements.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and specifically belongs to an industrial intelligent vision algorithm cloud service system. Background Art

[0002] With the continuous maturity of technologies such as artificial intelligence and computer vision, the industry continues to develop. This has also greatly promoted the development of the industrial intelligent vision industry. Industrial intelligent vision refers to the use of visual equipment to convert the target into a digital signal based on pixel distribution, brightness, color and other information, and transmit it to a dedicated image processing system. The image processing system performs various operations on these signals to extract the characteristics of the target, and then controls the equipment action on site based on the results of the judgment. Intelligent vision is widely used in industrial production and is often used in four types of business applications throughout the entire production process: visual guidance and positioning, pattern recognition and detection, precise measurement and ranging, product appearance detection, etc. In general, the characteristics of industrial intelligent vision systems are to improve the degree of automation of production. Mainly in some dangerous working environments that are not suitable for manual operations or occasions where artificial vision is difficult to meet the requirements, intelligent vision is often used to replace artificial vision; in the process of mass industrial production, the efficiency and accuracy of using artificial vision to check product quality are low, and the use of intelligent vision detection methods greatly improves production efficiency and the degree of automation of production.

[0003] As the application of industrial intelligent vision becomes more and more extensive and the objects to be inspected become more and more complex, the application of industrial intelligent vision is transitioning from traditional industrial intelligent vision to AI industrial intelligent vision based on deep learning. The application of traditional industrial intelligent vision system is to process the digital images taken in the target perception area under the specific environment of given background, light source and acquisition optical equipment parameters, extract specific information data, and output it to the designated device as the basis for action. It relies on pre-defined fixed features. The vision engineer defines the target features and the threshold definition of numerical judgment based on the specific needs of the visual task. The limitation of this simple logic cannot be applied to work tasks with strong randomness and complex features, such as complex appearance defect detection that occurs randomly. Therefore, the industry is increasingly using deep learning that can solve such complex feature problems.

[0004] As an extension of traditional neural networks, deep learning methods have made great progress in semantic cognition problems such as speech, images, and natural language in recent years, providing a general framework for solving the problem of representation and understanding of visual big data. The content of images and videos is complex, including diverse scenes and a wide variety of objects. Under uncontrolled conditions, the content of images and videos varies greatly due to the influence of lighting, posture, occlusion, etc. The amount of image and video data is large and the feature dimension is high. Some applications require real-time processing, and the rapid development of deep learning methods provides an effective way to solve the above problems. The current general technical level of deep learning algorithms in the industry has been able to achieve a judgment accuracy of more than 95%. By balancing the missed judgment rate and the false judgment rate, the missed judgment rate can be more strictly controlled to reduce the missed judgment rate to below 100PPM and the false judgment rate to below 5%.

[0005] Although AI industrial intelligent vision based on deep learning can solve more complex problems, there are still the following bottlenecks that restrict faster development.

[0006] First, the algorithm development cost is high and the general commonality and reusability are poor. The problems encountered by various industrial enterprises in the actual production environment are complicated, and the labor and time cost of developing new algorithms for the problems they encounter is high. Moreover, the problems encountered by different enterprises may have the same commonality, and the required underlying algorithms are universal. Spending a lot of energy on developing each problem separately is time-consuming and labor-intensive, and the effect is often not good. Therefore, intelligent vision is insufficient in terms of versatility, and cannot be carried in some integrated applications. A device may only be suitable for one manufacturer or one industry, resulting in excessively high development costs. In terms of intelligence, when the inventory unit is high or the moving speed is fast, it is difficult for industrial intelligent vision to meet the requirements in terms of recognition accuracy. For example, in the field of sorting, industrial intelligent vision has been able to adapt well to small-scale and fixed sorting with fewer categories, but for the recognition and sorting of complex stacked objects, more than 90% are still completed manually; there is a lack of experience in algorithm application.

[0007] Second, there are more and more computer algorithms, some of which are suitable for classification scenarios, some for detection scenarios, and some require key technologies to be mastered according to specific scenarios, so they cannot adapt well to application scenarios with different demands. It is particularly difficult to confirm the most suitable and practical algorithms for different scenarios. Moreover, after obtaining the algorithm, how to accurately and effectively deploy the application algorithm and select the most appropriate parameters is also a major difficulty. Moreover, due to the rapid development of data science and machine learning, various intelligent vision application scenarios are increasing and machine learning algorithms are frequently replaced. Advanced algorithms from a year ago may be outdated a year later. How to facilitate industrial enterprises to adjust application models in a timely manner according to the latest scientific research findings, obtain better model effects, combine production and research, enable enterprises to use the most advanced algorithms, and ensure the advancement and timeliness of the models is still difficult. Summary of the invention

[0008] In order to solve the problems existing in the prior art, the present invention provides an industrial intelligent vision algorithm cloud service system and method for solving the above problems.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] An industrial intelligent vision algorithm cloud service system, comprising an industrial intelligent vision infrastructure layer, an industrial intelligent vision algorithm support layer and an industrial intelligent vision algorithm application layer;

[0011] The industrial vision infrastructure layer is used to provide the basic capabilities and guarantees for model operation, algorithm storage and data processing in the industrial intelligent vision algorithm support layer and the industrial intelligent vision algorithm application layer;

[0012] The construction of the industrial vision algorithm support layer is used to provide an algorithm basis for the industrial intelligent vision algorithm application layer;

[0013] The industrial intelligent vision algorithm application layer is used to select appropriate data processing algorithms and industrial vision models based on the service requirements of application scenario data processing obtained from users, and generate solutions for application scenarios for users to download and use.

[0014] Preferably, the industrial vision infrastructure layer includes an operating environment, a storage environment and a network resource library;

[0015] The operating environment is divided into a hardware operating environment and a software operating environment. The hardware operating environment includes a central processing unit and a graphics processing unit; the software operating environment includes an algorithm development and operating environment based on Java and Python languages;

[0016] The storage environment is a MySQL database and a MongoDB database; the MySQL database is used to store algorithm attribute information and industrial enterprise user information; the MongoDB database is used to store detailed documentation of each algorithm, including algorithm principles, algorithm application scenarios, training and testing documents, deployment plans, and pre-trained models;

[0017] The network resource library is used to connect to the network database and provide document retrieval.

[0018] Preferably, the industrial vision algorithm support layer includes a visual data processing algorithm library and an industrial intelligent vision algorithm model library; the visual data processing algorithm library includes algorithms for feature extraction of input images and processing before model training; the industrial intelligent vision algorithm model library includes a theoretical basis vision algorithm library and an industrial application vision algorithm library.

[0019] Preferably, the application scenario solution includes a detailed algorithm introduction document of the recommended algorithm model, a model training and testing document, a pre-trained model and a deployment document.

[0020] Preferably, the industrial intelligent vision algorithm application layer list displays all algorithms available for use, allowing users to select and download them.

[0021] Preferably, the industrial intelligent vision algorithm application layer independently recommends the algorithm required for each step based on the visual data mining steps in industrial applications, and then integrates the decisions of all steps to obtain several better overall solutions for recommendation to users.

[0022] An industrial intelligent visual algorithm cloud service method comprises the following steps:

[0023] Step 1: Obtain the data set collected by the user in his application scenario;

[0024] Step 2: Extracting manual features of the dataset based on the dataset uploaded by the user;

[0025] Step 3: Input the extracted data set features into the intelligent visual algorithm recommendation model to recommend the industrial visual preprocessing algorithm and visual algorithm model required to process the data set to the user, and generate an application scenario solution;

[0026] Step 4: Provide users with application scenario solutions, and provide users with solutions for downloading and using through the application's long interactive interface;

[0027] Steps 1 to 4 are used by the industrial vision infrastructure layer to provide basic capabilities for operation, storage, and processing and basic operation guarantees.

[0028] Preferably, in step 3, performing visual algorithm recommendation specifically includes the following steps:

[0029] Step 3.1, feature extraction of user uploaded dataset;

[0030] Extract manual features of the dataset based on the dataset uploaded by the user;

[0031] Step 3.2, data preprocessing algorithm recommendation;

[0032] According to the features of the data set extracted in step 3.1, select a suitable data preprocessing algorithm for the user's application scenario, and preprocess the data provided by the user in step 3.1 for subsequent machine learning model training;

[0033] Step 3.3, visual algorithm model recommendation;

[0034] According to the dataset features extracted in step 3.1, select the appropriate visual algorithm model for the user's application scenario and provide an algorithm that effectively meets the user's visual application scenario requirements;

[0035] Step 3.4, solution generation;

[0036] Based on the recommendation results of steps 3.2 and 3.3, generate a personalized solution suitable for data set processing for the user's visual application scenario.

[0037] Furthermore, in step 3.1, the manual features of the data set include the industrial application scenario deployment location, scenario category, data type, positive-negative sample ratio, positive sample data volume, negative sample data volume, and whether negative samples are included.

[0038] Furthermore, in step 3.2, the preprocessing process includes unifying the data format, reducing data noise and simplifying the data.

[0039] Compared with the prior art, the present invention has the following beneficial technical effects:

[0040] The present invention provides an industrial intelligent visual algorithm cloud service system. By building an industrial visual algorithm support layer, integrating various advanced machine learning and deep learning models, as well as visual algorithm models under different application scenarios, a visual data processing algorithm library and an industrial visual algorithm model library are constructed to effectively ensure that the system can meet various visual business needs and improve the reusability of industrial visual algorithms. By building an industrial visual algorithm application layer, an intelligent recommendation method for visual algorithms is proposed. Combined with the personalized needs of industrial enterprise users, machine learning methods are used to automatically find and recommend suitable industrial visual algorithms, thereby improving the user experience of the system. Based on the difficulties of traditional industrial intelligent visual applications, and focusing on the transformation and upgrading of traditional industries and the development needs of advanced manufacturing industries, an industrial intelligent visual algorithm cloud service system is proposed to provide the manufacturing industry with full-process technical services from data processing, model development to decision-making assistance, breaking the technical barriers of the industry's use of visual algorithms and artificial intelligence technologies, promoting the deep integration of cross-industry technologies, and creating a manufacturing value chain of intelligent services, intelligent applications, and intelligent innovations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a framework diagram of an industrial intelligent vision algorithm cloud service system.

[0042] Figure 2 This is an example of a classification method for the industrial vision algorithm model library provided by the present invention.

[0043] Figure 3 This is the intelligent recommendation model framework for industrial intelligent visual algorithms based on machine learning proposed in this invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0045] An industrial intelligent vision algorithm cloud service system, comprising an industrial intelligent vision infrastructure layer, an industrial intelligent vision algorithm support layer and an industrial intelligent vision algorithm application layer;

[0046] The industrial vision infrastructure layer is used to provide the basic capabilities and basic operation guarantees for the operation, storage, and processing of the industrial intelligent vision algorithm support layer and the industrial intelligent vision algorithm application layer;

[0047] The construction of the industrial vision algorithm support layer is used to provide an algorithm basis for the industrial intelligent vision algorithm application layer;

[0048] The industrial intelligent vision algorithm application layer is used to select appropriate vision algorithms based on the acquired data requirements and generate application scenario solutions for download and use.

[0049] An industrial intelligent visual algorithm cloud service method of the present invention comprises the following steps:

[0050] Step 1: Obtain the data set collected by industrial enterprise users in their application scenarios through the interactive interface of the industrial intelligent vision algorithm application layer;

[0051] Step 2: The industrial vision algorithm application layer extracts manual features of the data set based on the data set uploaded by the industrial enterprise users;

[0052] Step 3: Input the data set features into the industrial intelligent algorithm recommendation model in the industrial vision algorithm application layer, and select and recommend data preprocessing algorithms and industrial intelligent vision algorithm models suitable for their application scenarios from the visual data processing algorithm library and industrial vision algorithm model library in the industrial vision algorithm support layer;

[0053] Step 4: Based on the recommended data preprocessing algorithm and industrial intelligent vision algorithm model, the industrial vision algorithm application layer provides industrial enterprise users with a complete, clear, and accurate description of its application scenario solution, including the introduction document of the recommended algorithm model, model training and testing documents, pre-trained models, and deployment documents, and provides users with solution download and use through the application layer interactive interface;

[0054] Steps 1 to 4 are used by the industrial vision infrastructure layer to provide basic capabilities for operation, storage, and processing and basic operation guarantees.

[0055] A method for constructing an industrial intelligent visual algorithm cloud service system comprises the following steps:

[0056] Step 1: Construction of industrial visual infrastructure layer;

[0057] The industrial vision infrastructure layer is the foundation for building an industrial intelligent vision algorithm cloud service system, providing the basic capabilities and basic operation guarantees for the entire system in terms of operation, storage, and processing. Specifically, the infrastructure layer includes the operating environment, storage environment, and network resource library.

[0058] The operating environment is divided into hardware operating environment and software operating environment. The hardware operating environment includes the central processing unit and the graphics processing unit; the software operating environment of this system includes the algorithm development and operating environment based on the Java language and the Python language. The storage environment uses the database to store data efficiently and in an organized manner, enabling the system to manage data faster and more conveniently. This system builds a MySQL database to store the category information and user information of the algorithm. This system builds a MongoDB database to store the detailed document introduction of each algorithm, including the algorithm principle, algorithm application scenarios, training and testing documents, deployment plans, pre-trained models, etc. The network resource library integration can access more than 70 Chinese and foreign network databases and massive electronic journals, providing excellent and convenient literature retrieval conditions.

[0059] Step 2: Construction of industrial vision algorithm support layer;

[0060] The industrial vision algorithm support layer provides the algorithm resource library and its update and iteration modules of the industrial intelligent vision algorithm cloud service system. The algorithm resource library provides an introduction to industrial vision algorithms, model training and testing steps, pre-trained model storage and download introduction, and usage plan introduction. Among them, the algorithm introduction provides a detailed introduction to each algorithm, including algorithm classification, algorithm principle, algorithm application scenario, etc.; the model training test records in detail the design of each algorithm training and test experiment, training and testing methods and processes, training and test results, etc.; the pre-trained model is to make the original manual parameter adjustment and reliance on machine learning experts enter the stage of large-scale, replicable large-scale industrial application. It provides storage of different trained models for the project, and users can directly download and deploy them; the usage plan introduction refers to an accurate and complete description of the use of each algorithm, providing a series of clear instructions for the use of the algorithm.

[0061] According to the steps of visual data mining and application, the algorithm resource library is divided into a visual data processing algorithm library and an industrial visual algorithm model library. Among them, the visual data processing algorithm library contains algorithms related to feature extraction of input images and processing before model training. The industrial visual algorithm model library contains a theoretical basis visual algorithm library and an industrial application visual algorithm library.

[0062] According to the different attributes of the data set, the theoretical foundation visual algorithm library includes the image theoretical foundation visual algorithm library and the video theoretical foundation visual algorithm library. The visual algorithm model for the image theoretical foundation visual algorithm library specifically includes algorithms for image classification, image detection and image segmentation according to the three directions of image analysis; among them, the image classification algorithm is an image processing method that distinguishes different categories of targets according to the different features reflected in the image information. It uses computers to quantitatively analyze images and classify images or each pixel or area in the image into one of several categories to replace human visual interpretation; the image detection algorithm reads local image data as input, performs target detection on objects in the image, obtains the confidence of the detected object category and the frame of the detected target, and finally displays the detection results. The image segmentation algorithm divides the image into several specific areas with unique properties and proposes the technology and process of the target of interest. For the video theoretical foundation visual algorithm library, considering the wide range of video detection needs in real application scenarios, the system includes video detection algorithms in the field of intelligent vision.

[0063] For the industrial application vision algorithm library, it includes algorithms related to small sample learning, single-class sample learning, and lightweight learning. Among them, the small sample learning algorithm is suitable for solving the situation where the real application scenario only contains a small number of training samples; the single-class sample learning algorithm is suitable for solving the problem that some tasks in real industrial scenarios can only collect a large number of normal samples but no negative samples; the lightweight learning algorithm provides a compressed model, so that the network carries fewer network parameters, thereby reducing memory requirements and providing faster reasoning speed.

[0064] The algorithm resource library update and iteration module uses manual updates to continuously investigate advanced visual algorithms from top academic conferences and journals, and updates the algorithms to the algorithm resource library.

[0065] Step 3: Construction of industrial vision algorithm application layer;

[0066] The industrial vision algorithm application layer provides users with two vision algorithm recommendation functions, allowing users to select industrial vision algorithms suitable for their application scenarios. The first vision algorithm recommendation function provides users with a list of all algorithms available for their application scenarios for users to select and download; the second vision algorithm intelligent recommendation function provides users with a vision algorithm intelligent recommendation method based on machine learning algorithms, which uses data sets with different attributes in various application scenarios as training data and a recommendation model trained with the performance of different candidate machine learning models on the data sets as labels. When a user provides a new data set, this recommendation method recommends an intelligent vision algorithm model with good performance on the data set. The following introduces the implementation details of the two recommendation methods.

[0067] The core idea of ​​the first visual algorithm recommendation method is to display the algorithms integrated in the system's industrial visual algorithm support layer to users through the front-end page, so that users can directly select and download them. This method is suitable for industrial enterprises with clear requirements and experience in industrial visual algorithm model development, and can be used by directly downloading the required model algorithms and deployment documents.

[0068] The core idea of ​​the second visual algorithm intelligent recommendation method is to use the principle of divide and conquer. According to the visual data mining steps in industrial applications, the algorithm required for each step is independently recommended, and then the decisions of all steps are integrated to obtain several better overall solutions for users to recommend. Specifically, it is divided into four steps: data set feature extraction, preprocessing algorithm recommendation, visual algorithm model recommendation, and solution generation.

[0069] Step 3.1, data set feature extraction step;

[0070] According to the data set collected in the application scenario uploaded by the industrial enterprise user, the manual features of the data set are designed and extracted. Manual features are artificially designed features, generally designed based on experience. The data set features extracted by the present invention include the industrial application scenario deployment location, scenario category, data type, positive and negative sample ratio, positive sample data volume, negative sample data volume, and whether negative samples are included.

[0071] Step 3.2, preprocessing algorithm recommendation;

[0072] The preprocessing algorithm is a process performed on the input data set before feature extraction and model training. Its main purpose is to eliminate irrelevant information in the data, enhance the detectability of relevant information and simplify the data to the maximum extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition. According to the specific application scenario data set, the use of appropriate data preprocessing algorithms is crucial to the operation effect of the visual algorithm model. The preprocessing algorithm of the present invention recommends selecting appropriate visual data preprocessing algorithms for different data sets based on the visual data processing algorithm library in the industrial intelligent vision support layer, such as image scaling, normalization, etc.

[0073] Step 3.3, visual algorithm model recommendation;

[0074] The visual algorithm model recommendation method recommends a visual algorithm model with good performance on a data set uploaded by an industrial enterprise user. The visual algorithm recommendation function in the present invention is provided by a machine learning model. When an industrial enterprise user uploads a new data set to a cloud service system, the application layer extracts the manual features of the data set and inputs them into the visual algorithm recommendation model trained in the cloud service system, and outputs a visual algorithm model suitable for the data set.

[0075] Step 3.4, solution generation;

[0076] The solution generation function recommends solutions suitable for industrial enterprise user data sets to users based on the results of the previous steps. Specifically, this module directly provides several optimal algorithm models for visual algorithm preprocessing and visual algorithm models. Not only that, it also includes detailed algorithm introduction documents, model training and testing documents, pre-trained models, and deployment documents for the recommended algorithm models. The algorithm introduction document provides a detailed introduction to each algorithm, including algorithm classification, algorithm principles, algorithm application scenarios, etc.; the model training and testing document records in detail the design of each algorithm training and test experiment, training and testing methods and processes, training and testing results, etc.; the pre-trained model provides storage and downloading of different trained models for the project, and users can directly download and deploy them; the deployment document is an accurate and complete description of each algorithm application solution, and provides a series of clear instructions for solving related problems.

[0077] This system deeply integrates machine vision, artificial intelligence and other technologies to serve the manufacturing industry in a systematic way; by building an industrial intelligent visual cloud service system, it provides the manufacturing industry with full-process technical services from data processing, model development to decision-making assistance, breaking the technical barriers of the industry using visual algorithms and artificial intelligence technologies, promoting the deep integration of cross-industry technologies, and creating a manufacturing value chain of intelligent services, intelligent applications, and intelligent innovation. Based on visual inspection technology, through the application of information and intelligent technologies and the implementation of demonstration applications, it has improved the production efficiency of the manufacturing industry, optimized the traditional decision-making and management models of the industry, and provided strong technical support for the development and value of the manufacturing industry. At the same time, it also provides an organizational framework for reference for other artificial intelligence systems.

[0078] The present invention builds an industrial vision algorithm support layer, which integrates various advanced machine learning and deep learning vision algorithm models, as well as vision algorithm models in different special application scenarios, to ensure that the system can meet various vision business needs. Through the standardized combination of various visual data and vision algorithm models, the standardization and reusability of system data and algorithms are improved.

[0079] An industrial vision algorithm application layer is constructed in the system, and a visual algorithm intelligent recommendation method is proposed to improve the user experience of the system. This system constructs a visual algorithm intelligent recommendation method based on machine learning. Compared with traditional users who spend a lot of time and energy searching for the models and deployment methods they need to use, the visual algorithm intelligent recommendation method proposed in this invention combines the personalized needs of industrial enterprises and uses machine learning methods to automatically find suitable visual algorithms.

[0080] Example

[0081] The present invention discloses an industrial intelligent vision algorithm cloud service system. Based on the difficulties of traditional industrial intelligent vision applications and centering on the transformation and upgrading of traditional industries and the development needs of advanced manufacturing industries, an industrial intelligent vision algorithm cloud service system is proposed to provide the manufacturing industry with full-process technical services from data processing, model development to decision support, breaking the technical barriers of the industry's use of visual algorithms and artificial intelligence technologies, promoting the deep integration of cross-industry technologies, and creating a manufacturing value chain of intelligent services, intelligent applications, and intelligent innovations. In this system, by constructing an industrial vision algorithm support layer, integrating various advanced machine learning and deep learning models, as well as visual algorithm models in different application scenarios, constructing a visual data processing algorithm library and a visual algorithm model library, effectively ensuring that the system can meet various visual business needs and improve the reusability of industrial vision algorithms. By constructing an industrial vision algorithm application layer, a visual algorithm intelligent recommendation method is proposed. Combined with the personalized needs of industrial enterprise users, machine learning methods are used to automatically find and recommend suitable industrial vision algorithms, thereby improving the user experience of the system. Figure 1 As shown, the framework mainly consists of three layers: industrial vision infrastructure layer, industrial vision algorithm support layer, and industrial vision algorithm application layer.

[0082] Industrial vision infrastructure layer construction:

[0083] The industrial vision infrastructure layer provides the basic operation guarantee of the industrial intelligent vision algorithm cloud service system, and provides the entire system with operation capabilities, data storage capabilities, and Internet paper resources. Specifically, the infrastructure layer includes the operation environment, storage environment, and network resource library.

[0084] The operating environment is divided into hardware operating environment and software operating environment. The hardware operating environment includes the central processing unit and the graphics processing unit. The hardware operating environment of this platform consists of two 14-core GeForce GTX TITAN X series high-performance image processing workstations and four dual-core E5-2680v4 rack servers; the software operating environment of this system includes algorithm development and operating environments based on Java and Python languages. The operating environment of Java language is JDK 1.8. The Python operating environment is deployed using Anaconda. It is a Python distribution for scientific computing, suitable for Python tools for data analysis, and contains Python and related very useful third-party libraries.

[0085] The storage environment uses the database to store data efficiently and in an organized manner, allowing the system to manage data faster and more conveniently. This system builds a MySQL database to store algorithm category information and user information. This system builds a MongoDB database to store detailed documentation of each algorithm, including algorithm principles, algorithm application scenarios, training and testing documents, deployment plans, pre-trained models, etc.

[0086] The network resource library integrates access to more than 70 Chinese and foreign network databases and massive electronic journals, providing excellent and convenient literature retrieval conditions.

[0087] Industrial vision algorithm support layer construction:

[0088] The industrial vision algorithm support layer summarizes the common technologies in the fields of image and video classification and detection in the field of computer vision, and builds an industrial vision algorithm resource library and update and iteration modules. The industrial vision algorithm resource library provides an introduction to industrial intelligent vision algorithms, model training and testing steps, pre-trained model storage and download introduction, and usage plan introduction. Among them, the algorithm introduction provides a detailed introduction to each algorithm, including algorithm classification, algorithm principle, algorithm application scenario, etc.; the model training and testing records in detail the design of each algorithm training and test experiment, training and testing methods and processes, training and test results, etc.; the pre-trained model is to make the original manual parameter adjustment and rely on machine learning experts enter the stage of large-scale, replicable large-scale industrial application. It provides storage of different trained models for the project, and users can directly download and deploy them; the usage plan introduction refers to an accurate and complete description of the use of each algorithm, providing a series of clear instructions for the use of the algorithm.

[0089] like Figure 2 As shown in the figure, the industrial vision algorithm resource library includes the theoretical basis vision algorithm library and the industrial application vision algorithm library, integrating the theoretical basis vision algorithms in the field of computer vision and the algorithms in industrial practical applications. At the same time, the support layer provides a continuous update and iteration mechanism for the industrial vision algorithm resource library, and manually updates the advanced algorithms newly proposed by the scientific research and academic community for various application scenarios in a timely manner. The algorithm classification method is shown in the attached Figure 2 shown.

[0090] For the theoretical basis visual algorithm library, the present invention includes the image theory basis visual algorithm library and the video theory basis visual algorithm library according to the different attributes of the data set. . For the image theory basis visual algorithm library, according to the three research directions of image data mining, it specifically includes image classification algorithm, image detection algorithm and image segmentation algorithm; among them, the image classification algorithm is an image processing method that distinguishes different categories of targets according to the different features reflected in the image information. It uses a computer to quantitatively analyze the image and classifies the image or each pixel or area in the image into one of several categories to replace human visual interpretation; the image detection algorithm reads the local image data as input, performs target detection on the object in the image, obtains the confidence of the detected object category and the frame of the detected target, and finally displays the detection result. The image segmentation algorithm divides the image into several specific areas with unique properties and proposes the technology and process of the target of interest. For the video theory basis visual algorithm library, considering the wide range of video data detection needs in real application scenarios, the present invention integrates industrial video detection algorithms.

[0091] For the industrial application visual algorithm library, the present invention includes a small sample learning algorithm, a single-class sample learning algorithm, and a lightweight learning algorithm. Among them, the small sample learning algorithm is suitable for solving the situation where the real application scenario only contains a small number of training samples; the single-class sample learning algorithm is suitable for solving the problem that some tasks in real industrial scenarios can only collect a large number of normal samples but no negative samples; the lightweight learning algorithm provides a compressed model, so that the network carries fewer network parameters, thereby reducing memory requirements and providing faster reasoning speed.

[0092] The algorithm resource library update and iteration module uses manual updates to continuously investigate advanced visual algorithms from top academic conferences and journals, and updates the algorithms to the algorithm resource library.

[0093] Industrial vision algorithm application layer construction:

[0094] The industrial vision algorithm application layer determines the type of industrial vision algorithm based on the user's application scenario, and proposes two vision algorithm recommendation functions to help users select applicable vision algorithms. The first vision algorithm recommendation function provides users with a list of all available algorithms for selection; the second vision algorithm intelligent recommendation function provides users with a vision algorithm recommendation method based on a machine learning algorithm, using the features of the training data set as training data and the performance of the candidate model on the data set as labels to train a classification model. When a user provides a new data set, this recommendation method recommends a vision algorithm model with good performance on the data set. The following introduces the implementation details of the two recommendation methods.

[0095] The core idea of ​​the first visual algorithm recommendation method is to display all the algorithms integrated in the system to users through the front-end page in a classified manner for direct selection. This method is suitable for industrial enterprise users who have clear requirements and experience in developing industrial visual algorithm models, and can be used by directly downloading the required model algorithms and deployment documents.

[0096] The core idea of ​​the second visual algorithm intelligent recommendation method is to use the divide-and-conquer principle to divide the visual problem processing steps in industrial applications, and in each step, independently select the appropriate algorithm according to the specific visual problem, and then integrate the decisions of all divided steps to obtain several better overall solutions and recommend them to users. Specifically, it is divided into four steps: data set feature extraction, preprocessing algorithm recommendation, visual algorithm model recommendation, and solution generation.

[0097] Dataset feature extraction steps

[0098] According to the data set provided by the user, manual features of the data set are designed and extracted. Manual features are artificially designed features, that is, the features themselves are directly designed, generally based on experience. The data set features extracted by the present invention include the industrial application scenario deployment location, industrial application scenario category, data type, positive and negative sample ratio, positive sample data volume, negative sample data volume, and whether negative samples are included. Specifically, the meaning of each feature is as follows:

[0099] The deployment locations of industrial application scenarios are divided into front-end deployment and back-end deployment. Front-end deployment refers to deploying the model to the business front-end for deployment, directly processing the image or video data collected by the front-end and providing services. Back-end deployment refers to uploading the image or video data collected by the front-end to the server and executing the decision model output results, and returning the results to the front-end.

[0100] Industrial application scenario categories include classification, detection, and segmentation. Image classification is an image processing method that distinguishes different categories of targets based on the different features reflected in the image information. It uses computers to perform quantitative analysis on images and classify images or each pixel or area in the image into one of several categories to replace human visual interpretation; image detection reads local image data as input, performs target detection on objects in the image, obtains the confidence of the detected object category and the frame of the detected target, and finally displays the detection results. Image segmentation is the technology and process of dividing an image into several specific regions with unique properties and proposing targets of interest. For video tasks, considering the large number of video detection needs in real application scenarios, video detection related algorithms in the field of intelligent vision are studied.

[0101] Data type refers to the type of data set used by the user, which includes two types, namely image data and video data.

[0102] The positive-negative sample ratio refers to the ratio of positive to negative samples in the dataset used by the user. For example, if there are 1,000 positive sample images and 500 negative sample images, the positive-negative sample ratio is 2. This feature is used to confirm whether the user dataset has unbalanced positive and negative samples, so as to decide whether a special visual algorithm model needs to be used.

[0103] The amount of positive sample data refers to the number of positive samples contained in the data set used by the user.

[0104] The amount of negative sample data refers to the number of negative samples contained in the dataset used by the user.

[0105] Whether it contains negative samples refers to whether the dataset used by the user contains the number of positive and negative samples.

[0106] Preprocessing algorithm recommendation

[0107] The preprocessing algorithm is a process performed on the input image before feature extraction, segmentation and matching. The main purpose is to eliminate irrelevant information in the data, enhance the detectability of relevant information and simplify the data to the maximum extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition. The preprocessing algorithm of the present invention is recommended based on the visual data processing algorithm library, which provides commonly used data preprocessing algorithms for different data types.

[0108] For image data, the data preprocessing algorithm provided by the present invention includes simple scaling, sample-by-sample mean reduction, and feature standardization. Simple scaling is to readjust the value of each dimension of the data so that the final data vector falls within the interval of [0, 1] or [-1, 1]. Sample-by-sample mean reduction subtracts the statistical mean of the data from each sample for stationary data (sample-by-sample calculation). For example, for images, this normalization removes the average brightness value of the image. In many cases, the illumination of the image is not of interest, but more attention is paid to its content. At this time, it is meaningful to remove the mean of the pixels for each data point. Feature standardization refers to making each dimension of the data have zero mean and unit variance.

[0109] For video data, the data preprocessing algorithms provided by the present invention include frame scaling and video cropping. Frame scaling is to calculate the pixel values ​​of the target image according to certain rules based on the pixel values ​​of the original image. Video cropping is to crop the original video data into video segments with the same size and number of frames, which is convenient for machine learning models to use.

[0110] Visual algorithm model recommendation

[0111] The visual algorithm recommendation model uses machine learning models as the core to provide personalized application scenario algorithm recommendations for industrial enterprise users. It is necessary to train the visual algorithm recommendation model based on machine learning. Therefore, we collect a large number of real industrial scenario data sets, extract manual features from these data sets as training data and input them into candidate classifiers to obtain the performance of each data set under different candidate classifiers. We use the performance of the candidate model on the data set as a label to train a visual algorithm recommendation model for use in the cloud service system.

[0112] As attached Figure 3 As shown in Figure 1, the industrial intelligent visual algorithm recommendation model mainly consists of two stages: training stage and online use stage. In the training stage, we collect a large number of historical industrial data sets D1, D2, ..., D m , and design a manual feature extraction function f to extract the manual features f(D1), f(D2), ..., f(D m ) and is input into the candidate classifier as training data. The candidate classifier includes n C1, C2, …, C n Then, the performance P of each data set under different candidate classifiers is obtained. ij , where i = 1, 2, ..., m, j = 1, 2, ..., n. Finally, the hand-crafted feature set f(D1), f(D2), ..., f(D m ) and the performance of the candidate model on the dataset P ij For the label, train a classification model C.

[0113] During the online use phase, when an industrial enterprise user provides a new dataset D, we extract the features f(D) of the dataset and then input them into the trained visual algorithm recommendation model C to recommend a visual algorithm model with good performance on the dataset.

[0114] The machine learning-based visual algorithm recommendation model is implemented by a decision tree, which characterizes the mapping relationship between the input data set features and the model. The model is fast, has a relatively small amount of calculation, and is easy to convert into classification rules. Moreover, it is suitable for processing continuous and categorical fields, does not require any domain knowledge and parameter assumptions, and is suitable for high-dimensional data. The decision tree is a commonly used classification method with a tree structure. Each internal node represents a test on an attribute, each branch represents a test output, and each leaf node represents a category. It is a supervised learning. Given a bunch of samples, each sample has a set of attributes and a category. These categories are determined in advance. By training the classifier, the decision tree gives the correct classification for the newly appeared object.

[0115] The present invention uses the ID3 algorithm in the decision tree to divide the data set. The principle of dividing the data is to make the disordered data more orderly. The information before and after the data division changes into information gain. Knowing how to calculate the information gain, calculate the information gain obtained by dividing the data set by each feature value, and the feature with the highest information gain is the best choice.

[0116] The definition of information is l(x i )=-log2 p(x i ), where p(x i ) is the probability of selecting the classification. In order to calculate entropy, the present invention needs to calculate the expected value of information contained in all possible values, which is obtained by the following formula:

[0117]

[0118] The smaller the value of Ent(D), the higher its purity; when calculating information entropy, it is agreed that: if p=0, then plog2 p=0, the minimum value of H is 0, and the maximum value is log2 n.

[0119] Information gain assumes that the discrete attribute a has V possible values ​​{a 1 , a 2 , …, a V}, if a is used to partition the data set D, V branch nodes will be generated, where the vth branch node contains all the nodes in D whose value is a on attribute a v The sample is denoted as D v According to the entropy formula, D is calculated v The information entropy of , and considering the different number of samples contained in different branch nodes, the branch nodes are given weights |D v | / |D|, that is, the more samples a branch node has, the greater its influence will be. Therefore, we can calculate the value obtained by dividing the sample set D with attribute a.

[0120] Information gain is defined as Generally speaking, the greater the information gain, the greater the purity improvement obtained by using attribute a to divide. * =arg a∈A max Gain(D, a)

[0121] Solution Generation

[0122] The solution generation function is based on the results of the previous steps, and recommends solutions suitable for industrial enterprise users in their application scenarios. Specifically, this module directly provides several optimal algorithm models for visual algorithm preprocessing and visual algorithm models, and provides detailed algorithm introduction documents, model training and testing documents, pre-trained models, and usage deployment documents for the recommended algorithm models. The algorithm introduction document provides a detailed introduction to each algorithm, including algorithm classification, algorithm principles, algorithm application scenarios, etc.; the model training and testing document records in detail the design of each algorithm training and test experiment, training and testing methods and processes, training and testing results, etc.; the pre-trained model provides storage and downloading of different trained models for the project, and users can directly download and deploy them; the deployment document is an accurate and complete description of each algorithm application solution, and provides a series of clear instructions for solving related problems.

[0123] The present invention discloses an industrial intelligent visual algorithm cloud service system. Based on the difficulties of traditional industrial intelligent visual applications, and focusing on the transformation and upgrading of traditional industries and the development needs of advanced manufacturing industries, an industrial intelligent visual algorithm cloud service system is proposed to provide the manufacturing industry with full-process technical services from data processing, model development to decision-making assistance, break the technical barriers of the industry using visual algorithms and artificial intelligence technologies, promote the deep integration of cross-industry technologies, and create a manufacturing value chain of intelligent services, intelligent applications, and intelligent innovation. In this system, by building an industrial visual algorithm support layer, integrating various advanced machine learning and deep learning models, as well as visual algorithm models under different application scenarios, constructing a visual data processing algorithm library and a visual algorithm model library, effectively ensuring that the system can meet various visual business needs and improve the reusability of industrial visual algorithms. By building an industrial visual algorithm application layer, a visual algorithm intelligent recommendation method is proposed, combined with the personalized needs of industrial enterprise users, using machine learning methods, automatically finding and recommending suitable industrial visual algorithms, thereby improving the user experience of the system. The framework mainly includes three layers: industrial visual infrastructure layer, industrial visual algorithm support layer, and industrial visual algorithm application layer. The industrial vision infrastructure layer provides the basic operation guarantee of the industrial intelligent vision algorithm cloud service system, and provides the platform operation capability, data storage capability, and Internet paper resources for the entire system. The industrial vision algorithm support layer summarizes the common technologies in the field of computer vision in terms of image and video classification, detection, etc. From the perspectives of theoretical basis and industrial application, it integrates the theoretical algorithms in the field of computer vision and the advanced algorithms of key applications in industrial applications, and builds a visual data processing algorithm library and an industrial intelligent vision algorithm model library. The industrial vision algorithm application layer determines the type of industrial vision algorithm based on the user's application scenario, and proposes a visual algorithm recommendation method based on a machine learning algorithm. It uses the characteristics of the training data set as the training data, and the performance of the candidate model on the data set as the label to train a classification model. When an industrial enterprise user provides a new data set, the recommendation model recommends a suitable and high-performance visual algorithm model for the data set.

Claims

1. An industrial intelligent visual algorithm cloud service system, characterized in that: It includes industrial intelligent vision infrastructure layer, industrial intelligent vision algorithm support layer and industrial intelligent vision algorithm application layer; The industrial intelligent vision infrastructure layer is used to provide the basic capabilities and guarantees for model operation, algorithm storage and data processing in the industrial intelligent vision algorithm support layer and the industrial intelligent vision algorithm application layer; The industrial intelligent vision infrastructure layer includes an operating environment; The operating environment is divided into a hardware operating environment and a software operating environment. The hardware operating environment includes a central processing unit and a graphics processing unit; the software operating environment includes an algorithm development and operating environment based on Java and Python languages; The industrial intelligent vision algorithm support layer is used to provide an algorithm basis for the industrial intelligent vision algorithm application layer; The industrial intelligent vision algorithm support layer includes a visual data processing algorithm library and an industrial intelligent vision algorithm model library; the visual data processing algorithm library includes algorithms for feature extraction of input images and processing before model training; the industrial intelligent vision algorithm model library includes a theoretical basis visual algorithm library and an industrial application visual algorithm library; The industrial intelligent vision algorithm application layer is used to select appropriate data processing algorithms and industrial vision models based on the service requirements of application scenario data processing obtained from users, recommend industrial vision preprocessing algorithms and vision algorithm models required for processing data sets to users, and generate application scenario solutions for users to download and use; The industrial intelligent vision algorithm application layer independently recommends the algorithm required for each step based on the visual data mining steps in industrial applications, and then integrates the decisions of all steps to obtain several better overall solutions for users to recommend.

2. The industrial intelligent visual algorithm cloud service system according to claim 1, characterized in that: The industrial intelligent vision infrastructure layer also includes a storage environment and a network resource library; The storage environment is a MySQL database and a MongoDB database; the MySQL database is used to store algorithm attribute information and industrial enterprise user information; the MongoDB database is used to store detailed documentation of each algorithm, including algorithm principles, algorithm application scenarios, training and testing documents, deployment plans, and pre-trained models; The network resource library is used to connect to the network database and provide document retrieval.

3. The industrial intelligent visual algorithm cloud service system according to claim 1, characterized in that: The application scenario solution includes a detailed algorithm introduction document of the recommended algorithm model, model training and testing documents, pre-trained models and deployment documents.

4. The industrial intelligent visual algorithm cloud service system according to claim 1, characterized in that: The industrial intelligent vision algorithm application layer list displays all available algorithms for users to select and download.

5. An industrial intelligent visual algorithm cloud service method, based on an industrial intelligent visual algorithm cloud service system as described in claim 1, characterized in that: The following steps are included: Step 1: Obtain the data set collected by the user in his application scenario; Step 2: Extracting manual features of the dataset based on the dataset uploaded by the user; Step 3: Input the extracted data set features into the intelligent visual algorithm recommendation model to recommend the industrial visual preprocessing algorithm and visual algorithm model required to process the data set to the user, and generate an application scenario solution; In step 3, the visual algorithm recommendation specifically includes the following steps: Step 3.1, feature extraction of user uploaded dataset; Extract manual features of the dataset based on the dataset uploaded by the user; Step 3.2, data preprocessing algorithm recommendation; According to the features of the data set extracted in step 3.1, select a suitable data preprocessing algorithm for the user's application scenario, and preprocess the data provided by the user in step 3.1 for subsequent machine learning model training; Step 3.3, visual algorithm model recommendation; According to the dataset features extracted in step 3.1, select the appropriate visual algorithm model for the user's application scenario and provide an algorithm that effectively meets the user's visual application scenario requirements; Step 3.4, solution generation; Based on the recommendation results of step 3.2 and step 3.3, generate a personalized solution suitable for data set processing for the user's visual application scenario; Step 4: Provide users with application scenario solutions and provide users with solutions for downloading and using through the application's long interactive interface; Steps 1 to 4 are used by the industrial intelligent vision infrastructure layer to provide basic capabilities for operation, storage, and processing and basic operation guarantees.

6. The industrial intelligent visual algorithm cloud service method according to claim 5, characterized in that: In step 3.1, the manual features of the data set include the deployment location of the industrial application scenario, the scenario category, the data type, the ratio of positive and negative samples, the amount of positive sample data, the amount of negative sample data, and whether negative samples are included.

7. The industrial intelligent visual algorithm cloud service method according to claim 5, characterized in that: In step 3.2, the preprocessing process includes unifying the data format, reducing data noise and simplifying the data.

Images