Graphical programming method and device based on AI

Abstract

The invention discloses a graphical programming method and device based on AI. The method comprises the steps that a graphical programming interface is displayed, and the graphical programming interface comprises a programming area used for editing graphical codes and a stage area used for displaying the execution effect of the graphical codes, the programming area comprises a code area used for displaying the graphical codes and a programming instruction building block box used for providing programming instruction building blocks for the code area. A deep learning model used for picture classification is trained through the graphical programming interface, the trained deep learning model serves as picture classification building blocks to be deployed into the programming instruction building block box, and the picture classification building blocks can be dragged to the code area to generate the graphical codes. The technical problem that the picture classification programming process is complex and tedious during graphical programming in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI), and more specifically, to a graphical programming method and device based on AI. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, image classification has gradually become an important part of many practical applications, such as security monitoring, medical diagnosis, smart retail, etc. However, the current image classification training process requires a high level of technical skills from the user, especially for users without a machine learning background. The whole process is complicated and cumbersome. Therefore, a simpler and more intuitive solution is urgently needed so that ordinary users can easily complete the image classification training task.

[0003] With regard to the technical problem that the image classification programming process is complicated and tedious in the prior art graphical programming, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide an AI-based graphical programming method and device to at least solve the technical problem that the image classification programming process in graphical programming in the prior art is complicated and cumbersome.

[0005] According to one aspect of an embodiment of the present application, an AI-based graphical programming method is provided, comprising: displaying a graphical programming interface, the graphical programming interface comprising a programming area for editing graphical code and a stage area for displaying the execution effect of the graphical code, wherein the programming area comprises a code area for displaying the graphical code and a programming instruction building block box for providing programming instruction building blocks for the code area; training a deep learning model for image classification through the graphical programming interface, and deploying the trained deep learning model as an image classification building block in the programming instruction building block box, wherein the image classification building block can be dragged to the code area to generate the graphical code.

[0006] According to another aspect of an embodiment of the present application, there is also provided an AI-based graphical programming device, including: a display module, configured to display a graphical programming interface, the graphical programming interface including a programming area for editing graphical code, a programming instruction building block box for providing graphical code for the programming area, and a stage area for displaying the execution effect of the graphical code; a training module, configured to generate a deep learning model for image classification through the graphical programming interface, and deploy the trained deep learning model as a picture classification building block box to the programming instruction building block box, wherein the picture classification building block box can be dragged to the programming area to generate graphical code for image classification.

[0007] According to another aspect of the embodiments of the present application, there is also provided an electronic device, comprising: a processor, and a memory for storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the method in any of the above embodiments.

[0008] According to another aspect of the embodiments of the present application, a non-transitory machine-readable medium storing computer instructions is also provided, where the computer instructions are used to enable a computer to execute the method in any of the above embodiments.

[0009] In an embodiment of the present application, a display module is configured to display a graphical programming interface, wherein the graphical programming interface includes a programming area for editing graphical codes and a stage area for displaying the execution effect of the graphical codes, wherein the programming area includes a code area for displaying the graphical codes and a programming instruction block box for providing programming instruction blocks for the code area; a training module is configured to train a deep learning model for image classification through the graphical programming interface, and deploy the trained deep learning model as an image classification block into the programming instruction block box, wherein the image classification block can be dragged to the code area to generate the graphical code. Through the above scheme, the technical problem that the image classification programming process is complicated and cumbersome in the prior art during graphical programming is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1 is a flowchart of an AI-based graphical programming method according to an embodiment of the present application;

[0012] Figure 2 is an architectural diagram of a graphical programming system in a Web browser according to an embodiment of the present application;

[0013] Figure 3 is a graphical programming interface diagram according to an embodiment of the present application;

[0014] Figure 4 is a flowchart of another AI-based graphical programming method according to an embodiment of the present application;

[0015] Figure 5 is a flowchart of another AI-based graphical programming method according to an embodiment of the present application;

[0016] Figure 6 is a flowchart of a method for image preprocessing according to an embodiment of the present application;

[0017] Figure 7 is a structural schematic diagram of an AI-based graphical programming device according to an embodiment of the present application;

[0018] Figure 8 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] According to an embodiment of the present application, an embodiment of an AI-based graphical programming method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0022] Figure 1 is an AI-based graphical programming method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0023] Step S102, displaying a graphical programming interface, wherein the graphical programming interface includes a programming area for editing graphical codes and a stage area for displaying the execution effect of the graphical codes, wherein the programming area includes a code area for displaying the graphical codes and a programming instruction block box for providing programming instruction blocks for the code area.

[0024] Graphical programming is a programming method that uses visual elements (such as graphics, blocks, nodes, etc.) to represent programming logic and operations instead of traditional text code. With graphical programming, users can build programs by dragging and combining these visual elements such as programming instruction blocks.

[0025] Programming instruction blocks are represented in the form of blocks, each of which represents a command or a logical structure, such as conditional judgment, loop, etc. Users write programs by dragging these blocks. Graphical programming is event-driven. Users can create code blocks that are triggered when a certain event occurs. For example, "when a button is clicked" or "when a gesture is recognized." In the graphical programming environment, users can create and control characters (sprites) to perform various actions on the stage. Characters can move, rotate, change appearance, etc. Blocks can be connected and nested to form complex logical structures. For example, a conditional judgment block can be nested in a loop block.

[0026] This application provides an intuitive and user-friendly graphical programming interface, making it easier for users to edit and debug codes without having to deeply understand the details of the programming language, thereby lowering the threshold for programming.

[0027] Step S104, training a deep learning model for image classification through the graphical programming interface, and deploying the trained deep learning model as an image classification building block into the programming instruction building block box, wherein the image classification building block can be dragged to the code area to generate the graphical code.

[0028] First, in response to receiving the image data set imported through the graphical programming interface, the image data set is preprocessed. By automatically preprocessing the image data set, the burden of manual data processing by the user is reduced, the consistency and quality of the data are ensured, and the effect of model training is improved.

[0029] Subsequently, in response to receiving a model selected through the graphical programming interface, the selected model is used as the deep learning model; or in response to receiving model parameters custom configured through the graphical programming interface, the deep learning model is generated based on the custom configured model parameters. The present application allows users to select predefined models or customize model parameters according to their needs, provides flexibility and controllability, meets the needs of different users, and thus improves the personalization and pertinence of model training.

[0030] Then, in response to receiving the training parameters set through the graphical programming interface, the training parameters are set for the deep learning model, wherein the training parameters include at least one of the following: learning rate, batch size, training rounds, data enhancement strategy, and regularization strategy. The present application provides a training parameter setting option, and the user can adjust various parameters in the training process according to specific needs, thereby optimizing model performance and improving training effects.

[0031] Based on the set training parameters, the preprocessed image data set is used to train the deep learning model, and the training process of the deep learning model is displayed in real time on the graphical programming interface. For example, a visualization chart is set on the graphical programming interface, wherein the visualization chart displays the training progress and performance indicators of the deep learning model in real time, wherein the performance indicators include at least one of the following: loss function value and accuracy. The present application displays the training progress and performance indicators in real time visually, allowing users to intuitively monitor and adjust the training process, thereby promptly discovering and solving problems in training and improving the transparency and efficiency of training.

[0032] In some embodiments, while using the preprocessed image data set to train the deep learning model, the method further includes: in response to receiving a dynamic adjustment parameter input through the graphical programming interface, dynamically adjusting the model parameters of the deep learning model based on the dynamic adjustment parameter. The present application makes the model training process more flexible and intelligent by dynamically adjusting the model parameters, and can be optimized according to real-time feedback to improve the adaptability and performance of the model.

[0033] Finally, in response to the image classification block in the programming instruction block box being dragged to the code area, the graphical code is generated in the code area; in response to running the graphical code, the image classification result corresponding to the graphical code is displayed in the stage area. This application improves development efficiency and user experience by intuitively displaying the editing and running results of the graphical code on the interface, allowing users to conveniently view and verify the execution effect of the code.

[0034] In graphical programming, the existing image classification training process usually involves many tedious steps, which is not user-friendly for non-professional users and requires users to have strong machine learning and programming capabilities. In addition, processes such as data preparation, model training, and evaluation require complex configuration and a lot of code writing. At the same time, existing solutions often lack good integrated interfaces, causing users to switch between multiple tools and platforms.

[0035] This application, by providing an intuitive graphical programming interface, converts complex machine learning operation steps into simple drag and click operations, so that users can complete image classification tasks without writing a lot of code, significantly reducing the threshold for use. In addition, non-professional users can complete model training and deployment through a graphical interface without having to have in-depth machine learning and programming knowledge, greatly improving the ease of use and popularity of graphical programming. In addition, this solution supports automated data preprocessing to ensure the consistency and quality of data, thereby improving the effect of model training and reducing the burden of users manually processing data. Users can select predefined models or customize model parameters as needed to meet the needs of different scenarios, enhancing the flexibility and personalization of model training. Through the graphical interface, the training progress and performance indicators are displayed in real time. Users can intuitively monitor the training process and dynamically adjust the model parameters as needed to optimize the training effect and improve model performance. Finally, this application integrates various links such as data preparation, model training, evaluation and deployment. Users do not need to switch between multiple tools and platforms, providing a complete one-stop solution, improving work efficiency and user experience.

[0036] This application provides a graphical programming system in a web browser to simplify the image classification training process. Through an intuitive graphical interface, users can easily complete data preparation, model selection and configuration, training monitoring, and model evaluation and deployment.

[0037] Figure 2 is an architectural diagram of a graphical programming system in a Web browser according to an embodiment of the present application. Figure 2 As shown, the system includes a front-end interface 12 , a back-end engine 14 and a model library 16 .

[0038] The front-end interface 12 is a graphical programming interface for user operation interaction, which mainly includes modules such as data import, model selection, training configuration, training management, and evaluation and deployment. The graphical programming interface provides an intuitive and user-friendly way for users to create and edit programs by dragging and dropping programming instruction blocks. In addition, the front-end interface 12 is also responsible for real-time feedback of user operations and displaying the status and results of program execution.

[0039] The backend engine 14 is used to process frontend user instructions, perform model training, evaluation and deployment tasks, and provide real-time feedback during the training process. Specifically, it is used to receive programming instructions from the frontend interface 12, parse these instructions, and convert them into executable code or instruction sets. The backend engine 14 also includes a runtime environment in which user-defined programs can be executed and process data and events generated during program execution. In addition, the backend engine 14 can also interact with external systems and services to implement more complex functions and tasks.

[0040] The model library 16 contains a variety of models that can be used for different programming tasks and application scenarios. Each model in the model library 16 is pre-trained and optimized and can be directly applied to the user's programming project. These models can include machine learning models, data analysis models, simulation models, etc. Users can select appropriate models from the model library 16 and integrate them into their own programs to achieve specific functions and goals. Users can also generate and train models through a graphical programming interface and store the trained models in the model library 16. In addition, the model library 16 also supports model updates and expansions. Users can add new models or update existing models as needed to maintain the advancement and flexibility of the system.

[0041] Specifically, the front-end interface 12 is a graphical programming interface, for example, Figure 3 As shown, it includes a programming area 32 (also called a console) for editing graphical codes, a stage area 36 for displaying the execution effect of the graphical codes, and a character bar 34 for providing the stage area 36 with stage characters and backgrounds.

[0042] The programming area 32 is the main operation area for users to perform graphical programming, and includes two components. The first is the code area 322, which is used to display and edit graphical codes. Users can create and modify program logic in this area by dragging and connecting programming instruction blocks. The second is the programming instruction building block box 324, which provides various programming instruction blocks. Users can drag these blocks into the code area 322 to build a program. The building block box contains multiple categories of building blocks, such as the camera building block box 3242, which covers various programming needs from basic operations to advanced functions.

[0043] The stage area 36 is used to display the execution effect of the graphical code. The user can directly observe the program running results by clicking the start button of the stage area 36. Whether it is animation, interaction or data display, the stage area 36 provides a visual platform so that users can instantly see the actual effect of the code they wrote. This is very important for debugging and improving the program, because users can find and correct errors by observing the output effect.

[0044] The backend engine 14 includes a data import module, a training configuration module, a training management module and an evaluation and deployment module.

[0045] The data import module supports users to upload and manage image datasets required for classification training; it provides data preprocessing functions such as image resizing and format conversion.

[0046] The training configuration module provides an interface for setting training parameters, including learning rate, batch size, training epochs, etc. It also supports advanced configurations such as data enhancement and regularization strategies.

[0047] The training management module is used to monitor the training process in real time, and displays the training progress and performance indicators, such as loss function value, accuracy, etc., through visual charts. In addition, it allows users to dynamically adjust parameters during the training process to improve training efficiency and model performance.

[0048] The evaluation and deployment module is used to evaluate the accuracy, recall rate, F1 value and other performance of the trained model; it provides a one-click deployment function to quickly integrate the model into the Web application; and it supports model version management and online updates.

[0049] The present application embodiment provides another AI-based graphical programming method, which is applied to the graphical programming system as described above. Figure 4 As shown, the method comprises the following steps:

[0050] Step S402, data import.

[0051] The user enters the graphical programming interface through the graphical programming platform browser, clicks the button to import image dataset on the graphical programming interface, selects and uploads the image dataset, and the background engine automatically preprocesses the images in the image dataset.

[0052] Step S404: model selection.

[0053] Users can select a suitable model from the pre-trained model library or customize model parameters through a graphical programming interface.

[0054] Step S406, training configuration.

[0055] The user configures the training parameters through the graphical programming interface and starts the training process.

[0056] Step S408: training monitoring.

[0057] Users can monitor the training progress and performance indicators in real time, understand the model training status through the visual interface, and make necessary parameter adjustments.

[0058] Step S410: evaluation and deployment.

[0059] After training is completed, the system automatically evaluates the model and provides a comprehensive performance report. Users can quickly integrate the trained model into the web application through the one-click deployment function.

[0060] This application lowers the threshold for users to use by providing an intuitive and simple interface. In addition, it provides a rich selection of models and configurations to meet the different needs of users. Moreover, during the training process, it provides real-time monitoring and intuitive adjustment mechanisms to enhance the user's control over model training. Finally, through the one-click deployment function, it can simplify the transition process from model training to application and improve user efficiency.

[0061] The present application embodiment provides another AI-based graphical programming method, which is applied to the graphical programming system as described above. Figure 5 As shown, the method comprises the following steps:

[0062] Step S502: in response to receiving the image data set imported through the graphical programming interface, preprocessing the image data set.

[0063] Provide a file upload control in the graphical programming interface to allow users to select and upload image datasets. For example, the interface contains an "Import Image Dataset" button. After the user clicks the button, select a local image dataset file to upload. The image selected by the user is sent to the backend engine for processing. The front end can use JavaScript to upload files and send the selected file to the server through the form data (FormData) object.

[0064] Use a web framework (such as Flask or Django) to implement an API interface for receiving file uploads on the back end. After the back-end engine, such as a server, receives a file upload request, it parses the file data in the request and saves the file to a specified directory on the server.

[0065] The images in the uploaded image dataset are preprocessed. The preprocessing method will be described in detail below and will not be repeated here.

[0066] Step S504, determining a deep learning model.

[0067] In response to receiving a model selected through the graphical programming interface, the selected model is used as the deep learning model; or in response to receiving model parameters custom configured through the graphical programming interface, the deep learning model is generated based on the custom configured model parameters.

[0068] Step S506, setting training parameters and performing training.

[0069] In response to receiving the training parameters set through the graphical programming interface, the training parameters are set for the deep learning model, wherein the training parameters include at least one of the following: learning rate, batch size, training rounds, data enhancement strategy, and regularization strategy. Based on the set training parameters, the preprocessed image data set is used to train the deep learning model. The method for training the deep learning model will be described in detail below and will not be repeated here.

[0070] Step S508, monitoring training.

[0071] A visualization chart is set in the graphical programming interface, wherein the visualization chart displays the training progress and performance indicators of the deep learning model in real time, wherein the performance indicators include at least one of the following: loss function value and accuracy. In response to receiving a dynamic adjustment parameter input through the graphical programming interface, dynamically adjusting the model parameters of the deep learning model based on the dynamic adjustment parameter

[0072] Step S510: evaluation and deployment.

[0073] After training is completed, the system automatically evaluates the model and provides a comprehensive performance report. Users can quickly integrate the trained model into the web application through the one-click deployment function.

[0074] This application uses a graphical interface, and the system greatly reduces the user's usage threshold, so that even users without a machine learning background can easily get started, and simplifies the steps of data preparation, model configuration, training monitoring and model deployment, thereby greatly shortening the time of the entire process. The front-end user interface is highly integrated with the back-end engine. Users can complete all operations on the same platform without switching tools, and the system supports a variety of pre-trained model selections and custom configurations to meet different types of image classification needs. In addition, the system provides efficient model training and real-time feedback mechanisms to ensure the accuracy and performance of the model in various scenarios. This application provides a comprehensive, simple and efficient solution for the Web-side image classification training process, allowing users to seamlessly prepare data, train models, monitor and adjust, and evaluate deployment, greatly improving user experience and work efficiency.

[0075] This application also provides a method for image preprocessing, such as Figure 6 As shown, the following steps are included:

[0076] Step S602, performing standardization processing on the image.

[0077] Normalize the image and scale the pixel values ​​to the range of [0,1]. Normalization can accelerate the convergence of the model.

[0078] Step S604: perform data enhancement processing on the image.

[0079] For example, randomly rotate the image by a certain angle; randomly enlarge or reduce the image; randomly translate the image horizontally or vertically; randomly cut certain parts of the image; randomly flip the image horizontally or vertically; randomly adjust the brightness, contrast and saturation of the image.

[0080] Step S606: randomly erase part of the picture.

[0081] Random erasing simulates missing areas in the image by randomly selecting rectangular areas in the image and randomly setting their pixel values ​​to zero.

[0082] Step S608, denoising the image.

[0083] First, a Gaussian kernel is generated and normalized; then, the image is denoised based on the offset value of the pixel value at each pixel position of the image, the horizontal gradient of the image at each pixel position, the vertical gradient of the image at each pixel position, and the noise intensity parameter, wherein the noise intensity parameter is used to control the sensitivity to the difference in the pixel value of the image during the denoising process. Specifically, the following steps are included:

[0084] Step S6082, generate a Gaussian kernel and normalize the Gaussian kernel.

[0085] Assume that the image has been loaded as a matrix I, where I(i,j) represents the pixel value of the image at position (i,j). Select the size and standard deviation of the Gaussian kernel. Select the size of the Gaussian kernel (such as 3x3, 5x5, 7x7, etc.) and the standard deviation σ. The larger the Gaussian kernel size, the stronger the blurring effect, but more details may be lost at the same time. The larger the standard deviation σ, the more significant the blurring effect. Generate a Gaussian kernel. Generate a Gaussian kernel matrix K according to the Gaussian function formula. For a Gaussian kernel of size k×k, the center position is (k / / 2, k / / 2):

[0086]

[0087] in, σ is the standard deviation, which determines the width of the Gaussian distribution, (x,y) is the coordinate of the pixel in the image relative to the center of the kernel, and k is the size of the Gaussian kernel.

[0088] Then, normalize the Gaussian kernel. In order to ensure that the weighted sum of the Gaussian kernel is 1, it needs to be normalized:

[0089]

[0090] Step S6084, perform denoising processing.

[0091] Multi-scale Gaussian blur is achieved by applying multiple Gaussian kernels of different scales. The standard deviation σ of each Gaussian kernel is different, which can capture the image features at different scales, thereby improving the denoising effect. The improved denoising processing formula is:

[0092]

[0093] N=e -η·I(i+m ,j+n)-I(i ,j)

[0094] Among them, I(i,j) is the pixel value of the original image at position (i,j), m,n are the offsets of the horizontal and vertical coordinates relative to the center of the Gaussian kernel, tx is the horizontal gradient of the image at each pixel position, ty is the vertical gradient of the image at each pixel position, β is the gradient weight, and N is the noise intensity parameter, which is used to control the sensitivity of the pixel value difference of the image during the denoising process. It is obtained based on the noise intensity η, the pixel value of the image at position (i,j), and the offset value of the pixel value of the image at position (i,j).

[0095] This embodiment performs denoising on the image based on the pixel value offset, horizontal gradient, vertical gradient and noise intensity parameter at each pixel position, which can make the denoising process more detailed and accurate. Among them, the noise intensity parameter is used to control the sensitivity to the pixel value difference in the denoising process, which helps to retain more image details and edge information while effectively removing noise, making the denoised image clearer and more natural.

[0096] Step S6086, handle boundary issues.

[0097] In the denoising process, if the pixels at the boundary do not have enough neighboring pixels for calculation, the boundary can be mirrored and expanded so that the pixels at the boundary have enough neighboring pixels. Another example is to fill them with a constant value (such as 0).

[0098] Step S6088, verify the effect.

[0099] The processed images need to be verified to ensure that the denoising effect is significant and the main image features are preserved. The following methods can be used for verification: Calculate the signal-to-noise ratio (SNR) or peak signal-to-noise ratio (PSNR) before and after processing to quantify the denoising effect.

[0100] The signal-to-noise ratio (SNR) is calculated as:

[0101]

[0102] The peak signal-to-noise ratio (PSNR) is calculated as:

[0103]

[0104] Wherein, L is the maximum pixel value of the picture (for an 8-bit picture, L=255), and M and N represent the height and width of the picture.

[0105] The present application also provides an AI-based graphical programming device, such as Figure 7 As shown, it includes: a display module 72 is configured to display a graphical programming interface, the graphical programming interface includes a programming area for editing graphical codes and a stage area for displaying the execution effect of the graphical codes, wherein the programming area includes a code area for displaying the graphical codes and a programming instruction building block box for providing programming instruction building blocks for the code area; a training module 74 is configured to train a deep learning model for image classification through the graphical programming interface, and deploy the trained deep learning model as an image classification building block into the programming instruction building block box, wherein the image classification building block can be dragged to the code area to generate the graphical code.

[0106] It should be noted that: the AI-based graphical programming device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the AI-based graphical programming device provided in the above embodiment and the AI-based graphical programming method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0107] Figure 8 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0108] like Figure 8 As shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0109] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0110] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, various functions defined in the method and apparatus of the present application are executed. In some embodiments, the electronic device may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0111] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0112] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0114] As another aspect, the present application further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device.

[0115] The computer-readable medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments. For example, the electronic device can implement each step of the method embodiment.

[0116] As another aspect, the present application further provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method of the embodiment of the present application when executed by the at least one processor.

[0117] As another aspect, the present application further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0118] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0119] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0123] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0124] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A graphical programming method based on AI, characterized in that: include: Displaying a graphical programming interface, the graphical programming interface comprising a programming area for editing graphical codes and a stage area for displaying the execution effect of the graphical codes, wherein the programming area comprises a code area for displaying the graphical codes and a programming instruction building block box for providing programming instruction building blocks for the code area; A deep learning model for image classification is trained through the graphical programming interface, and the trained deep learning model is deployed as an image classification building block in the programming instruction building block box, wherein the image classification building block can be dragged to the code area to generate the graphical code.

2. The method according to claim 1, characterized in that: Training a deep learning model for image classification through the graphical programming interface includes: In response to receiving the image data set imported through the graphical programming interface, preprocessing the image data set; In response to receiving the training parameters set through the graphical programming interface, setting the training parameters for the deep learning model, wherein the training parameters include at least one of the following: a learning rate, a batch size, a training round, a data augmentation strategy, and a regularization strategy; Based on the set training parameters, the deep learning model is trained using the preprocessed image data set, and the training process of the deep learning model is displayed in real time on the graphical programming interface.

3. The method according to claim 2, characterized in that Displaying the training process of the deep learning model in real time on the graphical programming interface, including: setting a visualization chart on the graphical programming interface, wherein the visualization chart displays the training progress and performance indicators of the deep learning model in real time, wherein the performance indicators include at least one of the following: loss function value and accuracy rate.

4. The method according to claim 3, characterized in that While using the preprocessed image data set to train the deep learning model, the method also includes: in response to receiving dynamic adjustment parameters input through the graphical programming interface, dynamically adjusting model parameters of the deep learning model based on the dynamic adjustment parameters.

5. The method according to claim 2, characterized in that: Before using the preprocessed image data set to train the deep learning model, the method also includes: in response to receiving a model selected through the graphical programming interface, using the selected model as the deep learning model; or in response to receiving model parameters custom configured through the graphical programming interface, generating the deep learning model based on the custom configured model parameters.

6. The method according to any one of claims 1 to 5, characterized in that After deploying the trained deep learning model as an image classification building block into the programming instruction building block box, the method further includes: In response to the picture classification building block in the programming instruction building block box being dragged to the code area, generating the graphical code in the code area; In response to running the graphical code, the picture classification result corresponding to the graphical code is displayed in the stage area.

7. A graphical programming device based on AI, characterized in that: include: A display module is configured to display a graphical programming interface, wherein the graphical programming interface includes a programming area for editing graphical codes and a stage area for displaying the execution effect of the graphical codes, wherein the programming area includes a code area for displaying the graphical codes and a programming instruction block box for providing programming instruction blocks for the code area; The training module is configured to train a deep learning model for image classification through the graphical programming interface, and deploy the trained deep learning model as an image classification building block into the programming instruction building block box, wherein the image classification building block can be dragged to the code area to generate the graphical code.

8. An electronic device, characterized in that: include: a memory configured to store a computer program; A processor, configured to cause a computer to execute the method according to any one of claims 1 to 6 when the program is executed.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed, the computer is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images