Design method of large model and small model combined image processing system

Abstract

The invention discloses a design method of an image processing system combining a large model and a small model, and relates to the technical field of image processing of artificial intelligence and large model application. Comprising the following steps: establishing an image processing system based on the combination of a large model and a small model, decoding an image into a pixel matrix through a small model processing module, converting an image format into an RGB format suitable for model input, normalizing and standardizing the decoded image through a preprocessing module, loading configuration parameters and the small model through a reasoning module, and outputting the image. Inputting the preprocessed image into a small model for reasoning to obtain the output of the small model, and performing corresponding size conversion on the output of the small model through a post-processing module, so that the detected target coordinate corresponds to the pixel size of the original picture; a detected target information result and detection confidence are integrated into a unified protocol through an output protocol module and are provided for a large model prediction module, and analysis and explanation are performed through the large model prediction module according to context learning and in combination with large model processing capability. And a final explanation and result is formed and returned to the http server for visual display.

Description

Technical Field

[0001] The present invention discloses a design method for an image processing system combining a large model and a small model, and relates to the technical field of image processing for artificial intelligence and large model applications. Background Art

[0002] With the rapid development of artificial intelligence technology, the efficiency, accuracy and intelligence of image processing have become the focus of industry development. Large models are widely studied and applied for their powerful data processing capabilities and high-precision recognition effects, and can handle complex image tasks. However, large models are often accompanied by high energy consumption and deployment difficulties, which limit their widespread use in practical applications. In particular, large models have huge parameters and require high-performance computing resources for training and reasoning. They have high computational complexity and slow reasoning speed, and are even more difficult to meet the requirements of real-time scenarios. At the same time, small models have advantages on resource-constrained devices with their low computing cost, small number of parameters and easy deployment. They are suitable for scenarios with high real-time requirements, but the accuracy of small models is relatively low, the generalization ability is relatively weak, and they are not as capable as large models in processing complex image tasks. Existing research technologies that combine large and small models to improve image processing performance and efficiency mainly focus on technologies such as model compression and knowledge distillation, and achieve the collaborative work of large and small models through different methods. However, these methods have problems such as inflexible combination, complex parameter adjustment, and lack of versatility. Summary of the invention

[0003] In response to the problems of the prior art, the present invention provides a design method for an image processing system that combines a large model with a small model. Through this design method, the system can process images more efficiently, reduce computing costs, and improve the accuracy and response speed of image processing. At the same time, the plug-in small model improves the adaptability and scalability of the system.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides a design method for an image processing system combining a large model and a small model, and establishes an image processing system based on the combination of a large model and a small model. The image processing system includes a small model processing module and a large model prediction module. The small model processing module includes a preprocessing module, an inference module, a post-processing module and an output protocol module.

[0006] The small model processing module decodes the image into a pixel matrix and converts the image format into an RGB format suitable for model input. The decoded image is normalized and standardized through the preprocessing module. The configuration parameters and small model are loaded through the inference module. The preprocessed image is input into the small model for inference to obtain the output of the small model. The output of the small model is converted to the corresponding size through the post-processing module so that the detected target coordinates correspond to the original image pixel size. The detected target information results and detection confidence are integrated into a unified protocol through the output protocol module and provided to the large model prediction module.

[0007] The target information results and detection confidence of the small model are received through the large model prediction module, and analysis and interpretation are performed based on context learning and combined with the processing capabilities of the large model to form the final interpretation and results and return them to the http server for visual display.

[0008] Furthermore, the training of the small model in the reasoning module of the design method of the image processing system combining a large model and a small model comprises the following steps:

[0009] Data preparation phase: According to the specific task, relevant images are collected from different sources, and then the data is annotated, cleaned, and the data diversity is enhanced. Different data sets are divided. The data sets include training sets, validation sets, and test sets. Data preparation is completed.

[0010] Model selection and training: Select the appropriate model architecture according to the actual situation, build a small model, configure training parameters, load the prepared data, and train to obtain a small model.

[0011] Furthermore, the design method of the image processing system combining a large model and a small model also includes:

[0012] Evaluate the small model: Use the validation set data to evaluate the performance of the model.

[0013] Small model optimization: Use hyperparameter tuning, model fine-tuning, and regularization to prevent overfitting of the small model, and use the test set data to perform the final performance evaluation of the small model.

[0014] Small model standardization: Export the model to the unified ONNX format for image reasoning.

[0015] Furthermore, the design method of an image processing system combining a large model and a small model encapsulates the small model processing module into a plug-in, so that the small model processing module interacts with the large model prediction module as an independent module, and defines a unified plug-in interface, including input and output format protocols and interfaces, to realize the interaction between the large model prediction module and different small model processing modules.

[0016] The present invention also provides an image processing system combining a large model and a small model, wherein the image processing system comprises a small model processing module and a large model prediction module, wherein the small model processing module comprises a preprocessing module, a reasoning module, a post-processing module and an output protocol module.

[0017] The small model processing module decodes the image into a pixel matrix and converts the image format into an RGB format suitable for model input. The preprocessing module normalizes and standardizes the decoded image. The inference module loads the configuration parameters and the small model, inputs the preprocessed image into the small model for inference to obtain the output of the small model. The post-processing module converts the output of the small model into the corresponding size so that the detected target coordinates correspond to the original image pixel size. The output protocol module integrates the detected target information results and detection confidence into a unified protocol and provides it to the large model prediction module.

[0018] The large model prediction module receives the target information results and detection confidence of the small model, analyzes and interprets them based on context learning and the processing capabilities of the large model, and forms the final explanation and results and returns them to the http server for visual display.

[0019] Furthermore, the image processing system combining a large model and a small model further includes a training module, and the training module trains the small model in the reasoning module, including the following steps:

[0020] Data preparation phase: According to the specific task, relevant images are collected from different sources, and then the data is annotated, cleaned, and the data diversity is enhanced. Different data sets are divided. The data sets include training sets, validation sets, and test sets. Data preparation is completed.

[0021] Model selection and training: Select the appropriate model architecture according to the actual situation, build a small model, configure training parameters, load the prepared data, and train to obtain a small model.

[0022] Furthermore, the training module of the image processing system combining a large model and a small model also evaluates the small model: the performance of the model is evaluated using the validation set data,

[0023] Optimize the small model: Use hyperparameter tuning, model fine-tuning, and regularization to prevent the small model from overfitting, and use the test set data to perform the final performance evaluation of the small model.

[0024] Standardized small model: Export the model to the unified ONNX format for image reasoning.

[0025] Furthermore, the image processing system combining a large model and a small model encapsulates the small model processing module into a plug-in, so that the small model processing module interacts with the large model prediction module as an independent module, and defines a unified plug-in interface, including input and output format protocols and interfaces, to realize the interaction between the large model prediction module and different small model processing modules.

[0026] The benefits of the present invention are:

[0027] An image processing system based on small models and large models is designed. The image data source to be processed is sent to the small model processing module, and preliminary knowledge and prediction results are obtained through the image decoding module, preprocessing module, reasoning module, and post-processing module of the small model. The large model combines the prediction results and context learning, and outputs the final prediction results and explanations to the http server for visual display, overcoming the limitations of existing technologies, achieving more efficient computing and memory utilization, improving efficiency, and reducing costs. The application update and maintenance of the small model are more flexible and efficient, and can quickly adapt to new data and tasks, enhancing the adaptability and scalability of the system. For specific tasks, the small model is properly optimized and adjusted, combined with the performance of the large model, not only to obtain better performance in supervision tasks, but also to solve the instability problem of context learning, thereby improving the real-time and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0029] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0030] Example 1

[0031] The present invention provides a design method for an image processing system combining a large model and a small model, and establishes an image processing system based on the combination of a large model and a small model. The image processing system includes a small model processing module, a training module and a large model prediction module, and the small model processing module includes a preprocessing module, an inference module, a post-processing module and an output protocol module.

[0032] The training module trains the small model in the inference module, including the following steps:

[0033] Data preparation phase: According to the specific task, relevant images are collected from different sources, and then the data is annotated, cleaned, and the data diversity is enhanced. Different data sets are divided. The data sets include training sets, validation sets, and test sets. Data preparation is completed.

[0034] Model selection and training: Select the appropriate model architecture according to the actual situation, build a small model, configure training parameters, load the prepared data, and train to obtain a small model.

[0035] Evaluate the small model: Use the validation set data to evaluate the performance of the model.

[0036] Small model optimization: Use hyperparameter tuning, model fine-tuning, and regularization to prevent overfitting of the small model, and use the test set data to perform the final performance evaluation of the small model.

[0037] Small model standardization: Export the model to the unified ONNX format for image reasoning.

[0038] The small model processing module decodes the image into a pixel matrix and converts the image format into an RGB format suitable for model input. The decoded image is normalized and standardized through the preprocessing module. The configuration parameters and the small model are loaded through the inference module. The preprocessed image is input into the small model for inference to obtain the output of the small model. The output of the small model is converted to the corresponding size through the post-processing module so that the detected target coordinates correspond to the pixel size of the original image. The detected target information results and detection confidence are integrated into a unified protocol through the output protocol module and provided to the large model prediction module.

[0039] The small model processing module can be further encapsulated into a plug-in so that the small model processing module can interact with the large model prediction module as an independent module, and a unified plug-in interface can be defined, including input and output format protocols and interfaces, to realize the interaction between the large model prediction module and different small model processing modules.

[0040] The target information results and detection confidence of the small model are received through the large model prediction module, and analysis and interpretation are performed based on context learning and combined with the processing capabilities of the large model to form the final interpretation and results and return them to the http server for visual display.

[0041] Example 2

[0042] The present invention also provides an image processing system combining a large model and a small model, wherein the image processing system comprises a small model processing module and a large model prediction module, wherein the small model processing module comprises a preprocessing module, a reasoning module, a post-processing module and an output protocol module.

[0043] The small model processing module decodes the image into a pixel matrix and converts the image format into an RGB format suitable for model input. The preprocessing module normalizes and standardizes the decoded image. The inference module loads the configuration parameters and the small model, inputs the preprocessed image into the small model for inference to obtain the output of the small model. The post-processing module converts the output of the small model into the corresponding size so that the detected target coordinates correspond to the original image pixel size. The output protocol module integrates the detected target information results and detection confidence into a unified protocol and provides it to the large model prediction module.

[0044] The large model prediction module receives the target information results and detection confidence of the small model, analyzes and interprets them based on context learning and the processing capabilities of the large model, and forms the final explanation and results and returns them to the http server for visual display.

[0045] As the information interaction and execution process between the modules in the above-mentioned system are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0046] Similarly, the present invention is based on the image processing system of the small model and the large model, and the image data source to be processed is sent to the small model processing module, and the preliminary knowledge and prediction results are obtained through the image decoding module, preprocessing module, reasoning module, and post-processing module of the small model. The large model combines the prediction results and context learning, outputs the final prediction results and explanations to the http server for visual display, overcomes the limitations of the prior art, achieves more efficient calculation and memory utilization, improves efficiency, and reduces costs. The application update and maintenance of the small model are more flexible and efficient, and can quickly adapt to new data and tasks, enhancing the adaptability and scalability of the system. For specific tasks, the small model is properly optimized and adjusted, combined with the performance of the large model, not only to obtain better performance in supervision tasks, but also to solve the instability problem of context learning, thereby improving the real-time and reliability of the system.

[0047] It should be noted that not all steps and modules in the above-mentioned processes and system structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0048] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A design method for an image processing system combining a large model and a small model, characterized in that An image processing system based on the combination of a large model and a small model is established. The image processing system includes a small model processing module and a large model prediction module. The small model processing module includes a preprocessing module, an inference module, a post-processing module and an output protocol module. The small model processing module decodes the image into a pixel matrix and converts the image format into an RGB format suitable for model input. The decoded image is normalized and standardized through the preprocessing module. The configuration parameters and small model are loaded through the inference module. The preprocessed image is input into the small model for inference to obtain the output of the small model. The output of the small model is converted to the corresponding size through the post-processing module so that the detected target coordinates correspond to the original image pixel size. The detected target information results and detection confidence are integrated into a unified protocol through the output protocol module and provided to the large model prediction module. The target information results and detection confidence of the small model are received through the large model prediction module, and analysis and interpretation are performed based on context learning and combined with the processing capabilities of the large model to form the final interpretation and results and return them to the http server for visual display.

2. The design method of an image processing system combining a large model and a small model according to claim 1, characterized in that Training the small model in the inference module includes the following steps: Data preparation phase: According to the specific task, relevant images are collected from different sources, and then the data is annotated, cleaned, and the data diversity is enhanced. Different data sets are divided. The data sets include training sets, validation sets, and test sets. Data preparation is completed. Model selection and training: Select the appropriate model architecture according to the actual situation, build a small model, configure training parameters, load the prepared data, and train to obtain a small model.

3. The design method of an image processing system combining a large model and a small model according to claim 1, characterized in that Also includes: Evaluate the small model: Use the validation set data to evaluate the performance of the model. Small model optimization: Use hyperparameter tuning, model fine-tuning, and regularization to prevent overfitting of the small model, and use the test set data to perform the final performance evaluation of the small model. Small model standardization: Export the model to the unified ONNX format for image reasoning.

4. The method for designing an image processing system combining a large model and a small model according to claim 1, characterized in that The small model processing module is encapsulated into a plug-in so that the small model processing module can interact with the large model prediction module as an independent module. A unified plug-in interface is defined, including input and output format protocols and interfaces, to realize the interaction between the large model prediction module and different small model processing modules.

5. An image processing system combining a large model and a small model, characterized in that The image processing system includes a small model processing module and a large model prediction module. The small model processing module includes a preprocessing module, an inference module, a post-processing module and an output protocol module. The small model processing module decodes the image into a pixel matrix and converts the image format into an RGB format suitable for model input. The preprocessing module normalizes and standardizes the decoded image. The inference module loads the configuration parameters and the small model, inputs the preprocessed image into the small model for inference to obtain the output of the small model. The post-processing module converts the output of the small model into the corresponding size so that the detected target coordinates correspond to the original image pixel size. The output protocol module integrates the detected target information results and detection confidence into a unified protocol and provides it to the large model prediction module. The large model prediction module receives the target information results and detection confidence of the small model, analyzes and interprets them based on context learning and the processing capabilities of the large model, and forms the final explanation and results and returns them to the http server for visual display.

6. The image processing system according to claim 5, characterized in that: It also includes a training module, which trains the small model in the inference module, including the following steps: Data preparation phase: According to the specific task, relevant images are collected from different sources, and then the data is annotated, cleaned, and the data diversity is enhanced. Different data sets are divided. The data sets include training sets, validation sets, and test sets. Data preparation is completed. Model selection and training: Select the appropriate model architecture according to the actual situation, build a small model, configure training parameters, load the prepared data, and train to obtain a small model.

7. The image processing system according to claim 5, characterized in that: The training module also evaluates the small model: the performance of the model is evaluated using the validation set data. Optimize the small model: Use hyperparameter tuning, model fine-tuning, and regularization to prevent the small model from overfitting, and use the test set data to perform the final performance evaluation of the small model. Standardized small model: Export the model to the unified ONNX format for image reasoning.

8. The image processing system according to claim 5, characterized in that: The small model processing module is encapsulated into a plug-in so that the small model processing module can interact with the large model prediction module as an independent module. A unified plug-in interface is defined, including input and output format protocols and interfaces, to realize the interaction between the large model prediction module and different small model processing modules.

Images