An image separation and recognition system based on computer vision model

Abstract

The present invention relates to the field of image processing, and in particular to an image separation and recognition system based on a computer vision model, comprising an image receiving module, an image recognition module, a model scheduling module and a recognition result output module; the image receiving module is used to receive an image and pre-process the image; the model scheduling comprises a model library, and the model scheduling module is used to schedule a visual model; the image recognition module is used to recognize the image and separate various parts in the image according to the image received by the image receiving module and the model scheduled by the model scheduling module; the recognition result output module is used for the recognition result of the image recognition module; the scheme is conducive to evaluating the reliability of different visual models by calculating the confidence indicators of different visual models, and is conducive to obtaining a more reliable recognition result by combining the recognition results of different visual models and fusing the images based on this.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image separation and recognition system based on a computer vision model. Background Art

[0002] With the continuous development of computer vision technology and the widespread application of deep learning algorithms, image separation and recognition systems have become one of the important tools for solving a variety of practical problems. This system uses computer vision models to analyze and process images, separate and identify different objects or features in the image, and provides efficient and accurate solutions for various industries.

[0003] For example, the prior art of CN117037042A discloses a method and related device for automatic numbering of photovoltaic strings based on a visual recognition model, wherein the method comprises: obtaining an image to be numbered, wherein the image to be numbered comprises at least one photovoltaic string; partitioning the image to be numbered to obtain at least one partitioned image; performing background separation on each of the partitioned images to obtain background contours of the partitions and photovoltaic string contours; inputting each of the photovoltaic string contours into a photovoltaic string visual recognition model to obtain segmentation results of each photovoltaic panel; and numbering the photovoltaic panels in the image to be numbered based on the segmentation results of each photovoltaic panel.

[0004] Another typical example is an image detection model and method applied in the field of industrial visual inspection, as disclosed in the prior art of CN114821200B. The image detection model includes a first feature extractor and a first label classifier obtained by using a preset image detection training model according to a preset training method; the preset image detection training model includes a second feature extractor, a gradient flipping layer, a second classifier, a first domain discriminator, and a second domain discriminator. Among them, the second feature extractor, the gradient flipping layer, and the second classifier constitute an adversarial module for separating known and unknown class images in the target domain; the second classifier, the third classifier, the first domain discriminator, and the second domain discriminator constitute a weighted module for adding weights to the target domain data to promote data separation and avoid negative transfer of the image detection model.

[0005] Let's look at a method for extracting image contours based on a visual model, such as the prior art disclosed in CN110378920A. The method includes: firstly, the color channels of the input RGB image are separated and processed by color single antagonism, and the grayscale channel is processed by local standard deviation to obtain three types of images, and then the three types of images are input into the visual cortex model, and these images are processed by cell models step by step, which includes the direction and position of the cells, and finally the images of several directions and positions are pooled to obtain the final image contour.

[0006] The existing image separation and recognition systems generally only use a single visual model for image recognition, resulting in large recognition errors. The present invention is made to solve the common problems in the field. Summary of the invention

[0007] The purpose of the present invention is to propose an image separation and recognition system based on a computer vision model in view of the current shortcomings.

[0008] In order to overcome the shortcomings of the prior art, the present invention adopts the following technical solutions:

[0009] An image separation and recognition system based on a computer vision model comprises an image receiving module, an image recognition module, a model scheduling module and a recognition result output module; the image receiving module is used to receive an image and pre-process the image; the model scheduling comprises a model library and the model scheduling module is used to schedule a visual model; the image recognition module is used to recognize the image and separate the parts in the image according to the image received by the image receiving module and the model scheduled by the model scheduling module; the recognition result output module is used for the recognition result of the image recognition module;

[0010] The image recognition module includes a confidence index calculation unit, which is used to calculate the confidence index of the recognition results of each visual model. The confidence index is used to characterize the accuracy of the recognition results of each visual model. The larger the confidence index, the higher the accuracy of the recognition results of the visual model.

[0011] Furthermore, the image receiving module includes an image receiving unit and a preprocessing unit, the image receiving unit is used to receive images sent by users, and the preprocessing unit is used to preprocess the images received by the image receiving unit and send the preprocessed input images to the image recognition module, and the preprocessing includes image compression and image noise reduction.

[0012] Furthermore, the image recognition module also includes a scheduling signal sending unit, an image recognition unit, an image fusion unit and a similarity calculation unit; the scheduling signal sending module is used to send a scheduling signal to the model scheduling module, the image recognition unit is used to recognize the input image according to the model scheduled by the model scheduling module, the image fusion unit is used to perform image fusion on the recognition result of the visual model, and the similarity calculation unit is used to calculate the similarity between each image in the image recognition process and the image fusion process.

[0013] Furthermore, the recognition result output module includes a numbering unit, a packaging unit and an image output unit. The numbering unit is used to number the images contained in the recognition results of the image recognition module, the packaging unit is used to package the numbered images, and the image output unit is used to output the packaged images.

[0014] Furthermore, the model scheduling module includes a model library, a signal recognition unit and a scheduling unit. The model library is used to store various computer vision models. The model library includes two or more sub-model libraries, and different sub-model libraries are used to store different categories of visual models. The signal recognition unit is used to identify the scheduling signal sent by the image recognition module and command the scheduling unit to schedule according to the scheduling signal. The scheduling unit is used to schedule the model from the model library to the image recognition module according to the command of the signal recognition unit.

[0015] Furthermore, the workflow of the image separation and recognition system includes the following steps:

[0016] S1, the image receiving module receives the image sent by the user and sends the pre-processed input image to the image recognition module;

[0017] S2, the image recognition module sends a scheduling signal to the model scheduling unit, and the model scheduling unit schedules the model for use by the image recognition module;

[0018] S3, the image recognition module processes the input image according to the visual model to obtain the recognition result;

[0019] S4, the recognition result output unit sends the recognition result of the image recognition module to the user.

[0020] Furthermore, the image recognition module processes the input image including the following steps:

[0021] S31, using the scheduled visual model to recognize the input image, and obtain the recognition results of each model, wherein the recognition results include a primary image and a secondary image;

[0022] S32, calculating the confidence index of the recognition result of each visual model according to the past recognition accuracy index of each visual model and the preliminary recognition result;

[0023] S33, fusing each main image according to the confidence index to obtain a fused image;

[0024] S34, respectively comparing the image similarity between the fused image and the main image corresponding to the model with the largest confidence index and the input image;

[0025] S35, selecting an image with a greater similarity to the corresponding image from the main image corresponding to the fused image and the model with the largest confidence index, and allowing the model with the largest confidence index to recognize the image to obtain a final recognition result.

[0026] Furthermore, the fusion of each main image according to the confidence index includes the following steps:

[0027] S331, creating an image template of the same size according to the size of the input image;

[0028] S332, calculating the gray value of each pixel in the image template according to the confidence index;

[0029] S333, filling the image template according to the calculation result to obtain a fused image;

[0030] S334, denoising the fused image.

[0031] The beneficial effects achieved by the present invention are: 1. By calculating the confidence indicators of different visual models, it is helpful to evaluate the reliability of different visual models. By combining the recognition results of different visual models and fusing the images based on this, it is helpful to obtain more reliable recognition results.

[0032] 2. By comparing the image similarity between the fused image and the main image corresponding to the model with the largest confidence index and the input image, it is helpful to determine whether the fused image is more reliable than the image before fusion. If the image similarity between the fused image and the input image is higher, it proves that the fusion is effective, which is conducive to obtaining more reliable recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate the same parts.

[0034] Figure 1 It is a schematic diagram of the structure of the present invention.

[0035] Figure 2 It is the work flow chart of the present invention.

[0036] Figure 3 The present invention is a flowchart of the image recognition module processing an input image.

[0037] Figure 4 The figure is a flow chart of fusing each main image in the present invention. DETAILED DESCRIPTION

[0038] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0039] Embodiment 1: According to Figure 1 , Figure 2 , Figure 3 and Figure 4 This embodiment provides an image separation and recognition system based on a computer vision model, including an image receiving module, an image recognition module, a model scheduling module and a recognition result output module; the image receiving module is used to receive an image and pre-process the image; the model scheduling includes a model library, and the model scheduling module is used to schedule a visual model; the image recognition module is used to recognize the image and separate the parts in the image according to the image received by the image receiving module and the model scheduled by the model scheduling module; the recognition result output module is used for the recognition result of the image recognition module;

[0040] The image recognition module includes a confidence index calculation unit, which is used to calculate the confidence index of the recognition results of each visual model. The confidence index is used to characterize the accuracy of the recognition results of each visual model. The larger the confidence index, the higher the accuracy of the recognition results of the visual model.

[0041] Furthermore, the image receiving module includes an image receiving unit and a preprocessing unit, the image receiving unit is used to receive images sent by users, and the preprocessing unit is used to preprocess the images received by the image receiving unit and send the preprocessed input images to the image recognition module, and the preprocessing includes image compression and image noise reduction.

[0042] Furthermore, the image recognition module also includes a scheduling signal sending unit, an image recognition unit, an image fusion unit and a similarity calculation unit; the scheduling signal sending module is used to send a scheduling signal to the model scheduling module, the image recognition unit is used to recognize the input image according to the model scheduled by the model scheduling module, the image fusion unit is used to perform image fusion on the recognition result of the visual model, and the similarity calculation unit is used to calculate the similarity between each image in the image recognition process and the image fusion process.

[0043] Specifically, the recognition result of the visual model includes a main image and multiple secondary images. The main image is an image obtained by recognizing and processing various parts of the input image, and the secondary image is a result of separating and processing various parts of the main image. Both the main image and the secondary image are grayscale images.

[0044] Furthermore, the recognition result output module includes a numbering unit, a packaging unit and an image output unit. The numbering unit is used to number the images contained in the recognition results of the image recognition module, the packaging unit is used to package the numbered images, and the image output unit is used to output the packaged images.

[0045] Furthermore, the model scheduling module includes a model library, a signal recognition unit and a scheduling unit. The model library is used to store various computer vision models. The model library includes two or more sub-model libraries, and different sub-model libraries are used to store different categories of visual models. The signal recognition unit is used to identify the scheduling signal sent by the image recognition module and command the scheduling unit to schedule according to the scheduling signal. The scheduling unit is used to schedule the model from the model library to the image recognition module according to the command of the signal recognition unit.

[0046] Specifically, the visual model includes but is not limited to convolutional neural network (CNN), generative adversarial network (GAN), recurrent neural network (RNN), YOLO series, RCNN series, etc.

[0047] Furthermore, the workflow of the image separation and recognition system includes the following steps:

[0048] S1, the image receiving module receives the image sent by the user and sends the pre-processed input image to the image recognition module;

[0049] S2, the image recognition module sends a scheduling signal to the model scheduling unit, and the model scheduling unit schedules the model for use by the image recognition module;

[0050] S3, the image recognition module processes the input image according to the visual model to obtain the recognition result;

[0051] S4, the recognition result output unit sends the recognition result of the image recognition module to the user.

[0052] Furthermore, the image recognition module processes the input image including the following steps:

[0053] S31, using the scheduled visual model to recognize the input image, and obtain the recognition results of each model, wherein the recognition results include a primary image and a secondary image;

[0054] S32, calculating the confidence index of the recognition result of each visual model according to the past recognition accuracy index of each visual model and the preliminary recognition result;

[0055] The confidence index can be calculated by the following formula:

[0056] ;

[0057] in, is the confidence index, e is the natural constant, is the recognition accuracy weight, To identify the accuracy index, is the image difference weight, num is the number of pixels contained in the main image, is the average gray value of each pixel in the main image, is the average gray value of each pixel in the input image, I is the number of secondary images in the recognition result, is the number of pixels contained in the ith secondary image, The average gray value of each pixel in the ith secondary image, is the average gray value of each pixel at the corresponding position of the i-th secondary image in the input image;

[0058] Specifically, for each visual model, there is only one primary image obtained from the recognition result, but there are two or more secondary images. Since different visual models obtain different recognition results, the number of corresponding secondary images is also different;

[0059] Specifically, the result obtained by combining the secondary images in the recognition result is the primary image, the number of pixels of the primary image is the same as the number of pixels of the input image, and the positions of the pixels correspond one to one, so each secondary image has a corresponding position in the input image;

[0060] Specifically, It can be obtained by the following method: input multiple sample images into the corresponding visual model, and the technicians in this field judge whether the main image of the corresponding recognition result is correct, and calculate the corresponding accuracy, and use the accuracy as The value of

[0061] Specifically, different visual models correspond to and The value of is different. The value of can be the reciprocal of the difference between the publication date of the corresponding model and the current date. The larger the difference, the older the model. The value of can be I. The larger the I is, the more secondary images there are, the more samples are used to calculate the confidence index, the more information the secondary images can obtain, and the greater the image gap weight.

[0062] S33, fusing each main image according to the confidence index to obtain a fused image;

[0063] S34, respectively comparing the image similarity between the fused image and the main image corresponding to the model with the largest confidence index and the input image;

[0064] The similarity of the images can be obtained by calculating cosine similarity or Euclidean distance, etc. The above methods belong to the prior art and will not be described in detail here;

[0065] S35, selecting an image with a greater similarity to the corresponding image from the main image corresponding to the fused image and the model with the largest confidence index, and allowing the model with the largest confidence index to recognize the image to obtain a final recognition result.

[0066] Furthermore, the fusion of each main image according to the confidence index includes the following steps:

[0067] S331, creating an image template of the same size according to the size of the input image;

[0068] S332, calculating the gray value of each pixel in the image template according to the confidence index;

[0069] The grayscale value of each pixel in the image template can be calculated by the following formula:

[0070] ;

[0071] in, The coordinates in the image template are The gray value of the pixel, J is the number of visual models used, is the sum of all confidence indicators, is the confidence index corresponding to the j-th visual model, The coordinates in the main image corresponding to the jth visual model are The gray value of the pixel;

[0072] S333, filling the image template according to the calculation result to obtain a fused image;

[0073] S334, denoising the fused image.

[0074] The beneficial effects of this solution are as follows: 1. By calculating the confidence indicators of different visual models, it is helpful to evaluate the reliability of different visual models. By combining the recognition results of different visual models and fusing the images based on this, it is helpful to obtain more reliable recognition results.

[0075] 2. By comparing the image similarity between the fused image and the main image corresponding to the model with the largest confidence index and the input image, it is helpful to determine whether the fused image is more reliable than the image before fusion. If the image similarity between the fused image and the input image is higher, it proves that the fusion is effective, which is conducive to obtaining more reliable recognition results.

[0076] Embodiment 2: This embodiment should be understood to include all the features of any of the above embodiments and further improve upon them. , and It can be obtained in the following ways:

[0077] for , which can be obtained by the following formula:

[0078] ;

[0079] in, is the difference between the publication date of the visual model and the current date, is the basic weight of the visual model. The basic weight is ranked by those skilled in the art according to past usage results and is set between 0 and 1. For example, the effect of the visual model using YOLOv1 is generally not as good as that of the visual model using YOLOv2. If the stan corresponding to YOLOv1 is set to 0.4, the stan corresponding to YOLOv2 can be set to 0.5;

[0080] for , which can be obtained by the following formula:

[0081] ;

[0082] Wherein, M is the number of example images used to calculate the accuracy, and N is the number of regions divided by the ideal main image of the example image, and the ideal main image is manually divided by those skilled in the art; is the number of regions where the ideal main image of the mth example image differs from the actual main image, where the actual main image is obtained by the visual model; is the number of pixels of the mth example image; is the number of pixels that the nth region of the ideal main image of the mth example image differs from the region that is most similar in shape in the actual main image;

[0083] for , which can be obtained by the following formula:

[0084] ;

[0085] Where I is the number of secondary images in the recognition result, is the average grayscale value of the reference image, where the reference image is an image in which the grayscale values ​​of all pixels are 122, It is the average gray value of the pixels of the main image after the reference image is recognized by the visual model.

[0086] The beneficial effects of this embodiment are as follows: by taking multiple elements into consideration to obtain weight values, it is beneficial to improve the comprehensiveness of the weight values; by comparing the ideal main image of the example image with the actual main image to obtain the recognition accuracy index, it is beneficial for the accuracy index to more accurately characterize the accuracy of the visual model.

[0087] The above disclosed contents are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made by using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated with the development of technology. The above units are only examples. Those skilled in the art can use corresponding units according to different designs according to actual needs when implementing this solution.

Claims

1. An image separation and recognition system based on a computer vision model, characterized in that: It includes an image receiving module, an image recognition module, a model scheduling module and a recognition result output module; the image receiving module is used to receive an image and pre-process the image; the model scheduling includes a model library and the model scheduling module is used to schedule a visual model; the image recognition module is used to recognize the image and separate the parts in the image according to the image received by the image receiving module and the model scheduled by the model scheduling module; the recognition result output module is used for the recognition result of the image recognition module; The image recognition module includes a confidence index calculation unit, which is used to calculate the confidence index of the recognition result of each visual model, and the confidence index is used to characterize the accuracy of the recognition result of each visual model. The larger the confidence index, the higher the accuracy of the recognition result of the visual model; The image recognition module processes the input image and includes the following steps: S31, using the scheduled visual model to recognize the input image, and obtain the recognition results of each model, wherein the recognition results include a primary image and a secondary image; S32, calculating the confidence index of the recognition result of each visual model according to the past recognition accuracy index of each visual model and the preliminary recognition result; S33, fusing each main image according to the confidence index to obtain a fused image; S34, respectively comparing the image similarity between the fused image and the main image corresponding to the model with the largest confidence index and the input image; S35, selecting an image with a greater similarity to the corresponding image from the main image corresponding to the fused image and the model with the largest confidence index, and allowing the model with the largest confidence index to recognize the image to obtain a final recognition result; The main image is an image obtained by identifying and processing various parts of the input image, and the secondary image is a result of separating and processing various parts of the main image. Both the main image and the secondary image are grayscale images.

2. The image separation and recognition system based on computer vision model according to claim 1, characterized in that: The image receiving module includes an image receiving unit and a preprocessing unit. The image receiving unit is used to receive an image sent by a user. The preprocessing unit is used to preprocess the image received by the image receiving unit and send the preprocessed input image to the image recognition module. The preprocessing includes image compression and image noise reduction.

3. The image separation and recognition system based on computer vision model according to claim 2, characterized in that: The image recognition module also includes a scheduling signal sending unit, an image recognition unit, an image fusion unit and a similarity calculation unit; the scheduling signal sending module is used to send a scheduling signal to the model scheduling module, the image recognition unit is used to recognize the input image according to the model scheduled by the model scheduling module, the image fusion unit is used to perform image fusion on the recognition result of the visual model, and the similarity calculation unit is used to calculate the similarity between each image in the image recognition process and the image fusion process.

4. The image separation and recognition system based on computer vision model according to claim 3, characterized in that: The recognition result output module includes a numbering unit, a packaging unit and an image output unit. The numbering unit is used to number the images included in the recognition results of the image recognition module, the packaging unit is used to package the numbered images, and the image output unit is used to output the packaged images.

5. The image separation and recognition system based on computer vision model according to claim 4, characterized in that: The model scheduling module includes a model library, a signal recognition unit and a scheduling unit. The model library is used to store various computer vision models. The model library includes two or more sub-model libraries, and different sub-model libraries are used to store different categories of visual models. The signal recognition unit is used to identify the scheduling signal sent by the image recognition module and command the scheduling unit to schedule according to the scheduling signal. The scheduling unit is used to schedule the model from the model library to the image recognition module according to the command of the signal recognition unit.

6. The image separation and recognition system based on computer vision model according to claim 5, characterized in that: The workflow of the image separation and recognition system includes the following steps: S1, the image receiving module receives the image sent by the user and sends the pre-processed input image to the image recognition module; S2, the image recognition module sends a scheduling signal to the model scheduling unit, and the model scheduling unit schedules the model for use by the image recognition module; S3, the image recognition module processes the input image according to the visual model to obtain the recognition result; S4, the recognition result output unit sends the recognition result of the image recognition module to the user.

7. The image separation and recognition system based on computer vision model according to claim 6, characterized in that: The fusion of each main image according to the confidence index includes the following steps: S331, creating an image template of the same size according to the size of the input image; S332, calculating the gray value of each pixel in the image template according to the confidence index; S333, filling the image template according to the calculation result to obtain a fused image; S334, denoising the fused image.

Images