Image recognition processing method and system, electronic equipment and medium
By using deep learning models for dejitter and segmentation processing in image recognition processing, the problems of inefficient image processing and insufficient accuracy are solved, and more efficient and more accurate detection of tobacco proportion is achieved.
Patent Information
- Application Number
- CN202311335868.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-16
- Publication Date
- 2025-04-18
AI Technical Summary
In the prior art, there are problems such as low image processing efficiency and low processing result accuracy.
By acquiring the original image of the tobacco, the first deep learning model is used for debounce processing, and then the original segmentation result and image outline are obtained using the second deep learning model, and finally the segmentation result and proportion of the tobacco are obtained.
Improve image processing efficiency and obtain more accurate results for tobacco proportion.
Smart Images
Figure CN119850939A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and relates to an image recognition processing method, and in particular to an image recognition processing method, system, electronic device and medium. Background Art
[0002] Image recognition processing is an important research direction in the field of computer vision. Its goal is to enable computers to understand and interpret image information like humans. In the past few decades, this field has made significant progress, especially after the introduction of deep learning models. Deep learning is a machine learning method that imitates the working mechanism of the human brain. It enables computers to learn and extract useful information from large amounts of data by building a multi-layer neural network model. In image recognition processing, deep learning models such as convolutional neural networks (CNNs) have become the mainstream method. Through structures such as convolutional layers, pooling layers, and fully connected layers, CNNs can effectively extract local and global features of images, thereby achieving accurate recognition of image content. In addition, with the development of technology, some new deep learning models have also been applied to image recognition processing. For example, generative adversarial networks (GANs) can generate extremely realistic artificial images.
[0003] In general, deep learning models have played an important role in the field of image recognition and processing, and with the continuous advancement of technology, their application prospects are even broader. However, there are still problems such as low image processing efficiency and low processing result accuracy. Summary of the invention
[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide an image recognition processing method, system, electronic device and medium, which are used to solve the problems of low image processing efficiency and low processing result accuracy in the prior art.
[0005] In a first aspect, the present application provides an image recognition and processing method, which includes: obtaining an original image of tobacco shreds; processing the original image using a first deep learning model to obtain a de-jittered image; processing the de-jittered image using a second deep learning model to obtain an original segmentation result and an image contour; obtaining a segmentation result of tobacco shreds using the original segmentation result and the image contour; and obtaining a tobacco shred ratio using the segmentation result.
[0006] In this application, the original image is de-shaked to eliminate the jitter effect on the image, and then the second deep learning model is used to obtain the original segmentation result and the image contour, and then the two are used to obtain the segmentation result, and the tobacco percentage is finally obtained using the segmentation result. This image recognition processing method improves the image processing efficiency and can obtain more accurate tobacco percentage results.
[0007] In an implementation of the first aspect, the original image is processed using a first deep learning model to obtain a de-jittered image, including: extracting features of the original tobacco image using the first deep learning model to obtain features of different scales; performing feature screening on the features of different scales; and obtaining the de-jittered image using the screened features.
[0008] In an implementation of the first aspect, the convolution layer of the first deep learning model outputs the features of different scales and multiple feature maps, and the image recognition processing method further includes: transferring the features of different scales output by the convolution layer to the feature map output by the convolution layer in an upper layer.
[0009] In this implementation, the output of the convolutional layer of the first deep learning model is used as the input of the next convolutional layer, which can screen effective image features during the image feature extraction process and improve the generation quality of the de-shaking image.
[0010] In an implementation of the first aspect, obtaining the de-jittered image using the filtered features also includes: fusing the filtered features with the feature map to obtain a fusion result; performing splicing and interpolation processing based on the fusion result to obtain processed output features; and fusing the output features with the original image of the tobacco to obtain the de-jittered image.
[0011] In an implementation of the first aspect, using a second deep learning model to process the de-jittered image to obtain an original segmentation result and an image contour includes: using a main task of the second deep learning model to process the de-jittered image to obtain an original segmentation result; and using a subtask of the second deep learning model to extract a contour of the de-jittered image to obtain an image contour.
[0012] In this implementation, image contour extraction is used as a subtask of the second deep learning model, which enables the model to learn more edge details and improve image processing accuracy.
[0013] In an implementation manner of the first aspect, using the original segmentation result and the image contour to obtain the segmentation result includes: using the original segmentation result and the image contour to perform fusion processing to obtain the segmentation result.
[0014] In an implementation of the first aspect, obtaining the tobacco percentage using the segmentation result includes: obtaining a segmented binary image based on the segmentation result; extracting features of the segmented binary image using the second deep learning model; and obtaining the tobacco percentage using a fully connected layer of the second deep learning model and features of the segmented binary image.
[0015] In a second aspect, the present application provides an image recognition and processing system, which includes: an image acquisition module for acquiring an original image of tobacco shreds; a de-jittering module for processing the original image using a first deep learning model to obtain a de-jittered image; an image segmentation module for processing the de-jittered image using a second deep learning model to obtain an original segmentation result and an image contour; an image processing module for obtaining a segmentation result of tobacco shreds using the original segmentation result and the image contour; and a result acquisition module for obtaining a tobacco shred proportion using the segmentation result.
[0016] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the electronic device performs the image recognition processing method described in any one of the first aspects.
[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image recognition processing method described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 Shown is a schematic diagram of an application scenario of the tobacco ratio detection device described in the present application.
[0019] Figure 2 Shown is a flowchart of the image recognition processing method described in an embodiment of the present application.
[0020] Figure 3 Shown is a flowchart of the image recognition processing method described in an embodiment of the present application.
[0021] Figure 4 Shown is a schematic diagram of the image recognition processing method described in an embodiment of the present application.
[0022] Figure 5 Shown is a flowchart of the image recognition processing method described in an embodiment of the present application.
[0023] Fig. 6A Shown is a flowchart of the image recognition processing method described in an embodiment of the present application.
[0024] Figure 6B Shown is a schematic diagram of the image recognition processing method described in an embodiment of the present application.
[0025] Fig. 7A Shown is a flowchart of the image recognition processing method described in an embodiment of the present application.
[0026] Figure 7B Shown is a schematic diagram of the image recognition processing method described in an embodiment of the present application.
[0027] Figure 8 Shown is a structural schematic diagram of the image recognition processing system described in an embodiment of the present application.
[0028] Fig. 9 Shown is a schematic diagram of the structure of an electronic device described in an embodiment of the present application.
[0029] Component number description
[0030] 1. Tobacco percentage detection device
[0031] 11. Camera
[0032] 12. Vibration tank proportion detection device
[0033] 13 Vibration trough
[0034] 800 Image Recognition Processing System
[0035] 810 Image acquisition module
[0036] 820 De-jitter module
[0037] 830 Image Segmentation Module
[0038] 840 Image Processing Module
[0039] 850 Result Acquisition Module
[0040] 900 Electronic equipment
[0041] 910 Memory
[0042] 920 processor
[0043] 930 Display
[0044] Steps S11 to S15
[0045] Steps S121 to S123
[0046] Steps S1231 to S1233
[0047] Steps S131 to S132
[0048] Steps S151 to S153 DETAILED DESCRIPTION
[0049] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0050] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0051] In the cigarette production process, tobacco making is one of the most important links. In order to ensure the quality of tobacco, complex procedures are used to process tobacco in the making stage. In order to avoid abnormal situations in the production process, a special manual inspection will be set up in the workshop to check whether the current production line is in normal operation. However, the manual inspection method cannot guarantee real-time performance, and repeated inspections lead to a waste of scarce human resources in the workshop. Artificial intelligence technology replaces the traditional manual inspection method. By analyzing the production images collected in the production workshop, real-time alarms for abnormal situations on the production line are realized. In the making stage, the production line needs to count the amount of tobacco passing through the vibration trough in real time, and the jitter of the vibration trough affects the image quality collected by the camera, resulting in a low accuracy of the final tobacco percentage.
[0052] At least to address the above-mentioned problems, an embodiment of the present application provides an image recognition and processing method, which includes: obtaining an original image of tobacco shreds; processing the original image using a first deep learning model to obtain a de-jittered image; processing the de-jittered image using a second deep learning model to obtain an original segmentation result and an image contour; obtaining a segmentation result of tobacco shreds using the original segmentation result and the image contour; and obtaining a tobacco shred proportion using the segmentation result.
[0053] In the embodiment of the present application, the original image is de-jittered to eliminate the jitter effect on the image, and then the second deep learning model is used to obtain the original segmentation result and the image contour, and then the two are used to obtain the segmentation result, and the segmentation result is used to finally obtain the tobacco percentage. This image recognition processing method improves the processing efficiency of tobacco images and can obtain more accurate tobacco percentage results.
[0054] Figure 1 The diagram shows an application scenario of the tobacco percentage detection device described in the present application. The tobacco percentage detection device 1 can be used to implement the image recognition processing method provided in the embodiment of the present application, but the application scenario of the image recognition processing method provided in the embodiment of the present application is not limited to Figure 1 The vibration tank proportion detection device 1 shown. Figure 1 As shown, the tobacco percentage detection device 1 includes a camera 11, a vibration slot percentage detection device 12 and a vibration slot 13. The image recognition processing method provided in the embodiment of the present application can be applied to the vibration slot percentage detection device 12.
[0055] in, Figure 1 The vibration tank proportion detection device 12 can be a processor cluster or a cloud computing center composed of a single processor or multiple processors, and the specifics are not limited here. Figure 1 Only one camera 11, one vibration slot ratio detection device 12 and one vibration slot 13 are shown, but it should be understood that Figure 1 The examples are only used to understand this solution, and the specific number of vibration slots and vibration slot body ratio detection devices should be flexibly determined based on actual conditions.
[0056] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.
[0057] The following embodiments of the present application provide an image recognition processing method, for example, Figure 1 It is realized by the vibration slot proportion detection device 12 shown. Figure 2 The flowchart of the image recognition processing method described in the embodiment of the present application is shown as follows: Figure 2 As shown, the image recognition processing method includes steps S11 to S15.
[0058] Step S11, obtaining an original image of the tobacco shreds. The original image of the tobacco shreds is an image of the tobacco shreds in the vibration tank. Specifically, a camera is installed above the vibration tank to collect the original image of the tobacco shreds.
[0059] In some possible implementations, camera parameters, including shutter speed, aperture, gain, etc., are set according to the lighting conditions and jitter effects of the installation position of the camera above the vibration tank to obtain the original image of the tobacco.
[0060] Step S12, using the first deep learning model to process the original image to obtain a de-shaking image, wherein the de-shaking image is an image obtained by processing the original image using the first deep learning model to eliminate the effect of the vibration groove shaking.
[0061] Step S13: Process the de-shaking image using a second deep learning model to obtain an original segmentation result and an image contour.
[0062] Step S14, obtaining a segmentation result of the tobacco shreds by using the original segmentation result and the image contour.
[0063] Step S15, using the segmentation result to obtain the tobacco shred proportion, wherein the tobacco shred proportion is the proportion of tobacco shreds in the vibration tank body.
[0064] In the embodiment of the present application, the original image is de-jittered to eliminate the jitter effect on the image, and then the second deep learning model is used to obtain the original segmentation result and the image contour, and then the two are used to obtain the segmentation result, and the tobacco percentage is finally obtained using the segmentation result. This image recognition processing method improves the image processing efficiency and can obtain a more accurate tobacco percentage result.
[0065] Figure 3 The flowchart of the image recognition processing method described in the embodiment of the present application is shown. Figure 3 As shown, the step S12 includes the following steps S121 to S123.
[0066] Step S121, using the first deep learning model to extract features of the original image of the shredded tobacco to obtain features of different scales. The first deep learning model is an image generation model based on a convolutional neural network.
[0067] Step S122: performing feature screening on the features of different scales.
[0068] In some possible implementations, the extracted semantic features c of different scales are grouped into i To the upsampled feature maps u of the same size respectively i The feature map is filtered in spatial dimensions based on the semantic features of different scales c i and u i-1 Generate a feature heat map and compare the feature heat map with the feature map u i The above is only a possible implementation method of the embodiment of the present application, and the present application is not limited thereto.
[0069] Step S123, using the filtered features to obtain the de-shaking image.
[0070] Figure 4 The schematic diagram of the image recognition processing method described in the embodiment of the present application is shown as follows: Figure 4As shown, the convolution layer of the first deep learning model outputs the features of different scales and multiple feature maps, and the image processing method further includes: starting from the second convolution layer, passing the features of different scales output by the convolution layer to the feature map output by the convolution layer in the upper layer. Optionally, when the first deep learning model extracts the features of the de-jittered image, the input of the first convolution layer comes from all subsequent convolution layers, the input of the second convolution layer comes from all convolution layers after the second layer, and the output of each layer will also be directly passed to all subsequent convolution layers.
[0071] Figure 5 The flowchart of the image recognition processing method described in the embodiment of the present application is shown. Figure 5 As shown, the step S123 also includes the following steps S1231 to S1233.
[0072] Step S1231, using the filtered features to fuse with the feature map to obtain a fusion result.
[0073] Step S1232, performing splicing and interpolation processing according to the fusion result to obtain processed output features.
[0074] Step S1233: perform fusion processing on the output feature and the original tobacco image to obtain the de-jittered image.
[0075] In some possible implementations, the filtered features are fused with the feature map to obtain a fusion result. The fusion result is spliced at a size of 1 / 4 of the original image, restored to the original image resolution through two layers of interpolation, and a tanh activation function is used to limit the dynamic range after generation. The output features are fused with the normalized original image of the tobacco shreds to obtain the de-jittered image. It should be noted that the above is only a possible implementation of the embodiment of the present application, and the present application is not limited to this.
[0076] Fig. 6A The flowchart of the image recognition processing method described in the embodiment of the present application is shown. Fig. 6A As shown, the step S13 includes the following steps S131 to S132.
[0077] Step S131: Use the main task of the second deep learning model to process the de-shaking image to obtain an original segmentation result.
[0078] Step S132: Use the subtask of the second deep learning model to perform contour extraction on the de-shaking image to obtain the image contour.
[0079] In some possible implementations, Figure 6BThe schematic diagram of the image recognition processing method described in the embodiment of the present application is shown as follows: Figure 6B As shown, the second deep learning model is used to extract the features of the de-jittered image, and based on the extracted image features, deconvolution is used to restore the resolution of the image and output a feature map. The feature map is passed through a softmax function to predict the probability map of each pixel being tobacco, which is used as the original segmentation result.
[0080] The subtask of the second deep learning model is to extract the contour of the input image. According to the feature map obtained by the second deep learning model, the contour map of the input image is obtained by interpolation and convolution to obtain the image contour.
[0081] In an embodiment of the present application, using the original segmentation result and the image contour to obtain the segmentation result includes: using the original segmentation result and the image contour to perform fusion processing to obtain the segmentation result.
[0082] Fig. 7A The flowchart of the image recognition processing method described in the embodiment of the present application is shown. Fig. 7A As shown, the step S15 includes the following steps S151 to S153.
[0083] Step S151, obtaining a segmented binary image according to the segmentation result.
[0084] Step S152: Use the second deep learning model to extract the segmented binary image features.
[0085] Step S153, using the fully connected layer of the second deep learning model and the features of the segmented binary image to obtain the tobacco percentage.
[0086] In some possible implementations, Figure 7B The schematic diagram of the image recognition processing method described in the embodiment of the present application is shown as follows: Figure 7B As shown, according to the segmentation result, a segmented binary image is obtained, and then the features of the segmented binary image are extracted using the second deep learning model, and the tobacco percentage is obtained using the fully connected layer of the second deep learning model.
[0087] Figure 8 The structure diagram of the image recognition processing system described in the embodiment of the present application is shown. Figure 8 As shown, the image recognition processing system 800 includes an image acquisition module 810 , a de-shaking module 820 , an image segmentation module 830 , an image processing module 840 and a result acquisition module 850 .
[0088] The image acquisition module 810 is used to acquire the original image of tobacco.
[0089] The de-jittering module 820 is used to process the original image using a first deep learning model to obtain a de-jittered image.
[0090] The image segmentation module 830 is used to process the de-shaking image using a second deep learning model to obtain an original segmentation result and an image contour.
[0091] The image processing module 840 is used to obtain a segmentation result of the tobacco shreds by using the original segmentation result and the image contour.
[0092] The result acquisition module 850 is used to obtain the tobacco percentage using the segmentation result.
[0093] It should be noted that the modules 810 to 850 included in the image recognition processing system 800 are Figure 2 Steps S11 to S15 in the image recognition processing method are in one-to-one correspondence and are not described in detail here.
[0094] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.
[0095] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.
[0096] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0097] An embodiment of the present application also provides an electronic device. Fig. 9 The structure diagram of the electronic device 900 described in the embodiment of the present application is shown. Fig. 9 As shown, in this embodiment, the electronic device 900 includes a memory 910 and a processor 920 .
[0098] The memory 910 is used to store computer programs; preferably, the memory 910 includes: ROM, RAM, disk, USB flash drive, memory card or optical disk, etc., various media that can store program codes.
[0099] Specifically, the memory 910 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 900 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 910 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.
[0100] The processor 920 is connected to the memory 910 and is used to execute the computer program stored in the memory 910 so that the electronic device 900 executes the image recognition processing method described in any embodiment of the present application.
[0101] Optionally, the processor 920 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
[0102] Optionally, the electronic device 900 in this embodiment may further include a display 930. The display 930 is communicatively connected to the memory 910 and the processor 920, and is used to display a graphical user interface (GUI) interaction interface related to the image recognition processing method described in the embodiment of the present application.
[0103] The embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image recognition processing method described in any embodiment of the present application is implemented.
[0104] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.
[0105] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. An image recognition processing method, characterized in that: include: Obtaining the original image of tobacco; Processing the original image using a first deep learning model to obtain a de-shaking image; Processing the de-shaking image using a second deep learning model to obtain an original segmentation result and an image contour; Using the original segmentation result and the image contour, obtaining a segmentation result of the tobacco shreds; The tobacco shred proportion is obtained using the segmentation result.
2. The image recognition processing method according to claim 1, characterized in that: Processing the original image using the first deep learning model to obtain a de-shaking image includes: Extracting features of the original image of the shredded tobacco using the first deep learning model to obtain features of different scales; Performing feature screening on the features of the different scales; The de-shaking image is obtained by using the filtered features.
3. The image recognition processing method according to claim 2, characterized in that: The convolution layer of the first deep learning model outputs the features of different scales and the plurality of feature maps, and the image recognition processing method further includes: The features of different scales output by the convolutional layer are transferred to the feature map output by the convolutional layer in the upper layer.
4. The image recognition processing method according to claim 2, characterized in that: Using the filtered features, obtaining the de-shaking image includes: Using the filtered features to fuse with the feature map to obtain a fusion result; Performing splicing and interpolation processing according to the fusion result to obtain processed output features; The output feature is used to perform fusion processing with the original tobacco image to obtain the de-jittered image.
5. The image recognition processing method according to claim 1, characterized in that: Processing the de-shaking image using the second deep learning model to obtain the original segmentation result and the image contour includes: Processing the de-shaking image using the main task of the second deep learning model to obtain an original segmentation result; The subtask of the second deep learning model is used to extract the contour of the de-shaking image to obtain the image contour.
6. The image recognition processing method according to claim 1, characterized in that: Using the original segmentation result and the image contour to obtain the segmentation result includes: using the original segmentation result and the image contour to perform fusion processing to obtain the segmentation result.
7. The image recognition processing method according to claim 1, characterized in that: Using the segmentation result to obtain the tobacco percentage includes: According to the segmentation result, obtaining a segmented binary image; Extracting the segmented binary image features using the second deep learning model; The tobacco percentage is obtained by utilizing the fully connected layer of the second deep learning model and the features of the segmented binary image.
8. An image recognition processing system, characterized in that: include: An image acquisition module is used to acquire the original image of the tobacco; A de-jittering module, configured to process the original image using a first deep learning model to obtain a de-jittered image; An image segmentation module, used to process the de-shaking image using a second deep learning model to obtain an original segmentation result and an image contour; An image processing module, used to obtain a segmentation result of the tobacco shreds by using the original segmentation result and the image contour; The result acquisition module is used to obtain the tobacco shred proportion by using the segmentation result.
9. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer programs; A processor, wherein the processor is used to execute the computer program stored in the memory so that the electronic device executes the image recognition processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image recognition processing method according to any one of claims 1 to 7 is implemented.