Image recognition methods, devices, computer-readable media and electronic devices

By extracting and training images using a hybrid approach of semantic and frequency domain information, the problem of decreased accuracy in image recognition models when the scene changes is solved, thereby improving the accuracy of image recognition and the ability to recognize images outside the domain.

CN116993992BActive Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211318878.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-26
Publication Date
2026-11-17
Estimated Expiration
2042-10-26

AI Technical Summary

Technical Problem

Existing image recognition models suffer from decreased accuracy when the scene changes significantly, making it difficult to effectively recognize images outside the domain.

Method used

By extracting semantic and frequency domain information from the first and second sample images, semantic mixed data and frequency domain mixed data are generated and used to train a preset classification model, thereby improving the richness of the model's learning data.

Benefits of technology

It improves the accuracy of image recognition and the ability to recognize images outside the domain, reduces interference information in the image recognition process, and enhances the generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116993992B_ABST
    Figure CN116993992B_ABST
Patent Text Reader

Abstract

This application discloses an image recognition method, apparatus, computer-readable medium, and electronic device. The method includes: extracting semantic information from a first sample image and a second sample image respectively to obtain first semantic information corresponding to the first sample image and second semantic information corresponding to the second sample image; mixing the first semantic information and the second semantic information to obtain semantically mixed data; extracting frequency domain information from the first sample image and the second sample image respectively to obtain first frequency domain information corresponding to the first sample image and second frequency domain information corresponding to the second sample image; mixing the first frequency domain information and the second frequency domain information to obtain frequency domain mixed data; and training a preset classification model based on the semantically mixed data and the frequency domain mixed data to obtain an image recognition model for determining the category of the image to be recognized. The technical solution of this application increases the richness of model training data and improves the accuracy of image recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an image recognition method, apparatus, computer-readable medium, and electronic device. Background Technology

[0002] Image recognition is a technology that uses computers to process, analyze, and understand images to identify targets and objects contained within them. Models developed based on deep learning algorithms are widely used in image recognition. To obtain a model capable of accurate image recognition, it is typically necessary to train the model using a large number of sample images. However, models trained in this way can usually accurately identify images with similar scenes to the sample images. When the scene between the image to be recognized and the sample images differs significantly, the model's accuracy decreases, indicating a need for improvement.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide an image recognition method, apparatus, computer-readable medium, and electronic device to optimize the problem of low image recognition accuracy in related technologies.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0006] According to one aspect of the embodiments of this application, an image recognition method is provided, comprising:

[0007] Semantic information is extracted from the first sample image and the second sample image respectively to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image.

[0008] The first semantic information and the second semantic information are mixed to obtain semantically mixed data;

[0009] Frequency domain information is extracted from the first sample image and the second sample image respectively to obtain the first frequency domain information corresponding to the first sample image and the second frequency domain information corresponding to the second sample image.

[0010] The first frequency domain information and the second frequency domain information are mixed to obtain frequency domain mixed data;

[0011] The preset classification model is trained based on the semantic mixed data and the frequency domain mixed data to obtain an image recognition model, which is used to determine the category of the image to be recognized.

[0012] According to one aspect of the embodiments of this application, an image recognition device is provided, comprising:

[0013] The semantic information extraction module is used to extract semantic information from the first sample image and the second sample image respectively, so as to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image.

[0014] A semantic information mixing module is used to mix the first semantic information and the second semantic information to obtain semantically mixed data;

[0015] The frequency domain information extraction module is used to extract frequency domain information from the first sample image and the second sample image respectively, to obtain the first frequency domain information corresponding to the first sample image and the second frequency domain information corresponding to the second sample image;

[0016] A frequency domain information mixing module is used to mix the first frequency domain information and the second frequency domain information to obtain frequency domain mixed data;

[0017] The model training module is used to train a preset classification model based on the semantic mixed data and the frequency domain mixed data to obtain an image recognition model, which is used to determine the category of the image to be recognized.

[0018] In one embodiment of this application, the first semantic information includes first foreground information and first background information, and the second semantic information includes second foreground information and second background information; the semantic information mixing module is specifically used for:

[0019] The first foreground information and the second foreground information are mixed to obtain mixed foreground information;

[0020] The first background information and the second background information are mixed to obtain mixed background information;

[0021] The mixed foreground information and the mixed background information are combined to obtain the semantic mixed data.

[0022] In one embodiment of this application, the first frequency domain information includes first low-frequency information and first high-frequency information, and the second frequency domain information includes second low-frequency information and second high-frequency information; the frequency domain information mixing module is specifically used for:

[0023] The first low-frequency information and the second low-frequency information are mixed to obtain mixed low-frequency information;

[0024] The first high-frequency information and the second high-frequency information are mixed to obtain mixed high-frequency information;

[0025] The mixed low-frequency information and the mixed high-frequency information are mixed to obtain the target mixed data;

[0026] The target mixed data is subjected to inverse frequency domain transformation to obtain the frequency domain mixed data.

[0027] In one embodiment of this application, the frequency domain information extraction module includes:

[0028] The frequency domain transformation unit is used to perform frequency domain transformation on the first sample image to obtain the frequency domain data of the first sample image;

[0029] An amplitude calculation unit is used to calculate the amplitude of the first sample image based on the frequency domain data of the first sample image;

[0030] The frequency domain information determination unit is used to obtain the first frequency domain information corresponding to the first sample image based on the amplitude of the first sample image.

[0031] In one embodiment of this application, the first frequency domain information includes first low-frequency information and first high-frequency information; the frequency domain information determining unit is specifically used for:

[0032] The real part of the amplitude of the first sample image is taken as the first low-frequency information of the first sample image;

[0033] The imaginary part of the amplitude of the first sample image is used as the first high-frequency information of the first sample image.

[0034] In one embodiment of this application, the model training module includes:

[0035] The label mixing unit is used to mix the first sample category label of the first sample image and the second sample category label of the second sample image to obtain mixed data category labels;

[0036] The model training unit is used to train the preset classification model using the semantic mixed data and the frequency domain mixed data as training data, and the category label of the mixed data as the category label of the training data, to obtain an image recognition model.

[0037] In one embodiment of this application, the label mixing unit is specifically used for:

[0038] The first label data is obtained by linearly combining the first sample category label and the second sample category label using a first weight;

[0039] The second label data is obtained by linearly combining the first sample category label and the second sample category label using a second weight;

[0040] The mixed data category label is obtained by linearly combining the first label data and the second label data using a third weight.

[0041] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image recognition method as described above.

[0042] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions to cause the electronic device to perform the image recognition method as described above.

[0043] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image recognition method as described above.

[0044] In the technical solution provided in this application embodiment, semantic information extraction and frequency domain information extraction are performed on the first sample image and the second sample image respectively, and the extracted first semantic information and second semantic information are mixed to obtain semantic mixed data, and the extracted first frequency domain information and second frequency domain information are mixed to obtain frequency mixed data. This increases the richness of image semantic information and frequency domain information in the model training data, so that the model has more learning data, thereby improving the accuracy of image recognition.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0047] Figure 1An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0048] Figure 2 A flowchart illustrating an embodiment of the image recognition method provided in this application is shown schematically.

[0049] Figure 3 A flowchart illustrating an embodiment of the image recognition method provided in this application is shown schematically.

[0050] Figure 4A A flowchart illustrating the model training phase provided in one embodiment of this application is shown schematically.

[0051] Figure 4B A flowchart illustrating the model testing phase provided in one embodiment of this application is shown schematically.

[0052] Figure 5 A schematic diagram illustrating an image blending process provided in one embodiment of this application is shown.

[0053] Figure 6 A schematic diagram of the hardware framework of the technical solution of this application is shown.

[0054] Figure 7 A schematic block diagram of the image recognition device provided in an embodiment of this application is shown.

[0055] Figure 8 A schematic diagram of a computer system architecture suitable for implementing the embodiments of this application is shown. Detailed Implementation

[0056] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0057] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0058] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0059] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0060] Figure 1 An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0061] like Figure 1 As shown, the system architecture 100 may include terminal devices 110, a network 120, and a server 130. Terminal devices 110 may include smartphones, tablets, laptops, smart voice interaction devices, smart home appliances, in-vehicle terminals, etc. Server 130 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal devices 110 and server 130, such as a wired communication link or a wireless communication link.

[0062] Depending on the implementation requirements, the system architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

[0063] For example, the technical solution of this application is implemented by terminal device 110. On one hand, terminal device 110 extracts semantic information from the first sample image and the second sample image respectively, to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image; the extraction of semantic information of the image is equivalent to semantic segmentation of the image, grouping the pixels in the image that express different semantic meanings. For example, extracting semantic information from the foreground and background of the image can obtain the foreground information, background information, etc. corresponding to the image. Then, terminal device 110 mixes the first semantic information and the second semantic information to obtain semantic mixed data; that is, the semantic information of the two sample images is mixed to generate semantic mixed data, which corresponds to a new image.

[0064] On the other hand, the terminal device 110 extracts frequency domain information from the first sample image and the second sample image respectively, obtaining first frequency domain information corresponding to the first sample image and second frequency domain information corresponding to the second sample image. Frequency domain information extraction refers to transforming the image to the frequency domain and then extracting image features from the frequency domain, such as extracting high-frequency information and low-frequency information. Then, the terminal device 110 mixes the first and second frequency domain information to obtain mixed frequency domain data; that is, it mixes the frequency domain information of the two sample images to generate mixed frequency domain data, which also corresponds to a new image.

[0065] Finally, the terminal device 110 trains the preset classification model based on the semantic mixed data and the frequency domain mixed data to obtain an image recognition model. This image recognition model is used to determine the category of the image to be recognized. In other words, the new image obtained from semantic information extraction and the new image obtained from frequency domain information extraction are used as training data. This training data is then used to train the preset classification model to obtain an image recognition model for identifying image categories. Identifying image categories refers to recognizing the category of the target object contained in the image; for example, image categories include cats, dogs, machinery, buildings, etc.

[0066] The image recognition method provided in this application can effectively improve the accuracy of image recognition. Image recognition is an important part of computer vision (CV) technology. Computer vision is a science that studies how to enable machines to "see," or more specifically, it refers to machine vision that uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and further performs image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0067] The image recognition method provided in this application will be described in detail below with reference to specific embodiments.

[0068] Figure 2 A flowchart illustrating an embodiment of the image recognition method provided in this application is shown. The method includes steps 210 to 250, as detailed below:

[0069] Step 210: Extract semantic information from the first sample image and the second sample image respectively to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image.

[0070] Specifically, semantic information extraction from images can be achieved through semantic segmentation algorithms, such as region-based semantic segmentation methods, fully convolutional network semantic segmentation methods, and weakly supervised or unsupervised semantic segmentation methods.

[0071] In one embodiment of this application, an unsupervised saliency region extraction algorithm S(x) is used to extract and separate the foreground and background information of the image. The semantic information extraction process can be represented by the following equation (1):

[0072] S(x i ) = v i ,σ i (1)

[0073] Where, x i Let v represent the i-th sample image. i Represents the sample image x i Foreground information, σ i Represents the sample image x iBackground information. The extraction result of the unsupervised salient region extraction algorithm S(x) depends only on the predefined contrastive learning paradigm, without the need to introduce additional semantic annotation information during training and learning. That is, there is no need to manually annotate the training samples, thus greatly reducing the training cost.

[0074] Let the first sample image be denoted as sample image x. i Then the first semantic information includes the first foreground information v i and the first background information σ i Let the second sample image be denoted as sample image x. j Then the second semantic information includes the second foreground information v j Second background information ρ j .

[0075] Step 220: Mix the first semantic information and the second semantic information to obtain semantically mixed data.

[0076] Specifically, the mixing of the first and second semantic information can be a linear combination of the first and second semantic information. For example, the first and second semantic information can be multiplied by their respective weights and then added together to obtain the mixed result, which is semantic mixed data. Semantic mixed data is equivalent to the result of mixing a portion of image data extracted from two sample images, and it still constitutes an image. Therefore, semantic mixed data can also be called semantic mixed image.

[0077] In one embodiment of this application, the first semantic information includes first foreground information and first background information, and the second semantic information includes second foreground information and second background information. The mixing of semantic information can be a mixture of any one of the first foreground information and the first background information with any one of the second foreground information and the second background information; alternatively, one of the first semantic information can be mixed with one of the second semantic information to obtain a first semantic mixing result, and the other of the first semantic information can be mixed with the other of the second semantic information to obtain a second semantic mixing result. Then, the first semantic mixing result and the second semantic mixing result are mixed to obtain semantically mixed data.

[0078] Step 230: Extract frequency domain information from the first sample image and the second sample image respectively to obtain the first frequency domain information corresponding to the first sample image and the second frequency domain information corresponding to the second sample image.

[0079] Specifically, semantic information extraction from images involves extracting image information in the spatial domain. However, some information in an image has more obvious features in the frequency domain and is easier to extract. Therefore, the image is transformed to the frequency domain for frequency domain information extraction. For example, the texture features of an image are high-frequency information in the frequency domain. Transforming the image to the frequency domain makes it easier to extract the texture features.

[0080] In one embodiment of this application, the frequency domain information extraction process of the first sample image includes: performing a frequency domain transformation on the first sample image to obtain frequency domain data of the first sample image; calculating the amplitude of the first sample image based on the frequency domain data of the first sample image; and obtaining the first frequency domain information corresponding to the first sample image based on the amplitude of the first sample image.

[0081] Specifically, in this embodiment, Fourier transform is used to realize the frequency domain transformation of the image, as shown in the following equation (2):

[0082]

[0083] Where F(x)(u,v) represents the frequency domain data of image x after transformation to the frequency domain, or it can also be denoted as F(x); H represents the height of image x in pixels; W represents the width of image x in pixels; x(h,w) represents the pixel point in image x with coordinates (h,w); (u,v) is used to indicate that the corresponding data is frequency domain data. u < α*H, v < α*W.

[0084] The frequency domain data F(x) of image x can ultimately be expressed in the form of a complex number, namely: F(x) = R(x) + jI(x), where R(x) is the real part of F(x) and I(x) is the imaginary part of F(x).

[0085] After obtaining the frequency domain data of the image, the image amplitude is calculated based on the frequency domain data, and the amplitude data is the frequency domain information of the image.

[0086] Step 240: Mix the first frequency domain information and the second frequency domain information to obtain frequency domain mixed data.

[0087] Specifically, the mixing of frequency domain information is similar to the mixing of speech information. The mixing of the first and second frequency domain information can be a linear combination of the first and second frequency domain information. For example, the first and second frequency domain information can be multiplied by their respective weights and then added together to obtain the mixed result, which is frequency domain mixed data. Frequency domain mixed data is equivalent to the result of mixing data from two sample images extracted from the frequency domain, and it still constitutes an image. Therefore, frequency domain mixed data can also be called frequency domain mixed image.

[0088] In one embodiment of this application, the first frequency domain information includes first low-frequency information and first high-frequency information, and the second frequency domain information includes second low-frequency information and second high-frequency information. The mixing of frequency domain information can be a mixture of any one of the first low-frequency information and the first high-frequency information with any one of the second low-frequency information and the second high-frequency information; alternatively, one of the first frequency domain information can be mixed with one of the second frequency domain information to obtain a first frequency domain mixing result, and the other of the first frequency domain information can be mixed with the other of the second frequency domain information to obtain a second frequency domain mixing result. Then, the first frequency domain mixing result and the second frequency domain mixing result are mixed to obtain frequency domain mixed data.

[0089] It should be noted that the semantic information extraction in step 210 and the frequency domain information extraction in step 230 can be performed in parallel or sequentially; the semantic information mixing in step 220 and the frequency domain information mixing in step 240 can also be performed in parallel or sequentially. When the aforementioned steps are performed sequentially, this application does not limit the order in which the steps are performed.

[0090] Step 250: Train the preset classification model based on semantic mixed data and frequency domain mixed data to obtain an image recognition model. The image recognition model is used to determine the category of the image to be recognized.

[0091] Specifically, training a pre-defined classification model using semantic mixed data and frequency domain mixed data means training the model with both semantic mixed data and frequency domain mixed data. Alternatively, it can use the first sample image, the second sample image, semantic mixed data, and frequency domain mixed data as training data. The trained model is an image recognition model, which can be used to classify the category of the image to be recognized.

[0092] In the technical solution provided in this application embodiment, semantic information extraction and frequency domain information extraction are performed on the first sample image and the second sample image respectively, and the extracted first semantic information and second semantic information are mixed to obtain semantic mixed data, and the extracted first frequency domain information and second frequency domain information are mixed to obtain frequency mixed data. This increases the richness of image semantic information and frequency domain information in the model training data, so that the model has more learning data, thereby improving the accuracy of image recognition.

[0093] Figure 3 A flowchart illustrating an embodiment of the image recognition method provided in this application is shown schematically. This embodiment is a further refinement of the above embodiment. Figure 3 As shown, the image recognition method provided in this application includes steps 301 to 311, as detailed below:

[0094] Step 301: Extract semantic information from the first sample image and the second sample image respectively to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image; the first semantic information includes the first foreground information and the first background information, and the second semantic information includes the second foreground information and the second background information.

[0095] Step 302: Mix the first foreground information and the second foreground information to obtain mixed foreground information.

[0096] Step 303: Mix the first background information and the second background information to obtain mixed background information.

[0097] Step 304: Mix the foreground information and background information to obtain semantically mixed data.

[0098] In this embodiment, the semantic information extraction of an image involves segmenting the image into foreground and background information; therefore, the first semantic information includes the first foreground information v. i and the first background information σ i The second semantic information includes the second foreground information v j Second background information σ j The semantic information mixing operation can be represented by the following equation (3):

[0099]

[0100] Among them, M λ The function (a,b) represents a convex combination of data a and data b, that is, a linear combination with weight λ, M. λ (a,b)=λa+(1-λ)b. Represents semantically mixed data; This indicates that the first foreground information v i Second foreground information v j Mixing, λ υ Indicates the foreground mixing weight; This indicates that the first background information σ i Second background information σ j Mixing, λ σ This represents the background blending weight.

[0101] Foreground information is often crucial in determining image category and is effective information in image classification and recognition. Background information has a smaller impact on image category, but it often interferes with the determination of image category in image classification and recognition. Therefore, background information is interference or limiting information in image recognition. This application separates the foreground and background information of an image through semantic information extraction, and then obtains a semantically mixed image by mixing the semantic information of the image. This is equivalent to obtaining sample images with different background information (i.e., scene information), increasing the richness of the background information of the sample images, giving the model more learning data, and thus improving the accuracy of image recognition.

[0102] Step 305: Extract frequency domain information from the first sample image and the second sample image respectively to obtain the first frequency domain information corresponding to the first sample image and the second frequency domain information corresponding to the second sample image; the first frequency domain information includes the first low frequency information and the first high frequency information, and the second frequency domain information includes the second low frequency information and the second high frequency information.

[0103] In this embodiment, the frequency domain information extraction of the image involves separating the high-frequency and low-frequency information of the image; therefore, the first frequency domain information includes the first low-frequency information k. i and the first high-frequency information l i The second frequency domain information includes the second low-frequency information k. j Second high frequency information l j .

[0104] When extracting frequency domain information, the first step is to transform the image from the spatial domain to the frequency domain through frequency domain transformation to obtain frequency domain data. The calculation method for the frequency domain data can be referred to in formula (2) above, and will not be repeated here. Then, the amplitude is calculated based on the frequency domain data. The amplitude can be expressed in complex form. The imaginary part of the amplitude is the high-frequency information of the image, and the real part of the amplitude is the low-frequency information of the image. The image amplitude is calculated as shown in formula (4):

[0105]

[0106] Where A(x)(u,v) represents the amplitude of image x, which can also be denoted as A(x). R(x) is the real part of the frequency domain data F(x), and I(x) is the imaginary part of the frequency domain data F(x).

[0107] The amplitude A(x) of image x can ultimately be expressed in complex form, i.e., A(x) = K + jL. Then, the real part K of amplitude A(x) corresponds to the low-frequency information of the image, and the imaginary part L of amplitude A(x) corresponds to the high-frequency information of the image.

[0108] Step 306: Mix the first low-frequency information and the second low-frequency information to obtain mixed low-frequency information.

[0109] Step 307: Mix the first high-frequency information and the second high-frequency information to obtain mixed high-frequency information.

[0110] Step 308: Mix the mixed low-frequency information and the mixed high-frequency information to obtain the target mixed data.

[0111] Step 309: Perform inverse frequency domain transformation on the target mixed data to obtain frequency domain mixed data.

[0112] In this embodiment, the mixing operation of frequency domain information can be represented by the following equation (5):

[0113]

[0114] in, Represents mixed frequency domain data; This indicates that the first low-frequency information k i Second low-frequency information k j The resulting mixed low-frequency information, λ k Indicates low-frequency mixed weights; This indicates that the first high-frequency information l i Second high frequency information l j The resulting high-frequency information, λ l This indicates high-frequency mixed weights. That is, the target mixed data. Represents the inverse frequency domain transform function. This indicates that the target mixed data is subjected to inverse frequency domain transformation to obtain frequency domain mixed data. The inverse frequency domain transform can be achieved through the inverse Fourier transform.

[0115] The limiting information that interferes with image recognition also includes texture information in the image. Texture information in the image typically manifests as high-frequency features in the frequency domain. Therefore, the high-frequency information extracted from the frequency domain is the limiting information that interferes with image recognition, while low-frequency information is the effective information that is beneficial for image recognition. This application separates the low-frequency and high-frequency information of the image through frequency domain information extraction, and then obtains a frequency-mixed image by mixing the frequency domain information of the image. This is equivalent to obtaining sample images with different texture information, increasing the richness of the texture information of the sample images, giving the model more learning data, and thus improving the accuracy of image recognition.

[0116] In one embodiment of this application, since high-frequency information and background information are both limited information of the image, while low-frequency information and foreground information are both valid information of the image, when performing semantic information mixing and frequency domain information mixing, the mixing weights of the limited information and the mixing weights of the valid information can be set to be the same, i.e., the foreground mixing weight λ. υ With low-frequency mixed weight λ kSame, background blending weight λ σ With high-frequency mixed weight λ l same.

[0117] Step 310: Mix the first sample category label of the first sample image and the second sample category label of the second sample image to obtain mixed data category labels.

[0118] Specifically, during model training, the training data needs to have category labels, which represent the image category corresponding to the training data. When the first sample image and the second sample image are mixed, the corresponding sample category labels need to be mixed in the same way to obtain the mixed data category labels. That is, the mixed data category labels are the category labels of semantic mixed data or frequency domain mixed data.

[0119] In one embodiment of this application, the process of mixing sample category labels includes: linearly combining a first sample category label and a second sample category label using a first weight to obtain first label data; linearly combining the first sample category label and the second sample category label using a second weight to obtain second label data; and linearly combining the first label data and the second label data using a third weight to obtain mixed data category labels.

[0120] Specifically, the mixture of sample category labels can be represented by the following equation (6):

[0121]

[0122] Among them, y i Represents the first sample image x i The first sample category label, y j Represents the second sample image x j The second sample category label; λ1 represents the first weight, λ2 represents the second weight, and α represents the third weight.

[0123] In one embodiment of this application, to ensure that the category labels of the mixed data are consistent with the corresponding semantic mixed data or frequency domain mixed data, the first and second weights of the label mixing can be set to be the same as the corresponding mixing weights. For example, when calculating the category labels of the semantic mixed data, the first weight λ1 is set to be the same as the foreground mixing weight λ. v Similarly, the second weight λ2 is set to the blending weight λ with the background. σ The same applies. When calculating the category label for the frequency domain mixed data, the first weight λ1 is set to the same as the low-frequency mixing weight λ. k Similarly, the second weight λ2 is set to be the same as the high-frequency mixing weight λ. l same.

[0124] In one embodiment of this application, low-frequency information and foreground information are considered as valid information, while high-frequency information and background information are considered as restricted information, and the foreground mixing weight λ is... υ With low-frequency mixed weight λ k Same, background blending weight λ σ With high-frequency mixed weight λ l If they are the same, then the first weight λ1 and the foreground mixing weight λ are the same. υ and low-frequency mixed weight λ k The three are the same: the second weight λ2, the background mixing weight λ. σ and high-frequency hybrid weight λ l If all three are the same, then the category label for the mixed data only needs to be calculated once, which means that the category label for the semantic mixed data is the same as the category label for the frequency domain mixed data.

[0125] In one embodiment of this application, it can be seen from equation (6) that the first tag data is λ1y i +(1-λ1)y j Set the first weight λ1 to the blending weight λ with the foreground weight. υ Or low-frequency mixed weight λ k If they are the same, then the first label data is equivalent to the mixed valid information and still belongs to valid information. The second label data is λ2y. i +(1-λ2)y j The second weight λ2 is set to the blending weight λ with the background. σ Or high-frequency mixed weight λ l If they are the same, then the second label data is equivalent to the mixed restricted information, and still belongs to restricted information. Mixed data category labels It is obtained by mixing the first label data and the second label data, so the mixed data category label In other words, the data is obtained by mixing valid and restricted information. By changing the third weight α, the category label of the mixed data can be adjusted. The ratio of effective information to restricted information in the mixed data effectively suppresses the restricted information. For example, when the third weight α = 1, the mixed data category labels... The data is equal to the first label data, which means that only valid information is considered and restricted information is completely suppressed.

[0126] Since restricted information is interference information in the image recognition process, suppressing restricted information by adjusting the weights is equivalent to reducing interference information in the image recognition process, thereby improving the accuracy and precision of image recognition.

[0127] Step 311: Using semantic mixed data and frequency domain mixed data as training data, and using the mixed data category labels as the category labels of the training data, train the preset classification model to obtain the image recognition model. The image recognition model is used to determine the category of the image to be recognized.

[0128] Specifically, semantic mixed data and frequency domain mixed data are input into a pre-defined classification model as training data. The pre-defined classification model performs feature extraction and mapping processing on the input training data, predicting the image category corresponding to the training data, i.e., outputting the predicted category label. Then, the predicted category label is compared with the mixed data category label corresponding to the training data. When the difference between the two is large, it indicates that the model's prediction result is not accurate enough. At this time, a loss function is constructed based on the predicted category label and the mixed data category label. The model parameters are updated according to the loss function, and the updated classification model is trained again using training data until the model converges, resulting in a trained image recognition model. This image recognition model can be used for the classification processing of images to be recognized. The convergence condition of the model can be that the difference between the predicted category label and the mixed data category label is within a preset range, or the number of training iterations reaches a threshold, or other conditions that indicate that the model can accurately identify the image category.

[0129] In this embodiment, the scene information or background information of an image is called a domain. Two images with the same image category but different background information (scene information) are considered out-of-domain images. For example, image 1, a cat taken at the beach, and image 2, a cat taken in a forest. Image 1 and image 2 have the same image category, i.e., both are images of cats; however, image 1 and image 2 have different background information: image 1 has a beach background, while image 2 has a forest background. In related technologies, when training an image recognition model, samples with beach background information are used. The trained model will then use "beach" as a factor to identify "cat." When image 2 with a forest background is input, the model will not be able to identify the category of image 2 well, meaning the model has a weak ability to recognize out-of-domain images.

[0130] The technical solution provided in this application not only increases the richness of model training data by mixing semantic information and frequency domain information, but also suppresses restricted information in the image recognition process and reduces interference information in the image recognition process by adjusting the corresponding mixed weight data, thereby improving the accuracy of image recognition and the ability of the image recognition model to recognize images outside the domain.

[0131] Figure 4A The flowchart illustrating a model training phase provided in one embodiment of this application is shown in the illustration. Figure 4A As shown, the model training phase includes:

[0132] Step 411: Input image. This image is a sample image, which can be obtained from a publicly available image dataset.

[0133] Step 412, Image Mixing. This involves extracting and mixing semantic information from the sample images to obtain semantically mixed data; extracting and mixing frequency domain information from the sample images to obtain frequency domain mixed data; and simultaneously mixing the category labels of the sample images to obtain mixed data category labels.

[0134] For example, Figure 5 A schematic diagram illustrating an image blending process provided in one embodiment of this application is shown. Figure 5 As shown, the image blending process includes:

[0135] Step 501: Obtain the input sample images. The input images include a first sample image x1 and a second sample image x2. The first sample category label y1 of the first sample image x1 represents a cat, and the second sample category label y2 of the second sample image x2 represents a dog.

[0136] Step 502: Decouple information from the sample image. Information decoupling refers to segmenting and extracting different information in the image. In this embodiment, information decoupling includes semantic information decoupling and frequency domain information decoupling. Semantic information decoupling is semantic information extraction, which is performed using the function S(x i ) represents; frequency domain information decoupling is frequency domain information extraction, which is achieved through the function F(x) i )express.

[0137] Semantic information decoupling is performed on the first sample image x1 to obtain first foreground information v1 and first background information σ1; semantic information decoupling is performed on the second sample image x2 to obtain second foreground information v2 and second background information σ2.

[0138] Frequency domain information decoupling is performed on the first sample image x1 to obtain the first low-frequency information. and the first high frequency information Frequency domain information decoupling is performed on the second sample image x2 to obtain the second low-frequency information. Second high frequency information

[0139] Step 503: Image Blending. The image blending process includes semantic information blending, frequency domain information blending, and category label blending. During image blending, the first weight λ1 and the foreground blending weight λ... v and low-frequency mixed weight λ k The three are the same: the second weight λ2, the background mixing weight λ. σ and high-frequency hybrid weight λ l All three are the same. Foreground blending weight λυ (including the first weight λ1 and the low-frequency mixed weight λ) k ), background mixing weight λ σ (including the second weight λ2 and the high-frequency hybrid weight λ) l The first weight and the second weight α are collectively referred to as the mixed weight.

[0140] Semantic information mixing includes: mixing the first foreground information v1 of the first sample image x1 and the second foreground information v2 of the second sample image x2. The mixing process can be referred to in equation (3) to obtain the mixed foreground information. This is the effective information (or effective feature) for image recognition. The first background information σ1 of the first sample image x1 and the second background information σ2 of the second sample image x2 are mixed. The mixing process can be referred to in equation (3) to obtain the mixed background information. This is the restricted information (or restricted feature) for image recognition. This information belongs to the spatial domain, and therefore can also be called spatially restricted feature. Then, the mixed foreground information... and mixed background information Mixing to obtain semantically mixed data

[0141] Frequency domain information mixing includes: the first low-frequency information of the first sample image x1 Second low-frequency information of the second sample image x2 The mixing process can be referenced in equation (5) to obtain mixed low-frequency information. This is the effective information (or effective feature) for image recognition. The first high-frequency information of the first sample image x1... Second high-frequency information of the second sample image x2 The mixing process can be referenced in equation (3) to obtain the mixed high-frequency information. This is the restricted information (or restricted feature) for image recognition. This information belongs to the spatial domain, and therefore can also be called spatially restricted feature. Then, the low-frequency information is mixed... and mixed high-frequency information The data is mixed and then subjected to an inverse Fourier transform to obtain the frequency domain mixed data.

[0142] The category label mixing can be referred to the relevant description in equation (6) above, and will not be repeated here. In this embodiment, semantically mixed data and frequency domain mixed data They have the same category label.

[0143] Step 413: Classification Network Training. The training data obtained from image mixing in Step 402 is input into a preset classification model, which predicts the category of the input data. For example... Figure 5As shown, semantically mixed data and frequency domain mixed data Input classification network.

[0144] Step 414: Output category prediction results. The preset classification model outputs category prediction results for the training data. The category prediction results are compared with the category labels of the mixed data, and the model parameters are updated until the model converges, thus obtaining the image recognition model.

[0145] Figure 4B A flowchart illustrating a model testing phase provided in one embodiment of this application is shown schematically. Figure 4B As shown, the model testing phase includes:

[0146] Step 421: Input image. This image is a test image and is an image of a known image category.

[0147] Step 422: Classification Network Prediction. This classification network is a pre-trained image recognition model obtained during the training phase. The test image is input into the image recognition model to predict the category.

[0148] Step 423: Output the category prediction result. The image recognition model outputs the predicted category of the test image and compares it with the known category of the test image to determine the accuracy of the image recognition model. When the accuracy of the image recognition model is low, it can return to the training phase to retrain the model.

[0149] Figure 6 A schematic diagram of the hardware framework of the technical solution of this application is shown. For example... Figure 6 As shown, the hardware framework applying the technical solution of this application includes front-end A, back-end, and front-end B. Front-end A receives test images for the image recognition task and transmits them to the back-end. The back-end uses the technical solution of this application to train a model, obtains an image recognition model, and tests the image recognition model. After passing the test, the image recognition model is used to execute the image recognition task. Finally, the back-end outputs the image recognition result to front-end B, and the relevant operators of front-end B are informed of the image recognition result.

[0150] The following shows the recognition performance of the technical solution of this application on out-of-domain images on publicly available data in the NICO++ dataset.

[0151] The NICO++ dataset consists of two sub-datasets: Track-1 and Track-2. Track-1 contains 88,866 images from six different domains (water, grass, rocks, outdoors, autumn, and low light), categorized into 60 classes. Track-2 contains 57,266 additional images from the same six domains, also categorized into 60 classes.

[0152] To demonstrate the generalization ability of the technical solution in this application for image recognition outside the domain, cross-validation is used for image recognition. Specifically, an image recognition model is trained using the Track-1 dataset and tested using the Track-2 dataset, and an image recognition model is trained using the Track-2 dataset and tested using the Track-1 dataset.

[0153] During the training phase, the image recognition method used employed the same experimental settings, with model parameters including: a batch size of 128, a training period of 200, and an optimizer using SGD (Stochastic Gradient Descent) (where the initial learning rate was 0.1, and the learning rate was decayed using a cosine annealing algorithm).

[0154] During the testing phase, accuracy (ACC) was used to measure the performance of different image recognition methods in out-of-domain image recognition.

[0155] The classification models used in the experiment include ResNet-34, WideResNet-50, and DenseNet-121. Conventional solutions directly input the data from the training dataset (Track-1 or Track-2) into the classification model for training. However, the solution in this application first decouples the information and mixes the images in the training dataset, and then uses the mixed data as training data to train the classification model.

[0156] Table 1. Classification Models and Accuracy Rate (ACC) (%) Data

[0157]

[0158] Table 1 shows the specific performance of the image recognition accuracy (ACC) of the technical solution of this application and the general technical solution when using different classification models. In Table 1, the "Model" column only contains the model name, which indicates the general technical solution using the corresponding model. The model name + "w / Ours" indicates the technical solution of this application. For example, ResNet-34 indicates the general technical solution using the ResNet-34 classification model, and ResNet-34w / Ours indicates the technical solution of this application using the ResNet-34 classification model.

[0159] As shown in Table 1, regardless of whether the image recognition model is trained using the Track-1 dataset and tested using the Track-2 dataset, or the image recognition model is trained using the Track-2 dataset and tested using the Track-1 dataset, the image recognition accuracy ACC (%) of the technical solution of this application is higher than that of the ordinary technical solution. It can be seen that the technical solution of this application can effectively improve the accuracy of out-of-domain image recognition.

[0160] Table 2 shows the image recognition accuracy (ACC) of the proposed solution using the ResNet-34 classification model at different third weights α. The third weight α is used to adjust the proportion of effective information (effective features) in the mixed data category labels, and is therefore also called the effective feature weight.

[0161] Table 2 Effective Feature Weights and Accuracy (ACC) (%) Data

[0162]

[0163] As shown in Table 2, adjusting the effective feature weight α to suppress restricted features in the image recognition process can effectively improve image recognition accuracy. For example, when the training dataset is Track-1, setting α = 0.2 results in higher image classification accuracy, as more restricted features can be suppressed. When the training dataset is Track-2, setting α = 0.6 also results in higher image classification accuracy, with more restricted features being suppressed.

[0164] In related technologies, image recognition also employs direct image blending and image blending methods based on image region cropping. The direct image blending method sets weights to sum two sample images and then sums the class labels of the sample images according to these weights to form the model training data. However, the direct image blending method cannot further provide more effective features for recognizing out-of-domain images and may generate images and labels that mislead the model's learning.

[0165] Image blending methods based on image region cropping randomly crop a sample image and fill the cropped region into another image. Correspondingly, class labels are accumulated by setting weights to form the model training data. While this method can blend pixel information from different locations in an image, this random region cropping method may add invalid local image information to the blended image. During training, the labels are affected by the corresponding weights, failing to provide correct guidance for the model.

[0166] Table 3 presents the image recognition accuracy (ACC) data comparing the technical solution of this application with the aforementioned technical solutions. In Table 3, WideResNet-50 represents the common technical solution for image recognition using the WideResNet-50 classification model, WideResNet-50w / Mixup represents the direct image mixing method for image recognition using the WideResNet-50 classification model, WideResNet-50w / CutMix represents the image mixing method based on image region cropping using the WideResNet-50 classification model, and WideResNet-50w / Ours represents the technical solution of this application for image recognition using the WideResNet-50 classification model.

[0167] As can be seen from Table 3, the image recognition accuracy of the technical solution of this application exceeds that of the existing image mixing method in both cross-validation methods, which shows that the technical solution of this application can effectively improve the image recognition accuracy.

[0168] Table 3. Accuracy data for different technical solutions

[0169]

[0170] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0171] The following describes an apparatus embodiment of this application, which can be used to perform the image recognition method described in the above embodiments of this application. Figure 7 A schematic block diagram of the image recognition device provided in an embodiment of this application is shown. Figure 7 As shown, the image recognition device includes:

[0172] The semantic information extraction module 710 is used to extract semantic information from the first sample image and the second sample image respectively, to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image.

[0173] The semantic information mixing module 720 is used to mix the first semantic information and the second semantic information to obtain semantic mixed data;

[0174] The frequency domain information extraction module 730 is used to extract frequency domain information from the first sample image and the second sample image respectively, to obtain the first frequency domain information corresponding to the first sample image and the second frequency domain information corresponding to the second sample image.

[0175] The frequency domain information mixing module 740 is used to mix the first frequency domain information and the second frequency domain information to obtain frequency domain mixed data;

[0176] The model training module 750 is used to train a preset classification model based on the semantic mixed data and the frequency domain mixed data to obtain an image recognition model, which is used to determine the category of the image to be recognized.

[0177] In one embodiment of this application, the first semantic information includes first foreground information and first background information, and the second semantic information includes second foreground information and second background information; the semantic information mixing module 720 is specifically used for:

[0178] The first foreground information and the second foreground information are mixed to obtain mixed foreground information;

[0179] The first background information and the second background information are mixed to obtain mixed background information;

[0180] The mixed foreground information and the mixed background information are combined to obtain the semantic mixed data.

[0181] In one embodiment of this application, the first frequency domain information includes first low-frequency information and first high-frequency information, and the second frequency domain information includes second low-frequency information and second high-frequency information; the frequency domain information mixing module 740 is specifically used for:

[0182] The first low-frequency information and the second low-frequency information are mixed to obtain mixed low-frequency information;

[0183] The first high-frequency information and the second high-frequency information are mixed to obtain mixed high-frequency information;

[0184] The mixed low-frequency information and the mixed high-frequency information are mixed to obtain the target mixed data;

[0185] The target mixed data is subjected to inverse frequency domain transformation to obtain the frequency domain mixed data.

[0186] In one embodiment of this application, the frequency domain information extraction module 730 includes:

[0187] The frequency domain transformation unit is used to perform frequency domain transformation on the first sample image to obtain the frequency domain data of the first sample image;

[0188] An amplitude calculation unit is used to calculate the amplitude of the first sample image based on the frequency domain data of the first sample image;

[0189] The frequency domain information determination unit is used to obtain the first frequency domain information corresponding to the first sample image based on the amplitude of the first sample image.

[0190] In one embodiment of this application, the first frequency domain information includes first low-frequency information and first high-frequency information; the frequency domain information determining unit is specifically used for:

[0191] The real part of the amplitude of the first sample image is taken as the first low-frequency information of the first sample image;

[0192] The imaginary part of the amplitude of the first sample image is used as the first high-frequency information of the first sample image.

[0193] In one embodiment of this application, the model training module 750 includes:

[0194] The label mixing unit is used to mix the first sample category label of the first sample image and the second sample category label of the second sample image to obtain mixed data category labels;

[0195] The model training unit is used to train the preset classification model using the semantic mixed data and the frequency domain mixed data as training data, and the category label of the mixed data as the category label of the training data, to obtain an image recognition model.

[0196] In one embodiment of this application, the label mixing unit is specifically used for:

[0197] The first label data is obtained by linearly combining the first sample category label and the second sample category label using a first weight;

[0198] The second label data is obtained by linearly combining the first sample category label and the second sample category label using a second weight;

[0199] The mixed data category label is obtained by linearly combining the first label data and the second label data using a third weight.

[0200] The specific details of the image recognition devices provided in the various embodiments of this application have been described in detail in the corresponding method embodiments, and will not be repeated here.

[0201] Figure 8 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0202] It should be noted that, Figure 8The computer system 800 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0203] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM). The random access memory 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output interface 805 (I / O interface) is also connected to the bus 804.

[0204] The following components are connected to the input / output interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a local area network card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0205] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit 801, it performs various functions defined in the system of this application.

[0206] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0207] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0208] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0209] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0210] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0211] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An image recognition method characterized by, include: Semantic information is extracted from the first sample image and the second sample image respectively to obtain the first semantic information corresponding to the first sample image and the second semantic information corresponding to the second sample image; the first semantic information includes the first foreground information and the first background information, and the second semantic information includes the second foreground information and the second background information. The first foreground information and the second foreground information are mixed to obtain mixed foreground information; the first background information and the second background information are mixed to obtain mixed background information. The mixed foreground information and the mixed background information are combined to obtain semantically mixed data; Frequency domain information is extracted from the first sample image and the second sample image respectively to obtain the first frequency domain information corresponding to the first sample image and the second frequency domain information corresponding to the second sample image; the first frequency domain information includes first low frequency information and first high frequency information, and the second frequency domain information includes second low frequency information and second high frequency information. The first low-frequency information and the second low-frequency information are mixed to obtain mixed low-frequency information; the first high-frequency information and the second high-frequency information are mixed to obtain mixed high-frequency information; the mixed low-frequency information and the mixed high-frequency information are mixed to obtain target mixed data. Perform an inverse frequency domain transform on the target mixed data to obtain frequency domain mixed data; First label data is obtained by linearly combining the first sample category label of the first sample image and the second sample category label of the second sample image using a first weight; second label data is obtained by linearly combining the first sample category label and the second sample category label using a second weight; and mixed data category label is obtained by linearly combining the first label data and the second label data using a third weight. Using the semantic mixed data and the frequency domain mixed data as training data, and using the category label of the mixed data as the category label of the training data, a preset classification model is trained to obtain an image recognition model, which is used to determine the category of the image to be recognized.

2. The image recognition method of claim 1, wherein, The semantic information extraction includes: An unsupervised saliency region extraction algorithm is used to extract and separate foreground and background information of an image.

3. The image recognition method of claim 1, wherein, Mixing the first foreground information and the second foreground information to obtain mixed foreground information includes: mixing the first foreground information and the second foreground information according to the foreground mixing weight to obtain mixed foreground information; Mixing the first background information and the second background information to obtain mixed background information includes: mixing the first background information and the second background information according to the background mixing weight to obtain mixed background information; The category label of the mixed data is the category label of the semantic mixed data; wherein, when calculating the category label of the semantic mixed data, the first weight is set to be the same as the foreground mixing weight, and the second weight is set to be the same as the background mixing weight.

4. The image recognition method of claim 1, wherein, Frequency domain information is extracted from the first sample image to obtain the first frequency domain information corresponding to the first sample image, including: The first sample image is transformed in the frequency domain to obtain the frequency domain data of the first sample image; The amplitude of the first sample image is calculated based on the frequency domain data of the first sample image; The first frequency domain information corresponding to the first sample image is obtained based on the amplitude of the first sample image.

5. The image recognition method of claim 4, wherein, The first frequency domain information includes first low-frequency information and first high-frequency information; the first frequency domain information corresponding to the first sample image is obtained based on the amplitude of the first sample image, including: The real part of the amplitude of the first sample image is taken as the first low-frequency information of the first sample image; The imaginary part of the amplitude of the first sample image is used as the first high-frequency information of the first sample image.

6. The image recognition method of claim 1, wherein, Mixing the first low-frequency information and the second low-frequency information to obtain mixed low-frequency information includes: mixing the first low-frequency information and the second low-frequency information according to the low-frequency mixing weight to obtain mixed low-frequency information; Mixing the first high-frequency information and the second high-frequency information to obtain mixed high-frequency information includes: mixing the first high-frequency information and the second high-frequency information according to the high-frequency mixing weight to obtain mixed high-frequency information; The category label of the mixed data is the category label of the frequency domain mixed data; wherein, when calculating the category label of the frequency domain mixed data, the first weight is set to be the same as the low-frequency mixed weight, and the second weight is set to be the same as the high-frequency mixed weight.

7. The image recognition method according to claim 1, characterized in that, Mixing the first foreground information and the second foreground information to obtain mixed foreground information includes: mixing the first foreground information and the second foreground information according to the foreground mixing weight to obtain mixed foreground information; Mixing the first background information and the second background information to obtain mixed background information includes: mixing the first background information and the second background information according to the background mixing weight to obtain mixed background information; Mixing the first low-frequency information and the second low-frequency information to obtain mixed low-frequency information includes: mixing the first low-frequency information and the second low-frequency information according to the low-frequency mixing weight to obtain mixed low-frequency information; Mixing the first high-frequency information and the second high-frequency information to obtain mixed high-frequency information includes: mixing the first high-frequency information and the second high-frequency information according to the high-frequency mixing weight to obtain mixed high-frequency information; In this approach, low-frequency information and foreground information are considered valid information, while high-frequency information and background information are considered restricted information. The first weight, the foreground mixing weight, and the low-frequency mixing weight are all set to be the same, and the second weight, the background mixing weight, and the high-frequency mixing weight are all set to be the same.

8. An image recognition device, characterized in that, include: The semantic information extraction module is used to extract semantic information from the first sample image and the second sample image respectively, to obtain first semantic information corresponding to the first sample image and second semantic information corresponding to the second sample image; the first semantic information includes first foreground information and first background information, and the second semantic information includes second foreground information and second background information. A semantic information mixing module is used to mix the first foreground information and the second foreground information to obtain mixed foreground information; and to mix the first background information and the second background information to obtain mixed background information. The mixed foreground information and the mixed background information are combined to obtain semantically mixed data; The frequency domain information extraction module is used to extract frequency domain information from the first sample image and the second sample image respectively, to obtain first frequency domain information corresponding to the first sample image and second frequency domain information corresponding to the second sample image; the first frequency domain information includes first low frequency information and first high frequency information, and the second frequency domain information includes second low frequency information and second high frequency information. A frequency domain information mixing module is used to mix the first low-frequency information and the second low-frequency information to obtain mixed low-frequency information; mix the first high-frequency information and the second high-frequency information to obtain mixed high-frequency information; and mix the mixed low-frequency information and the mixed high-frequency information to obtain target mixed data. Perform an inverse frequency domain transform on the target mixed data to obtain frequency domain mixed data; The model training module is used to linearly combine the first sample category label of the first sample image and the second sample category label of the second sample image using a first weight to obtain first label data; to linearly combine the first sample category label and the second sample category label using a second weight to obtain second label data; and to linearly combine the first label data and the second label data using a third weight to obtain mixed data category labels. Using the semantic mixed data and the frequency domain mixed data as training data, and using the category label of the mixed data as the category label of the training data, a preset classification model is trained to obtain an image recognition model, which is used to determine the category of the image to be recognized.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image recognition method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor executes the executable instructions to cause the electronic device to perform the image recognition method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, include: The computer program product or computer program includes computer instructions stored in a computer-readable storage medium; The processor of the computer device reads the computer instructions from a computer-readable storage medium, and executes the computer instructions to cause the computer device to perform the image recognition method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Model training method and device and computer storage medium

    CN113255433A

  • Video clustering method and device thereof

    CN113515668A