Image scene recognition method and device based on artificial intelligence and electronic equipment

Abstract

The invention provides an image scene recognition method and device based on artificial intelligence and electronic equipment, and is applied to the technical field of data processing. The method comprises the steps of obtaining a training sample set, to-be-recognized image information, image purpose information and a shooting influence factor; preprocessing the training sample set to generate a training sample set with identification information; acquiring a preset image scene recognition model matched with the identification information; training a preset image scene recognition model based on the training sample set with the identification information to generate a target image scene recognition model; processing the to-be-recognized image information to generate category information to which the to-be-recognized image belongs; processing the image purpose information and the shooting influence factor matched with the to-be-recognized image information to generate an image scene recognition influence factor; and based on the target image scene recognition model, processing the category information of the to-be-recognized image and the image scene recognition influence factor to which the to-be-recognized image belongs, and generating an image scene recognition result.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image scene recognition method, device and electronic equipment based on artificial intelligence. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0003] With the rapid development of computer technology and Internet technology, images are becoming more and more popular as a communication medium. Many aspects of people's work and life involve image understanding, and the primary task of image understanding is to perform scene recognition. Scene recognition is a technology that realizes human visual functions by using computers. The purpose of scene recognition is to enable computers to process images and automatically identify scenes in images. With the development of deep learning, it is becoming more and more common to use trained image recognition models to solve scene recognition problems. However, the image recognition of related technologies mainly extracts features for a specific object and cannot effectively perform high-accuracy scene recognition.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present application is to provide an image scene recognition method, device and electronic device based on artificial intelligence, which at least overcomes the problems existing in the prior art to a certain extent. The image scene recognition method integrates various information, including training sample sets, image information to be identified, image usage information and shooting influencing factors. The training sample set is preprocessed to generate identification information, and a preset image scene recognition model is obtained and trained to obtain a target model. The image to be identified is preprocessed to determine category information, and the image usage and shooting influencing factors are processed to obtain the image scene recognition influencing factors. Based on the target model, the category information and influencing factors are smoothed to generate image coordinate factors, and a multi-channel image is generated through operations such as convolution, and graphic features are obtained. Finally, the image scene recognition results are generated, providing an efficient and accurate solution for image scene recognition.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present invention.

[0007] According to one aspect of the present application, there is provided an image scene recognition method based on artificial intelligence, comprising: obtaining a training sample set, image information to be recognized, image usage information and shooting influencing factors matching the image information to be recognized, wherein the shooting image factor is generated based on the historical shooting images of the photographer; preprocessing the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image; obtaining a preset image scene recognition model matching the identification information, wherein the preset image scene recognition model comprises an upsampling model and a downsampling model, wherein the upsampling model and the downsampling model are connected by Skip The preset image scene recognition model is used to obtain feature information of photographic images at different spatial scales; the preset image scene recognition model is trained based on the training sample set with identification information to generate a target image scene recognition model; the image information to be recognized is processed to generate category information to which the image to be recognized belongs; the image usage information and the shooting influencing factors matching the image information to be recognized are processed to generate image scene recognition influencing factors; the category information and the image scene recognition influencing factors to which the image to be recognized belongs are processed based on the target image scene recognition model to generate an image scene recognition result.

[0008] Another aspect of the present application is an image scene recognition device based on artificial intelligence, characterized in that it includes: an acquisition module, used to acquire a training sample set, image information to be recognized, image usage information and shooting influence factors matching the image information to be recognized, wherein the shooting image factor is generated based on the historical shooting images of the photographer; acquiring a preset image scene recognition model matching the identification information, wherein the preset image scene recognition model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a SkipConnection connection layer, and the preset image scene recognition model is used to acquire feature information of the photographic image at different spatial scales; A processing module is used to preprocess the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image; train the preset image scene recognition model based on the training sample set with identification information to generate a target image scene recognition model; process the image information to be recognized to generate category information to which the image to be recognized belongs; process the image usage information and the shooting influencing factors that match the image information to be recognized to generate an image scene recognition influencing factor; process the category information and the image scene recognition influencing factor to which the image to be recognized belongs based on the target image scene recognition model to generate an image scene recognition result.

[0009] According to another aspect of the present application, an electronic device is characterized in that it includes: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-mentioned artificial intelligence-based image scene recognition method by executing the executable instructions.

[0010] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a second processor, the above-mentioned image scene recognition method based on artificial intelligence is implemented.

[0011] According to another aspect of the present application, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a third processor, the above-mentioned image scene recognition method based on artificial intelligence is implemented.

[0012] The present application provides an artificial intelligence-based image scene recognition method, device and electronic device, in which a server obtains a training sample set, information of images to be identified, information on image usage and shooting influencing factors, and uses advanced technical means to achieve accurate image scene recognition. First, obtain various information such as a training sample set, wherein the shooting image factor is generated based on the photographer's historical shooting images. The training sample set is preprocessed, including feature extraction to generate an original feature library, and then processed to obtain a first and second feature data set. The first feature data set is processed by a classifier to generate a first prediction result, and the second feature data set is trained by a preset algorithm to generate a second prediction result, thereby generating identification information for characterizing the spatial distribution characteristics of the photographic image.

[0013] Next, a preset image scene recognition model matching the identification information is obtained. The model is connected by an upsampling model and a downsampling model through a SkipConnection connection layer, and the feature information of the photographic image at different spatial scales can be obtained. Then, the preset model is trained using a training sample set with identification information. It is divided into a training set and a validation set, and multiple data groups are extracted from the training set for training to generate a trained model. The validation set is then used to process the model to obtain the validation result. If the conditions are met, it is determined to be the target image scene recognition model. For the image information to be identified, preprocessing is first performed to generate partition image information in the target format, extract color information, perform grayscale and binarization processing, generate the coordinates of the outermost contour endpoints and the partition contour area information, and finally determine the category information to which the image to be identified belongs. The image usage information and shooting influencing factors are processed to generate event type information, and then the image processing influencing factors and weight information are obtained. The image scene recognition influencing factors are generated in combination with the preset image processing influencing feature set.

[0014] Based on the target image scene recognition model, the category information of the image to be recognized and the image scene recognition influencing factors are smoothed to generate image coordinate factors, and convolution processing is performed on them to generate feature information of multiple spatial scales, and the graphic features of the image to be recognized are obtained. Multi-channel images are generated to characterize different spatial distributions, and finally, the image coordinate factors are combined to generate image scene recognition results, providing an efficient and accurate solution for image scene recognition.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of an image scene recognition method based on artificial intelligence provided by an embodiment of the present application is shown; Figure 2 A schematic diagram of the structure of an image scene recognition device based on artificial intelligence provided by an embodiment of the present application is shown; Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown; Figure 4 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0018] Combine the following Figure 1 To describe the image scene recognition method based on artificial intelligence according to the exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0019] In one implementation, the present application also proposes an image scene recognition method, device and electronic device based on artificial intelligence. Figure 1 The following schematically shows a flow chart of an image scene recognition method based on artificial intelligence according to an embodiment of the present application. Figure 1 As shown, the method is applied to a server, comprising: S101, obtaining a training sample set, image information to be identified, image usage information matching the image information to be identified, and shooting influencing factors.

[0020] In one implementation, first, a large number of images need to be collected as training sample sets. These images can come from a variety of sources, such as works by professional photographers, photos on social media, gallery websites, etc. These images can be obtained through web crawlers, data purchases, or user uploads. For each image, some relevant information needs to be recorded, such as the model of the shooting device, shooting parameters (such as focal length, aperture, shutter speed, etc.), description of the shooting scene, etc. This information will help to better understand the characteristics of the image and the shooting environment in the subsequent training process.

[0021] When a user uses a camera app to take a photo, the camera device automatically obtains the image information to be identified. This information includes the image's pixel value, color distribution, texture features, etc. By analyzing this information, you can get a preliminary understanding of the image's content and quality. For example, you can determine whether the shooting scene is a special scene such as night scene, sunrise, sunset, etc. by analyzing the color distribution of the image. If the colors in the image are mainly blue and black, and the brightness is low, it is likely a night scene.

[0022] In addition to the image information to be identified, it is also necessary to obtain information related to the purpose of the image. For example, if the user takes a landscape photo, the possible uses may be for social media sharing, making photo albums, printing photos, etc. Different uses have different requirements for image quality and style. If it is used for social media sharing, the image may require more vivid colors and clear details to attract more attention and likes. If it is used to make a photo album, the image may require softer colors and natural light and shadow effects to create a warm atmosphere.

[0023] Shooting influence factors are generated based on the photographer's historical images. These factors can reflect the photographer's shooting habits, preferences and style. For example, if the photographer often takes landscape photos, it can be inferred that he may be more interested in natural scenery, and more attention can be paid to the expression of natural scenery during autofocus, scene recognition and image optimization. Shooting influence factors can also include information such as the photographer's shooting equipment, shooting location, shooting time, etc. By analyzing this information, we can better understand the photographer's shooting environment and needs, thereby providing him with a more personalized shooting experience.

[0024] By obtaining the training sample set, the image information to be identified, the image usage information matching the image information to be identified, and the shooting influencing factors, more intelligent autofocus, scene recognition and image optimization can be achieved, providing users with a better photography and videography experience.

[0025] S102: pre-process the training sample set to generate a training sample set with identification information.

[0026] In one implementation, it is assumed that there is a training sample set containing a large number of photographic images, which are from different shooting scenes, equipment and photographers. In order to better utilize these images for training, they need to be preprocessed. First, some basic processing can be performed on the images, such as resizing, cropping, denoising, etc., to ensure the quality and consistency of the images. Then, feature extraction can be performed on the images to generate a training sample set with identification information.

[0027] Perform feature extraction on the training sample set to generate an original feature library. Feature extraction is to extract useful information from the image for subsequent processing and analysis. In this example, some common feature extraction methods can be used, such as SIFT (Scale-Invariant Feature Transform), HOG (Histogram of Oriented Gradients), etc. By performing feature extraction on each image in the training sample set, an original feature library can be obtained, which contains the feature information of all images.

[0028] The original feature library is processed to generate a first feature data set and a second feature data set. In order to improve the training efficiency and accuracy, the original feature library can be further processed to generate two different feature data sets. One method is to randomly divide the original feature library into two parts, one as the first feature data set and the other as the second feature data set. Another method is to divide the features according to their importance or relevance, with important features forming the first feature data set and other features forming the second feature data set.

[0029] The first feature data set is processed based on the classifier to generate a first prediction result. A classifier is an algorithm that can classify data. In this example, some common classifiers can be used, such as support vector machine (SVM), random forest (RandomForest), etc. The first feature data set is input into the classifier, and the classifier classifies the image according to the feature information to generate a first prediction result. This prediction result can be information such as the category of the image, the shooting scene, and the shooting device.

[0030] The second feature data set is trained based on a preset algorithm to generate a second prediction result. In addition to using a classifier for processing, other algorithms can also be used to train the second feature data set to generate a second prediction result. For example, a deep learning algorithm, such as a convolutional neural network (CNN), can be used to train the second feature data set. By continuously adjusting the parameters of the network, the network can better learn the features of the image, thereby generating a more accurate second prediction result.

[0031] The first prediction result and the second prediction result are processed to generate identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image. The first prediction result and the second prediction result are fused to obtain a more comprehensive prediction result. Then, identification information can be generated based on this prediction result, such as encoding the image category, shooting scene, shooting equipment and other information into a vector as identification information. In addition, the first prediction result and the second prediction result can be analyzed to extract key information therein, such as the color distribution, texture features, object shape and the like of the image. Then, identification information can be generated based on these key information, such as encoding the color distribution into a histogram, encoding the texture features into a matrix, encoding the object shape into a contour map and the like. By preprocessing the training sample set, extracting features, dividing the data set, classifier processing, algorithm training and result processing, a training sample set with identification information can be generated, providing strong support for subsequent photographic image analysis and processing.

[0032] S103: Acquire a preset image scene recognition model that matches the identification information.

[0033] In one embodiment, when a user takes a photo, the software first analyzes the identification information of the photo. The identification information is generated by performing operations such as preprocessing and feature extraction on the photo, and it can characterize the spatial distribution characteristics of the photographic image. Based on the identification information, the software obtains a preset image scene recognition model that matches it from a preset model library. Different identification information may correspond to different scene types, such as landscapes, people, animals, etc., so different models are required for accurate scene recognition.

[0034] The preset image scene recognition model includes an upsampling model and a downsampling model. The upsampling model and the downsampling model are connected through the SkipConnection connection layer. The preset image scene recognition model is used to obtain feature information of photographic images at different spatial scales. The role of the downsampling model is to gradually reduce the resolution of the image and extract features of the image at different scales. For example, in a convolutional neural network, through a series of convolutional layers and pooling layers, the size of the image can be gradually reduced while extracting more abstract features. These features contain information such as texture, color, and shape of the image, which can help the model recognize different scene elements. The upsampling model, on the contrary, gradually restores the resolution of the image and maps the low-resolution feature map back to the high-resolution space. Through the upsampling operation, the model can perform scene recognition at a higher resolution and improve the recognition accuracy. The upsampling model usually uses methods such as deconvolution layers or interpolation to increase the size of the image. The SkipConnection connection layer plays a role in connecting the upsampling model and the downsampling model. It allows feature information at different scales to be fused, thereby better utilizing the multi-scale features of the image. For example, at a certain intermediate layer, the features extracted by the downsampling model can be directly concatenated with the features of the corresponding layer in the upsampling model, which can retain more detailed information and improve the performance of the model.

[0035] The preset image scene recognition model is used to obtain the feature information of photographic images at different spatial scales. Through the process of downsampling and upsampling, the model can consider both the global features and local details of the image, thereby more accurately identifying the scene of the image. For example, for a landscape photo, the model can extract large-scale features such as mountains, rivers, and sky through the downsampling model, while capturing small-scale details such as leaves, flowers, and clouds through the upsampling model. In this way, the model can understand the content of the image more comprehensively and accurately determine that the scene of the photo is natural scenery. By obtaining the preset image scene recognition model that matches the identification information and using its unique structure to obtain the feature information of the photographic image at different spatial scales, it can provide powerful scene recognition capabilities for intelligent photography assistance software and bring users a better shooting experience.

[0036] S104: Training the preset image scene recognition model based on the training sample set with identification information to generate a target image scene recognition model.

[0037] In one embodiment, a training sample set with identification information is divided into a training set for training a preset image scene recognition model and a verification set for verifying the preset image scene recognition model based on a preset ratio; multiple data groups are extracted from the training set, wherein each data group contains a preset number of data samples, wherein at least one data sample includes identification information. There is a training sample set with identification information, and each image in this sample set has been preprocessed to generate identification information for characterizing the spatial distribution characteristics of the photographic image. We divide this sample set into a training set for training a preset image scene recognition model and a verification set for verifying the preset image scene recognition model according to a preset ratio, such as 7:3. For example, if we have 1,000 images with identification information, 700 of them will be divided into the training set for model training; the remaining 300 will be divided into the verification set for verifying the performance of the model.

[0038] The preset image scene recognition model is trained based on the data samples in the multiple data sets to generate a trained image scene recognition model. Multiple data sets are extracted from the training set, each of which contains a preset number of data samples. Assuming the preset number is 10, we randomly extract 10 images from the training set to form a data set. Moreover, at least one data sample includes identification information, which means that in each data set, at least one image has identification information so that the model can learn the characteristics of different scenes. The trained image scene recognition model is processed based on the validation set to generate a validation result. The preset image scene recognition model is trained using the data samples in the multiple data sets. This model may be a deep learning model, such as a convolutional neural network. During the training process, the model will continuously adjust its parameters to better fit the training data. For example, the model will learn image features in different scenes, such as features such as mountains, rivers, and trees in landscape scenes, and features such as faces, bodies, and actions in character scenes. Through continuous iterative training, the performance of the model will gradually improve.

[0039] If the data samples containing identification information in the verification results are pixels representing the image scene and the brightness matching the pixels of the image scene, the trained image scene recognition model is used as the target image scene recognition model. After the model has been trained on the training set for a certain number of rounds, we use the verification set to process the trained image scene recognition model and generate verification results. The verification set is a part of data independent of the training set, which is used to evaluate the performance of the model on unseen data. If the data samples containing identification information in the verification results are pixels representing the image scene and the brightness matching the pixels of the image scene, then we use the trained image scene recognition model as the target image scene recognition model. For example, if a landscape image in the verification set can accurately identify the scene elements such as mountains and rivers in the image, and give brightness information matching these scene elements, then it means that the model performs well on this image. If such results can be obtained for most images in the entire verification set, then we can consider that the trained model is an effective target image scene recognition model and can be used for actual image scene recognition tasks.

[0040] By dividing the training sample set with identification information, extracting multiple data sets for training, and using the validation set for verification, we can continuously optimize the image scene recognition model and ultimately obtain a target model that can accurately recognize image scenes.

[0041] S105: Process the image information to be identified to generate category information to which the image to be identified belongs.

[0042] In one implementation, the image information to be identified is preprocessed to generate partitioned image information in a target format, wherein the partitioned image information in the target format is used to characterize images of different sizes and different pixel brightnesses. Suppose we have a color photo to be identified, which contains a variety of objects and scenes. First, we preprocess the image information to be identified. The purpose of preprocessing is to convert the image into a target format for subsequent processing and analysis. The partitioned image information in the target format is used to characterize images of different sizes and different pixel brightnesses. For example, we can divide the image into multiple partitions of the same size, each with its own brightness value and color information. Doing so allows us to analyze different parts of the image in more detail, thereby better identifying objects and scenes in the image.

[0043] Perform feature extraction on the partition image information of the target format to generate the color information of the partition image. Perform feature extraction on the partition image information of the target format. The purpose of feature extraction is to extract useful information from the image for subsequent classification and recognition. In this example, we can extract the color information of the partition image. The color information can include the main color in the image, the distribution of colors, the contrast of colors, etc. By analyzing the color information, we can preliminarily judge the theme and style of the image. For example, if the main colors in the image are blue and white, it may be a snow scene photo; if the main colors in the image are red and yellow, it may be a sunset photo.

[0044] Grayscale the partition image information of the target format to generate a grayscale image. In order to further analyze the characteristics of the image, we can grayscale the partition image information of the target format to generate a grayscale image. The purpose of grayscale processing is to convert a color image into a grayscale image so that the brightness and contrast of the image can be better analyzed. In a grayscale image, the brightness value of each pixel ranges from 0 to 255, where 0 represents black and 255 represents white. By analyzing the brightness and contrast of the grayscale image, we can better understand the structure and texture of the image. Binarize the grayscale image to generate the coordinates of the outermost contour endpoints. In order to extract the contour of the object in the image, we can binarize the grayscale image. The purpose of binarization processing is to convert the grayscale image into a binary image, in which there are only two colors, black and white. In a binary image, we can more easily identify the contour of the object. By analyzing the binary image, we can generate the coordinates of the outermost contour endpoints. The coordinates of the outermost contour endpoints can be used to describe the shape and size of the contour of the object in the image. For example, we can determine the size and shape of an object by calculating the perimeter and area of ​​its outermost contour.

[0045] Based on the coordinates of the endpoints of the outermost contour, the area information of the partition contour is generated. Based on the coordinates of the endpoints of the outermost contour, we can generate the area information of the partition contour. The area information of the partition contour can be used to describe the size and shape of the object in the image. For example, we can determine the size of the object by calculating the area of ​​the partition contour; and determine the type of the object by analyzing the shape of the partition contour. The area information of the partition contour and the color information of the partition image are processed to generate the category information to which the image to be identified belongs. The area information of the partition contour and the color information of the partition image are processed to generate the category information to which the image to be identified belongs. Category information can be used to describe the theme and content of the image. For example, we can determine whether the image is a landscape photo, a person photo, or an animal photo by analyzing the contour and color information of the object in the image. By preprocessing the image information to be identified, extracting features, graying, binarizing, generating contour coordinates and area information, and classifying, we can better understand and identify the content and theme in the image.

[0046] S106: Process the image usage information that matches the image information to be identified and the shooting influencing factor to generate an image scene identification influencing factor.

[0047] In one implementation, the image usage information that matches the image information to be identified is processed to generate event type information of the image to be identified, wherein the event type information of the image to be identified is used to characterize the image of the image to be identified. Assuming that we have an image to be identified, we first process the image usage information that matches the image. The image usage information may include being used for social media sharing, for photography exhibitions, for commercial advertising, and so on. If the image usage information indicates that the image is a travel photo for social media sharing, then the event type information of the image to be identified that can be generated through analysis is "travel." This event type information is used to characterize the subject content of the image to be identified, helping us better understand the meaning and background of the image.

[0048] The event type information of the image to be identified is processed based on the shooting image factor to generate an image processing influence factor and weight information corresponding to the image processing influence factor. Next, the event type information of the image to be identified is processed based on the shooting image factor. The shooting image factor can be generated based on the historical shooting images of the photographer. For example, if the photographer often takes landscape photos, there may be a specific shooting style and preference for landscape event types. Assume that when taking travel photos, the photographer usually pays attention to color vividness and picture clarity. Based on this shooting image factor, for the image to be identified of the "travel" event type, image processing influence factors such as "color enhancement" and "clarity improvement" can be generated. At the same time, corresponding weight information can also be assigned to each image processing influence factor, such as the weight of "color enhancement" is 0.6, and the weight of "clarity improvement" is 0.4, which means that when processing the image, the importance of color enhancement is relatively higher.

[0049] Based on the preset image processing influence feature set, the image processing influence factors and the weight information corresponding to the image processing influence factors are processed to generate image scene recognition influence factors. The preset image processing influence feature set may include various image processing features, such as contrast adjustment, saturation adjustment, sharpening, etc. By matching and analyzing the image processing influence factors and weight information with the preset image processing influence feature set, it is possible to determine which image processing features are most important for the current image to be recognized. For example, according to the two image processing influence factors of "color enhancement" and "clarity improvement" and their weight information, matching in the preset image processing influence feature set may determine that the image scene recognition influence factors are "moderately increase saturation" and "apply mild sharpening". These image scene recognition influence factors will guide subsequent image autofocus, scene recognition and image optimization processes, so that the processed images are more in line with the needs of the photographer and the purpose of the image.

[0050] S107: Process the category information of the image to be recognized and the image scene recognition influencing factor based on the target image scene recognition model to generate an image scene recognition result.

[0051] In one implementation, the category information of the image to be identified and the image scene identification influencing factor are smoothed based on the target image scene recognition model to generate an image coordinate factor. Assume that we have a target image scene recognition model that has been fully trained and can accurately identify the category and scene of the image. Now we have an image to be identified. First, based on this target image scene recognition model, the category information of the image to be identified (such as scenery, people or animals, etc.) and the image scene identification influencing factor (such as specific color adjustment requirements, clarity requirements, etc.) are smoothed. Smoothing can be imagined as a "harmonization" of this information to make the transition between different features more natural. For example, if the image is identified as a landscape category and there is a high color saturation requirement as a scene recognition influencing factor, then smoothing may adjust these features so that the change in color saturation is more uniform in the image and no abrupt areas appear. After smoothing, an image coordinate factor is generated, which can be regarded as a feature description of the image at different coordinate positions.

[0052] The image coordinate factors are convolved to generate feature information of multiple spatial scales. Next, the image coordinate factors are convolved. Convolution is like sliding a small "filter" on the image coordinate factors to extract different features. Through convolution, feature information of multiple spatial scales can be generated. For example, at a small scale, detailed features in the image can be extracted, such as the texture of leaves and facial expressions of people; at a large scale, the overall structural features of the image can be extracted, such as the outline of mountains and the shape of buildings. These multi-spatial scale feature information can more comprehensively describe the content and characteristics of the image.

[0053] Based on the preset feature division method, the feature information of multiple spatial scales is processed to generate the image scene recognition result. The preset feature division method can be classified according to different feature types, or divided according to specific thresholds. For example, if we divide the feature information of multiple spatial scales according to color features, texture features and shape features, and then judge the image scene according to the combination of different features. If the multi-spatial scale feature information of an image shows a large area of ​​green, the texture features of the leaves are obvious, and the shape presents an irregular natural shape, then the image scene can be judged as a forest. In this way, the final image scene recognition result is generated, which provides an accurate basis for subsequent image optimization and application.

[0054] In another embodiment, the feature information of multiple spatial scales is processed based on a preset feature division method to generate an image scene recognition result, including: obtaining the graphic features of the image to be identified based on the feature information of multiple spatial scales. Assuming that we have an image to be identified, we first obtain the graphic features of the image based on the feature information of multiple spatial scales. The feature information of multiple spatial scales contains various features of the image at different scales, such as the texture details of the image and the subtle changes in color at a small scale; the overall outline of the image and the shape of the main object at a large scale. For example, for a landscape photo, by analyzing the feature information of multiple spatial scales, the general outline of the mountain range can be extracted as one of the graphic features, and the texture features of the clouds in the sky and the color distribution features of the leaves in the forest can also be extracted. These graphic features can describe the content and structure of the image more specifically.

[0055] The graphic features of the image to be identified are processed to generate a multi-channel image, wherein the multi-channel image is used to characterize the image distribution of the image to be identified in different spaces. Next, the graphic features of the image to be identified are processed to generate a multi-channel image. The multi-channel image is used to characterize the image distribution of the image to be identified in different spaces. For example, the extracted graphic features can be mapped to different channels respectively. One channel can represent the color features of the image, another channel can represent the texture features of the image, and another channel can represent the shape features of the image, etc. In this way, each channel reflects the distribution of the image in a specific aspect. Taking a landscape photo as an example, the color channel may show the distribution of different colors such as the blue sky, green grass and mountains, and yellow flowers; the texture channel may show the soft texture of clouds, the delicate texture of leaves, the rough texture of rocks, etc.; the shape channel may show the undulating shape of mountains, the curved shape of rivers, the irregular shape of trees, etc.

[0056] The multi-channel image is processed based on the image coordinate factor to generate the image scene recognition result. The image coordinate factor has been generated in the previous step, which describes the characteristics of the image at different coordinate positions. By combining the image coordinate factor and the multi-channel image, the scene of the image can be judged more accurately. For example, the position of a certain area in the image is determined according to the image coordinate factor, and then the characteristics of the area in different channels are combined to determine which scene element this area belongs to. If an area appears blue and white in the color channel, a smooth texture in the texture channel, and an irregular circle in the shape channel, then it can be judged that this area is the sky. By performing such analysis on each area of ​​the entire image, the scene recognition result of the image is finally determined, such as judging whether it is a photo of snowy mountain scenery, a photo of sunset at the seaside, etc.

[0057] By acquiring training sample sets, information about images to be identified, information about image usage, and shooting influencing factors, accurate image scene recognition is achieved using advanced technical means. First, multiple information such as training sample sets is acquired, where shooting image factors are generated based on the photographer's historical shooting images. The training sample sets are preprocessed, including feature extraction to generate an original feature library, and then processed to obtain the first and second feature data sets. The first feature data set is processed by a classifier to generate a first prediction result, and the second feature data set is trained by a preset algorithm to generate a second prediction result, thereby generating identification information to characterize the spatial distribution characteristics of the photographic image.

[0058] Next, a preset image scene recognition model matching the identification information is obtained. The model is connected by an upsampling model and a downsampling model through a SkipConnection connection layer, and the feature information of the photographic image at different spatial scales can be obtained. Then, the preset model is trained using a training sample set with identification information. It is divided into a training set and a validation set, and multiple data groups are extracted from the training set for training to generate a trained model. The validation set is then used to process the model to obtain the validation result. If the conditions are met, it is determined to be the target image scene recognition model. For the image information to be identified, preprocessing is first performed to generate partition image information in the target format, extract color information, perform grayscale and binarization processing, generate the coordinates of the outermost contour endpoints and the partition contour area information, and finally determine the category information to which the image to be identified belongs. The image usage information and shooting influencing factors are processed to generate event type information, and then the image processing influencing factors and weight information are obtained. The image scene recognition influencing factors are generated in combination with the preset image processing influencing feature set.

[0059] Based on the target image scene recognition model, the category information of the image to be recognized and the image scene recognition influencing factors are smoothed to generate image coordinate factors, and convolution processing is performed on them to generate feature information of multiple spatial scales, and the graphic features of the image to be recognized are obtained. Multi-channel images are generated to characterize different spatial distributions, and finally, the image coordinate factors are combined to generate image scene recognition results, providing an efficient and accurate solution for image scene recognition.

[0060] In one embodiment, if Figure 2 As shown, the present application also provides an image scene recognition device based on artificial intelligence, comprising: The acquisition module 201 is used to acquire a training sample set, image information to be identified, image usage information and shooting influence factors matching the image information to be identified, wherein the shooting image factors are generated based on the historical shooting images of the photographer; acquire a preset image scene recognition model matching the identification information, wherein the preset image scene recognition model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a SkipConnection connection layer, and the preset image scene recognition model is used to acquire feature information of photographic images at different spatial scales; The processing module 202 is used to pre-process the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image; train the preset image scene recognition model based on the training sample set with identification information to generate a target image scene recognition model; process the image information to be recognized to generate category information to which the image to be recognized belongs; process the image usage information and the shooting influencing factors that match the image information to be recognized to generate an image scene recognition influencing factor; process the category information and the image scene recognition influencing factor to which the image to be recognized belongs based on the target image scene recognition model to generate an image scene recognition result.

[0061] The present application embodiment provides an electronic device, such as Figure 3 As shown, the electronic device 3 includes a first processor 300, a memory 301, a bus 302 and a communication interface 303. The first processor 300, the communication interface 303 and the memory 301 are connected via the bus 302; the memory 301 stores a computer program that can be run on the first processor 300, and when the first processor 300 runs the computer program, it executes the artificial intelligence-based image scene recognition method provided in any of the aforementioned embodiments of the present application.

[0062] The memory 301 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 303 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0063] The bus 302 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store programs, and the first processor 300 executes the programs after receiving the execution instruction. The image scene recognition method based on artificial intelligence disclosed in any implementation of the embodiment of the present application may be applied to the first processor 300, or implemented by the first processor 300.

[0064] The first processor 300 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 300. The above-mentioned first processor 300 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in a decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 301, and the first processor 300 reads the information in the memory 301 and completes the steps of the above method in combination with its hardware.

[0065] The electronic device provided by the above-mentioned embodiments of the present application and the artificial intelligence-based image scene recognition method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0066] The present application embodiment provides a computer-readable storage medium, such as Figure 4 As shown, the computer-readable storage medium 401 stores a computer program, and when the computer program is read and executed by the second processor 402, the aforementioned image scene recognition method based on artificial intelligence is implemented.

[0067] The technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which may be an air conditioner, a refrigeration device, a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0068] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the artificial intelligence-based image scene recognition method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0069] An embodiment of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a third processor to implement the method described above.

[0070] The computer program product provided by the above-mentioned embodiments of the present application and the artificial intelligence-based image scene recognition method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0071] Each embodiment in the present application is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for evaluating the embodiment of the image scene recognition method based on artificial intelligence, the electronic device, the electronic device, and the readable storage medium, since they are basically similar to the above-mentioned embodiment of the image scene recognition method based on artificial intelligence, the description is relatively simple, and the relevant parts can be referred to the partial description of the above-mentioned embodiment of the image scene recognition method based on artificial intelligence.

[0072] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. An image scene recognition method based on artificial intelligence, characterized in that: include: Acquire a training sample set, image information to be identified, image usage information matching the image information to be identified, and a shooting influence factor, wherein the shooting image factor is generated based on the historical images shot by the photographer; Preprocessing the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image; Acquire a preset image scene recognition model that matches the identification information, wherein the preset image scene recognition model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected via a SkipConnection connection layer, and the preset image scene recognition model is used to acquire feature information of a photographic image at different spatial scales; Training the preset image scene recognition model based on the training sample set with identification information to generate a target image scene recognition model; Processing the image information to be identified to generate category information to which the image to be identified belongs; Processing the image usage information and the shooting influencing factors that match the image information to be identified to generate an image scene identification influencing factor; Based on the target image scene recognition model, the category information of the image to be recognized and the image scene recognition influencing factor are processed to generate an image scene recognition result.

2. The method according to claim 1, characterized in that Preprocessing the training sample set to generate a training sample set with identification information includes: Extracting features from the training sample set to generate an original feature library; Processing the original feature library to generate a first feature data set and a second feature data set; Processing the first feature data set based on a classifier to generate a first prediction result; Training the second feature data set based on a preset algorithm to generate a second prediction result; The first prediction result and the second prediction result are processed to generate identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image.

3. The method according to claim 1, characterized in that The preset image scene recognition model is trained based on the training sample set with identification information to generate a target image scene recognition model, including: Dividing the training sample set with identification information into a training set for training the preset image scene recognition model and a verification set for verifying the preset image scene recognition model based on a preset ratio; Extracting a plurality of data groups from the training set, wherein each data group includes a preset number of data samples, wherein at least one data sample includes identification information; Training the preset image scene recognition model based on data samples in the multiple data groups to generate a trained image scene recognition model; Processing the trained image scene recognition model based on the verification set to generate a verification result; If the data samples containing identification information in the verification result are pixels representing the image scene and brightness matching the pixels of the image scene, the trained image scene recognition model is used as the target image scene recognition model.

4. The method according to claim 1, characterized in that Processing the image information to be identified to generate category information to which the image to be identified belongs includes: Preprocessing the image information to be identified to generate subarea image information in a target format, wherein the subarea image information in the target format is used to represent images of different sizes and different pixel brightness; Performing feature extraction processing on the subarea image information of the target format to generate color information of the subarea image; Performing grayscale processing on the partition image information of the target format to generate a grayscale image; Binarizing the grayscale image to generate the coordinates of the outermost contour endpoints; Generate partition contour area information based on the outermost contour endpoint coordinates; The subarea contour area information and the color information of the subarea image are processed to generate category information to which the image to be identified belongs.

5. The method according to claim 4, characterized in that Processing the image usage information and the shooting influencing factors that match the image information to be identified to generate the image scene identification influencing factors includes: Processing the image usage information that matches the image information to be identified to generate event type information of the image to be identified, wherein the event type information of the image to be identified is used to characterize the image of the image to be identified; Processing the event type information of the image to be identified based on the captured image factor to generate an image processing influence factor and weight information corresponding to the image processing influence factor; The image processing influence factor and the weight information corresponding to the image processing influence factor are processed based on a preset image processing influence feature set to generate an image scene recognition influence factor.

6. The method according to claim 5, characterized in that Processing the category information of the image to be identified and the image scene identification influencing factor based on the target image scene identification model to generate an image scene identification result, including: Based on the target image scene recognition model, smoothing is performed on the category information of the image to be recognized and the image scene recognition influencing factor to which it belongs, to generate an image coordinate factor; Performing convolution processing on the image coordinate factors to generate feature information of multiple spatial scales; The feature information of the multiple spatial scales is processed based on a preset feature division method to generate an image scene recognition result.

7. The method according to claim 6, characterized in that Processing the feature information of the multiple spatial scales based on a preset feature division method to generate an image scene recognition result includes: Acquire graphic features of the image to be identified based on the feature information of multiple spatial scales; Processing the graphic features of the image to be identified to generate a multi-channel image, wherein the multi-channel image is used to characterize the image distribution of the image to be identified in different spaces; The multi-channel image is processed based on the image coordinate factors to generate an image scene recognition result.

8. An image scene recognition device based on artificial intelligence, characterized in that: The device comprises: An acquisition module is used to acquire a training sample set, image information to be identified, image usage information and shooting influencing factors matching the image information to be identified, wherein the shooting image factors are generated based on the historical shooting images of the photographer; acquire a preset image scene recognition model matching the identification information, wherein the preset image scene recognition model includes an upsampling model and a downsampling model, the upsampling model and the downsampling model are connected through a Skip Connection layer, and the preset image scene recognition model is used to acquire feature information of photographic images at different spatial scales; A processing module is used to preprocess the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize the spatial distribution characteristics of the photographic image; train the preset image scene recognition model based on the training sample set with identification information to generate a target image scene recognition model; process the image information to be recognized to generate category information to which the image to be recognized belongs; process the image usage information and the shooting influencing factors that match the image information to be recognized to generate an image scene recognition influencing factor; process the category information and the image scene recognition influencing factor to which the image to be recognized belongs based on the target image scene recognition model to generate an image scene recognition result.

9. An electronic device, characterized in that: include: a first processor; and a memory for storing executable instructions of the first processor; Wherein, the first processor is configured to execute the artificial intelligence-based image scene recognition method described in any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the second processor, the image scene recognition method based on artificial intelligence described in any one of claims 1 to 7 is implemented.

Images