Image scene recognition method, device, equipment and readable storage medium

By extracting the global features and regional sub-graph features of the target image, combining attention mechanism and image recognition classification library labels, the problem of low accuracy in scene recognition in the prior art is solved, and the recognition accuracy is improved, especially in the lack of detection targets. in the scene.

CN113705293BActive Publication Date: 2024-11-08TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110218951.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-26
Publication Date
2024-11-08
Estimated Expiration
2041-02-26

AI Technical Summary

Technical Problem

The prior art in scene recognition, especially in seaside, forest and other scenarios, resulting in low accuracy and high failure rate.

Method used

The global features of the target image are extracted, and the area submap is extracted from the image based on the attention mechanism, and scene recognition is combined with global features and submap features, and scene category labels in the image recognition classification library are used for annotation to improve recognition accuracy.

Benefits of technology

Through the joint recognition of global features and sub-graph features, the scene recognition accuracy of the target image is improved, the recognition influence caused by unsupervised learning in traditional methods is avoided, and the recognition ability of scenes is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113705293B_ABST
    Figure CN113705293B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, equipment and readable storage medium for recognizing image scenes, and relates to the field of machine learning. The method comprises: acquiring a target image; extracting global features of the target image; extracting at least two regional subgraphs from the target image based on the global features and an image recognition classification library; and identifying and obtaining a scene category label corresponding to the target image based on the global features and the subgraph features of the regional subgraphs. In the scene recognition process of the target image, after extracting the global features of the target image, an attention mechanism is used to extract regional subgraphs from the target image based on the global features, so that the target image is subjected to scene recognition based on the subgraph features and the global features, that is, the reference content of the scene recognition includes not only a single entity in the target image, but also image regions in the target image that are related to each scene category label in the image recognition classification library, thereby improving the scene recognition accuracy of the target image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of machine learning, and in particular to a method, device, equipment and readable storage medium for recognizing image scenes. Background Art

[0002] Scene recognition refers to the recognition of depicted scenes in multimedia content, which can be applied to video scene recognition or image scene recognition. When applied to video scene recognition, scene recognition is performed on image frames in the video, and scene features are usually in the background environment of the recognized image.

[0003] In the related art, when implementing scene recognition, object detection is usually performed first, and the object detection result is used as preliminary information, so as to perform scene recognition on the image based on the object detection result to obtain the scene recognition result, thereby obtaining the scene label of the image.

[0004] However, since not all scenes have scene detection targets, for example, the key objects in scenes such as the seaside and forest are scattered and cannot be accurately identified, there is a great impact on the accuracy of scene recognition, and the scene recognition failure rate is high. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and readable storage medium for recognizing an image scene, which can improve the recognition accuracy of the image scene. The technical solution is as follows:

[0006] In one aspect, a method for recognizing an image scene is provided, the method comprising:

[0007] Acquire a target image, where the target image is an image scene to be recognized;

[0008] Extracting global features of the target image, wherein the global features are features obtained by extracting features of the target image as a whole;

[0009] Extracting at least two regional sub-images from the target image based on the global feature and an image recognition classification library, wherein the image recognition classification library includes scene category labels for annotating images;

[0010] Based on the global features and the sub-image features of the regional sub-image, a scene category label corresponding to the target image is identified.

[0011] In another aspect, a device for recognizing an image scene is provided, the device comprising:

[0012] An acquisition module, used to acquire a target image, wherein the target image is an image scene to be recognized;

[0013] An extraction module, used to extract global features of the target image, wherein the global features are features obtained by extracting features of the target image as a whole;

[0014] The extraction module is further used to extract at least two regional sub-images from the target image based on the global feature and an image recognition classification library, wherein the image recognition classification library includes scene category labels for annotating images;

[0015] The recognition module is used to identify the scene category label corresponding to the target image based on the global feature and the sub-image feature of the regional sub-image.

[0016] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement an image scene recognition method as described in any of the above-mentioned embodiments of the present application.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image scene recognition method as described in any of the above-mentioned embodiments of the present application.

[0018] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image scene recognition method described in any of the above embodiments.

[0019] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0020] In the scene recognition process of the target image, after extracting the global features of the target image, the attention mechanism is used to extract the regional subgraph from the target image based on the global features, so as to perform scene recognition on the target image based on the subgraph features and the global features. That is, the reference content of scene recognition not only includes a single entity in the target image, but also includes the image areas in the target image that are related to each scene category label in the image recognition classification library, thereby improving the scene recognition accuracy of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0023] Figure 2 is a flow chart of an image scene recognition method provided by an exemplary embodiment of the present application;

[0024] Figure 3 is based on Figure 2 A schematic diagram of a scene category label recognition process provided by the illustrated embodiment;

[0025] Figure 4 is a flow chart of an image scene recognition method provided by another exemplary embodiment of the present application;

[0026] Figure 5 is based on Figure 4 A schematic diagram of a mapping relationship between feature points and candidate sub-images in a target image provided by an illustrated embodiment;

[0027] Figure 6 is a flow chart of an image scene recognition method provided by another exemplary embodiment of the present application;

[0028] Figure 7 is based on Figure 6 A schematic diagram of a loss value calculation process of a scene recognition model provided by the illustrated embodiment;

[0029] Figure 8 is a structural block diagram of an image scene recognition device provided by an exemplary embodiment of the present application;

[0030] Fig. 9 is a structural block diagram of an image scene recognition device provided by another exemplary embodiment of the present application;

[0031] Fig.10 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0033] First, a brief introduction is given to the terms involved in the embodiments of this application:

[0034] Artificial Intelligence (AI): It is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0036] In the embodiments of the present application, computer vision technology is mainly involved in performing scene recognition on the image content. Schematically, after the target image is input into the scene recognition model, the scene recognition result of the target image is output, such as: the scene of the target image is recognized as a library scene.

[0037] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0038] Computer Vision (CV): Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further process images so that the computer processes them into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0039] Scene recognition: refers to the recognition of the background of an image or video. It is different from entity recognition. Entity recognition refers to the recognition of the category of objects appearing in an image or video, such as recognizing that the entity in the image belongs to a person or an animal, while scene recognition mainly recognizes the background features of an image or video, such as recognizing that the scene corresponding to the current image is a restaurant. In the related art, in the process of implementing scene recognition, scene recognition is realized based on multi-scale salient region feature learning, that is, firstly, object detection is performed on the image to obtain object detection results, and one or more parts of the image with objects are obtained from the object detection results; secondly, the key scene information is extracted through the object parts for recognition, so as to identify the scene information corresponding to the image.

[0040] Scene recognition belongs to high-level semantic recognition, which is usually more difficult than entity category recognition, because scene features are often in the background environment of image recognition, and conventional image recognition pre-training models focus on extracting features on specific entities or parts, which easily leads to overfitting of scene recognition to the foreground of the target scene, that is, the scene recognition model remembers the situation in certain scenes. In addition, in the scene recognition scheme, not all scenes have detection targets, such as: the seaside, forest, etc. usually do not have relatively specific detection entities, so scene recognition is more difficult.

[0041] This application proposes an end-to-end scene recognition method based on local image positioning and robust feature combination, which realizes scene recognition by performing self-supervised attention feature extraction in high-dimensional image space, supervised learning of multiple attention blocks, and finally using robust joint features. Since this method performs multi-task supervised learning on local positioning and recognition, and global recognition, optimizes global features through multi-feature combination, and optimizes local features through local feature classification and positioning, it avoids the problem of local influence on classification in scene recognition due to the lack of supervised learning of local block positioning and local block feature extraction in traditional scene recognition.

[0042] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application, such as Figure 1 As shown, the implementation environment includes a terminal 110 and a server 120 , wherein the terminal 110 and the server 120 are connected via a communication network 130 .

[0043] The terminal 110 is used to send multimedia content requiring scene recognition to the server 120, wherein the multimedia content includes image content or video content. The terminal 110 sends the multimedia content to the server 120 via the communication network 130, and instructs the server 120 to perform scene recognition on the multimedia content.

[0044] In the above content, multimedia content is sent from the terminal 110 to the server 120 as an example. In some embodiments, the multimedia content is content stored in the server 120 itself, or content sent to the server 120 by other servers.

[0045] The server 120 includes a scene recognition model 121, which has the following features: 1. Mining relevant features of local blocks of the image through the attention mechanism; 2. Extracting key scene information for recognition through the fusion of local attention and global features; 3. The model implements end-to-end scene recognition through three steps: inputting pictures, self-supervising attention learning and extraction, and combining recognition results. After receiving the multimedia content sent by the terminal 110, the server 120 inputs the multimedia content into the scene recognition model 121.

[0046] Among them, when the multimedia content is image content, the image content is directly input into the scene recognition model 121; when the multimedia content is video content, the video content is first decoded, and image frames are obtained from the video content, and then the image frames are input into the scene recognition model 121 for scene recognition, or after the video content is decoded, the decoded video stream is input into the scene recognition model 121 for scene recognition frame by frame, which is not limited in the embodiments of the present application.

[0047] In some embodiments, after the server 120 obtains the scene recognition result through the scene recognition model 121, the scene recognition result is fed back to the terminal 110 through the communication network 130. After receiving the scene recognition result, the terminal 110 displays the scene recognition result, or classifies the multimedia content based on the scene recognition result, etc., which is not limited to this in the embodiments of the present application.

[0048] It is worth noting that the image scene recognition method provided in the embodiment of the present application can be implemented by a terminal, can be implemented by a server, or can be implemented by a terminal and a server in collaboration. That is, in the above embodiment, the scene recognition model 121 is located in the server 120 as an example for explanation. In some embodiments, the above scene recognition model 121 can also be configured in the terminal 110, or part of the network in the scene recognition model 121 can be configured in the terminal 110, and the other part of the network can be configured in the server 120.

[0049] The terminal includes at least one of a smart phone, a tablet computer, a portable laptop computer, a desktop computer, a smart speaker, a smart wearable device, a smart face recognition device, etc. The server can be a physical server or a cloud server that provides cloud computing services, and the server can be implemented as a single server or a server cluster or distributed system composed of multiple servers. When the terminal and the server collaborate to implement the solution provided in the embodiment of the present application, the terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application.

[0050] In combination with the above description, the application scenarios of the embodiments of the present application are introduced.

[0051] First, the application scenarios of the scene recognition function.

[0052] In some embodiments, an application is installed in the terminal, and the application provides a scene recognition function. When the user uses the above application, the user uploads the target image or target video in the function interface of the scene recognition function, so that the terminal sends the target image or target video to the server for scene recognition. After the server performs scene recognition on the target image or target video, it feeds back the scene recognition result obtained, and the terminal displays the scene recognition result.

[0053] Second, the application scenarios of content classification.

[0054] Schematically, taking a video application as an example, there are n video contents stored in the server, where n is a positive integer. After the scene recognition model is used to perform scene recognition on the n video contents, the n video contents are classified according to the scene recognition results corresponding to the n video contents, so that in the video content selection interface of the terminal, the options corresponding to the n video contents are displayed in categories, and the user can view the video contents of the specified scene category in categories. Schematically, after the user selects the library scene in the video application, the candidate video contents corresponding to the library scene are displayed in the terminal interface, so that the user can select and view the target video content from the candidate video contents.

[0055] It is worth noting that in the distance of the above-mentioned application scenarios, the scene recognition function and the content classification function are used as examples for explanation. The image scene recognition method provided in the present application can also be applied to other application scenarios that require scene recognition and analysis of images, and the embodiments of the present application are not limited to this.

[0056] Based on the above content, the image scene recognition method provided by the embodiment of the present application is described. Figure 2 FIG. 1 is a flow chart of an image scene recognition method provided by an exemplary embodiment of the present application, and the method is applied to a server as an example for explanation. Figure 2 As shown, the method includes:

[0057] Step 201 : acquiring a target image, where the target image is an image scene to be recognized.

[0058] In some embodiments, the method of acquiring the target image includes at least one of the following methods.

[0059] First, the server receives video content uploaded by the terminal, and the video content includes at least one of short videos or regular videos, wherein short videos refer to videos uploaded on short video platforms with a duration less than the required duration, or videos played on short video playback platforms; regular videos refer to videos with no restrictions on video format or duration. After receiving the video content, the server needs to perform scene recognition on the video content, so it first decodes the video content to obtain decoded video frames, and then obtains specified frames (such as key frames, or image frames at specified positions) from the decoded video frames as target images for scene recognition.

[0060] Second, the server stores video content. Based on the video scene classification function, the stored video content needs to be classified into scenes, so that the server obtains the stored video for decoding, and obtains the specified video frames and identifies the scenes one by one.

[0061] Third, the server receives the image content uploaded by the terminal. The image content is the content that the user specifies needs to be recognized by the scene, so the image content is used as the target image for scene recognition.

[0062] In the above examples, the target image is taken as an example of an image that needs to be scene recognized in an actual application. In some embodiments, the target image can also be implemented as a sample image annotated with a sample category label, that is, the target image is an image to be image scene recognized, and the scene recognition model is trained based on the sample category label. The image scene of the target image is realized by the scene recognition model, and the target image itself is annotated with a sample category label as a sample. After the scene recognition model is used to perform scene recognition on the target image to obtain the scene category label, the scene recognition model is trained based on the scene category label and the sample category label (that is, the model parameters of the scene recognition model are adjusted).

[0063] In some embodiments, when the target image is implemented as a sample image, the target image is an image randomly selected from a sample image library, or an image selected in rounds from the sample image library.

[0064] Step 202: extract global features of the target image.

[0065] Global features are features obtained by extracting features of the target image as a whole.

[0066] In some embodiments, the scene recognition model includes a feature extraction network, and global features of the target image are extracted by inputting the entire target image into the feature extraction network.

[0067] In some embodiments, a ResNet101 network is used to extract global features of a target image. The ResNet101 network is implemented as a 5-layer convolutional neural network structure. For schematic illustration, please refer to the following Table 1.

[0068] Table 1

[0069]

[0070] Among them, taking "7×7, 64" as an example, 7×7 represents the convolution size, and 64 represents the number of channels. That is, in "1×1, 128", 1×1 represents the convolution size, and 128 represents the number of convolution channels.

[0071] That is, by inputting the target image into the feature extraction network shown in Table 1, the global features of the target image can be extracted.

[0072] Step 203: extract at least two region sub-images from the target image based on the global features and the image recognition classification library.

[0073] The image recognition classification library includes scene category labels for annotating images, wherein the scene category labels are set by developers, or the scene category labels are manually annotated on sample images and stored in the image recognition classification library before the sample images are trained on the scene recognition model.

[0074] In some embodiments, firstly, based on the global features and the image recognition classification library, the feature points in the global features are identified to obtain the candidate point scores corresponding to the feature points, and the confidence of the candidate sub-image in the target image is determined based on the candidate point scores, and there is a mapping relationship between the candidate sub-image and the feature points. At least two regional sub-images are determined from the candidate sub-image of the target image based on the confidence.

[0075] That is, in the process of extracting regional sub-images, it is first necessary to identify the feature points corresponding to the global features under different recognition dimensions, and the recognition process is based on the association between the feature points and the scene category labels in the image recognition classification library. Since the global features are identified under different recognition dimensions, the feature points are mapped to the corresponding candidate sub-images in the original target image, and the sizes of the candidate sub-images obtained by mapping the feature points under different dimensions are different. Based on the recognition of the feature points and the mapping relationship between the feature points and the target image, at least two regional sub-images are determined.

[0076] Step 204: Based on the global features and the sub-image features of the regional sub-image, a scene category label corresponding to the target image is identified.

[0077] First, feature extraction is performed on the regional sub-image to obtain sub-image features, and then recognition is performed based on global features and sub-image features to obtain scene category labels.

[0078] Among them, when extracting features from a regional sub-map, a feature extraction network as described in Table 1 above is used for extraction, or the feature extraction network shown in Table 1 above is a first feature extraction network, and a second feature extraction network is used to extract sub-map features of the regional sub-map, the network structures of the first feature extraction network and the second feature extraction network are the same or different, and the network parameters of the first feature extraction network and the second feature extraction network are the same or different.

[0079] In some embodiments, the global features are merged with the sub-image features, and the merged features are identified to obtain a scene category label.

[0080] Among them, the merging methods of the above-mentioned global features and sub-image features include: one-by-one splicing method, overall splicing method, etc., among which, the one-by-one splicing method refers to splicing the sub-image features of at least two regional sub-images with the global features respectively to obtain basic fusion features, thereby splicing at least two basic fusion features to obtain fusion features; the overall splicing method refers to splicing the sub-image features of at least two regional sub-images and the global features as a whole to obtain fusion features.

[0081] In the embodiment of the present application, the overall splicing method is used as an example for explanation. Figure 3 , which shows the recognition process of scene category labels provided by an exemplary embodiment of the present application. Figure 3 As shown in FIG. 1 , in the process, a target image 300 is first obtained, and a global feature 301 is extracted by a convolutional neural network 310; based on the global feature 301 of the target image 300, an attention position 330 is obtained by an attention extraction module 320, thereby extracting at least two regional sub-images 340 (such as Figure 3 4 regional sub-graphs are extracted as shown), and sub-graph features 341 of the regional sub-graph 340 are extracted by the convolutional neural network 310 (such as Figure 3 As shown, four sub-image features corresponding to the four regional sub-images are extracted, and a fusion feature 350 is obtained by merging the global feature 301 and the sub-image feature 341. Scene recognition is performed based on the fusion feature 350 to obtain a scene category label 360.

[0082] In some embodiments, after the attention position 330 is acquired by the attention extraction module 320 , the attention loss value 370 is obtained to train the attention extraction module 320 .

[0083] In summary, the image scene recognition method provided in the embodiment of the present application, for the scene recognition process of the target image, after extracting the global features of the target image, uses the attention mechanism to extract the regional subgraph from the target image based on the global features, so as to perform scene recognition on the target image based on the subgraph features and the global features. That is, the reference content of the scene recognition includes not only a single entity in the target image, but also image areas in the target image that are related to each scene category label in the image recognition classification library, thereby improving the scene recognition accuracy of the target image.

[0084] In some embodiments, a region sub-image is extracted from a target image via an attention mechanism. Figure 4 FIG. 1 is a flow chart of an image scene recognition method provided by another exemplary embodiment of the present application, and takes the method applied in a server as an example for explanation. Figure 4 As shown, the method includes:

[0085] Step 401: Acquire a target image, where the target image is an image scene to be recognized.

[0086] The method of acquiring the target image has been described in the above step 201 and will not be repeated here.

[0087] Step 402: extracting global features of the target image.

[0088] Global features are features obtained by extracting features of the target image as a whole.

[0089] In some embodiments, the scene recognition model includes a feature extraction network, and global features of the target image are extracted by inputting the entire target image into the feature extraction network.

[0090] Step 403: Based on the global features and the image recognition classification library, feature points in the global features are identified to obtain candidate point scores corresponding to the feature points.

[0091] In some embodiments, global feature classification is performed based on an image recognition classification library, thereby obtaining candidate point scores of feature points based on the global feature classification results.

[0092] Based on the feature extraction network shown in Table 1 above, a feature classification network is determined, and global feature classification is performed based on the feature classification network. The structure of the feature classification network is shown in Table 2 below.

[0093] Table 2

[0094]

[0095] After global feature classification is performed through the feature classification network shown in Table II, regional sub-images are extracted from the target image based on the attention mechanism.

[0096] In some embodiments, the candidate point scores of feature points in the global features are determined by a part extraction network based on an attention mechanism. Schematically, please refer to the following Table 3, which shows the network structure of the part extraction network.

[0097] Table 3

[0098]

[0099] Among them, the part extraction network includes two layers of networks, namely, down layer and Propost layer. The layer name of the downlayer layer is shown as Down1_y, and the layer name of the Propost layer is shown as Propost2_y.

[0100] The matrix size output by the Propost2_y layer is 6×9×15, where 6 represents the number of channels. The 128 channels of the previous network structure are compressed into 6 channels through the Propost2_y layer, and 9×15 represents the length and width of the space after convolution. The feature point in 9×15 represents the attention intensity of the spatial coordinates where the feature point is located. The feature point can be mapped to an area in the target image (i.e., a candidate sub-image). After reshaping, the attention intensity 6×9×15 matrix output by the Propost2_y layer is converted into 6×9×15=810 candidate point scores corresponding to the feature points.

[0101] It is worth noting that the above Table 3 uses 6 channels as an example for explanation. In some embodiments, the number of channels may be more or less, which may be based on the experimental experience of developers, and the embodiments of the present application do not limit this.

[0102] Step 404 : determining the confidence of the candidate sub-image in the target image based on the candidate point scores.

[0103] Since the global feature is a feature image obtained by upsampling or downsampling the target image, the feature points in the space after the global feature is convolved with 6 channels can be mapped to the target image to obtain the corresponding candidate sub-image.

[0104] In some embodiments, the candidate point scores corresponding to the feature points after channel convolution are normalized to obtain the confidence of the candidate sub-image corresponding to the feature points; or, the candidate point scores corresponding to the feature points after channel convolution are used as the confidence of the corresponding candidate sub-image.

[0105] Step 405 : determining at least two region sub-images from the candidate sub-images of the target image based on the confidence level.

[0106] In some embodiments, candidate sub-images of the target image are arranged in order based on confidence levels, a designated sub-image is determined from the ordered candidate sub-images, the designated sub-image being a sub-image that meets designated requirements among the candidate sub-images, a required sub-image is determined from the candidate sub-images based on an overlapping relationship between the designated sub-image and the candidate sub-images, the required sub-image being a sub-image whose overlapping relationship with the designated sub-image meets overlapping relationship requirements, and at least two regional sub-images are determined based on the required sub-image and the designated sub-image.

[0107] In some embodiments, from the candidate subgraphs arranged in order, a candidate subgraph with the highest confidence value is determined as the designated subgraph.

[0108] When determining the required subgraph based on the overlapping relationship, the overlap rate (Intersection over Union, IoU) between the specified subgraph and the candidate subgraph is determined. In response to the overlap rate between the specified subgraph and the candidate subgraph reaching the overlap rate threshold, the candidate subgraph is retained and determined as the required subgraph; and in response to the overlap rate between the specified subgraph and the candidate subgraph being less than the overlap rate threshold, the candidate subgraph is discarded.

[0109] Schematically, the feature points correspond to the candidate sub-images in the target image respectively. After the candidate sub-images are determined, at least two regional sub-images are obtained by the non-maximum suppression algorithm (Hard Non-Maximum Suppression, Hard NMS). Among them, the execution method of Hard NMS includes: sorting the confidence of each candidate sub-image given by the model from large to small, then retaining the largest one, and deleting all candidate sub-images whose IoU with the candidate sub-image with the maximum confidence is greater than the threshold. Schematically, there are 4 candidate sub-images: (box1, 0.8), (box2, 0.9), (box3, 0.7), (box4, 0.5), 0.8, 0.9, 0.7 and 0.5 are used to represent the confidence of each candidate sub-image corresponding to a certain scene category label. These four candidate sub-images are sorted from large to small according to the confidence to obtain: box2>box1>box3>box4. Keep the candidate box box2 with the highest confidence, calculate the IoU between the remaining three boxes and box2, and if the IoU is greater than a preset threshold, then delete the box. Assuming the preset threshold is 0.5, IoU(box1, box2) = 0.1 < 0.5, keep; IoU(box3, box2) = 0.7 < 0.5, delete; IoU(box4, box2) = 0.2 < 0.5, keep; thus box1, box2 and box4 are kept, and then repeat the above process for other scene category labels to sort, delete and keep.

[0110] For illustration, please refer to Figure 5 , the target image 500 is convolved through 6 channels in the first convolution layer, then pooled in the first pooling layer, then convolved in the second convolution layer, and pooled in the second pooling layer, and then classified based on the Softmax layer after passing through the fully connected layer. Among them, the feature points can be mapped to obtain the candidate sub-image 510 in the target image 500.

[0111] Step 406: Based on the global features and the sub-image features of the regional sub-image, a scene category label corresponding to the target image is identified.

[0112] First, sub-image features of at least two regional sub-images are extracted, and then recognition is performed based on global features and sub-image features to obtain scene category labels.

[0113] In some embodiments, the global features and the sub-image features are merged to obtain fused features, and scene recognition is performed on the fused features to obtain a scene category label corresponding to the target image.

[0114] Indicatively, the pooling results are first directly connected to obtain a (1+K)×2048 feature vector, where K is the number of region sub-images, and then the fully connected layer is used to predict the probability of belonging to N categories. The input of the fully connected layer is 1×(1+K)×2048, and the output is 1×N. This layer calculates the probability that all (global + local) features of the target image predict a certain scene category label, and finally obtains a classification result.

[0115] In summary, the image scene recognition method provided in the embodiment of the present application, for the scene recognition process of the target image, after extracting the global features of the target image, uses the attention mechanism to extract the regional subgraph from the target image based on the global features, so as to perform scene recognition on the target image based on the subgraph features and the global features. That is, the reference content of the scene recognition includes not only a single entity in the target image, but also image areas in the target image that are related to each scene category label in the image recognition classification library, thereby improving the scene recognition accuracy of the target image.

[0116] The method provided in this embodiment determines the regional sub-image from the target image based on the mapping relationship between the feature points and the candidate sub-images, and thereby identifies the scene category labels based on the merging of the sub-image features of the regional sub-image with the global features, thereby further improving the recognition accuracy of the scene category labels.

[0117] In some embodiments, the target image is a sample image annotated with a sample category label, that is, the scene recognition model is trained through the target image. Figure 6 FIG. 1 is a flow chart of an image scene recognition method provided by another exemplary embodiment of the present application, and takes the method applied in a server as an example for explanation. Figure 6 As shown, the method includes:

[0118] Step 601: Acquire a target image, where the target image is an image scene to be recognized.

[0119] The method of acquiring the target image has been described in the above step 201 and will not be repeated here.

[0120] Step 602: extract global features of the target image.

[0121] Global features are features obtained by extracting features of the target image as a whole.

[0122] In some embodiments, the scene recognition model includes a feature extraction network, and global features of the target image are extracted by inputting the entire target image into the feature extraction network.

[0123] Step 603: extract at least two region sub-images from the target image based on the global features and the image recognition classification library.

[0124] The image recognition classification library includes scene category labels for annotating images, wherein the scene category labels are set by developers, or the scene category labels are manually annotated on sample images and stored in the image recognition classification library before the sample images are trained on the scene recognition model.

[0125] The method of extracting the regional sub-image has been described in detail in the above steps 403 to 406, and will not be repeated here.

[0126] Step 604: Obtain image prediction results corresponding to the global features and the sub-image features of the regional sub-image through scene recognition model recognition.

[0127] Among them, the image prediction result includes: at least one of a global prediction result, an attention prediction result, a positioning prediction result and a sub-image prediction result. In this embodiment, the image prediction result including a global prediction result, an attention prediction result, a positioning prediction result and a sub-image prediction result is taken as an example for explanation.

[0128] Among them, the global prediction result refers to the prediction result obtained by scene recognition based on the global features of the target image; the attention prediction result refers to the prediction result obtained based on the global features and the sub-image features of the regional sub-image determined by the attention mechanism; the positioning prediction result refers to the positioning accuracy of the predicted regional sub-image; the sub-image prediction result refers to the prediction result obtained by scene recognition of the regional sub-image.

[0129] That is, the global features are predicted by the scene recognition model to obtain the global prediction results; the fusion features of the sub-image features and the global features are predicted by the scene recognition model to obtain the scene category label corresponding to the target image as the attention prediction result; the positioning accuracy of the sub-image features of at least two regional sub-images is predicted by the scene recognition model to obtain the positioning prediction result; the sub-image features of at least two regional sub-images are classified and predicted by the scene recognition model to obtain the sub-image prediction results.

[0130] Step 605: Based on the image prediction result and the sample category label, a loss value of the image prediction result is obtained.

[0131] In some embodiments, a first loss value is obtained based on the global prediction result and the sample category label; a second loss value is obtained based on the attention prediction result and the sample category label; a third loss value is obtained based on the positioning prediction result and the sample category label, and a fourth loss value is obtained based on the sub-image prediction result and the sample category label, thereby obtaining a loss value of the image prediction result based on the first loss value, the second loss value, the third loss value and the fourth loss value.

[0132] The first loss value is calculated as follows: the first loss value is calculated using the following formula 1, where the formula 1 is a cross entropy loss function for classification. The input is a target image with a labeled sample category label.

[0133] Formula 1:

[0134] Among them, y is the sample category label annotated by the target image, is the scene category label obtained by global prediction, and L is the first loss value.

[0135] The calculation method of the second loss value refers to the calculation method of the first loss value mentioned above, that is, the scene category label obtained by the global prediction of the above formula Replaced with the scene category label corresponding to the attention prediction result.

[0136] The third loss value is calculated as follows: after extracting the regional sub-image through the attention mechanism, a positioning accuracy prediction network is designed to calculate the attention positioning prediction to learn the final N categories, with the goal of making each attention output result category-aware. Among them, after at least two regional sub-images are input into the feature extraction network, they pass through the pooling layer and the positioning accuracy prediction network, and the output result is K×N, that is, the probability of K regional sub-images belonging to any of the N categories is predicted respectively, and finally K positioning loss values ​​are calculated to obtain the positioning accuracy loss of the regional sub-image (i.e., the third loss value). Among them, the third loss value is the average of the K positioning loss values, or the sum of the K positioning loss values.

[0137] The fourth loss value is calculated as follows: after extracting the regional subgraph through the attention mechanism, a subgraph prediction network is designed to predict the probability that the regional subgraph belongs to N categories, in order to enable the network to have the ability to recognize the local features of the regional subgraph. Among them, after at least two regional subgraphs are input into the feature extraction network, they pass through the pooling layer and the subgraph prediction network, and the output result is K×N, that is, the probability of K regional subgraphs belonging to any of the N categories is predicted respectively, and finally K classification loss values ​​are calculated and the subgraph recognition accuracy loss (that is, the fourth loss value mentioned above) is obtained. Among them, the fourth loss value is the sum of the K classification loss values.

[0138] In some embodiments, the loss value of the image prediction result is a weighted sum of the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0139] Formula 2: loss=a×Loss_cr+b×Loss_locate+c×Loss_part+d×Loss_all

[0140] Among them, loss represents the loss value of the image prediction result, Loss_cr represents the above-mentioned first loss value, Loss_all represents the above-mentioned second loss value, Loss_locate represents the above-mentioned third loss value, Loss_part represents the above-mentioned fourth loss value, and a is the first weight of the first loss value, d is the second weight of the second loss value, b is the third weight of the third loss value, and c is the fourth weight of the fourth loss value.

[0141] Step 606: Adjust the parameters of the scene recognition model based on the loss value of the image prediction result.

[0142] In some embodiments, multiple rounds of iterations are performed until the model converges.

[0143] Schematically, the stochastic gradient descent (SGD) method is used to solve the convolution parameters and bias parameters of the scene recognition model, and all parameters of the scene recognition model are set to a state that requires learning; in each iteration, m sample images are extracted; all sample images are forward calculated, the loss value is calculated and back-propagated to the scene recognition model (i.e., the convolutional neural network model), the gradient is calculated and the parameters of the scene recognition model are updated; the above process is iterated for multiple rounds.

[0144] It is worth noting that the neural network model shown in the above embodiment is only a schematic example, and the embodiments of the present application do not limit the specific structure and specific parameter settings of the model.

[0145] For illustration, please refer to Figure 7 , which shows a schematic diagram of the loss value calculation process of a scene recognition model provided by an exemplary embodiment of the present application, such as Figure 7 As shown, first, the sample image 700 marked with the sample category label 710 is subjected to feature extraction through the feature extraction network 701 to obtain global features, and then global feature classification is performed through the global feature classification network 702 to obtain the global prediction result 703, and the first loss value is obtained based on the global prediction result 703 and the sample category label 710.

[0146] At least two regional sub-images 705 are obtained from the sample image 700 based on the global features through the part extraction network 704 based on the attention mechanism, and the sub-image features of the regional sub-image 705 are extracted through the feature extraction network 701, and the sub-image features are merged with the global features to obtain the fused features 706, and the fused features 706 are predicted through the global attention classification prediction network 707 to obtain the global prediction result 708, thereby obtaining the second loss value based on the global prediction result 708 and the sample category label 710.

[0147] After extracting the sub-graph features, a positioning prediction result 711 is predicted by the positioning accuracy prediction network 709 , and a third loss value is obtained based on the positioning prediction result 711 and the sample category label 710 .

[0148] After extracting the sub-graph features, a sub-graph prediction result 713 is predicted by the sub-graph prediction network 712 , and a fourth loss value is obtained based on the sub-graph prediction result 713 and the sample category label 710 .

[0149] Thus, based on the first loss value, the second loss value, the third loss value and the fourth loss value, the loss value of the image prediction result is obtained, and the scene recognition model is trained based on the loss value.

[0150] In summary, the image scene recognition method provided in the embodiment of the present application, for the scene recognition process of the target image, after extracting the global features of the target image, uses the attention mechanism to extract the regional subgraph from the target image based on the global features, so as to perform scene recognition on the target image based on the subgraph features and the global features. That is, the reference content of the scene recognition includes not only a single entity in the target image, but also image areas in the target image that are related to each scene category label in the image recognition classification library, thereby improving the scene recognition accuracy of the target image.

[0151] The model uses attention learning to extract local features of the target image that are easily ignored globally and combines them with global features for recognition and classification, improving the model's ability to describe richer features of different scenes, thereby optimizing the recognition effect of the scene.

[0152] The overall process is learned end-to-end, which avoids the difficulty of optimizing the model in stages and the problem that feature learning in different stages cannot be shared.

[0153] Figure 8 is a structural block diagram of an image scene recognition device provided by an exemplary embodiment of the present application, such as Figure 8 As shown, the device comprises:

[0154] An acquisition module 810 is used to acquire a target image, where the target image is an image scene to be recognized;

[0155] An extraction module 820 is used to extract global features of the target image, where the global features are features extracted from the entire target image;

[0156] The extraction module 820 is further used to extract at least two regional sub-images from the target image based on the global feature and an image recognition classification library, wherein the image recognition classification library includes scene category labels for annotating images;

[0157] The identification module 830 is used to identify and obtain a scene category label corresponding to the target image based on the global feature and the sub-image feature of the regional sub-image.

[0158] In some embodiments, the recognition module 830 is further used to recognize feature points in the global features based on the global features and the image recognition classification library to obtain candidate point scores corresponding to the feature points;

[0159] like Fig. 9 As shown, the extraction module 820 includes:

[0160] A determination unit 821, configured to determine the confidence of a candidate sub-image in the target image based on the candidate point score, wherein there is a mapping relationship between the candidate sub-image and the feature point;

[0161] The determining unit 821 is further configured to determine the at least two region sub-images from the candidate sub-images of the target image based on the confidence level.

[0162] In some embodiments, the extraction module 820 further includes:

[0163] A sorting unit 822, configured to arrange the candidate sub-images of the target image in order based on the confidence level;

[0164] The determining unit 821 is further configured to determine a designated subgraph from the candidate subgraphs arranged in order, wherein the designated subgraph is a subgraph in the candidate subgraphs that meets a designated requirement;

[0165] The determining unit 821 is further configured to determine a required subgraph from the candidate subgraphs based on the overlapping relationship between the specified subgraph and the candidate subgraphs, wherein the required subgraph is a subgraph whose overlapping relationship with the specified subgraph meets the overlapping relationship requirement;

[0166] The determining unit 821 is further configured to determine the at least two regional sub-graphs based on the required sub-graph and the specified sub-graph.

[0167] In some embodiments, the determining unit 821 is further configured to determine, from among the candidate subgraphs arranged in order, a candidate subgraph having the highest confidence value as the designated subgraph.

[0168] In some embodiments, the determining unit 821 is further configured to determine an overlap ratio between the designated sub-image and the candidate sub-image;

[0169] The determination unit 821 is further configured to, in response to the overlap rate between the designated subgraph and the candidate subgraph reaching an overlap rate threshold, retain the candidate subgraph and determine it as the required subgraph; and, in response to the overlap rate between the designated subgraph and the candidate subgraph being less than the overlap rate threshold, discard the candidate subgraph.

[0170] In some embodiments, the extraction module 820 is further used to extract the sub-image features of the at least two region sub-images;

[0171] The device also includes:

[0172] A merging module 840, configured to merge the sub-image feature with the global feature to obtain a fused feature;

[0173] The recognition module 830 is further used to perform scene recognition on the fused features to obtain a scene category label corresponding to the target image.

[0174] In some embodiments, the device is installed with a scene recognition model, and the target image is a sample image annotated with a sample category label;

[0175] The recognition module 830 is further configured to obtain, by using the scene recognition model, an image prediction result corresponding to the global feature and the sub-image feature of the regional sub-image; and obtain a loss value of the image prediction result based on the image prediction result and the sample category label;

[0176] The device also includes:

[0177] The adjustment module 850 is used to adjust the parameters of the scene recognition model based on the loss value of the image prediction result.

[0178] In some embodiments, the recognition module 830 is further used to perform global feature prediction on the global feature through the scene recognition model to obtain a global prediction result;

[0179] The recognition module 830 is further used to perform attention classification prediction on the fusion feature obtained by merging the sub-image feature with the global feature through the scene recognition model, and obtain the scene category label corresponding to the target image as the attention prediction result;

[0180] The recognition module 830 is further configured to predict the positioning accuracy of the sub-image features of the at least two regional sub-images by using the scene recognition model to obtain a positioning prediction result;

[0181] The recognition module 830 is further configured to perform classification prediction on the sub-image features of the at least two regional sub-images through the scene recognition model to obtain a sub-image prediction result.

[0182] In some embodiments, the recognition module 830 is also used to obtain a first loss value based on the global prediction result and the sample category label; to obtain a second loss value based on the attention prediction result and the sample category label; to obtain a third loss value based on the positioning prediction result and the sample category label; to obtain a fourth loss value based on the sub-image prediction result and the sample category label; and to obtain the loss value of the image prediction result based on the first loss value, the second loss value, the third loss value and the fourth loss value.

[0183] To summarize, the image scene recognition device provided in the embodiment of the present application, for the scene recognition process of the target image, extracts the global features of the target image, and then uses the attention mechanism to extract the regional subgraph from the target image based on the global features, so as to perform scene recognition on the target image based on the subgraph features and the global features. That is, the reference content of the scene recognition includes not only a single entity in the target image, but also includes the image areas in the target image that are related to each scene category label in the image recognition classification library, thereby improving the scene recognition accuracy of the target image.

[0184] It should be noted that the image scene recognition device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the image scene recognition device provided in the above embodiment belongs to the same concept as the image scene recognition method embodiment. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0185] Fig.10 A schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application is shown. The computer device can be implemented as above Figure 1 The server 120 shown. Specifically:

[0186] The computer device 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including a random access memory (RAM) 1002 and a read only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the central processing unit 1001. The computer device 1000 also includes a mass storage device 1006 for storing an operating system 1013, application programs 1014, and other program modules 1015.

[0187] The mass storage device 1006 is connected to the central processing unit 1001 through a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1006 and its associated computer readable media provide non-volatile storage for the computer device 1000. That is, the mass storage device 1006 may include a computer readable medium (not shown) such as a hard disk or a compact disc read only memory (CD-ROM) drive.

[0188] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory 1004 and mass storage device 1006 can be collectively referred to as memory.

[0189] According to various embodiments of the present application, the computer device 1000 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 1000 can be connected to the network 1012 through the network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 can be used to connect to other types of networks or remote computer systems (not shown).

[0190] The above-mentioned memory also includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.

[0191] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the image scene recognition method provided by the above-mentioned method embodiments.

[0192] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the image scene recognition method provided by the above-mentioned method embodiments.

[0193] The embodiments of the present application also provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the image scene recognition method described in any of the above embodiments.

[0194] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0195] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0196] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for recognizing an image scene, characterized in that: Applied to a scene recognition model, the method comprises: Acquire a target image, where the target image is an image scene to be recognized; Extracting global features of the target image, wherein the global features are features obtained by extracting features of the target image as a whole; Extracting at least two regional sub-images from the target image based on the global feature and an image recognition classification library, wherein the image recognition classification library includes scene category labels for annotating images; The image prediction results corresponding to the global features and the sub-image features of the regional sub-images are obtained by identifying the scene recognition model; the image prediction results include positioning prediction results, and the positioning prediction results are obtained by predicting the positioning accuracy of the sub-image features of the at least two regional sub-images by the scene recognition model; Based on the image prediction result and the sample category label marked in the target image, a loss value of the image prediction result is obtained; the loss value includes a third loss value, and the third loss value is obtained based on the positioning prediction result and the sample category label marked in the target image; Parameters of the scene recognition model are adjusted based on the loss value of the image prediction result.

2. The method according to claim 1, characterized in that The step of extracting at least two regional sub-images from the target image based on the global features and the image recognition classification library comprises: Based on the global features and the image recognition classification library, feature points in the global features are identified to obtain candidate point scores corresponding to the feature points; Determining the confidence of a candidate sub-image in the target image based on the candidate point score, where there is a mapping relationship between the candidate sub-image and the feature point; The at least two region sub-images are determined from the candidate sub-images of the target image based on the confidence level.

3. The method according to claim 2, characterized in that The determining the at least two region sub-images from the candidate sub-images of the target image based on the confidence level comprises: Arranging the candidate sub-images of the target image in order based on the confidence level; Determine a designated subgraph from the candidate subgraphs arranged in order, the designated subgraph being a subgraph that meets designated requirements in the candidate subgraphs; Based on the overlapping relationship between the designated subgraph and the candidate subgraph, determining a required subgraph from the candidate subgraphs, the required subgraph being a subgraph whose overlapping relationship with the designated subgraph meets the overlapping relationship requirement; The at least two area sub-maps are determined based on the required sub-map and the specified sub-map.

4. The method according to claim 3, characterized in that The determining of the designated subgraph from the candidate subgraphs arranged in order comprises: From the candidate subgraphs arranged in order, a candidate subgraph with the highest confidence value is determined as the designated subgraph.

5. The method according to claim 3, characterized in that: The step of determining the required subgraph from the candidate subgraphs based on the overlapping relationship between the designated subgraph and the candidate subgraphs includes: Determining an overlap ratio between the designated sub-image and the candidate sub-image; In response to an overlap rate between the designated subgraph and the candidate subgraph reaching an overlap rate threshold, retaining the candidate subgraph and determining it as the required subgraph; In response to the overlap rate between the designated sub-image and the candidate sub-image being less than the overlap rate threshold, the candidate sub-image is discarded.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Extracting the sub-image features of the at least two region sub-images; Merging the sub-image feature with the global feature to obtain a fused feature; Perform scene recognition on the fused features to obtain a scene category label corresponding to the target image.

7. The method according to claim 1, characterized in that The image prediction result corresponding to the global feature and the sub-image feature of the regional sub-image obtained by identifying the scene recognition model further includes: Performing global feature prediction on the global features by using the scene recognition model to obtain a global prediction result; Performing attention classification prediction on the fusion feature formed by merging the sub-image feature with the global feature through the scene recognition model, and obtaining a scene category label corresponding to the target image as an attention prediction result; The sub-image features of the at least two region sub-images are classified and predicted by using the scene recognition model to obtain a sub-image prediction result.

8. The method according to claim 7, characterized in that The step of obtaining a loss value of the image prediction result based on the image prediction result and the sample category label further includes: Obtaining a first loss value based on the global prediction result and the sample category label; Obtaining a second loss value based on the attention prediction result and the sample category label; Obtaining a fourth loss value based on the sub-graph prediction result and the sample category label; A loss value of the image prediction result is obtained based on the first loss value, the second loss value, the third loss value and the fourth loss value.

9. An image scene recognition device, characterized in that: The device has a scene recognition model, and the device includes: An acquisition module, used to acquire a target image, wherein the target image is an image scene to be recognized; An extraction module, used to extract global features of the target image, wherein the global features are features obtained by extracting features of the target image as a whole; The extraction module is further used to extract at least two regional sub-images from the target image based on the global feature and an image recognition classification library, wherein the image recognition classification library includes scene category labels for annotating images; An identification module, configured to obtain image prediction results corresponding to the global features and the sub-image features of the regional sub-images through the scene recognition model; the image prediction results include positioning prediction results, and the positioning prediction results are obtained by predicting the positioning accuracy of the sub-image features of the at least two regional sub-images through the scene recognition model; based on the image prediction results and the sample category labels annotated in the target image, a loss value of the image prediction results is obtained; the loss value includes a third loss value, and the third loss value is obtained based on the positioning prediction results and the sample category labels annotated in the target image; An adjustment module is used to adjust parameters of the scene recognition model based on the loss value of the image prediction result.

10. The device according to claim 9, characterized in that The recognition module is further used to recognize feature points in the global features based on the global features and the image recognition classification library to obtain candidate point scores corresponding to the feature points; The extraction module comprises: a determination unit, configured to determine the confidence of a candidate sub-image in the target image based on the candidate point score, wherein there is a mapping relationship between the candidate sub-image and the feature point; The determining unit is further configured to determine the at least two region sub-images from the candidate sub-images of the target image based on the confidence level.

11. The device according to claim 10, characterized in that The extraction module further includes: A sorting unit, configured to arrange the candidate sub-images of the target image in order based on the confidence level; The determining unit is further configured to determine a designated subgraph from the candidate subgraphs arranged in order, wherein the designated subgraph is a subgraph in the candidate subgraphs that meets designated requirements; The determining unit is further configured to determine a required subgraph from the candidate subgraphs based on an overlapping relationship between the specified subgraph and the candidate subgraphs, wherein the required subgraph is a subgraph whose overlapping relationship with the specified subgraph meets an overlapping relationship requirement; The determining unit is further configured to determine the at least two area sub-graphs based on the required sub-graph and the specified sub-graph.

12. The device according to claim 11, characterized in that The determining unit is further configured to: From the candidate subgraphs arranged in order, a candidate subgraph with the highest confidence value is determined as the designated subgraph.

13. The device according to claim 11, characterized in that The determining unit is further configured to: Determining an overlap ratio between the designated sub-image and the candidate sub-image; In response to an overlap rate between the designated subgraph and the candidate subgraph reaching an overlap rate threshold, retaining the candidate subgraph and determining it as the required subgraph; In response to the overlap rate between the designated sub-image and the candidate sub-image being less than the overlap rate threshold, the candidate sub-image is discarded.

14. The device according to any one of claims 9 to 13, characterized in that: The extraction module is further used to extract the sub-image features of the at least two regional sub-images; The device also includes: A merging module, used to merge the sub-image feature with the global feature to obtain a fused feature; The recognition module is further used to perform scene recognition on the fused features to obtain a scene category label corresponding to the target image.

15. The device according to claim 9, characterized in that The identification module is also used for: Performing global feature prediction on the global features by using the scene recognition model to obtain a global prediction result; Performing attention classification prediction on the fusion feature formed by merging the sub-image feature with the global feature through the scene recognition model, and obtaining a scene category label corresponding to the target image as an attention prediction result; The sub-image features of the at least two region sub-images are classified and predicted by using the scene recognition model to obtain a sub-image prediction result.

16. The device according to claim 15, characterized in that The identification module is also used for: Obtaining a first loss value based on the global prediction result and the sample category label; Obtaining a second loss value based on the attention prediction result and the sample category label; Obtaining a fourth loss value based on the sub-graph prediction result and the sample category label; A loss value of the image prediction result is obtained based on the first loss value, the second loss value, the third loss value and the fourth loss value.

17. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the image scene recognition method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the image scene recognition method according to any one of claims 1 to 8.