Visual scene recognition method, device and electronic device based on implicit saliency anchor vector

Abstract

The present invention provides a visual scene recognition method, device and electronic device based on implicit saliency anchor vectors. In the method, saliency information of local features can be extracted, and the extracted saliency information is associated with the anchor vector to obtain the saliency enhanced anchor vector, that is, the saliency information can be captured. The saliency enhanced anchor vector improves the scene representation performance. In addition, the saliency information has the appearance stability and invariance, so it can improve the dynamic scene adaptability of the network. Even when the appearance changes, consistent scene matching results can be obtained based on the saliency enhanced global descriptor, which has good robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a visual scene recognition method, device and electronic equipment based on implicit saliency anchor point vectors. Background Art

[0002] The traditional visual scene recognition process is generally as follows: extract the feature map corresponding to the original image through the backbone network, where each pixel in the feature map is a local feature, and then fuse all the local features in the feature map (i.e., global feature fusion) to obtain a scene descriptor for characterizing the scene corresponding to the above original image, and then use the above scene descriptor to match the scene.

[0003] The above-mentioned visual scene recognition processes utilize salient regions differently. The global feature fusion process ignores the fine-grained capture of salient regions, limiting the robustness of the model in complex environments. In addition, the model has poor adaptability to dynamic scenes. For example, when faced with appearance changes such as lighting, weather or seasonal changes, traditional methods find it difficult to provide consistent scene matching results, i.e., the scene recognition accuracy is poor.

[0004] In summary, in the process of visual scene recognition, how to capture the salient information of salient areas, improve the scene representation performance, and improve the dynamic scene adaptability of the network have become technical problems that need to be solved urgently. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a visual scene recognition method, device and electronic device based on implicit saliency anchor point vector, so as to alleviate the technical problems of the prior art in the process of visual scene recognition, namely, the inability to capture the saliency information of salient areas, the poor representation performance of scene descriptors, and the poor adaptability of the network to dynamic scene recognition.

[0006] In a first aspect, an embodiment of the present invention provides a visual scene recognition method based on an implicit saliency anchor vector, comprising:

[0007] Use the pre-trained backbone network to extract features from the original scene image to obtain local features;

[0008] Using a feature mixing-based anchor vector representation generation module to perform anchor vector representation on the local feature to obtain an anchor vector;

[0009] An implicit scene saliency information association module is used to extract saliency information from the local features, and the extracted saliency information is associated with the anchor vector to obtain an anchor vector with enhanced saliency;

[0010] The saliency enhanced anchor point vectors are concatenated and normalized to obtain a saliency enhanced global descriptor, and then visual scene recognition is performed based on the saliency enhanced global descriptor.

[0011] Furthermore, an anchor vector representation generation module based on feature mixing is used to perform anchor vector representation on the local feature, including:

[0012] Using a feature mixing module to extract saliency information of the local features;

[0013] The saliency information is aggregated by dimensionality reduction to obtain the anchor point vector.

[0014] Furthermore, the saliency information is subjected to dimension reduction aggregation, including:

[0015] Using a dimensionality reduction linear projection layer to perform dimensionality reduction processing on the saliency information to obtain dimensionality reduced saliency information;

[0016] The aggregated linear projection layer is used to aggregate the saliency information after dimension reduction to obtain the anchor point vector.

[0017] Furthermore, the implicit scene saliency information association module includes: a saliency information extraction unit and an association unit, using the implicit scene saliency information association module to extract saliency information from the local features, and associating the extracted saliency information with the anchor point vector, including:

[0018] Using the saliency information extraction unit to extract saliency information from the local features to obtain the saliency information;

[0019] The extracted saliency information is associated with the anchor vector using the association unit to obtain the saliency enhanced anchor vector.

[0020] Furthermore, visual scene recognition is performed based on the saliency enhanced global descriptor, including:

[0021] Calculating the similarity between the global descriptor after saliency enhancement and the preset global descriptor after saliency enhancement in the scene recognition relationship, wherein the scene recognition relationship is the relationship between the preset global descriptor after saliency enhancement and the target scene;

[0022] A target preset saliency enhanced global descriptor corresponding to the saliency enhanced global descriptor is determined according to the similarity, and then a target scene corresponding to the target preset saliency enhanced global descriptor is used as the recognized scene.

[0023] Furthermore, the similarity includes: cosine similarity.

[0024] In a second aspect, an embodiment of the present invention provides a visual scene recognition device based on an implicit saliency anchor vector, comprising:

[0025] A feature extraction unit, used to extract features from the original scene image using a pre-trained backbone network to obtain local features;

[0026] An anchor vector characterization unit, used for performing anchor vector characterization on the local feature by using an anchor vector characterization generation module based on feature mixing to obtain an anchor vector;

[0027] A saliency information extraction and association unit, configured to extract saliency information from the local features using an implicit scene saliency information association module, and associate the extracted saliency information with the anchor vector to obtain an anchor vector with enhanced saliency;

[0028] The splicing and normalization unit is used to splice and normalize the saliency enhanced anchor point vectors to obtain a saliency enhanced global descriptor, and then perform visual scene recognition based on the saliency enhanced global descriptor.

[0029] Furthermore, the anchor vector representation unit is also used for:

[0030] Using a feature mixing module to extract saliency information of the local features;

[0031] The saliency information is aggregated by dimensionality reduction to obtain the anchor point vector.

[0032] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0033] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0034] In an embodiment of the present invention, a visual scene recognition method based on implicit saliency anchor vectors is provided, comprising: using a pre-trained backbone network to extract features from an original scene image to obtain local features; using an anchor vector representation generation module based on feature mixing to represent the local features with anchor vectors to obtain anchor vectors; using an implicit scene saliency information association module to extract saliency information from the local features, and associating the extracted saliency information with the anchor vectors to obtain saliency enhanced anchor vectors; splicing and normalizing the saliency enhanced anchor vectors to obtain saliency enhanced global descriptors, and then performing visual scene recognition based on the saliency enhanced global descriptors. From the above description, it can be seen that in the visual scene recognition method based on implicit saliency anchor vector of the present invention, saliency information can be extracted from local features, and the extracted saliency information can be associated with the anchor vector to obtain the anchor vector after saliency enhancement, that is, the saliency information can be captured. The anchor vector after saliency enhancement improves the scene representation performance. In addition, the saliency information has the appearance stability and invariance, so it can improve the dynamic scene adaptability of the network. Even when the appearance changes, consistent scene matching results can be obtained based on the global descriptor after saliency enhancement. It has good robustness and alleviates the technical problems of the prior art in the process of visual scene recognition, such as the inability to capture the saliency information of the saliency area, the poor representation performance of the scene descriptor, and the poor dynamic scene recognition adaptability of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A flowchart of a visual scene recognition method based on implicit saliency anchor vector provided by an embodiment of the present invention;

[0037] Figure 2 A flow chart of the global descriptor from local feature extraction to saliency enhancement provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of an anchor vector representation generation module based on feature mixing provided in an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a visual scene recognition device based on implicit saliency anchor vector provided by an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In the process of visual scene recognition, traditional technologies are unable to capture the salient information of salient areas, the representation performance of scene descriptors is poor, and the network's adaptability to dynamic scene recognition is poor.

[0043] Based on this, in the visual scene recognition method based on implicit saliency anchor vector of the present invention, saliency information can be extracted from local features, and the extracted saliency information is associated with the anchor vector, so as to obtain the saliency enhanced anchor vector, that is, the saliency information can be captured. The saliency enhanced anchor vector improves the scene representation performance. In addition, the saliency information has the appearance stability and invariance, so it can improve the dynamic scene adaptability of the network. Even when the appearance changes, consistent scene matching results can be obtained based on the saliency enhanced global descriptor, which has good robustness.

[0044] To facilitate understanding of the present embodiment, a visual scene recognition method based on implicit saliency anchor point vector disclosed in an embodiment of the present invention is first introduced in detail.

[0045] Embodiment 1:

[0046] According to an embodiment of the present invention, an embodiment of a visual scene recognition method based on implicit saliency anchor vectors is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] Figure 1 is a flow chart of a visual scene recognition method based on implicit saliency anchor vector according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0048] Step S102, using a pre-trained backbone network to extract features from the original scene image to obtain local features;

[0049] Step S104, using a feature mixing-based anchor vector representation generation module to perform anchor vector representation on the local features to obtain an anchor vector;

[0050] Specifically, the anchor vector can provide a compact representation of the global information of the scene, and the anchor vector effectively expresses the main semantic areas in the local features.

[0051] Step S106, using an implicit scene saliency information association module to extract saliency information from local features, and associating the extracted saliency information with an anchor vector to obtain an anchor vector with enhanced saliency;

[0052] Specifically, the above association can be a tightly coupled association achieved by addition, multiplication, concatenation, attention mechanism, MLP, 1×1 convolution, etc. The above saliency-enhanced anchor vector is an implicit saliency anchor vector, which can represent the scene compactly and efficiently, significantly improving the recognition performance and adaptability of the model. The saliency-enhanced anchor vector is actually an anchor vector that has the ability to represent the saliency information of the scene.

[0053] Step S108, concatenating and normalizing the saliency enhanced anchor point vectors to obtain a saliency enhanced global descriptor, and then performing visual scene recognition based on the saliency enhanced global descriptor.

[0054] For details, please refer to Figure 2 ,The method of the present invention is applicable to scene recognition tasks such as autonomous ,driving, augmented reality, and intelligent monitoring.

[0055] In an embodiment of the present invention, a visual scene recognition method based on implicit saliency anchor vectors is provided, comprising: using a pre-trained backbone network to extract features from an original scene image to obtain local features; using an anchor vector representation generation module based on feature mixing to represent the local features with anchor vectors to obtain anchor vectors; using an implicit scene saliency information association module to extract saliency information from the local features, and associating the extracted saliency information with the anchor vectors to obtain saliency enhanced anchor vectors; splicing and normalizing the saliency enhanced anchor vectors to obtain saliency enhanced global descriptors, and then performing visual scene recognition based on the saliency enhanced global descriptors. From the above description, it can be seen that in the visual scene recognition method based on implicit saliency anchor vector of the present invention, saliency information can be extracted from local features, and the extracted saliency information can be associated with the anchor vector to obtain the anchor vector after saliency enhancement, that is, the saliency information can be captured. The anchor vector after saliency enhancement improves the scene representation performance. In addition, the saliency information has the appearance stability and invariance, so it can improve the dynamic scene adaptability of the network. Even when the appearance changes, consistent scene matching results can be obtained based on the global descriptor after saliency enhancement. It has good robustness and alleviates the technical problems of the prior art in the process of visual scene recognition, such as the inability to capture the saliency information of the saliency area, the poor representation performance of the scene descriptor, and the poor dynamic scene recognition adaptability of the network.

[0056] The above content briefly introduces the visual scene recognition method based on implicit saliency anchor point vector of the present invention. The specific contents involved are described in detail below.

[0057] In an optional embodiment of the present invention, an anchor vector representation generation module based on feature mixing is used to represent the anchor vector of the local feature, which specifically includes the following steps:

[0058] (1) Using feature mixing module to extract saliency information of local features;

[0059] Specifically, refer to Figure 3 , Figure 3 The specific structure of the feature mixing module is shown in FIG.

[0060] (2) Perform dimension reduction aggregation on the salient information to obtain the anchor vector.

[0061] The specific steps include:

[0062] (21) Using a dimensionality reduction linear projection layer to reduce the dimensionality of the saliency information, and obtaining the saliency information after dimensionality reduction;

[0063] (22) An aggregated linear projection layer is used to aggregate the saliency information after dimensionality reduction to obtain an anchor vector.

[0064] For details, please refer to Figure 3 In the process of traditional technology, the anchor vectors are randomly generated. For different original scene images, the randomly generated anchor vectors are used. Then, in the subsequent learning process, the randomly generated anchor vectors are targeted for learning the corresponding original scene images to characterize the corresponding original scene images. In the present invention, a specific anchor vector can be generated for each original scene image, that is, by Figure 3 By using the process in , an anchor vector that can better represent the salient information in the original scene image can be obtained, that is, the accuracy of the anchor vector is better (compared with the randomly generated anchor vector), and the ability to represent the salient information is better, that is, the initialization method of the anchor vector in the present invention (i.e. Figure 3 The anchor vector obtained in the process of obtaining the anchor vector can better represent the scene and improve the performance of the subsequent network.

[0065] In an optional embodiment of the present invention, the implicit scene saliency information association module includes: a saliency information extraction unit and an association unit, and the implicit scene saliency information association module is used to extract saliency information from local features, and the extracted saliency information is associated with an anchor vector, specifically including the following steps:

[0066] (1) Using a saliency information extraction unit to extract saliency information from local features to obtain saliency information;

[0067] Specifically, the saliency information extraction unit may be a feature mixing module or a multi-layer convolutional layer. The embodiment of the present invention does not impose any specific limitation on this. Any method that can extract saliency information may be used.

[0068] (2) The extracted saliency information is associated with the anchor vector using an association unit to obtain the saliency enhanced anchor vector.

[0069] Specifically, associating saliency information with an anchor vector can enhance the representation capability of the anchor vector, that is, the representation capability of the anchor vector after saliency enhancement is better.

[0070] In an optional embodiment of the present invention, visual scene recognition is performed based on the saliency enhanced global descriptor, specifically comprising the following steps:

[0071] (1) calculating the similarity between the global descriptor after saliency enhancement and the preset global descriptor after saliency enhancement in the scene recognition relationship, wherein the scene recognition relationship is the relationship between the preset global descriptor after saliency enhancement and the target scene;

[0072] (2) Determine the target preset saliency enhanced global descriptor corresponding to the saliency enhanced global descriptor according to the similarity, and then use the target scene corresponding to the target preset saliency enhanced global descriptor as the recognized scene.

[0073] Specifically, the above similarity includes: cosine similarity.

[0074] The method of the present invention is tested on public scene recognition benchmark datasets (such as Tokyo24 / 7, Nordland, Pitts30k, etc.), and the scene recognition performance of the present invention is significantly better than the existing methods. The experimental results show that:

[0075] In scenes with complex lighting and perspective changes, the accuracy of the present invention is improved by more than 10%;

[0076] The dimension of the global descriptor is significantly reduced (2048 dimensions), and the computational efficiency is improved by 30%.

[0077] The method of the present invention has the following effects:

[0078] Dynamic capture of salient regions: Through anchor vector generation and subsequent adjustment (from anchor vector to saliency-enhanced anchor vector), the ability to focus on salient information in complex scenes is enhanced.

[0079] Improve matching robustness: The adaptability to appearance changes is significantly improved, showing stronger robustness in dynamic scenes.

[0080] Embodiment 2:

[0081] An embodiment of the present invention also provides a visual scene recognition device based on an implicit saliency anchor vector. The visual scene recognition device based on an implicit saliency anchor vector is mainly used to execute the visual scene recognition method based on an implicit saliency anchor vector provided in Embodiment 1 of the present invention. The following is a specific introduction to the visual scene recognition device based on an implicit saliency anchor vector provided in an embodiment of the present invention.

[0082] Figure 4 is a schematic diagram of a visual scene recognition device based on an implicit saliency anchor vector according to an embodiment of the present invention, such as Figure 4 As shown, the device mainly includes: a feature extraction unit 10, an anchor vector representation unit 20, a saliency information extraction and association unit 30, and a splicing normalization unit 40, wherein:

[0083] A feature extraction unit, used to extract features from the original scene image using a pre-trained backbone network to obtain local features;

[0084] An anchor vector representation unit, used for performing anchor vector representation on local features by using an anchor vector representation generation module based on feature mixing to obtain an anchor vector;

[0085] A saliency information extraction and association unit, used to extract saliency information from local features using an implicit scene saliency information association module, and associate the extracted saliency information with an anchor vector to obtain an anchor vector with saliency enhancement;

[0086] The splicing and normalization unit is used to splice and normalize the saliency enhanced anchor point vectors to obtain a saliency enhanced global descriptor, and then perform visual scene recognition based on the saliency enhanced global descriptor.

[0087] In an embodiment of the present invention, a visual scene recognition device based on implicit saliency anchor vector is provided, comprising: using a pre-trained backbone network to extract features from an original scene image to obtain local features; using an anchor vector representation generation module based on feature mixing to represent the local features with anchor vectors to obtain anchor vectors; using an implicit scene saliency information association module to extract saliency information from the local features, and associating the extracted saliency information with the anchor vector to obtain an anchor vector with saliency enhancement; splicing and normalizing the anchor vectors with saliency enhancement to obtain a global descriptor with saliency enhancement, and then performing visual scene recognition based on the global descriptor with saliency enhancement. From the above description, it can be seen that in the visual scene recognition device based on implicit saliency anchor vector of the present invention, saliency information can be extracted from local features, and the extracted saliency information can be associated with the anchor vector, so as to obtain the anchor vector after saliency enhancement, that is, the saliency information can be captured. The anchor vector after saliency enhancement improves the scene representation performance. In addition, the saliency information has the appearance stability and invariance, so it can improve the dynamic scene adaptability of the network. Even when the appearance changes, consistent scene matching results can be obtained based on the global descriptor after saliency enhancement. It has good robustness and alleviates the technical problems of the prior art in the process of visual scene recognition, such as the inability to capture the saliency information of the saliency area, the poor representation performance of the scene descriptor, and the poor dynamic scene recognition adaptability of the network.

[0088] Optionally, the anchor vector representation unit is further used to: extract saliency information of local features by using a feature mixing module; and perform dimension reduction aggregation on the saliency information to obtain an anchor vector.

[0089] Optionally, the anchor vector representation unit is further used to: use a dimensionality reduction linear projection layer to perform dimensionality reduction processing on the saliency information to obtain dimensionality reduced saliency information; use an aggregation linear projection layer to aggregate the dimensionality reduced saliency information to obtain an anchor vector.

[0090] Optionally, the implicit scene saliency information association module includes: a saliency information extraction unit and an association unit, and the saliency information extraction and association unit is also used to: use the saliency information extraction unit to extract saliency information from local features to obtain saliency information; use the association unit to associate the extracted saliency information with the anchor point vector to obtain the anchor point vector with saliency enhancement.

[0091] Optionally, the splicing normalization unit is also used to: calculate the similarity between the global descriptor after saliency enhancement and the preset global descriptor after saliency enhancement in the scene recognition relationship, wherein the scene recognition relationship is the relationship between the preset global descriptor after saliency enhancement and the target scene; determine the target preset global descriptor after saliency enhancement corresponding to the global descriptor after saliency enhancement based on the similarity, and then use the target scene corresponding to the target preset global descriptor after saliency enhancement as the recognized scene.

[0092] Optionally, the similarity includes: cosine similarity.

[0093] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0094] like Figure 5 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the visual scene recognition method based on implicit saliency anchor vectors as described above.

[0095] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned visual scene recognition method based on implicit saliency anchor point vector.

[0096] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0097] Corresponding to the above-mentioned visual scene recognition method based on implicit saliency anchor point vectors, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned visual scene recognition method based on implicit saliency anchor point vectors.

[0098] The visual scene recognition device based on implicit saliency anchor vector provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0099] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0100] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0102] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0103] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the visual scene recognition method based on implicit saliency anchor vectors described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0104] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0105] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A visual scene recognition method based on implicit saliency anchor vector, characterized in that: include: Use the pre-trained backbone network to extract features from the original scene image to obtain local features; Using a feature mixing-based anchor vector representation generation module to perform anchor vector representation on the local feature to obtain an anchor vector; An implicit scene saliency information association module is used to extract saliency information from the local features, and the extracted saliency information is associated with the anchor vector to obtain an anchor vector with enhanced saliency; The saliency enhanced anchor point vectors are concatenated and normalized to obtain a saliency enhanced global descriptor, and then visual scene recognition is performed based on the saliency enhanced global descriptor.

2. The method according to claim 1, characterized in that The anchor vector representation generation module based on feature mixing is used to represent the local feature with an anchor vector, including: Using a feature mixing module to extract saliency information of the local features; The saliency information is aggregated by dimensionality reduction to obtain the anchor point vector.

3. The method according to claim 2, characterized in that Performing dimension reduction aggregation on the saliency information, including: Using a dimensionality reduction linear projection layer to perform dimensionality reduction processing on the saliency information to obtain dimensionality reduced saliency information; The aggregated linear projection layer is used to aggregate the saliency information after dimension reduction to obtain the anchor point vector.

4. The method according to claim 1, characterized in that: The implicit scene saliency information association module includes: a saliency information extraction unit and an association unit, using the implicit scene saliency information association module to extract saliency information from the local feature, and associating the extracted saliency information with the anchor point vector, including: Using the saliency information extraction unit to extract saliency information from the local features to obtain the saliency information; The extracted saliency information is associated with the anchor vector using the association unit to obtain the saliency enhanced anchor vector.

5. The method according to claim 1, characterized in that Performing visual scene recognition based on the saliency enhanced global descriptor includes: Calculating the similarity between the global descriptor after saliency enhancement and the preset global descriptor after saliency enhancement in the scene recognition relationship, wherein the scene recognition relationship is the relationship between the preset global descriptor after saliency enhancement and the target scene; A target preset saliency enhanced global descriptor corresponding to the saliency enhanced global descriptor is determined according to the similarity, and then a target scene corresponding to the target preset saliency enhanced global descriptor is used as the recognized scene.

6. The method according to claim 5, characterized in that The similarity includes: cosine similarity.

7. A visual scene recognition device based on implicit saliency anchor vector, characterized in that: include: A feature extraction unit, used to extract features from the original scene image using a pre-trained backbone network to obtain local features; An anchor vector characterization unit, used for performing anchor vector characterization on the local feature by using an anchor vector characterization generation module based on feature mixing to obtain an anchor vector; A saliency information extraction and association unit, configured to extract saliency information from the local features using an implicit scene saliency information association module, and associate the extracted saliency information with the anchor vector to obtain an anchor vector with enhanced saliency; The splicing and normalization unit is used to splice and normalize the saliency enhanced anchor point vectors to obtain a saliency enhanced global descriptor, and then perform visual scene recognition based on the saliency enhanced global descriptor.

8. The device according to claim 7, characterized in that The anchor vector representation unit is also used for: Using a feature mixing module to extract saliency information of the local features; The saliency information is aggregated by dimensionality reduction to obtain the anchor point vector.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to execute the method according to any one of claims 1 to 6.

Images