Outdoor shooting auxiliary system and method based on image recognition

Abstract

The invention provides an outdoor shooting auxiliary system and method based on image recognition, and the method comprises the steps: respectively extracting the spatial distribution characteristics of shooting posture nodes in a visible light image and an infrared thermal image, and determining the distribution difference loss of the shooting posture nodes among different heterogeneous images; when the image is interfered by strong light in an image adopting process, carrying out fuzzy registration on element labels of shooting attitude nodes in different heterogeneous images based on all distribution difference losses to obtain affiliation confidence coefficients of the shooting attitude nodes; according to all the attribution confidence coefficients, determining pixel constraint conditions of shooting attitude nodes of different heterogeneous images during interference suppression; and performing hierarchical constraint fusion on the spatial distribution characteristics of the shooting attitude nodes in different heterogeneous images based on the cross-modal fusion attention among different heterogeneous images and the pixel constraint condition of the shooting attitude nodes to obtain the fusion characteristics of the shooting attitude. By means of the scheme, layered constraint fusion of shooting posture node features among different heterogeneous images can be achieved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and more specifically, to an outdoor shooting assistance system and method based on image recognition. Background Art

[0002] In outdoor shooting assistance systems, accurate recognition of shooting posture is of great significance to improving training quality and combat efficiency. Image recognition, as a key means to achieve posture recognition, can quickly and contactlessly extract key action nodes by collecting the shooter's posture image information, and provide the system with real-time and accurate posture feedback. In complex outdoor environments, analyzing shooting posture with the help of image recognition can not only improve the intelligence of the training process, but also assist instructors or systems in automatically evaluating the standardization and stability of shooting actions. It is one of the important directions for the construction of intelligent shooting assistance systems.

[0003] In the existing technology, the application of image recognition methods to identify shooting postures in outdoor environments still faces significant technical obstacles. Strong sunlight, shadow changes and fluctuations in weather conditions will seriously interfere with the quality of visible light images, making key posture node information blurred or missing, and difficult to support high-precision recognition. Although infrared thermal images can effectively compensate for these problems, cross-modal fusion and registration between different types of images is still a major problem. Existing technologies usually rely on traditional registration methods, but these methods are difficult to accurately deal with the distribution differences between images under complex environmental conditions, especially strong light interference, resulting in poor registration effect, thereby affecting the accuracy of the final posture recognition. Therefore, how to achieve hierarchical constrained fusion of shooting posture node features between different heterogeneous images has become a difficult problem faced by the industry. Summary of the Invention

[0004] The present application provides an outdoor shooting assistance system and method based on image recognition, which can realize the hierarchical constraint fusion of shooting posture node features between different heterogeneous images.

[0005] In a first aspect, the present application provides a shooting posture image recognition method for an outdoor shooting assistance system to perform image recognition of an outdoor shooter's shooting posture, comprising the following steps:

[0006] Synchronously capture visible light images and infrared thermal images of the shooter's shooting posture in an outdoor shooting environment;

[0007] performing multi-scale convolution on the visible light image and the infrared thermal image respectively, extracting spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image, and then determining the distribution difference loss of the same shooting posture node between different heterogeneous images;

[0008] When the image is interfered by strong light during acquisition, the feature labels of the shooting posture nodes in different heterogeneous images are fuzzy registered based on all distribution difference losses to obtain the attribution confidence of the shooting posture nodes in different heterogeneous images. Then, the pixel constraint conditions of the shooting posture nodes in different heterogeneous images during interference suppression are determined based on all the attribution confidences.

[0009] Based on the self-attention mechanism, cross-modal fusion attention of shooting posture nodes between different heterogeneous images is generated. The spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraints of the shooting posture nodes to obtain the fusion features of the shooting posture.

[0010] Preferably, extracting the spatial distribution features of the shooting posture nodes in the visible light image and the infrared thermal image, and then determining the distribution difference loss of the same shooting posture node between different heterogeneous images specifically includes:

[0011] Construct a convolutional neural network module with multi-scale receptive fields;

[0012] Extracting spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image through a convolutional neural network module;

[0013] Determining a node label vector of a shooting posture node in the visible light image according to a spatial distribution feature corresponding to the visible light image;

[0014] Determining a node label vector of a shooting posture node in the infrared thermal image according to a spatial distribution feature corresponding to the infrared thermal image;

[0015] The distribution difference loss of the same shooting posture node between different heterogeneous images is determined through all node label vectors.

[0016] Preferably, fuzzy registration is performed on the element labels of the shooting posture nodes in different heterogeneous images based on all distribution difference losses to obtain the attribution confidence of the shooting posture nodes in different heterogeneous images, specifically including:

[0017] The fuzzy membership of the shooting posture nodes between different heterogeneous images is determined by all distribution difference losses;

[0018] According to all the fuzzy memberships, fuzzy registration reasoning is performed on the element labels of the shooting posture nodes in different heterogeneous images to obtain the label credibility between the shooting posture nodes;

[0019] The attribution confidence of the shooting posture nodes in different heterogeneous images is determined by all label confidences.

[0020] Preferably, the pixel constraint conditions for determining the shooting posture nodes of different heterogeneous images during interference suppression based on all attribution confidences specifically include:

[0021] Based on all the attribution confidences, the interference suppression parameters of the shooting posture node pixels in different heterogeneous images are modeled with multi-target collaborative constraints to obtain a set of pixel constraint conditions.

[0022] Based on a multi-objective optimization algorithm, the interference suppression parameters of the pixel constraint condition set are optimized to obtain the optimal suppression parameters of the shooting posture node pixels in different heterogeneous images;

[0023] The pixel constraints of shooting posture nodes of different heterogeneous images during interference suppression are determined through all optimal pixel suppression parameters.

[0024] Preferably, generating cross-modal fusion attention of shooting posture nodes between different heterogeneous images based on the self-attention mechanism specifically includes:

[0025] Generate the attention weight matrix of shooting posture nodes in different heterogeneous images based on the self-attention mechanism;

[0026] The feature attention of shooting posture nodes between different heterogeneous images is fused through all attention weight matrices to obtain the cross-modal fused attention of shooting posture nodes between different heterogeneous images.

[0027] Preferably, the visible light image is collected by a camera.

[0028] Preferably, the infrared thermal image is collected by an infrared thermal imager.

[0029] In a second aspect, the present application provides an outdoor shooting assistance system based on image recognition, the outdoor shooting assistance system based on image recognition comprising an image recognition unit, the image recognition unit comprising:

[0030] An acquisition module, for synchronously acquiring visible light images and infrared thermal images of a shooter's shooting posture in an outdoor shooting environment;

[0031] a processing module, configured to perform multi-scale convolution on the visible light image and the infrared thermal image, respectively, to extract spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image, and further determine distribution difference losses of the same shooting posture node between different heterogeneous images;

[0032] The processing module is further configured to perform fuzzy registration on the element labels of the shooting posture nodes in different heterogeneous images based on all distribution difference losses when the image is interfered by strong light during the image acquisition process, thereby obtaining the attribution confidence of the shooting posture nodes in the different heterogeneous images, and then determining the pixel constraint conditions of the shooting posture nodes in the different heterogeneous images during interference suppression based on all the attribution confidences;

[0033] An execution module is used to generate cross-modal fusion attention of shooting posture nodes between different heterogeneous images based on the self-attention mechanism. The spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraints of the shooting posture nodes to obtain the fusion features of the shooting posture.

[0034] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a code, and the processor is configured to obtain the code and execute the above-mentioned shooting posture image recognition method.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image recognition method for the above-mentioned shooting posture is implemented.

[0036] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0037] In an embodiment of the present application, first, a visible light image and an infrared thermal image of the shooter's shooting posture in an outdoor shooting environment are synchronously collected; multi-scale convolution is performed on the visible light image and the infrared thermal image respectively, and the spatial distribution characteristics of the shooting posture nodes in the visible light image and the infrared thermal image are extracted, and then the distribution difference loss of the same shooting posture node between different heterogeneous images is determined; when the image is interfered by strong light during the image acquisition process, the element labels of the shooting posture nodes in different heterogeneous images are fuzzy aligned based on all distribution difference losses to obtain the attribution confidence of the shooting posture nodes in different heterogeneous images, and then the pixel constraint conditions of the shooting posture nodes in different heterogeneous images during interference suppression are determined based on all the attribution confidences; based on the self-attention mechanism, cross-modal fusion attention of the shooting posture nodes between different heterogeneous images is generated, and the spatial distribution characteristics of the shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraint conditions of the shooting posture nodes to obtain the fusion characteristics of the shooting posture.

[0038] It can be seen that the present application performs hierarchical constraint fusion on the spatial distribution features of shooting posture nodes in different heterogeneous images through cross-modal fusion attention and pixel constraints of shooting posture nodes to obtain the fusion features of shooting posture; first, by synchronously collecting visible light images and infrared thermal images, the system can use thermal imaging to supplement the missing image information caused by lighting when facing an outdoor environment with drastic changes in lighting, thereby establishing a redundant but complementary multimodal image foundation; secondly, the spatial distribution features of shooting posture nodes in visible light images and infrared thermal images are extracted respectively, and then the same shooting posture node is determined in different Distribution difference loss between heterogeneous images. By performing multi-scale convolution on images of different modalities, the system can obtain the spatial distribution characteristics of posture nodes at different scales, and further analyze the distribution difference loss under different modalities, thereby realizing the explicit difference characterization of node features between heterogeneous images; then, based on the distribution difference loss of shooting posture nodes between different heterogeneous images, the feature labels of shooting posture nodes in different heterogeneous images are fuzzy registered to obtain the attribution confidence of shooting posture nodes in different heterogeneous images, and the image of shooting posture nodes in different heterogeneous images during interference suppression is determined based on all the attribution confidences. Under strong light interference conditions, by introducing a fuzzy registration mechanism, the confidence level of the posture nodes is constructed according to the distribution difference loss of the shooting posture nodes in different heterogeneous images, thereby avoiding the misjudgment of key nodes caused by direct registration failure, and then forming a pixel-level constraint based on confidence, so that the fusion has higher fault tolerance and stability; finally, through cross-modal fusion attention and pixel constraints of shooting posture nodes, the spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused to obtain the fusion features of shooting posture, and the self-attention mechanism is used to generate cross-modal fusion attention. While paying attention to the semantic correlation between different heterogeneous images, combined with the aforementioned pixel constraints, the posture node features in different images are hierarchically constrained to achieve a fusion process from coarse to fine and from low to high, and gradually improve the semantic consistency and spatial alignment accuracy of the fusion results. The further fused shooting posture features significantly enhance the system's robustness and accuracy of posture recognition in complex environments while maintaining the diversity of the original images, and achieve the technical goal of realizing hierarchical constraint fusion of shooting posture nodes between heterogeneous images; in summary, the present application scheme can realize hierarchical constraint fusion of shooting posture node features between different heterogeneous images. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is an exemplary flow chart of a method for image recognition of a shooting posture according to some embodiments of the present application;

[0040] Figure 2 is a flowchart of the working application of the outdoor shooting assistance system according to some embodiments of the present application;

[0041] Figure 3 is a schematic diagram of a process for determining attribution confidence according to some embodiments of the present application;

[0042] Figure 4 is a schematic structural diagram of an image recognition unit according to some embodiments of the present application;

[0043] Figure 5 It is a structural diagram of a computer device for implementing an image recognition method for shooting posture according to some embodiments of the present application. DETAILED DESCRIPTION

[0044] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] refer to Figure 1 , which is an exemplary flow chart of a method for image recognition of a shooting posture according to some embodiments of the present application. The method 100 for image recognition of a shooting posture mainly includes the following steps:

[0046] In step 101, a visible light image and an infrared thermal image of a shooter's shooting posture in an outdoor shooting environment are synchronously collected.

[0047] It should be noted that the synchronous acquisition in this application refers to the use of an acquisition device with a time synchronization function to simultaneously acquire different types of image data (such as visible light images and infrared thermal images) at the same time to ensure that the two images are accurately aligned in time. Through synchronous acquisition, it can be ensured that the image data acquired at the same time node has a consistent timestamp, so that images of different modalities can be correctly matched in subsequent processing and analysis, avoiding the problem of inconsistent image features caused by acquisition time difference; it should also be noted that in this application, the visible light image is acquired by a camera, and the camera can specifically be a visible light camera, which is not specifically limited here. In addition, in this application, the infrared thermal image is acquired by an infrared thermal imager.

[0048] In specific implementation, the synchronous acquisition of visible light images and infrared thermal images of the shooter's shooting posture in an outdoor shooting environment can be achieved in the following manner, namely: it can be achieved by configuring a dual-spectrum imager with a synchronous acquisition function, and the dual-spectrum imager integrates a visible light camera and an infrared thermal imager. The visible light camera is used to acquire visible light images of the shooting posture, and the infrared thermal imager is used to acquire infrared thermal images of the shooting posture. In the dual-spectrum imager, the visible light camera and the infrared thermal imager can be started simultaneously through a time synchronization control program, and image acquisition is automatically started when the shooter starts to raise the gun, thereby completing the synchronous acquisition.

[0049] In some embodiments, reference Figure 2 As shown, this figure is a work application flow chart of the outdoor shooting assistance system in some embodiments of the present application. The figure shows the work application process of the outdoor shooting assistance system. In the outdoor shooting environment 110, the shooter performs shooting behavior, and his posture and other related information are collected by the visible light camera 120 and the infrared thermal imager 130 respectively. The former is used to obtain visible light images, and the latter is used to obtain infrared thermal images, so as to cope with complex lighting conditions. Then, the two types of image data collected will be transmitted to the data processor 140, which realizes the fusion processing of different modal images, so as to extract accurate shooting posture features. Finally, the visualization module 150 intuitively displays the processed posture evaluation results. By fusing visible light and infrared thermal image information, the entire workflow of the system is completed, thereby providing auxiliary support for outdoor shooting.

[0050] In step 102, multi-scale convolution is performed on the visible light image and the infrared thermal image respectively to extract the spatial distribution characteristics of the shooting posture nodes in the visible light image and the infrared thermal image, and then determine the distribution difference loss of the same shooting posture node between different heterogeneous images.

[0051] It should be noted that, in the present application, performing multi-scale convolution on the visible light image and the infrared thermal image respectively means using convolution kernels of different sizes to perform convolution processing on the visible light image and the infrared thermal image respectively, and extracting feature information of each image under different scale convolutions.

[0052] In some embodiments, extracting the spatial distribution features of the shooting posture nodes in the visible light image and the infrared thermal image, and then determining the distribution difference loss of the same shooting posture node between different heterogeneous images can be achieved by using the following steps:

[0053] Construct a convolutional neural network module with multi-scale receptive fields;

[0054] Extracting spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image through a convolutional neural network module;

[0055] Determining a node label vector of a shooting posture node in the visible light image according to a spatial distribution feature corresponding to the visible light image;

[0056] Determining a node label vector of a shooting posture node in the infrared thermal image according to a spatial distribution feature corresponding to the infrared thermal image;

[0057] The distribution difference loss of the same shooting posture node between different heterogeneous images is determined through all node label vectors.

[0058] It should be noted that the shooting posture node in this application refers to the feature point used to characterize the position of the key parts of the shooter's body (such as shoulders, elbows, knees, and wrists) in the image space; the spatial distribution feature in this application is a feature that measures the position structure layout of the shooting posture node in the image; the node label vector in this application is used to represent the position distribution feature of the shooting posture node in the image space; the distribution difference loss in this application is a quantitative indicator used to measure the spatial position difference of the same shooting posture node between different heterogeneous images.

[0059] In the specific implementation, first, constructing a convolutional neural network module with a multi-scale receptive field can be achieved in the following way: the HRNet architecture with multi-scale feature extraction capability can be selected. In this embodiment, first, a backbone network that maintains a high-resolution feature flow is built, and a standard convolution operation is performed on the input image in the initial stage of the network to extract basic features; secondly, multiple parallel branches are designed (the convolution kernel sizes used in each branch are different), each branch processes feature maps of different resolutions, and multi-scale modeling is achieved by increasing the stride or downsampling layer by layer. Then, through the information exchange between multiple branches, that is, the multi-resolution fusion mechanism, the features of different scales complement each other to enhance the spatial structure expression capability; finally, the fused multi-scale features are sent to the high-resolution output layer to generate a distribution feature map of the shooting posture nodes with rich spatial perception capabilities; the spatial distribution features of the shooting posture nodes in the visible light image and the infrared thermal image are extracted by the convolutional neural network module, which can be achieved in the following way, namely: the visible light image and the infrared thermal image are respectively used as The input parameters are input into the convolutional neural network module, and the distribution information of the shooting posture nodes in the visible light image and the infrared thermal image is output respectively through the output layer of the convolutional neural network module, and then the distribution information is used as the spatial distribution feature of the shooting posture nodes in the corresponding image; determining the node label vector of the shooting posture node in the visible light image according to the spatial distribution feature corresponding to the visible light image can be implemented in the following manner, namely: for the visible light image, the node labels of the shooting posture nodes in the spatial distribution feature can be arranged into a vector according to the distribution position of the pre-marked shooting posture nodes, and the vector is used as the node label vector of the shooting posture node in the visible light image; determining the node label vector of the shooting posture node in the infrared thermal image according to the spatial distribution feature corresponding to the infrared thermal image can be implemented in the following manner, namely: for the infrared thermal image, the node labels of the shooting posture nodes in the spatial distribution feature can be arranged into a vector according to the distribution position of the pre-marked shooting posture nodes, and the vector is used as the node label vector of the shooting posture node in the infrared thermal image.

[0060] Preferably, in the above embodiment, determining the distribution difference loss of the same shooting posture node between different heterogeneous images through all node label vectors can be achieved by using the following steps:

[0061] The distribution difference entropy of shooting posture nodes between different heterogeneous images is determined by all node label vectors;

[0062] For each shooting posture node, extract the distribution difference of the shooting posture node in different heterogeneous images;

[0063] The distribution difference loss of the shooting posture node between different heterogeneous images is determined by the distribution difference amount and the distribution difference entropy, and then the distribution difference loss of each shooting posture node between different heterogeneous images is obtained.

[0064] It should be noted that the distribution difference entropy in this application is an indicator used to measure the uncertainty of the position distribution of shooting posture nodes between different heterogeneous images.

[0065] In specific implementation, determining the distribution difference entropy of shooting posture nodes between different heterogeneous images by all node label vectors can be achieved in the following manner, namely: the Euclidean distance can be used as the difference value between every two corresponding elements of two node label vectors, and all the difference values are substituted into the information entropy calculation formula, and then the calculated information entropy is used as the distribution difference entropy of shooting posture nodes between different heterogeneous images; extracting the distribution difference amount of the position of the shooting posture node in different heterogeneous images can be achieved in the following manner, namely: the Euclidean distance between the distribution positions of the shooting posture nodes in different heterogeneous images can be used as the distribution difference amount of the position; determining the distribution difference loss of the shooting posture node between different heterogeneous images by the distribution difference amount and the distribution difference entropy can be achieved in the following manner, namely: for each shooting posture node, the distribution difference amount of the node in different images and the distribution difference entropy of all shooting posture nodes can be weightedly calculated, and the weighted result is used as the distribution difference loss of the node between different heterogeneous images, wherein the weighted weight can be set in advance according to historical calculation data, which will not be repeated here. Through the above method, the distribution difference loss of each shooting posture node between different heterogeneous images can be obtained.

[0066] It should be noted that the embodiment of the present application not only quantifies the spatial position differences of shooting posture nodes between different modal images by introducing distribution modeling of node label vectors, but also introduces the evaluation of the overall distribution stability of nodes by fusing distribution difference entropy. Compared with the existing technology that is only based on position or feature similarity alignment, it can more comprehensively reflect the uncertainty and offset trend of nodes under cross-modal images, thereby achieving more robust and higher-precision shooting posture recognition in a strong interference environment.

[0067] In step 103, when the image is interfered by strong light during the image acquisition process, the feature labels of the shooting posture nodes in different heterogeneous images are fuzzy aligned based on all distribution difference losses to obtain the attribution confidence of the shooting posture nodes in different heterogeneous images, and then the pixel constraint conditions of the shooting posture nodes in different heterogeneous images during interference suppression are determined based on all the attribution confidences.

[0068] It should be noted that strong light interference in this application refers to the phenomenon of overexposure, light spots or missing details in the image under the action of high-intensity light sources. In outdoor shooting environments, due to direct sunlight, reflections from objects or other high-intensity light sources, local areas of the collected images may be overexposed, unevenly bright or lost in details, thereby affecting the recognition accuracy of the shooting posture node features in the image.

[0069] In some embodiments, reference Figure 3 As shown in FIG, this figure is a schematic diagram of the process of determining the attribution confidence in some embodiments of the present application. In this embodiment, the element labels of the shooting posture nodes in different heterogeneous images are fuzzy registered based on all distribution difference losses. The attribution confidence of the shooting posture nodes in different heterogeneous images can be obtained by the following steps:

[0070] In step 1031, the fuzzy membership of the shooting posture nodes between different heterogeneous images is determined by all distribution difference losses;

[0071] In step 1032, fuzzy registration reasoning is performed on the element labels of the shooting posture nodes in different heterogeneous images according to all fuzzy memberships to obtain the label credibility between the shooting posture nodes;

[0072] In step 1033 , the attribution confidence of the shooting posture nodes in different heterogeneous images is determined through all label confidences.

[0073] It should be noted that the fuzzy membership in this application refers to the probability value of the shooting posture node's belonging to the real node distribution in different modal images; the element label in this application refers to the annotation information used to describe the spatial position, category and characteristic attributes of the shooting posture node in the image; the label credibility in this application is an indicator to measure the degree of matching reliability between shooting posture node labels in different modal images.

[0074] In the specific implementation, first, the fuzzy membership of the shooting posture node between different heterogeneous images is determined by all the distribution difference losses, which can be achieved in the following way, namely: the distribution difference loss of each shooting posture node between different heterogeneous images obtained by the above calculation can be used as the input variable of the fuzzy system. In order to convert the distribution difference loss into a measurable uncertainty membership relationship, a fuzzy membership function library is constructed. Commonly used function forms include Gaussian function and S-type membership function, where the smaller the difference, the larger the corresponding membership value. In the embodiment of the present application, the fuzzy membership function adopts the Gaussian function, and other fuzzy membership functions can also be used in other embodiments. No specific limitation is made here. For the Gaussian function The center is set to zero difference, and the standard deviation is set according to the statistics of the training images, so that the membership can be mapped to the possibility that the shooting posture node belongs to the target node in different heterogeneous image modes (i.e., visible light image and infrared thermal image). This process makes the registration between nodes no longer a hard judgment of "whether it matches", but a fuzzy judgment of "matching degree"; secondly, according to all the fuzzy memberships, the element labels of the shooting posture nodes in different heterogeneous images are fuzzy registered and inferred, and the label credibility between the shooting posture nodes can be obtained in the following way, namely: after calculating the fuzzy membership of the shooting posture nodes in all modal images, the fuzzy inference system (Fuzzy Inference System) in the prior art is used. Inference System (FIS) takes different fuzzy membership inputs for fusion decision-making. In this process, the position coordinates, confidence, and edge gradient label attributes of the shooting posture node in each modal image are used as input elements (i.e., element labels). Combined with the corresponding fuzzy membership, the rule base is used for reasoning. For example, "If the membership of image A is high and the difference loss of image B is large, the label of image A is preferred." After reasoning, a continuous value of label credibility is output to characterize the label matching confidence level of the shooting posture node between different image modalities, avoiding matching failure due to distortion of single label features; then, the confidence level of all labels in different heterogeneous images is determined. The attribution confidence of the shooting posture node can be achieved in the following way: taking the label credibility of the shooting posture nodes in all modal images as the evaluation index, the maximum credibility attribution method is used to perform the final attribution inference on the label of each node, and the attribution confidence of the shooting posture nodes in different heterogeneous images is obtained. For example, after normalizing the label credibility of each shooting posture node in each image modality, the one with the largest confidence is selected as the node attribution, or a fusion label is synthesized in proportion as the final representation of the node. This process realizes the alignment of image nodes under fuzzy conditions, effectively alleviating the label mismatch problem caused by inter-modal interference (such as overexposure of visible light images and blur of infrared thermal images).

[0075] It should be noted that by introducing a fuzzy membership and label credibility inference mechanism based on distribution difference loss in strong light interference scenarios, the problem of node information loss in visible light images caused by overexposure, reflection, etc. can be effectively alleviated. By integrating the relatively stable node distribution features in infrared thermal images and combining fuzzy registration to achieve flexible alignment of node element labels, the system can still judge the credible source of node information in each modal image based on fuzzy membership when facing severe distortion of local images, and then accurately generate attribution confidence. Compared with traditional technologies that rely on a single modality or rigid feature matching, the present application solution significantly improves the stability of shooting posture recognition under strong light interference.

[0076] In some embodiments, determining pixel constraints of shooting posture nodes of different heterogeneous images during interference suppression based on all attribution confidences can be implemented using the following steps:

[0077] Based on all the attribution confidences, the interference suppression parameters of the shooting posture node pixels in different heterogeneous images are modeled with multi-target collaborative constraints to obtain a set of pixel constraint conditions.

[0078] Based on a multi-objective optimization algorithm, the interference suppression parameters of the pixel constraint condition set are optimized to obtain the optimal suppression parameters of the shooting posture node pixels in different heterogeneous images;

[0079] The pixel constraints of shooting posture nodes of different heterogeneous images during interference suppression are determined through all optimal pixel suppression parameters.

[0080] It should be noted that the interference suppression parameters in this application refer to the control factors used to adjust the pixel characteristics of the shooting posture node area in the image to reduce the impact of strong light interference; the optimal suppression parameters in this application refer to the optimal parameter combination obtained through the optimization process, which is used to reduce interference in the image and enhance the feature information of the shooting posture node; the pixel constraint conditions in this application refer to the adjustment limits set for the image pixel area in order to ensure the accuracy of the characteristics of the shooting posture node under interference suppression during the image processing process.

[0081] In the specific implementation, first, the interference suppression parameters of the shooting posture node pixels in different heterogeneous images are modeled with multi-target collaborative constraints according to all the attribution confidences, and the pixel constraint condition set can be implemented in the following way, namely: the calculated attribution confidence can be used to evaluate the credibility of each shooting posture node in the visible light image and infrared thermal image. The higher the attribution confidence, the more reliable the information of the node in the image modality. The pixel feature parameters of the corresponding area of each posture node are extracted from each image, such as brightness value (L), gradient amplitude (G), local contrast (C) and infrared thermal response intensity (T), and the attribution confidence is set. The weight coefficient is set (here the first attribution confidence can be normalized, and the normalized value is used as the weight coefficient of the pixel feature parameter of the corresponding area of the posture node), and then the pixel feature parameters after weighting are combined into an interference sensitive feature vector. Through all the interference sensitive feature vectors, a multi-objective collaborative constraint model is established. The goal is to retain the effective node pixel information to the greatest extent while minimizing the pseudo response or distortion area caused by strong light interference. The modeling process can be implemented by normalized linear weighting or Pareto frontier construction, and the condition set obtained by modeling is used as the pixel constraint condition set; then, the multi-objective optimization algorithm is used to optimize the said The interference suppression parameter optimization is performed based on the pixel constraint condition set to obtain the optimal suppression parameters of the shooting posture node pixels in different heterogeneous images. This can be achieved in the following way: the constraint condition set is used as the optimization input, and a multi-objective evolutionary optimization algorithm in the prior art is introduced, such as NSGA-II (non-dominated sorting genetic algorithm). The pixel features of each shooting posture node in different modal images are used as decision variables. The optimization goals are set as follows: minimizing brightness saturation, maximizing heat map edge clarity, and maximizing the SNR (signal-to-noise ratio) of the posture area. In the iterative process, the optimal pixel control parameter set that meets the constraints is found through mutation, crossover, and selection mechanisms. The optimal pixel control parameter combination is used as the optimal suppression parameter of the shooting posture node pixel. The optimization result is the optimal interference suppression strategy for each node under each image modality, such as exposure compensation value, filtering threshold and image enhancement gain, so as to accurately control the visual response of the local image area of the node. Finally, the pixel constraint conditions of the shooting posture node in different heterogeneous images during interference suppression are determined by all the optimal pixel suppression parameters. This can be achieved in the following way: the optimal pixel suppression parameters output by the optimization are mapped to the corresponding shooting posture node in the image, and the pixel constraint conditions of the shooting posture node in different heterogeneous images are established through a unified pixel adjustment strategy.It should be noted that the pixel adjustment strategy in this application refers to a technical method for suppressing interference and enhancing features by controlling image pixel properties when the shooting posture node area is subject to interference (such as strong light interference) in the image. Its core lies in using optimal pixel suppression parameters to adaptively adjust the brightness, contrast, and texture clarity of the node area. In terms of technical implementation, this pixel adjustment strategy combines thermal feature enhancement in infrared thermal images with brightness suppression in visible light images based on the attribution confidence of the local area. Local histogram equalization is used to adjust the pixels in the shooting posture node area to enhance the stability and distinguishability of the posture features in interference environments. Through this pixel adjustment strategy, a consistent visual representation of the shooting posture node area can be formed in multimodal images, making the subsequent fusion recognition process more robust and accurate.

[0082] It should be noted that the present application scheme accurately formulates pixel constraints for interference suppression for shooting posture nodes in different heterogeneous images by combining attribution confidence and multi-objective optimization algorithm. Compared with the existing technology, it has the following significant advantages: First, the reliability of node features is quantified by attribution confidence, avoiding information loss or excessive suppression that may be caused by global adjustments in traditional methods; second, the use of multi-objective collaborative constraint modeling and optimization algorithm can accurately balance interference suppression and feature retention, and perform personalized processing on the local features of each shooting posture node, thereby improving the stability and accuracy of posture recognition; finally, the optimized pixel suppression parameters can adaptively respond to strong light interference, avoiding insufficient or excessive suppression effects in traditional single modes, and enhancing robustness in complex environments.

[0083] In step 104, based on the self-attention mechanism, cross-modal fusion attention of shooting posture nodes between different heterogeneous images is generated, and the spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraints of the shooting posture nodes to obtain the fusion features of the shooting posture.

[0084] In some embodiments, generating cross-modal fusion attention for shooting posture nodes between different heterogeneous images based on the self-attention mechanism can be implemented by the following steps:

[0085] Generate the attention weight matrix of shooting posture nodes in different heterogeneous images based on the self-attention mechanism;

[0086] The feature attention of shooting posture nodes between different heterogeneous images is fused through all attention weight matrices to obtain the cross-modal fused attention of shooting posture nodes between different heterogeneous images.

[0087] It should be noted that the cross-modal fusion attention in this application is an importance weight indicator that measures the significance of feature representation of the same shooting posture node in different modal images.

[0088] In the specific implementation, first, the attention weight matrix of the shooting posture nodes in different heterogeneous images is generated based on the self-attention mechanism. This can be achieved in the following way: the visible light image and the infrared thermal image are input into the pre-trained feature extraction network (such as HRNet) respectively to obtain the high-dimensional representation tensor F of the shooting posture node. vis 、F ir , where each node feature constitutes a vector. For the node features of each image modality, the (Query, Q), key (Key, K) and value (Value, V) matrices are constructed through linear transformation: For example, Q = W q F, K = W k F, V = W v F, and using the standard self-attention mechanism formula Calculate the attention weight matrix A within the modality vis and A ir , where d refers to the dimension of the Key vector, which represents the relative importance of the node features in the image modality. The attention weight matrices of the two modalities are used to weight the value vectors of the nodes to obtain the feature attention representation of each modality: F′ vis =A vis ·V vis , F′ ir =A ir ·V ir Then, all the attention weight matrices are used to fuse the feature attention of the shooting posture nodes between different heterogeneous images, and the cross-modal fusion attention of the shooting posture nodes between different heterogeneous images can be obtained. vis and F′ ir Perform feature fusion (such as weighted averaging, in this embodiment, the weights are set to the same value) to obtain the representation F of cross-modal fusion attention fusion , this representation takes into account the importance of nodes in both modes and their contextual relevance, and enhances the model's robust expression of shooting posture node features in complex interference scenarios. It should be further explained that W in this application q 、W k 、W v Represent the learnable weight matrices used to map input features to Query, Key, and Value spaces, respectively. They are essentially linear transformation parameters used to generate attention weights in the self-attention mechanism. Their functions are as follows: q , converting node features into Query expressions, indicating the content that the current node wants to “focus on”; W k, converting node features into Key expressions to represent the “information provided” by the current node; W v , converting node features into Value expressions, which are the information that is finally weighted and fused.

[0089] In some embodiments, the spatial distribution features of shooting posture nodes in different heterogeneous images are subjected to hierarchical constraint fusion by the cross-modal fusion attention and the pixel constraint of the shooting posture node to obtain the fusion feature of the shooting posture, which can be achieved by the following steps:

[0090] The pixel constraints of the shooting posture nodes are used to suppress and compensate the cross-modal fusion attention of the shooting posture nodes between different heterogeneous images, and the compensated cross-modal attention weights are obtained.

[0091] Perform hierarchical semantic alignment on the spatial distribution features of shooting posture nodes in different heterogeneous images to generate structural feature maps under each modality;

[0092] The structural feature maps under each modality are weightedly fused according to the compensated attention weights to obtain the fused features of the shooting posture.

[0093] It should be noted that the suppression compensation in this application refers to reducing the response attention of the interference area and enhancing the response attention of the effective area; the hierarchical semantic alignment in this application refers to structural matching of the node semantic information corresponding to different modal images at multiple feature levels.

[0094] In the specific implementation, first, the cross-modal fusion attention of the shooting posture nodes between different heterogeneous images is suppressed and compensated through the pixel constraints of the shooting posture nodes, and the compensated cross-modal attention weights can be implemented in the following way, namely: the cross-modal fusion attention is suppressed and compensated through the pixel constraints of the shooting posture nodes. Technically, pixel mask weighting can be used to suppress redundant attention areas, and give higher weights to the effective areas to generate compensated cross-modal attention weights. It should be further explained that pixel mask weighted suppression refers to the introduction of a pixel mask map generated by the pixel constraints of the shooting posture nodes on the basis of the cross-modal attention distribution, and structural intervention is performed on the attention distribution through element-by-element weighted operations. Specifically, a corresponding binary or continuous value mask is constructed based on the optimal pixel suppression parameters determined in each heterogeneous image. The mask value represents the credibility of the current pixel in the target posture area, and then the mask is Pixel-by-pixel multiplication is performed with the original cross-modal attention matrix to achieve explicit suppression of attention to redundant or irrelevant areas, while enhancing the response strength of key areas. It should also be noted that the method of the present application not only retains the dynamic weight distribution advantage of the attention mechanism, but also introduces spatial constraints to ensure that attention is focused on high-confidence shooting node areas, thereby improving the accuracy and stability of cross-modal feature fusion; secondly, the spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically semantically aligned to generate structural feature maps under each modality, which can be achieved in the following way: taking into account the natural differences in geometric distribution and perceptual scale between visible light images and infrared thermal images, in order to achieve their consistent expression in a unified feature space, this embodiment uses a multi-scale feature extraction network (such as U-Net) to extract shallow geometric structure information and deep semantic features of each modality image respectively, and introduces a differentiable spatial transformation module (such as Spatial The Transformer Network (STN) performs affine alignment operations such as rotation, translation, and scaling on the feature maps of each modality to achieve consistency in the spatial structure of multimodal features. After the above alignment process, the obtained feature structures of each modality are uniformly represented as a structural feature map, that is, the shooting posture features represented on the basis of clear scale hierarchy and spatial distribution alignment under different image modalities. Then, the structural feature maps of each modality are weighted fused according to the compensated attention weights to obtain the fused features of the shooting posture. This can be achieved by: an adaptive fusion module (such as Attention-Guided Fusion) can be used to perform channel-by-channel weighted summation of the structural feature maps of each modality according to the strength of attention. Residual connections are introduced in the fusion process to preserve the original feature structure and avoid the disappearance of key semantic information due to excessive weighting. This fusion mechanism ensures that the stable expression of key posture nodes in each modal image can be accurately extracted under interference conditions (such as strong illumination changes), and ultimately outputs a fused shooting posture feature map with strong cross-modal consistency and clear semantics.

[0095] It should be noted that the present application scheme realizes the joint recognition of shooting posture nodes under multimodal information by synchronously collecting visible light images and infrared thermal images, and uses multi-scale convolution to extract spatial distribution features, effectively capturing posture details at different scales. In the case of strong light interference, the association of cross-modal node elements is realized through the fuzzy registration method driven by distribution difference loss, and pixel-level constraints are further constructed based on the attribution confidence to enhance the accuracy of node matching between images. Finally, the self-attention mechanism and hierarchical constraint fusion strategy are introduced to take into account spatial consistency and semantic continuity while fusing different modal structure information, thereby significantly improving the robustness and accuracy of shooting posture recognition in complex outdoor environments.

[0096] On the other hand, in some embodiments, the present application provides an outdoor shooting assistance system based on image recognition, the outdoor shooting assistance system based on image recognition includes an image recognition unit, referring to Figure 4 , which is a schematic diagram of the structure of an image recognition unit according to some embodiments of the present application. The image recognition unit 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described as follows:

[0097] Acquisition module 401, in this application, acquisition module 401 is mainly used to synchronously acquire visible light images and infrared thermal images of the shooter's shooting posture in an outdoor shooting environment;

[0098] Processing module 402, in the present application, is used to perform multi-scale convolution on the visible light image and the infrared thermal image, respectively, to extract spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image, and then determine the distribution difference loss of the same shooting posture node between different heterogeneous images;

[0099] The processing module 402 in the present application is further configured to perform fuzzy registration on the element labels of the shooting posture nodes in different heterogeneous images based on all distribution difference losses when the image is interfered by strong light during the image acquisition process, thereby obtaining the attribution confidence of the shooting posture nodes in the different heterogeneous images, and then determining the pixel constraint conditions of the shooting posture nodes in the different heterogeneous images during interference suppression based on all the attribution confidences;

[0100] Execution module 403. In this application, execution module 403 is mainly used to generate cross-modal fusion attention of shooting posture nodes between different heterogeneous images based on the self-attention mechanism. The spatial distribution characteristics of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraints of the shooting posture nodes to obtain the fusion characteristics of the shooting posture.

[0101] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory storing a code, and the processor being configured to obtain the code and execute the above-mentioned shooting posture image recognition method.

[0102] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing a method for image recognition of shooting posture according to some embodiments of the present application. The method for image recognition of shooting posture in the above embodiment can be Figure 5 The computer device 500 shown in FIG. 5 is implemented as shown in FIG. 5 . The computer device 500 includes at least one processor 501 , a communication bus 502 , a memory 503 , and at least one communication interface 504 .

[0103] The processor 501 may be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0104] The communication bus 502 may be used to transmit information between the aforementioned components.

[0105] The memory 503 may be a read-only memory (ROM) or other static storage device that can store static information and instructions, a random access memory (RAM) or other dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 503 may be independent and connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0106] The memory 503 is used to store program code for executing the solution of the present application, and is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The image recognition method for shooting posture in the above embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0107] The communication interface 504 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0108] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0109] The aforementioned computer device may be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of computer device.

[0110] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned shooting posture image recognition method.

[0111] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0112] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A shooting posture image recognition method for an outdoor shooting assistance system to perform image recognition on the shooting posture of an outdoor shooter, characterized in that: The method comprises the following steps: Synchronously capture visible light images and infrared thermal images of the shooter's shooting posture in an outdoor shooting environment; performing multi-scale convolution on the visible light image and the infrared thermal image respectively, extracting spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image, and then determining the distribution difference loss of the same shooting posture node between different heterogeneous images; When the image is interfered by strong light during acquisition, the feature labels of the shooting posture nodes in different heterogeneous images are fuzzy registered based on all distribution difference losses to obtain the attribution confidence of the shooting posture nodes in different heterogeneous images. Then, the pixel constraint conditions of the shooting posture nodes in different heterogeneous images during interference suppression are determined based on all the attribution confidences. Based on the self-attention mechanism, cross-modal fusion attention of shooting posture nodes between different heterogeneous images is generated. The spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraints of the shooting posture nodes to obtain the fusion features of the shooting posture.

2. The method according to claim 1, wherein Extracting the spatial distribution features of the shooting posture nodes in the visible light image and the infrared thermal image, and then determining the distribution difference loss of the same shooting posture node between different heterogeneous images specifically includes: Construct a convolutional neural network module with multi-scale receptive fields; Extracting spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image through a convolutional neural network module; Determining a node label vector of a shooting posture node in the visible light image according to a spatial distribution feature corresponding to the visible light image; Determining a node label vector of a shooting posture node in the infrared thermal image according to a spatial distribution feature corresponding to the infrared thermal image; The distribution difference loss of the same shooting posture node between different heterogeneous images is determined through all node label vectors.

3. The method according to claim 1, wherein Based on all distribution difference losses, the feature labels of shooting posture nodes in different heterogeneous images are fuzzy registered to obtain the attribution confidence of shooting posture nodes in different heterogeneous images. Specifically, The fuzzy membership of the shooting posture nodes between different heterogeneous images is determined by all distribution difference losses; According to all the fuzzy memberships, fuzzy registration reasoning is performed on the element labels of the shooting posture nodes in different heterogeneous images to obtain the label credibility between the shooting posture nodes; The attribution confidence of the shooting posture nodes in different heterogeneous images is determined by all label confidences.

4. The method according to claim 1, wherein The pixel constraints of the shooting posture nodes of different heterogeneous images during interference suppression are determined based on all the attribution confidences, including: Based on all the attribution confidences, the interference suppression parameters of the shooting posture node pixels in different heterogeneous images are modeled with multi-target collaborative constraints to obtain a set of pixel constraint conditions. Based on a multi-objective optimization algorithm, the interference suppression parameters of the pixel constraint condition set are optimized to obtain the optimal suppression parameters of the shooting posture node pixels in different heterogeneous images; The pixel constraints of shooting posture nodes of different heterogeneous images during interference suppression are determined through all optimal pixel suppression parameters.

5. The method according to claim 1, wherein The cross-modal fusion attention based on the self-attention mechanism to generate shooting posture nodes between different heterogeneous images specifically includes: Generate the attention weight matrix of shooting posture nodes in different heterogeneous images based on the self-attention mechanism; The feature attention of shooting posture nodes between different heterogeneous images is fused through all attention weight matrices to obtain the cross-modal fused attention of shooting posture nodes between different heterogeneous images.

6. The method according to claim 1, wherein The visible light image is collected by a camera.

7. The method according to claim 1, wherein The infrared thermal image is collected by an infrared thermal imager.

8. An outdoor shooting assistance system based on image recognition, the outdoor shooting assistance system based on image recognition comprising an image recognition unit, characterized in that: The image recognition unit includes: An acquisition module, for synchronously acquiring visible light images and infrared thermal images of a shooter's shooting posture in an outdoor shooting environment; a processing module, configured to perform multi-scale convolution on the visible light image and the infrared thermal image, respectively, to extract spatial distribution features of shooting posture nodes in the visible light image and the infrared thermal image, and further determine distribution difference losses of the same shooting posture node between different heterogeneous images; The processing module is further configured to perform fuzzy registration on the element labels of the shooting posture nodes in different heterogeneous images based on all distribution difference losses when the image is interfered by strong light during the image acquisition process, thereby obtaining the attribution confidence of the shooting posture nodes in the different heterogeneous images, and then determining the pixel constraint conditions of the shooting posture nodes in the different heterogeneous images during interference suppression based on all the attribution confidences; An execution module is used to generate cross-modal fusion attention of shooting posture nodes between different heterogeneous images based on the self-attention mechanism. The spatial distribution features of shooting posture nodes in different heterogeneous images are hierarchically constrained and fused by the cross-modal fusion attention and the pixel constraints of the shooting posture nodes to obtain the fusion features of the shooting posture.

9. A computer device comprising a memory and a processor, wherein the memory stores a code, wherein: The processor is configured to obtain the code and execute the image recognition method for shooting posture according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image recognition method for shooting posture according to any one of claims 1 to 7 is implemented.

Images