Model training method and device and target accessory identification method

Abstract

The invention discloses a model training method and device and a target accessory identification method. The method comprises the following steps: acquiring a historical image; adjusting positions and weights of sampling points in the historical image, and performing feature extraction on the historical image according to an adjustment result to obtain a first target feature map; performing multi-channel feature fusion on the first target feature map to obtain second target feature maps of different levels; performing feature fusion processing on the second target feature maps of different levels by using a region attention mechanism to obtain a third target feature map; and training a deep learning model by using the third target feature map to obtain a target accessory identification model, the target accessory identification model being used for identifying whether the target object wears the target accessory. The technical problem that the recognition precision of a related target accessory recognition model is low is solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision application technology, and more specifically, to a model training method, device, and target accessory recognition method. Background Art

[0002] In modern industrial and catering environments, the wearing of factory hard hats and kitchen chef hats is crucial to the safety and hygiene of personnel. However, related monitoring methods usually rely on manual supervision, which is prone to omissions and inefficient. With the development of computer vision and deep learning technology, automated wearing detection solutions have gradually become a viable alternative.

[0003] Existing solutions usually use deep learning target detection algorithms to detect the wearing of factory helmets and kitchen chef hats. Common algorithms include the YOLO series (You Only Look Once), SSD (Single Shot MultiBox Detector) and Faster R-CNN (Faster Region Convolutional Neural Network). These algorithms are usually based on convolutional neural networks (CNN) as the feature extraction backbone network, and use trained models to perform target detection on input images to identify whether the helmet or chef hat is worn correctly.

[0004] In these existing technical solutions, labeled data sets are used for training so that the model can recognize the wearing situation. During the training process, the model requires a large amount of labeled data and computing resources to improve its detection accuracy and robustness. After the training is completed, the model can detect the wearing situation in a real-time video stream or static image and output the detection results. Although these technologies can achieve automated detection to a certain extent, they still face the following technical problems in practical applications: Due to the complex environment of factories and kitchens and severe background interference, the relevant technical solutions may have false detection or missed detection during detection.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a model training method, device and target accessory recognition method to at least solve the technical problem of low recognition accuracy of the relevant target accessory recognition model.

[0007] According to one aspect of an embodiment of the present application, a model training method is provided, including: acquiring historical images, wherein the historical images include: a target object wearing a target accessory; adjusting the positions and weights of sampling points in the historical images, and performing feature extraction on the historical images according to the adjustment results to obtain a first target feature map; performing multi-channel feature fusion on the first target feature map to obtain second target feature maps of different levels; performing feature fusion processing on the second target feature maps of different levels using a regional attention mechanism to obtain a third target feature map; and training a deep learning model using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

[0008] Optionally, the positions and weights of sampling points in the historical image are adjusted, and features of the historical image are extracted based on the adjustment results to obtain a first target feature map, including: adaptively adjusting the positions of the sampling points based on local features of the historical image; assigning different weights to sampling points at different positions based on an adaptive weight function; and extracting features of the historical image based on the adjusted positions of the sampling points and the weights corresponding to the sampling points at different positions to obtain a first target feature map.

[0009] Optionally, the position of the sampling point is adaptively adjusted according to the local features of the historical image, including: inputting local feature information of the sampling point at the target position into a neural network to obtain a horizontal offset and a vertical offset of the sampling point at the target position output by the neural network; and adjusting the position of the sampling point according to the horizontal offset and the vertical offset of the sampling point at the target position.

[0010] Optionally, multi-channel feature fusion is performed on the first target feature map to obtain second target feature maps of different levels, including: performing dimensionality reduction processing on the first target feature map to obtain a first feature map, wherein the first target feature map includes multiple channels; performing weighted fusion on the first feature map to obtain a second feature map; performing nonlinear activation and regularization processing on the second feature map to obtain a third feature map; and performing feature processing on the third feature map using a 1×1 convolution kernel or a depth-separable convolution kernel to obtain a second target feature map.

[0011] Optionally, a regional attention mechanism is used to perform feature fusion processing on second target feature maps of different levels to obtain a third target feature map, including: calculating the attention weight of each region in the second target feature map of the i-th layer according to the importance coefficient of each region in the second target feature map of the i-th layer, where i is a positive integer not greater than n, and n is the number of levels of the second target feature map; performing weighted summation on each region in the second target feature map of the i-th layer according to the attention weight of each region in the second target feature map of the i-th layer to obtain a fourth feature map of the i-th layer; and fusing the fourth feature map of each layer to obtain a third target feature map.

[0012] Optionally, after fusing the fourth feature map of each layer to obtain the third target feature map, the method further includes: performing edge detection and boundary enhancement processing on the second target feature map of the j-th layer to obtain the fifth feature map of the j-th layer, where j is a positive integer not greater than n; and performing weighted fusion processing on the fifth feature map of each layer.

[0013] Optionally, after the regional attention mechanism is used to perform feature fusion processing on the second target feature maps of different levels to obtain the third target feature map, the method also includes: fusing the third target feature map of the kth layer with the third target feature map of the k-1th layer to obtain a first aggregation result, wherein u is a positive integer not greater than m, and m is the number of levels of the third target feature map; performing independent convolution operations on the features of each channel in the first aggregation result, and fusing the features of different channels through a 1×1 convolution kernel.

[0014] According to another aspect of an embodiment of the present application, a method for identifying a target accessory is also provided, including: obtaining an image to be identified, wherein the image to be identified includes: a target object; using a target accessory recognition model to identify whether the target object in the image to be identified is wearing a target accessory, wherein the target accessory recognition model is trained by the above-mentioned model training method.

[0015] According to another aspect of the embodiments of the present application, a model training device is also provided, including: an acquisition module for acquiring historical images, wherein the historical images include: a target object wearing a target accessory; an adjustment module for adjusting the positions and weights of sampling points in the historical images, and performing feature extraction on the historical images according to the adjustment results to obtain a first target feature map; a first fusion module for performing multi-channel feature fusion on the first target feature map to obtain second target feature maps of different levels; a second fusion module for performing feature fusion processing on second target feature maps of different levels using a regional attention mechanism to obtain a third target feature map; a training module for training a deep learning model using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

[0016] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, and the storage medium includes a stored program, wherein when the program runs, the device where the storage medium is located is controlled to execute the above model training method.

[0017] According to another aspect of an embodiment of the present application, an electronic device is provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above model training method is executed when the program is running.

[0018] According to another aspect of the embodiments of the present application, a computer program is also provided, wherein the above model training method is implemented when the computer program is executed by a processor.

[0019] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above model training method is implemented.

[0020] In an embodiment of the present application, a historical image is obtained, wherein the historical image includes: a target object wearing a target accessory; the position and weight of the sampling point in the historical image are adjusted, and features are extracted from the historical image according to the adjustment result to obtain a first target feature map; multi-channel feature fusion is performed on the first target feature map to obtain second target feature maps of different levels; feature fusion processing is performed on the second target feature maps of different levels using a regional attention mechanism to obtain a third target feature map; a deep learning model is trained using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory, thereby achieving the purpose of jointly modeling features of different scales, thereby improving the recognition ability of the target, thereby achieving the technical effect of improving the recognition accuracy of the target accessory recognition model, and further solving the technical problem of low recognition accuracy of the relevant target accessory recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 is a flow chart of a model training method according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of another model training method according to an embodiment of the present application;

[0024] Figure 3 is a structural diagram of a model training device according to an embodiment of the present application;

[0025] Figure 4 It is a hardware structure block diagram of a computer terminal according to a model training method of an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to an embodiment of the present application, a method embodiment of a model training method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] Figure 1 is a flow chart of a model training method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0030] Step S101, obtaining a historical image, wherein the historical image includes: a target object wearing a target accessory.

[0031] Target accessories include: chef hats, hard hats, masks, goggles, etc.

[0032] First, collect a series of historical images containing target objects (such as workers or chefs) wearing target accessories (such as hard hats or chef hats). These images can come from surveillance cameras in factories or kitchens, or from public datasets. It is important that these images cover a variety of lighting conditions, background environments, and accessory states to ensure that the model can generalize to a variety of real-world scenarios.

[0033] Step S102, adjusting the positions and weights of the sampling points in the historical image, and performing feature extraction on the historical image according to the adjustment result to obtain a first target feature map.

[0034] Dynamically adjust the receptive field size of the convolution kernel and the weight of the sampling point to more effectively capture local feature changes in the image. This process adaptively adjusts the position and weight of each sampling point according to the characteristics of the input image to generate the first target feature map. Through the above steps, the model can pay more attention to important features while reducing the consumption of computing resources.

[0035] Step S103, performing multi-channel feature fusion on the first target feature map to obtain second target feature maps of different levels.

[0036] The first target feature map is subjected to lightweight multi-channel feature fusion, and features from different channels are integrated through dimensionality reduction and weighted fusion to obtain the second target feature map. The second target feature map contains multi-scale and multi-channel information, which enhances the feature expression ability and helps the model to more accurately identify target accessories in complex scenes.

[0037] Step S104, using the regional attention mechanism to perform feature fusion processing on the second target feature maps at different levels to obtain a third target feature map.

[0038] The second target feature map is further processed using a feature pyramid based on regional attention. By calculating the attention weights, the feature extraction of key areas in the image is strengthened to generate a third target feature map, which has stronger robustness and accuracy in detecting target accessories.

[0039] Step S105, using the third target feature map to train the deep learning model to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

[0040] The deep learning model is trained using the third target feature map as training data. At this stage, the model learns how to identify the target accessory from the feature map. Through back propagation and weight update, the model is gradually optimized to improve the recognition accuracy. After the training is completed, the target accessory recognition model obtained can be used for real-time or offline image analysis to determine whether the target object is wearing the target accessory correctly.

[0041] The entire process starts with collecting image data, and through multiple steps such as adaptive feature sampling, multi-channel feature fusion, regional attention mechanism and boundary enhancement, the feature representation is gradually optimized to improve the detection ability of the model. Ultimately, the trained model can be effectively used to monitor the wearing of factory helmets and kitchen chef hats to ensure the safety and hygiene of the working environment.

[0042] The following Figure 1 The steps shown are exemplary and explanatory.

[0043] According to some optional embodiments of the present application, the positions and weights of sampling points in historical images are adjusted, and features of the historical images are extracted based on the adjustment results to obtain a first target feature map. This can be achieved by the following method: adaptively adjusting the positions of the sampling points based on local features of the historical images; assigning different weights to sampling points at different positions based on an adaptive weight function; and extracting features of the historical images based on the adjusted positions of the sampling points and the weights corresponding to the sampling points at different positions to obtain a first target feature map.

[0044] Perform local feature analysis on each point to be sampled in the historical image, such as calculating the gradient, direction, texture information, etc. of the point to determine the feature importance of the point. For example, in an image, if a region contains edge or texture-rich information, this information may be particularly important for identifying target accessories. According to the results of the local feature analysis, the position of the sampling point is adjusted by an adaptive algorithm. Specifically, the algorithm moves the sampling point to a position that can capture more key information based on the importance of the feature, avoiding wasting computing resources in areas with weaker features. This step is achieved by learning the offset of the sampling point, which is determined by the local features of the point. Design an adaptive weight function, which is also based on the local features of the sampling point. The purpose of the weight function is to highlight those sampling points that contain important information and reduce the attention to background or unimportant features. The weight function may be a neural network that learns how to automatically assign weights based on the input features. Use the adjusted position and weights for feature extraction. For each sampling point, the convolution kernel extracts features based on its adjusted position and corresponding weight. This is equivalent to weighting the sampling points in the feature space, ensuring that the model can more effectively capture the key features in the image. Finally, the first target feature map is generated by weighted fusion of the features of all sampling points, which contains the feature information of all sampling points and highlights the important features.

[0045] According to some other optional embodiments of the present application, adaptively adjusting the position of the sampling point according to the local features of the historical image can be achieved by the following method: inputting the local feature information of the sampling point at the target position into the neural network to obtain the horizontal offset and the vertical offset of the sampling point at the target position output by the neural network; adjusting the position of the sampling point according to the horizontal offset and the vertical offset of the sampling point at the target position.

[0046] First, feature information is extracted from the local area of ​​the target location. These local features may include edges, textures, color information, etc., which can reflect the local features of the target accessory. The above information can be generated by a small convolutional neural network (CNN) or other feature extraction module, which can process local details in the image. The extracted local feature information is input into a specially designed neural network. The task of this neural network is to learn and predict the position adjustment of the sampling points, specifically the offset in the horizontal and vertical directions. The structure of the neural network may include convolutional layers, fully connected layers, etc., which are used to process feature information and generate position offset predictions. Based on the input local feature information, the neural network outputs the horizontal offset and vertical offset of each sampling point at the target location. The offset reflects the distance that the sampling point needs to move so that it is more accurately located at the key feature position of the target accessory. According to the offset output by the neural network, the position of each sampling point on the historical image is adjusted. The adjusted sampling point position will be closer to the boundary or key feature of the target accessory, so that more target information can be captured in the subsequent feature extraction process and the interference of background information can be reduced. The feature extraction is repeated using the adjusted position. In this stage, each sample point will be convolved on the feature map according to its new position to obtain an optimized feature representation.

[0047] In some optional embodiments of the present application, multi-channel feature fusion is performed on the first target feature map to obtain second target feature maps of different levels, which can be achieved by the following method: performing dimensionality reduction processing on the first target feature map to obtain a first feature map, wherein the first target feature map includes multiple channels; performing weighted fusion on the first feature map to obtain a second feature map; performing nonlinear activation and regularization processing on the second feature map to obtain a third feature map; and performing feature processing on the third feature map using a 1×1 convolution kernel or a depth-separable convolution kernel to obtain a second target feature map.

[0048] The first target feature map is subjected to dimensionality reduction processing to reduce the number of channels of the feature map, thereby reducing the computational complexity and the number of parameters. Dimensionality reduction can be achieved through a 1×1 convolutional layer or a linear transformation, the purpose of which is to retain key information while reducing the dimension of the data. The first feature map is output, and its number of channels is reduced compared to the first target feature map, but rich feature information is still retained. The feature map after dimensionality reduction is weighted fused, and a weight matrix needs to be learned or pre-set. The matrix is ​​used to adjust the contribution of each channel to integrate feature information at different levels and reduce redundancy. The second feature map is output, which is composed of the weighted fusion results of features from different channels, and can more effectively represent the features in the image with relatively small computational complexity. The second feature map is subjected to nonlinear activation processing, and activation functions such as ReLU, Leaky ReLU, etc. are used to increase the nonlinear expression ability of the model. At the same time, regularization processing is performed to avoid overfitting and ensure the generalization ability of the model. The third feature map is output, which has richer feature expression and better generalization performance after nonlinear activation and regularization. Use 1×1 convolution kernel or depth-wise separable convolution kernel for feature processing to further reduce computational complexity. 1×1 convolution kernel can recombine features, while depth-wise separable convolution kernel can efficiently perform convolution operations, reducing the amount of computation through the combination of depth-wise convolution and point-wise convolution. Output the second target feature map, which, after lightweight convolution processing, not only retains key information, but also further enhances the expression of features, and optimizes computational efficiency and model size.

[0049] As some optional embodiments of the present application, feature fusion processing is performed on second target feature maps of different levels using a regional attention mechanism to obtain a third target feature map. This can be achieved by the following method: according to the importance coefficient of each region in the second target feature map of the i-th layer, the attention weight of each region in the second target feature map of the i-th layer is calculated, where i is a positive integer not greater than n, and n is the number of levels of the second target feature map; according to the attention weight of each region in the second target feature map of the i-th layer, a weighted sum is performed on each region in the second target feature map of the i-th layer to obtain a fourth feature map of the i-th layer; the fourth feature map of each layer is fused to obtain a third target feature map.

[0050] The input is the second target feature map of the i-th layer, where i is a positive integer from 1 to n (n is the number of levels of the feature pyramid). First, the importance coefficient of each region is calculated through a lightweight neural network module, such as a convolutional layer plus global average pooling and a fully connected layer. This importance coefficient reflects the contribution of each region in the image to the target detection task. According to the calculated importance coefficient of each region, the corresponding attention weight is generated. The importance coefficient is converted into a weight matrix, each element in which corresponds to the weight of a region in the feature map. The softmax function can be used to achieve normalization to ensure that the sum of the weights is 1. According to the attention weight matrix of the i-th layer, the second target feature map of the i-th layer is weighted and summed. Specifically, in an element-by-element manner, the eigenvalue of each region is multiplied by its corresponding attention weight and summed to obtain a weighted feature map. This feature map highlights the regional features that are more critical to target detection and reduces the interference of background information. The fourth feature map generated by each layer is fused. The sizes of feature maps at different levels can be adjusted by upsampling or downsampling operations, and then weighted fusion is performed to obtain a comprehensive feature map, namely the third target feature map. The fusion process can be performed by weighted addition or concatenation to ensure that feature information at different scales is fully integrated.

[0051] Through the above steps, the feature pyramid based on regional attention can dynamically assign attention weights according to the importance of each region, and then weight the key areas in the feature map. Finally, through multi-layer feature fusion processing, a third target feature map that integrates multi-level information is obtained. This method can significantly improve the detection accuracy and robustness of the model when dealing with multi-scale targets and complex backgrounds.

[0052] In some optional embodiments of the present application, after the fourth feature map of each layer is fused to obtain the third target feature map, the following steps can also be performed: edge detection and boundary enhancement are performed on the second target feature map of the jth layer to obtain the fifth feature map of the jth layer, where j is a positive integer not greater than n; and weighted fusion is performed on the fifth feature map of each layer.

[0053] First, edge detection is performed on the second target feature map, usually using the Sobel operator, Canny edge detector or other edge detection methods to identify the boundary information of the target in the image. Next, boundary enhancement processing is performed, which involves weighting the detected edge information to enhance the model's sensitivity to the target boundary. The fifth feature map of the jth layer is output, which is a feature map with enhanced boundaries and highlighted edge information, which can more clearly express the boundaries of the target accessories. At all levels of the feature pyramid, the boundary enhancement feature maps of each layer are weighted and fused. Specifically, a weight matrix can be learned to adjust the contribution of feature maps at different levels to ensure that the network can comprehensively process boundary information from coarse to fine.

[0054] As other optional embodiments of the present application, after the regional attention mechanism is used to perform feature fusion processing on the second target feature maps of different levels to obtain the third target feature map, the following steps can also be performed: the third target feature map of the kth layer is fused with the third target feature map of the k-1th layer to obtain a first aggregation result, where u is a positive integer not greater than m, and m is the number of levels of the third target feature map; an independent convolution operation is performed on the features of each channel in the first aggregation result, and the features of different channels are fused through a 1×1 convolution kernel.

[0055] The feature maps of the two levels are fused to achieve the transmission and enhancement of feature information. Before fusion, the feature maps need to be size-aligned so that they can be directly weighted fused. The fusion method is to adjust the two feature maps to the same size by upsampling (if the low-level feature map is small) or downsampling (if the high-level feature map is large). Then, the weighted summation method is used to fuse and obtain the first aggregation result. The fusion weights can be pre-set or learned to reflect the importance of features at different levels. Output: The first aggregation result, which is a fused feature map containing the feature information of the kth layer and the k-1th layer.

[0056] Perform independent convolution operations on the features of each channel in the first aggregation result, usually using depthwise separable convolution, that is, first perform depthwise convolution to process each input channel independently, and then perform pointwise convolution to fuse the information of different channels. Next, further fuse the features of different channels through a 1×1 convolution kernel to integrate multi-channel information, while reducing the number of channels and computational complexity. The output feature map after the fusion of independent convolution and 1x1 convolution is denoted as, which integrates the information aggregated from feature maps at different levels, and the number of channels is optimized, providing a more lightweight and highly expressive feature representation for subsequent detection.

[0057] In summary, this application optimizes the feature extraction and feature pyramid structure of the YOLOv5 model to enhance the model's ability to recognize factory safety helmets and kitchen chef hats in complex backgrounds, reduce missed detections and false detections, and ensure the accuracy of the detection results. For targets of different sizes and shapes in complex environments such as factories and kitchens, the model's multi-scale feature extraction capabilities are improved so that it can better adapt to a variety of application scenarios. While maintaining high detection performance, by introducing lightweight feature extraction and feature fusion modules, the model's computing and storage requirements are reduced, enabling it to run efficiently in resource-constrained embedded devices or edge computing environments.

[0058] The embodiment of the present application also provides a method for identifying a target accessory, comprising: obtaining an image to be identified, wherein the image to be identified includes: a target object; using a target accessory identification model to identify whether the target object in the image to be identified is wearing the target accessory, wherein the target accessory identification model is Figure 1 The model training method shown is trained.

[0059] It should be noted that the preferred implementation of this embodiment can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0060] Figure 2 is a flowchart of another model training method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0061] Step S201, obtaining an image to be recognized.

[0062] The smart camera captures real-time video streams through the OpenCV library to obtain continuous image frames. This process ensures that the system can monitor the status of the monitored target object in real time.

[0063] Frame t =OpenCV.capture(t)

[0064] Among them, Frame t represents the image frame captured at time t.

[0065] Step S202, data enhancement and optimization of the image. Each frame of the image is preprocessed, including resizing and normalization. In the preprocessing stage, the image is adjusted to a size acceptable to the model, usually 640*640 pixels. Next, after normalization, the pixel values ​​are scaled to between 0 and 1 for network processing. Such a preprocessing process ensures the consistency and standardization of the image, so that the improved backbone network and feature pyramid can better extract image features.

[0066] Step S2021, resizing:

[0067] Image resized =Resize(Frame t ,640×640)

[0068] Step S2022, normalization:

[0069]

[0070] Step S203, use adaptive feature sampling convolution to extract and represent features of the preprocessed image. Adaptive Feature Sampling Convolution (AFSC) is an improved convolution operation that dynamically adjusts the position and weight of the sampling points so that the convolution kernel can more flexibly adapt to local feature changes in the image, thereby improving the accuracy of feature extraction. This method is particularly suitable for feature extraction of complex textures, irregular shapes and multi-scale targets. The traditional convolution operation slides the convolution kernel within a fixed receptive field, and the position of the sampling points is fixed, which makes it difficult to adapt to the feature differences in different regions of the input image. The core idea of ​​AFSC is to adaptively adjust the sampling points in the convolution operation so that the sampling points can be dynamically adjusted according to the local changes of the input features, thereby enhancing the focus on important features.

[0071] Step S2031, initial convolution operation: Assume that the input image feature map is I, the convolution kernel is K, and the standard convolution operation can be expressed as:

[0072]

[0073] Among them, O(i,j) is the output feature map, K(m,n) is the weight of the convolution kernel, I(i+m,j+n) is the pixel value at the corresponding position in the input feature map, and M and N are the sizes of the convolution kernel.

[0074] Step S2032, adaptive sampling point adjustment: In the adaptive feature sampling convolution, the position of each sampling point is no longer fixed, but is adaptively adjusted according to the local features of the input feature map. Assuming the offset of the sampling point is Δm and Δn, the convolution operation of adaptive sampling is expressed as:

[0075]

[0076] Among them, Δm(i,j) and Δn(i,j)) are the sampling point offsets adaptively adjusted according to the input features at position i and j.

[0077] Step S2033, calculation of sampling point offset: The calculation of sampling point offsets Δm(i,j) and Δn(i,j)) can be implemented by an offset prediction module. The module consists of a lightweight neural network, which inputs the feature information of the current position and outputs the corresponding offset:

[0078] Δm(i,j)=f offsetm (I(i,j))

[0079] Δn(i,j)=f offsetn (I(i,j))

[0080] Among them, f offsetm (I(i,j) and f offsetn (I(i,j)) is the output of the prediction module, which represents the offset in the horizontal and vertical directions respectively.

[0081] In step S2034, in addition to adjusting the sampling point positions, AFSC can also adaptively adjust the weight of each sampling point. By learning a weight function w(i,j), different weights are assigned to sampling points at different positions in the convolution operation:

[0082]

[0083] Among them, w(i+m,j+n) is an adaptive weight function, which indicates the importance of the current sampling point.

[0084] In step S2035, through adaptive feature sampling convolution, the output feature map O(i, j) can more accurately capture the key feature information in the input image. Especially when processing images with complex structures, AFSC performs particularly well.

[0085] Through the above process, AFSC can flexibly adjust the receptive field of the convolution kernel to adapt to different feature distributions, significantly improving the feature extraction capability of the model.

[0086] Step S204, improving the feature expression capability through a lightweight multi-channel feature fusion module. The lightweight multi-channel feature fusion module (LMCFFM) aims to achieve the purpose of improving the feature expression capability by effectively fusing and integrating multi-level and multi-scale feature maps. The design of this module takes into account reducing the computational complexity and the amount of parameters while maintaining the effectiveness of feature extraction, and is suitable for use when computing resources are limited. During the feature extraction process, the neural network generates multi-channel feature maps, which may capture information of different scales and different semantics at different levels. The core idea of ​​LMCFFM is to fuse features from different channels through lightweight operations, so that the final feature map contains more rich information without significantly increasing the computational overhead.

[0087] Step S2041, multi-channel feature map generation: In the process of feature extraction, different layers in the network will generate feature maps of multiple channels. Let the feature map generated by the i-th layer be denoted as F i , and these feature maps contain different numbers of channels C i .

[0088]

[0089] Among them, H i and W i Represent the height and width of the feature map, C i Indicates the number of channels.

[0090] Step S2042, feature dimensionality reduction: In order to reduce the computational complexity, firstly, each feature map is subjected to dimensionality reduction operation to reduce the number of its channels. Assuming that the dimensionality reduction operation is implemented by linear transformation, the output is the feature map F after dimensionality reduction. i ′ :

[0091] F i ′ =W i ×F i

[0092] Among them, W i To reduce the dimensionality of the matrix, a smaller matrix dimension is usually used to reduce the number of parameters.

[0093] Step S2043, feature fusion: feature map F after dimensionality reduction i ′ The fusion is performed by weighted fusion to integrate multi-scale information. Let the fused feature map be F fusion , the calculation process is:

[0094]

[0095] Among them, α i is the fusion weight, which indicates the importance weight of the feature map of the i-th layer in the fusion. This weight can be obtained through learning or distributed equally.

[0096] Step S2044, nonlinear activation and regularization: the fused feature map F fusion Perform nonlinear activation and regularization to enhance the expressiveness of features. Commonly used nonlinear activation functions include ReLU, Leaky ReLU, etc. The activated feature map is recorded as:

[0097] F activated =σ(F fusion )

[0098] Among them, σ represents a nonlinear activation function.

[0099] Step S2045, in order to make the entire module as lightweight as possible and reduce the number of parameters and the amount of calculation, LMCFFM usually uses simplified convolution operations (such as 1x1 convolution) or depth-separable convolution for feature processing. In this way, while maintaining the feature fusion effect, the computational overhead is minimized.

[0100] Step S2046, the final output feature map F output It contains fused information from multiple channels and multiple levels, which can better characterize the features of the input image, and after lightweight processing, it will not significantly increase the computational complexity:

[0101] F output =F activated

[0102] Through this series of steps, the lightweight multi-channel feature fusion module (LMCFFM) can effectively fuse multi-level features from different channels and improve the feature expression ability of the model without significantly increasing the amount of computation.

[0103] Step S205, introduce the regional attention mechanism to enhance the expressiveness of the feature pyramid. The feature pyramid based on regional attention (RAFP) aims to enhance the expressiveness of the feature pyramid by introducing the regional attention mechanism, and improve the recognition accuracy of the target detection model in complex scenes. This mechanism optimizes the extraction and fusion of features by focusing on areas with important semantic information in the image, thereby improving the detection performance. The traditional feature pyramid structure usually relies on fixed levels and scales to fuse features at different levels, while RAFP enables the feature pyramid to adaptively focus on important areas in the image through the regional attention mechanism. This method not only improves the expressiveness of the features, but also enhances the target detection effect of the model under complex backgrounds.

[0104] Step S2051, feature pyramid generation: extract feature maps from different layers of the convolutional network, which correspond to different scales and semantic levels. Let the feature map extracted from the lth layer be F l , whose dimensions are:

[0105]

[0106] Among them, H l and W l Represent the height and width of the feature map, C l Indicates the number of channels.

[0107] Step S2052, regional attention mechanism: for each feature map F lApply the regional attention mechanism to adaptively focus on the key areas in the image. The regional attention mechanism weights each region in the feature map by calculating the attention weight of the region. Let the attention weight be A l , calculated as:

[0108] A l =softmax(W a ×F l )

[0109] Among them, W a is the attention weight matrix, and softmax represents the normalization of the attention distribution of each region in the feature map. Regional attention weight A l Is a l A matrix with the same spatial dimension, reflecting the importance of each region.

[0110] Step S2053, weighted feature map: using regional attention weight A l Weight the feature map to obtain the weighted feature map F weighted :

[0111] F weighted =A l ☉F l

[0112] Among them, ⊙ represents the element-by-element multiplication operation. The weighted feature map will highlight the key area information in the image.

[0113] Step S2054, multi-scale feature fusion: The weighted feature map F weighted Fusion is performed from feature maps of different scales. First, adjust each feature map to the same size for fusion. Set the aligned feature map as F′ weighted , calculated as:

[0114] F′ weighted =Resize(F weighted )

[0115] Next, weighted fusion is performed to integrate weighted feature maps of different scales. Let the fused feature map be F fusion :

[0116]

[0117] Among them, β l is the fusion weight of each scale feature map, and Resize means resizing the feature map.

[0118] Feature Pyramid Output: The fused feature map F fusionAs the output of the final feature pyramid, these feature maps combine attention information from different scales and regions to better describe the objects in the image.

[0119] F output =F fusion

[0120] Step S206, introduces a boundary enhancement mechanism so that the model can pay more attention to the edge details of the target. Boundary enhancement-based feature fusion (BEFF) aims to improve the accuracy of target recognition by enhancing the boundary information of the target in the image and improving the effect of feature fusion. This method introduces a boundary enhancement mechanism in the feature fusion process, so that the model can pay more attention to the edge details of the target, reduce background interference, and improve detection performance. Traditional feature fusion methods mainly rely on weighted fusion of feature maps at different levels. However, in target detection, boundary information is crucial for accurate target identification. BEFF introduces a boundary enhancement mechanism to emphasize the boundary features of the target in the image, improve the effect of multi-scale feature fusion, and thus improve recognition performance.

[0121] Step S2061, feature map generation: extract feature maps from different layers of the convolutional neural network. Suppose the feature map extracted from the lth layer is F l , whose dimensions are:

[0122]

[0123] Among them, H l and W l Represent the height and width of the feature map, C l Indicates the number of channels.

[0124] Step S2062, boundary enhancement module: for each feature map F l The boundary enhancement module is applied to emphasize the boundary information of the target. The boundary enhancement module usually includes edge detection and boundary enhancement operations. Edge detection can be achieved by calculating the gradient of the feature map to obtain the edge map E l :

[0125]

[0126] in, Represents the gradient calculation operation. The boundary enhancement operation combines the edge map with the original feature map to generate an enhanced feature map F enhanced :

[0127] F enhanced =F l +λ×E l

[0128] Among them, λ is the weight coefficient of boundary enhancement, which is used to adjust the influence of boundary information.

[0129] Step S2063, multi-scale feature fusion: the feature map F after boundary enhancement enhanced Fusion is performed from feature maps of different scales. First, adjust each feature map to the same size for fusion. Set the aligned feature map as F′ enhanced , calculated as:

[0130] F′ enhanced =Resize(F enhanced )

[0131] Next, weighted fusion is performed to integrate enhanced feature maps of different scales. Let the fused feature map be F fusion :

[0132]

[0133] Among them, βl is the fusion weight of each scale feature map, and Resize means resizing the feature map.

[0134] Step S2064, boundary enhancement feature fusion output: The fused feature map F fusion As the final boundary enhancement feature fusion output, these feature maps combine boundary enhancement information and can better describe the target boundaries and details in the image.

[0135] F output =F fusion

[0136] By introducing a boundary enhancement mechanism, boundary-enhanced feature fusion (BEFF) can highlight the boundary information in the image, improve the effect of multi-scale feature fusion, and enhance the accuracy and robustness of target detection. This method is particularly suitable for processing complex scenes where boundary information has an important impact on target recognition.

[0137] Step S207, recursively aggregate features at multiple levels to improve detection performance. The core idea of ​​lightweight recursive feature aggregation (LRFA) is to recursively aggregate features at multiple levels during feature extraction to improve detection performance. In target detection tasks, features at different levels have different spatial resolutions and semantic information. LRFA gradually integrates low-level detail information and high-level semantic information through recursive aggregation to achieve better detection results. LRFA recursively aggregates features at different levels and combines it with a lightweight design to reduce computational overhead while improving the expressiveness of features. This recursive feature aggregation method enables the model to integrate more information at each level, thereby achieving better results in the detection process.

[0138] Step S2071, initial feature extraction: extract feature maps from each layer of the convolutional neural network, assuming that the feature map extracted from the lth layer is F l , whose dimensions are:

[0139]

[0140] Among them, H l and W l Represent the height and width of the feature map, C l Indicates the number of channels.

[0141] Step S2072, recursive feature aggregation: Recursive feature aggregation is to merge low-level features with high-level features layer by layer. The core of recursive feature aggregation is to gradually merge features from different levels through multiple iterations.

[0142] For the feature map F of the lth layer l , first perform a preliminary aggregation operation and fuse it with the feature map of the previous layer (layer l-1). The fusion operation can be achieved by element-by-element addition or weighted addition:

[0143] F agg (l) = α × F l +β×Resize(F agg (l-1))

[0144] Among them, F agg (l) represents the aggregated feature map of the lth layer, α and β are weighting coefficients, which are used to adjust the contribution of features at different levels; Resize means resizing the feature map to match the size of feature maps at different levels. In a recursive way, the feature map of each layer is fused with the aggregated feature map of the previous layer to form the final recursive feature aggregation result.

[0145] Step S2073, lightweight design: In order to reduce computational overhead, LRFA adopts lightweight design, such as using depthwise separable convolution to replace traditional convolution, thereby reducing computational complexity. Depthwise separable convolution decomposes the standard convolution into two parts: depthwise convolution and point convolution: Depthwise convolution: Perform convolution operation on each channel independently. Point convolution: Fusion of information from different channels through 1×1 convolution. This lightweight convolution operation can significantly reduce computational cost while maintaining good feature expression capabilities.

[0146] Final output: feature map F after recursive aggregation agg(l) will be used as the final detection feature map for subsequent detection tasks. Through lightweight recursive feature aggregation (LRFA), the effective fusion of features at different levels can be achieved, and the accuracy of target detection can be improved while ensuring the lightweight of the model.

[0147] Preferably, on the basis of recursive feature aggregation, a multi-scale feature enhancement module is further introduced to improve the recognition ability of the target by jointly modeling features of different scales. Specifically, multiple parallel convolution paths can be introduced to process feature maps of different scales respectively and fuse them at the end. In this way, the detection ability of small and large targets can be enhanced while keeping the model complexity low.

[0148] In addition, a dynamic weight allocation mechanism is introduced in the feature fusion process to adaptively adjust the fusion weight according to the characteristics of the input feature map (such as edge information, texture information, etc.). This method can avoid the limitations of fixed weight allocation and better fuse feature information at different levels. Fusion weights can be dynamically allocated by introducing attention mechanisms (such as channel attention and spatial attention). In addition, combined with self-supervised learning, the feature extraction network is pre-trained in an unsupervised manner, and then the pre-trained feature extraction network is used for target detection tasks. This method can effectively utilize a large amount of unlabeled data, thereby improving the expressiveness of features and the generalization ability of the model. Specifically, feature pre-training can be achieved through self-supervised methods such as contrastive learning.

[0149] In the above steps, this embodiment significantly improves the model's ability to detect small targets and targets in complex backgrounds through a combination of adaptive feature sampling convolution and boundary enhancement feature fusion, thereby improving detection accuracy. The lightweight multi-channel feature fusion and recursive feature aggregation modules greatly reduce the computational overhead, reduce the number of model parameters and the amount of calculation, making this technical solution suitable for embedded and mobile devices. Through the feature pyramid based on regional attention, the model can better capture the boundary information of the target in different scenarios, improve the detection ability of different types of targets, and thus improve the generalization of the model. The boundary enhancement feature fusion module can better distinguish the target boundary and background, reduce missed detection and false detection, and improve the target positioning accuracy of the model, especially in complex environments.

[0150] Figure 3 is a structural diagram of a model training device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0151] The acquisition module 31 is used to acquire historical images, wherein the historical images include: a target object wearing a target accessory.

[0152] The adjustment module 32 is used to adjust the positions and weights of the sampling points in the historical image, and perform feature extraction on the historical image according to the adjustment result to obtain a first target feature map.

[0153] The first fusion module 33 is used to perform multi-channel feature fusion on the first target feature map to obtain second target feature maps at different levels.

[0154] The second fusion module 34 is used to perform feature fusion processing on the second target feature maps at different levels by using the regional attention mechanism to obtain a third target feature map.

[0155] The training module 35 is used to train the deep learning model using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

[0156] Optionally, the adjustment module 32 is also used to perform the following steps: adaptively adjust the positions of the sampling points according to the local features of the historical image; assign different weights to the sampling points at different positions according to the adaptive weight function; and extract features of the historical image according to the positions of the adjusted sampling points and the weights corresponding to the sampling points at different positions to obtain a first target feature map.

[0157] Optionally, the adjustment module 32 is also used to perform the following steps: inputting the local feature information of the sampling point at the target position into the neural network to obtain the horizontal offset and the vertical offset of the sampling point at the target position output by the neural network; adjusting the position of the sampling point according to the horizontal offset and the vertical offset of the sampling point at the target position.

[0158] Optionally, the first fusion module 33 is also used to perform the following steps: perform dimensionality reduction processing on the first target feature map to obtain a first feature map, wherein the first target feature map includes multiple channels; perform weighted fusion on the first feature map to obtain a second feature map; perform nonlinear activation and regularization processing on the second feature map to obtain a third feature map; perform feature processing on the third feature map using a 1×1 convolution kernel or a depth-separable convolution kernel to obtain a second target feature map.

[0159] Optionally, the second fusion module 34 is also used to perform the following steps: calculating the attention weight of each area in the second target feature map of the i-th layer according to the importance coefficient of each area in the second target feature map of the i-th layer, where i is a positive integer not greater than n, and n is the number of levels of the second target feature map; performing weighted summation on each area in the second target feature map of the i-th layer according to the attention weight of each area in the second target feature map of the i-th layer to obtain the fourth feature map of the i-th layer; and fusing the fourth feature map of each layer to obtain the third target feature map.

[0160] Optionally, the model training device is also used to perform the following steps after fusing the fourth feature map of each layer to obtain the third target feature map: perform edge detection and boundary enhancement on the second target feature map of the jth layer to obtain the fifth feature map of the jth layer, where j is a positive integer not greater than n; and perform weighted fusion processing on the fifth feature map of each layer.

[0161] Optionally, the model training device is also used to perform feature fusion processing on second target feature maps of different levels using the regional attention mechanism to obtain a third target feature map, and then execute the following steps: fuse the third target feature map of the kth layer with the third target feature map of the k-1th layer to obtain a first aggregation result, wherein u is a positive integer not greater than m, and m is the number of levels of the third target feature map; perform independent convolution operations on the features of each channel in the first aggregation result, and fuse the features of different channels through a 1×1 convolution kernel.

[0162] It should be noted that the above Figure 3 The modules in the embodiment may be program modules (e.g., a set of program instructions for implementing a specific function) or hardware modules. For the latter, they may be expressed in the following forms, but are not limited thereto: the expression form of the above modules is a processor, or the functions of the above modules are implemented by a processor.

[0163] It should be noted that Figure 3 The preferred implementation of the illustrated embodiment can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0164] Figure 4 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a model training method. Figure 4 As shown, the computer terminal 40 may include one or more (402a, 402b, ..., 402n are used to illustrate) processors 402 (the processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 404 for storing data, and a transmission module 406 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4Different configurations are shown.

[0165] It should be noted that the one or more processors 402 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 40. As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0166] The memory 404 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the model training method in the embodiment of the present application. The processor 402 executes various functional applications and data processing by running the software programs and modules stored in the memory 404, that is, the above-mentioned model training method is realized. The memory 404 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 404 may further include a memory remotely arranged relative to the processor 402, and these remote memories may be connected to the computer terminal 40 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The transmission module 406 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 40. In one example, the transmission module 406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0168] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 40 .

[0169] It should be noted that, in some optional embodiments, the above Figure 4 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 4This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0170] It should be noted that Figure 4 The computer terminal shown is used to execute Figure 1 The model training method shown, therefore the relevant explanations in the execution method of the above commands are also applicable to the electronic device and will not be repeated here.

[0171] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above model training method.

[0172] A program for a non-volatile storage medium to perform the following functions: acquiring historical images, wherein the historical images include: a target object wearing a target accessory; adjusting the positions and weights of sampling points in the historical images, and extracting features from the historical images based on the adjustment results to obtain a first target feature map; performing multi-channel feature fusion on the first target feature map to obtain second target feature maps of different levels; performing feature fusion processing on the second target feature maps of different levels using a regional attention mechanism to obtain a third target feature map; and training a deep learning model using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to recognize whether the target object is wearing the target accessory.

[0173] An embodiment of the present application also provides an electronic device, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above model training method is executed when the program is running.

[0174] The processor is used to run a program that performs the following functions: obtaining historical images, wherein the historical images include: a target object wearing a target accessory; adjusting the position and weight of a sampling point in the historical image, and performing feature extraction on the historical image based on the adjustment result to obtain a first target feature map; performing multi-channel feature fusion on the first target feature map to obtain second target feature maps of different levels; performing feature fusion processing on the second target feature maps of different levels using a regional attention mechanism to obtain a third target feature map; and training a deep learning model using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

[0175] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0176] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0177] In the above-mentioned embodiments of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary protection measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0182] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a historical image, wherein the historical image includes: a target object wearing a target accessory; Adjusting the positions and weights of the sampling points in the historical image, and performing feature extraction on the historical image according to the adjustment result to obtain a first target feature map; Performing multi-channel feature fusion on the first target feature map to obtain second target feature maps at different levels; Using a regional attention mechanism to perform feature fusion processing on the second target feature maps at different levels to obtain a third target feature map; The deep learning model is trained using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

2. The method according to claim 1, characterized in that Adjusting the positions and weights of the sampling points in the historical image, and performing feature extraction on the historical image according to the adjustment result to obtain a first target feature map, including: Adaptively adjust the position of sampling points according to the local features of historical images; According to the adaptive weight function, different weights are assigned to sampling points at different locations: According to the adjusted positions of the sampling points and the weights corresponding to the sampling points at different positions, feature extraction is performed on the historical image to obtain the first target feature map.

3. The method according to claim 1, characterized in that Adaptively adjust the position of the sampling points according to the local features of the historical image, including: Inputting local feature information of the sampling point at the target position into a neural network, and obtaining a horizontal offset and a vertical offset of the sampling point at the target position output by the neural network; The position of the sampling point is adjusted according to the horizontal offset and the vertical offset of the sampling point at the target position.

4. The method according to claim 1, characterized in that: Performing multi-channel feature fusion on the first target feature map to obtain second target feature maps at different levels, including: Performing dimensionality reduction processing on the first target feature map to obtain a first feature map, wherein the first target feature map includes a plurality of channels; Performing weighted fusion on the first feature map to obtain a second feature map; Performing nonlinear activation and regularization processing on the second feature map to obtain a third feature map; The third feature map is feature processed by using a 1×1 convolution kernel or a depth-separable convolution kernel to obtain the second target feature map.

5. The method according to claim 1, characterized in that: The regional attention mechanism is used to perform feature fusion processing on the second target feature maps at different levels to obtain a third target feature map, including: Calculate the attention weight of each region in the second target feature map of the i-th layer according to the importance coefficient of each region in the second target feature map of the i-th layer, where i is a positive integer not greater than n, and n is the number of layers of the second target feature map; According to the attention weight of each area in the second target feature map of the i-th layer, weighted sum is performed on each area in the second target feature map of the i-th layer to obtain a fourth feature map of the i-th layer; The fourth feature map of each layer is fused to obtain a third target feature map.

6. The method according to claim 5, characterized in that After fusing the fourth feature map of each layer to obtain the third target feature map, the method further includes: performing edge detection and boundary enhancement processing on the second target feature map of the jth layer to obtain the fifth feature map of the jth layer, where j is a positive integer not greater than n; and performing weighted fusion processing on the fifth feature map of each layer.

7. The method according to claim 1, characterized in that After performing feature fusion processing on the second target feature maps at different levels by using the regional attention mechanism to obtain the third target feature map, the method further includes: The third target feature map of the kth layer is merged with the third target feature map of the k-1th layer to obtain a first aggregation result, wherein u is a positive integer not greater than m, and m is the number of layers of the third target feature map; An independent convolution operation is performed on the features of each channel in the first aggregation result, and the features of different channels are fused through a 1×1 convolution kernel.

8. A method for identifying a target accessory, characterized in that: include: Acquire an image to be identified, wherein the image to be identified includes: a target object; A target accessory recognition model is used to identify whether the target object in the image to be recognized is wearing a target accessory, wherein the target accessory recognition model is trained by the model training method described in any one of claims 1 to 7.

9. A model training device, characterized in that: include: An acquisition module, used for acquiring a historical image, wherein the historical image includes: a target object wearing a target accessory; An adjustment module, used for adjusting the positions and weights of the sampling points in the historical image, and performing feature extraction on the historical image according to the adjustment result to obtain a first target feature map; A first fusion module is used to perform multi-channel feature fusion on the first target feature map to obtain second target feature maps at different levels; A second fusion module is used to perform feature fusion processing on the second target feature maps at different levels by using a regional attention mechanism to obtain a third target feature map; A training module is used to train a deep learning model using the third target feature map to obtain a target accessory recognition model, wherein the target accessory recognition model is used to identify whether the target object is wearing the target accessory.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the model training method described in any one of claims 1 to 7.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the model training method described in any one of claims 1 to 7 when running.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the model training method described in any one of claims 1 to 7 is implemented.

Images