Multi-target detection method and device based on image recognition

Abstract

The application discloses a multi-target detection method and device based on image recognition, and relates to the technical field of target detection. The method comprises the following steps: constructing a multi-target detection network model based on an image recognition algorithm, wherein the multi-target detection network model comprises a backbone feature extraction network, an enhanced feature fusion neck network and a decoupling detection head; collecting real-time original images to be detected of a factory, and pre-processing the real-time original images to obtain real-time standard images after preprocessing; inputting the real-time standard images after preprocessing into the multi-target detection network model to perform multi-target detection, and obtaining real-time multi-target detection results of the factory. The application solves the problems of poor feature fusion effect, difficult hyperparameter optimization and complex background interference in the prior art.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a multi-target detection method and apparatus based on image recognition. Background Technology

[0002] With the development of Industry 4.0 and smart manufacturing, the demand for safe production and automated monitoring in factories is increasing. Image recognition-based multi-target detection technology is widely used in factories for scenarios such as worker helmet detection, violation identification, equipment status monitoring, and personnel counting. However, factory environments are usually complex, with problems such as large changes in lighting, numerous background clutter, severe target occlusion, and the coexistence of multi-scale targets, which poses a great challenge to the accuracy and robustness of detection algorithms.

[0003] Existing multi-object detection methods typically employ general-purpose deep learning models (such as the YOLO series, Faster R-CNN, etc.), but they suffer from the following problems in practical applications: 1) Poor feature fusion performance: Traditional feature pyramid networks often use simple addition or splicing when fusing deep semantic features and shallow detail features, lacking targeted guidance, resulting in insufficient detection capabilities for small targets and occluded targets.

[0004] 2) Difficulty in hyperparameter tuning: The performance of the model (such as the initial learning rate, loss weights, non-maximum suppression threshold, etc.) heavily depends on human experience for adjustment, which is time-consuming, laborious and difficult to achieve the global optimum, making it difficult for the model to achieve a balance between accuracy and speed.

[0005] 3) Complex background interference: Complex machinery and textures in the factory background are easily misidentified as targets, resulting in a high false alarm rate.

[0006] Therefore, there is an urgent need for a high-precision multi-target detection method that can adapt to complex factory environments and has adaptive optimization capabilities. Summary of the Invention

[0007] This invention provides a multi-target detection method and apparatus based on image recognition. This invention solves the problems of poor feature fusion, difficulty in hyperparameter tuning, and interference from complex backgrounds in existing technologies.

[0008] In a first aspect, embodiments of the present invention provide a multi-target detection method based on image recognition, the method comprising: Based on image recognition algorithms, a multi-target detection network model is constructed, which includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. Collect real-time raw images of the factory to be inspected, and preprocess the real-time raw images to obtain preprocessed real-time standard images; The preprocessed real-time standard image is input into the multi-object detection network model to perform multi-object detection, and the real-time multi-object detection results of the factory are obtained.

[0009] The technical solution provided in this application has at least the following beneficial effects: An enhanced feature fusion neck network based on a bidirectional guided feature fusion mechanism is adopted. A top-down path utilizes deep semantic features to semantically enhance shallow features, while a bottom-up path utilizes shallow texture features to compensate for spatial details in deep features. This cross-scale interactive weighting mechanism provides targeted guidance for the feature fusion process, simultaneously strengthening both semantic information and edge details. This significantly improves the network's feature representation ability for multi-scale targets, particularly greatly improving the feature extraction quality for small and occluded targets, thereby enhancing the detection accuracy for difficult targets. A multi-objective simulated annealing algorithm is introduced to automatically optimize model hyperparameters. This not only quickly escapes local optima but also maintains refined local exploitation capabilities in the later stages of the search, thus automatically finding a suitable equilibrium. The model achieves a globally optimal combination of hyperparameters that balances detection accuracy and inference speed, overcoming the shortcomings of existing technologies that heavily rely on human experience, are time-consuming and labor-intensive, and are difficult to optimize. It achieves optimal inference efficiency while ensuring high model accuracy. A dynamic focusing attention mechanism is embedded in the feature fusion neck network. This mechanism dynamically generates attention weight maps based on the local regional variance of the input features, which can automatically identify complex texture regions in the image. By suppressing smooth background regions (such as the sky and flat walls) and enhancing complex foreground regions (such as target edges), it can effectively filter out noise interference from complex machinery and background textures in the factory environment. This dynamic feature focusing processing enhances the contrast between the target and the background, significantly reduces the false alarm rate of the background, and improves the robustness of the model in complex factory environments.

[0010] In one alternative implementation, a multi-object detection network model is constructed based on an image recognition algorithm, including: Collect several historical raw images of the factory to be inspected, and preprocess the historical raw images to construct a training dataset; An initial multi-target detection network model is constructed using image recognition algorithms. The multi-target detection network model includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. A bidirectional guided feature fusion mechanism and a dynamic focusing attention mechanism are embedded in the enhanced feature fusion neck network, and an adaptive hard sample mining loss function is constructed. Using the training dataset, the initial multi-object detection network model, which is equipped with a bidirectional guided feature fusion mechanism, a dynamic focus attention mechanism, and an adaptive hard sample mining loss function, is iteratively trained to obtain the final multi-object detection network model.

[0011] In one optional implementation, several historical raw images of the factory to be detected are collected, and the historical raw images are preprocessed to construct a training dataset, including: Using surveillance cameras and industrial cameras, several historical raw images of the factory containing the target to be detected are collected, and the data of these historical raw images is cleaned to obtain several cleaned historical images. Image annotation is performed on several cleaned historical images to generate several annotated historical images containing target categories and bounding box coordinates; Data augmentation and preprocessing are performed on several labeled historical images to obtain a training dataset containing several preprocessed historical images.

[0012] In one alternative implementation, the enhanced feature fusion neck network is equipped with a semantic enhancement module, a spatial detail compensation module, and a feature fusion module built on a bidirectional guided feature fusion mechanism, as well as a feature focusing processing module built on a dynamic focusing attention mechanism.

[0013] In one alternative implementation, the formula for the adaptive hard sample mining loss function is:

[0014] In the formula, The loss value for adaptive hard sample mining is the total loss value; This is the classification loss value; This represents the regression loss value; This represents the confidence loss value. For dynamic modulation factor; For loss weights; j For sample indication; J The total number of samples;

[0015] In the formula, These are modulation parameters; It is a natural constant; This represents the IoU value between the predicted bounding box and the ground truth bounding box.

[0016] In one alternative implementation, the training dataset is used to iteratively train an initial multi-object detection network model equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function to obtain a final multi-object detection network model, including: The hyperparameters of an initial multi-object detection network model with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function are optimized using a multi-object simulated annealing algorithm to obtain the optimal hyperparameters. Based on the optimal hyperparameters, the hyperparameters of the initial multi-object detection network model are set to obtain the optimized multi-object detection network model. The training dataset is input into the optimized multi-object detection network model for iterative training. An adaptive hard sample mining loss function is used to calculate the total loss value for each training iteration until the total loss value is less than the total loss threshold, thus obtaining the final multi-object detection network model.

[0017] In one alternative implementation, a multi-objective simulated annealing algorithm is used to optimize the hyperparameters of an initial multi-objective detection network model equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function, to obtain the optimal hyperparameters, including: The hyperparameters of the initial multi-object detection network model, which is equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function, are encoded into the state vector of the multi-object simulated annealing algorithm. A multi-objective evaluation function is constructed and used as the energy function of the multi-objective simulated annealing algorithm. The chaotic sequence is generated using the Logistic mapping, and then mapped to the solution space of the hyperparameters to obtain the initial state of the multi-objective simulated annealing algorithm. Based on the candidate hyperparameters corresponding to the initial state, candidate multi-target detection network models are constructed, and inference is run on the validation set. The corresponding energy value is calculated using the energy function. Generate a new state with random perturbation in the neighborhood of the initial state, and run inference on the validation set to calculate the energy value corresponding to the new state using the energy function; Calculate the energy difference between the new state and the initial state, and update the initial state based on the energy difference to obtain the updated state; Repeat the state update steps until the number of updates reaches the update number threshold or the energy value of the state increases less than the preset increase threshold after several consecutive updates. The state of the last update is then taken as the optimal state. Decoding the state vector of the optimal state yields the optimal hyperparameters of the multi-object detection network model.

[0018] In one alternative implementation, the formula for the multi-objective evaluation function is:

[0019] In the formula, For state The corresponding multi-target evaluation value, i.e., energy value, of the multi-target detection network model; For state The accuracy value of the corresponding multi-target detection network model; For state The corresponding speed value of the multi-target detection network model; Weights for multi-objective evaluation;

[0020] In the formula, For state The mean accuracy of the corresponding multi-target detection network model calculated on the validation set;

[0021] In the formula, For state The number of frames transmitted per second for the corresponding multi-target detection network model; It is a local minimum.

[0022] In one optional implementation, the preprocessed real-time standard image is input into a multi-object detection network model for multi-object detection, yielding real-time multi-object detection results for the factory, including: The preprocessed real-time standard image is converted into an input tensor, and the input tensor is then fed into the multi-object detection network model. The backbone feature extraction network of the multi-object detection network model is used to extract low-level, mid-level, and high-level features of the input tensor. The semantic enhancement module in the neck network of the enhanced feature fusion of the multi-object detection network model is used to semantically enhance the mid-level and high-level features to obtain semantic features; The spatial detail compensation module in the neck network of the enhanced feature fusion of the multi-object detection network model is used to perform spatial detail compensation on semantic features, low-level features and high-level features to obtain compensated deep features. The feature fusion module in the neck network of the enhanced feature fusion multi-object detection network model is used to fuse semantic features and compensated deep features to obtain a fused feature map. The enhanced feature fusion neck network of the multi-object detection network model uses a feature focusing processing module to generate an attention weight map of the fused features, and then dynamically focuses it with the fused feature map to obtain a dynamically focused feature map. Using the decoupled detection head of the multi-object detection network model, prediction is made based on the dynamic focusing feature map to obtain the initial real-time multi-object detection results, including the probability vector of each preset box belonging to each category, the bounding box regression offset of each preset box, and the corresponding confidence. The initial real-time multi-target detection results are sequentially post-processed, non-maximum suppression is performed, and coordinate mapping is applied to obtain the final real-time multi-target detection results for the factory.

[0023] Secondly, embodiments of the present invention provide a multi-target detection device based on image recognition for implementing a multi-target detection method. The device includes: The model building unit is used to build a multi-target detection network model based on image recognition algorithms. The multi-target detection network model includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. The image acquisition unit is used to respond to the data fusion request from the application. It uses cross-source semantic mapping technology and ontology reasoning tree to perform semantic alignment on the multi-source heterogeneous data, identify and resolve deep semantic conflicts, and obtain standard data. The multi-target detection unit is used to input the preprocessed real-time standard image into the multi-target detection network model to perform multi-target detection and obtain the real-time multi-target detection results of the factory.

[0024] A third aspect of this invention provides an electronic device, which includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, such that the at least one processor can perform the method proposed in the first aspect of the present invention.

[0025] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first aspect of the present invention. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the steps of a multi-target detection method based on image recognition provided in an embodiment of the present invention; Figure 3 This is a functional unit diagram of a multi-target detection device based on image recognition provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0028] The present invention will be further described below with reference to the accompanying drawings.

[0029] Reference Figure 1 , Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0030] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0031] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0032] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and an electronic program for a multi-target detection device based on image recognition.

[0033] exist Figure 1In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device. The electronic device calls the electronic program of the multi-target detection device based on image recognition stored in the memory 1005 through the processor 1001 and executes the multi-target detection method based on image recognition provided in the embodiment of the present invention.

[0034] Reference Figure 2 The present invention provides a multi-target detection method based on image recognition, the method comprising: S201: Based on image recognition algorithms, construct a multi-target detection network model, which includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head; S202: Acquire the real-time raw image of the factory to be inspected, and preprocess the real-time raw image to obtain the preprocessed real-time standard image; S203: Input the preprocessed real-time standard image into the multi-target detection network model to perform multi-target detection and obtain the real-time multi-target detection results of the factory.

[0035] The technical solution provided in this application has at least the following beneficial effects: An enhanced feature fusion neck network based on a bidirectional guided feature fusion mechanism is adopted. A top-down path utilizes deep semantic features to semantically enhance shallow features, while a bottom-up path utilizes shallow texture features to compensate for spatial details in deep features. This cross-scale interactive weighting mechanism provides targeted guidance for the feature fusion process, simultaneously strengthening both semantic information and edge details. This significantly improves the network's feature representation ability for multi-scale targets, particularly greatly improving the feature extraction quality for small and occluded targets, thereby enhancing the detection accuracy for difficult targets. A multi-objective simulated annealing algorithm is introduced to automatically optimize model hyperparameters. This not only quickly escapes local optima but also maintains refined local exploitation capabilities in the later stages of the search, thus automatically finding a suitable equilibrium. The model achieves a globally optimal combination of hyperparameters that balances detection accuracy and inference speed, overcoming the shortcomings of existing technologies that heavily rely on human experience, are time-consuming and labor-intensive, and are difficult to optimize. It achieves optimal inference efficiency while ensuring high model accuracy. A dynamic focusing attention mechanism is embedded in the feature fusion neck network. This mechanism dynamically generates attention weight maps based on the local regional variance of the input features, which can automatically identify complex texture regions in the image. By suppressing smooth background regions (such as the sky and flat walls) and enhancing complex foreground regions (such as target edges), it can effectively filter out noise interference from complex machinery and background textures in the factory environment. This dynamic feature focusing processing enhances the contrast between the target and the background, significantly reduces the false alarm rate of the background, and improves the robustness of the model in complex factory environments.

[0036] In one alternative implementation, a multi-object detection network model is constructed based on an image recognition algorithm, including: S2011: Collect several historical original images of the factory to be inspected, and preprocess the historical original images to construct a training dataset; S2012: Using an image recognition algorithm, an initial multi-target detection network model is constructed, which includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head; It is worth noting that the backbone feature extraction network selects the Cross Stage Partial Darknet (CSPDarknet) as the backbone network, and builds CSPrayer (cross stage partial layer) in sequence. Each CSPrayer contains multiple basic Conv modules (Conv2d + BatchNorm + SiLU activation function). Downsampling is performed using a 3×3 convolutional layer with a stride of 2 to gradually reduce the feature map size and increase the number of channels, extracting feature maps of three different scales: low-level features, mid-level features, and high-level features, which correspond to downsampling ratios of 8x, 16x, and 32x, respectively. The decoupling head structure is divided into two parallel branches: a classification branch and a regression branch; Classification branch: Contains two 3×3 convolutional layers, followed by a 1×1 convolutional layer to output class probabilities; Regression branch: contains two 3×3 convolutional layers, followed by a 1×1 convolutional layer to output the coordinate offset of the bounding box; This decoupling design eliminates the conflict between the classification task and the localization task, thus accelerating the convergence speed. S2013: Embed a bidirectional guided feature fusion mechanism and a dynamic focusing attention mechanism in the enhanced feature fusion neck network, and construct an adaptive hard sample mining loss function; S2014: Using the training dataset, iteratively train the initial multi-object detection network model, which is equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function, to obtain the final multi-object detection network model.

[0037] In one optional implementation, several historical raw images of the factory to be detected are collected, and the historical raw images are preprocessed to construct a training dataset, including: S20111: Using surveillance cameras and industrial cameras, collect several historical original images of the factory containing the target to be detected, and clean the data of several historical original images to remove images with excessive blur (below the threshold based on the variance evaluation of the Laplacian operator), completely occluded or meaningless blank images, and obtain several cleaned historical images. S20112: Perform image annotation on several cleaned historical images to generate several annotated historical images containing target categories and bounding box coordinates; S20113: Perform data augmentation and preprocessing on several labeled historical images to obtain a training dataset containing several preprocessed historical images; In this embodiment, data augmentation and preprocessing are performed as follows: Geometric transformation: Randomly rotating the training image ( 15 Up to +15 Random scaling (scaling ratio 0.8 to 1.2), horizontal flipping (probability 0.5), and translation operations are used to increase the geometric invariance of the model; Mosaic Augmentation: The Mosaic data augmentation strategy is adopted. Four training images are randomly selected, scaled, cropped and stitched together to form a new image. This operation enriches the background detection context and effectively improves the detection performance of small targets without increasing the inference computation. Color jittering: Randomly adjusts the brightness, contrast, saturation, and hue of an image to simulate different lighting environments; Normalization: Normalizing pixel values ​​from 0 to 1. 255 linearly scaled to 0 The standard tensor of the model input is obtained by subtracting the mean of the ImageNet dataset (e.g., [0.485, 0.456, 0.406]) from the interval and dividing by the standard deviation (e.g., [0.229, 0.224, 0.225]).

[0038] In one alternative implementation, the enhanced feature fusion neck network is equipped with a semantic enhancement module, a spatial detail compensation module, and a feature fusion module built on a bidirectional guided feature fusion mechanism, as well as a feature focusing processing module built on a dynamic focusing attention mechanism.

[0039] In one alternative implementation, the formula for the adaptive hard sample mining loss function is:

[0040] In the formula, The loss value for adaptive hard sample mining is the total loss value; Here, is the classification loss value, and is the binary cross-entropy loss; For the regression loss value, CIoU (Complete IoU) loss is used, which comprehensively considers the overlapping area, center point distance and aspect ratio; For the confidence loss value, use BCEWithLogitsLoss; It is a dynamic modulation factor used to measure the "difficulty" of a sample; For loss weights; j For sample indication; J The total number of samples;

[0041] In the formula, These are modulation parameters; It is a natural constant; The IoU value between the predicted bounding box and the ground truth bounding box, when the predicted bounding box is very accurate ( When →1), the factor ≈1, updated with normal weights; when the predicted bounding box is extremely inaccurate ( When the factor reaches 0 (i.e., difficult samples), the factor... It is growing exponentially.

[0042] In one alternative implementation, the training dataset is used to iteratively train an initial multi-object detection network model equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function to obtain a final multi-object detection network model, including: S20141: The hyperparameters of the initial multi-object detection network model with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function are optimized using the multi-object simulated annealing algorithm to obtain the optimal hyperparameters. S20142: Based on the optimal hyperparameters, set the hyperparameters of the initial multi-object detection network model to obtain the optimized multi-object detection network model; S20143: Input the training dataset into the optimized multi-object detection network model for iterative training, and use the adaptive hard sample mining loss function to calculate the total loss value for each training session until the total loss value is less than the total loss threshold, thus obtaining the final multi-object detection network model.

[0043] In one alternative implementation, a multi-objective simulated annealing algorithm is used to optimize the hyperparameters of an initial multi-objective detection network model equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function, to obtain the optimal hyperparameters, including: S201411: The hyperparameters of the initial multi-object detection network model, which is equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function, are encoded into the state vector of the multi-object simulated annealing algorithm. The hyperparameters include the initial learning rate, the IoU threshold of non-maximum suppression, and the loss weight of the adaptive hard sample mining loss function. Initial learning rate, search range [ e 5 , e 2 ]; IoU threshold, search range [0.3, 0.7]; Loss weights of the adaptive hard sample mining loss function , are the weight coefficients of classification loss, regression loss and confidence loss respectively, with a search range of [0.1, 10.0][0.1, 10.0]; S201412: Construct a multi-objective evaluation function and use the multi-objective evaluation function as the energy function of the multi-objective simulated annealing algorithm; S201413: Use Logistic mapping to generate chaotic sequences and map the chaotic sequences to the solution space of hyperparameters to obtain the initial state of the multi-objective simulated annealing algorithm; The formula is:

[0044] In the formula, For the first n+ 1. n One chaotic variable; The stability coefficient is typically 4. This sequence is ergodic and random, ensuring that the initial population is uniformly distributed in the solution space, avoiding getting trapped in local optima, which is superior to traditional random initialization. n For chaotic variable indicators; S201414: Based on the candidate hyperparameters corresponding to the initial state, construct candidate multi-target detection network models, run inference on the validation set, and use the energy function to calculate the corresponding energy value; S201415: Generate a new state with random perturbation in the neighborhood of the initial state, and run inference on the validation set to calculate the energy value corresponding to the new state using the energy function; The formula is:

[0045] In the formula, For the new state; Given the current state, initially update to the initial state; for Levy Distributed random numbers; for Levy Step length, and ∈[1,2]; For the first t The convergence factor of the next update; t The number of times an update is performed; This is the step size scaling constant; The symbol for element-wise multiplication of vectors; It is a random perturbation vector;

[0046] In the formula, These are the maximum and minimum values ​​of the convergence factor; This represents the maximum number of iterations. , To adjust the parameters; It is the hyperbolic tangent function; S201416: Calculate the energy difference between the new state and the initial state, and update the initial state based on the energy difference to obtain the updated state; In this embodiment, the formula for the energy difference is:

[0047] In the formula, Energy difference; The energy values ​​for the new state and the current state; This is the energy function, i.e., the multi-objective evaluation function; S201417: Repeat the state update steps until the number of updates reaches the update number threshold or the energy value of the state after several consecutive updates is less than the preset improvement threshold. The state of the last update is taken as the optimal state. S201418: Decode the state vector of the optimal state to obtain the optimal hyperparameters of the multi-object detection network model.

[0048] In one alternative implementation, the formula for the multi-objective evaluation function is:

[0049] In the formula, For state The corresponding multi-target evaluation value, i.e., energy value, of the multi-target detection network model; For state The accuracy value of the corresponding multi-target detection network model; For state The corresponding speed value of the multi-target detection network model; Weights for multi-objective evaluation;

[0050] In the formula, For state The mean accuracy of the corresponding multi-target detection network model calculated on the validation set;

[0051] In the formula, For state The number of frames transmitted per second for the corresponding multi-target detection network model; It is a local minimum.

[0052] In one optional implementation, a real-time raw image of the factory to be inspected is acquired, and the real-time raw image is preprocessed to obtain a preprocessed real-time standard image, including: S2021: Connect to the factory's image acquisition equipment to perform video stream decoding and frame extraction to obtain the real-time raw image to be detected; In this embodiment, the device is accessed via RTSP (Real Time Streaming Protocol), ONVIF protocol, or USB direct connection to the image acquisition device of the surveillance camera or industrial camera. Streaming media capture: Use video capture tools (such as OpenCV's VideoCapture class) to establish a video stream reading channel and acquire image sequences in real time; Frame rate control: Set the frame extraction interval according to the actual computing power requirements; for example, extract 1 frame every 3 frames for processing to reduce the computing load, or process each frame directly to ensure high real-time performance. Format conversion: The acquired raw image (usually in YUV or BGR format) is decoded and converted into RGB color space to obtain the raw image matrix, which is the real-time raw image to be detected; S2022: Preprocess the real-time raw image to obtain a preprocessed real-time standard image; In this embodiment, image quality preprocessing: Histogram equalization: To address uneven lighting conditions (such as backlighting or nighttime), contrast-limited adaptive histogram equalization is applied to the brightness channel (V channel or grayscale image) of the real-time raw image to enhance the local contrast of the image and make the texture of targets in dark areas clearer. Denoising: Gaussian blur or bilateral filtering is applied to the image to remove high-frequency noise generated by the sensor while preserving target edge information; Distortion correction: If the acquisition device is a wide-angle or fisheye lens, distortion correction needs to be performed on the real-time raw image according to the preset camera intrinsic parameter matrix and distortion coefficients to restore the true geometric shape of the target; Size normalization and padding: Proportional scaling: Calculate the scaling ratio of the length and width of the real-time original image to the model input size, keep the aspect ratio unchanged, and scale the long side of the image to match the original size; Edge padding: Calculate the pixel difference in another dimension of the scaled image, and use a fixed color value (such as gray (114) or black (0)) to pad the right or bottom edge of the image to generate an image with a strictly square size; Data normalization: Numerical normalization: Normalizing the pixel values ​​of a square image from 0... The integer range of 255 is linearly mapped to 0.0. The floating-point range is 1.0; Standardization: Subtract the mean of the ImageNet dataset and divide by the standard deviation to obtain the preprocessed real-time standard image.

[0053] In one optional implementation, the preprocessed real-time standard image is input into a multi-object detection network model for multi-object detection, yielding real-time multi-object detection results for the factory, including: S2031: Convert the preprocessed real-time standard image into an input tensor, and input the input tensor into the multi-object detection network model; In this embodiment, the dimension transformation is performed by converting the dimension of the image data from (H,W,C) to (C,H,W) format and adding a batch dimension, resulting in an input tensor with a shape of (1,3,640,640). S2032: Uses the backbone feature extraction network of a multi-object detection network model to extract low-level, mid-level, and high-level features of the input tensor; S2033: Using the semantic enhancement module in the neck network of the enhanced feature fusion of the multi-object detection network model, semantic enhancement is performed on the mid-level and high-level features to obtain semantic features; It is worth noting that the semantic enhancement module upsamples high-level features (such as 80×80) using nearest neighbor interpolation, making their size consistent with that of mid-level features (such as 160×160). Perform a Concat (channel concatenation) operation between the upsampled features and the mid-layer features; Semantic weighting: A lightweight 1×1 convolutional layer is introduced to process the concatenation result and generate a semantic weight map, which is then normalized to [0,1] by the Sigmoid activation function; By using a normalized semantic weight map to weight the feature channels, the semantic features that play a key role in target judgment are strengthened. S2034: The spatial detail compensation module in the neck network of the enhanced feature fusion of the multi-object detection network model is used to perform spatial detail compensation on semantic features, low-level features and high-level features to obtain compensated deep features. It is worth noting that the semantic features are downsampled and then fused with the high-level features to obtain the fused high-level features. Edge information is extracted from shallow features, a detail weight map is generated through 3×3 convolution, and the detail weight map is injected into deep features to perform spatial detail compensation on the fused high-level features. This compensates for the edge and texture details lost by the deep features after multiple downsampling, resulting in compensated deep features. S2035: The feature fusion module in the enhanced feature fusion neck network of the multi-object detection network model performs feature fusion on semantic features and compensated deep features to obtain a fused feature map; S2036: The enhanced feature fusion neck network of the multi-object detection network model uses the feature focusing processing module to generate an attention weight map of the fused features, and dynamically focuses it with the fused feature map to obtain a dynamically focused feature map; It is worth noting that the feature focusing processing module divides the fused feature map into several local blocks and calculates the variance of the pixel values ​​within each local block. The larger the variance, the more complex the texture of the region (which may be the edge of a target or a complex background); the smaller the variance, the smoother the region (which may be the sky or a flat road). The calculated local variance map is used as the initial attention map, and linear interpolation is performed to restore it to the original feature map size. The restored variance map is concatenated with the original feature map in the channel dimension, and then activated by 1×1 convolution (dimensionality reduction), BatchNorm and SiLU in sequence. Then, a 1×1 convolution is used to generate the channel attention weight vector, and finally the Sigmoid function is used to generate the final attention weight map. The dynamic attention weight map is multiplied element-wise with the fused feature map to obtain the output feature, namely the dynamic focus feature map. This operation enables the network to dynamically suppress high-frequency background noise (such as the high-frequency variance caused by swaying leaves) based on the local complexity of the image content, while focusing on regions with target features. S2037: Using the decoupled detection head of the multi-object detection network model, prediction is made based on the dynamic focusing feature map to obtain the initial real-time multi-object detection results, including the probability vector of each preset box belonging to each category, the bounding box regression offset of each preset box, and the corresponding confidence. S2038: Perform post-processing, non-maximum suppression, and coordinate mapping sequentially on the initial real-time multi-target detection results to obtain the final real-time multi-target detection results of the factory. In this embodiment, post-processing: Decoding operation: Based on the regression offset and the base coordinates of the preset bounding box, the actual coordinates of the predicted bounding box in the input image coordinate system are calculated using the decoding formula; Confidence filtering: Multiply the target confidence score by the classification probability to obtain the final class confidence score; Filter out all candidate boxes with a confidence level greater than a preset threshold (e.g., 0.25); Nonmaximum suppression: Sorting: Sort the candidate boxes in descending order of category confidence scores; Overlap calculation: Select the box with the highest score as the reference, and calculate the intersection-union ratio of the remaining boxes with the reference; Redundancy removal: If the intersection-union ratio is greater than the IoU threshold in the optimal hyperparameters, the bounding box is considered to be the same target as the baseline detection, and it is suppressed (deleted). Loop processing: Repeat the above process for the remaining unsuppressed boxes until all boxes have been processed; Coordinate mapping and result output: Coordinate inversion: Map the coordinates of the predicted bounding box after non-maximum suppression filtering from the model input size (640×640) back to the actual pixel coordinate system of the original image. The added edge padding needs to be deducted and divided by the scaling ratio. Final output: Output the final detection result list, that is, the final real-time multi-object detection results. Each detection item includes: object category ID, confidence score, and pixel coordinates in the original image.

[0054] This invention also provides a multi-target detection device based on image recognition, referring to... Figure 3 The device may include the following units: The model building unit 301 is used to build a multi-target detection network model based on the image recognition algorithm. The multi-target detection network model includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. The image acquisition unit 302 is used to respond to the data fusion request from the application end, and uses cross-source semantic mapping technology and ontology reasoning tree to perform semantic alignment on the multi-source heterogeneous data accessed, identify and resolve deep semantic conflicts, and obtain standard data. The multi-target detection unit 303 is used to input the preprocessed real-time standard image into the multi-target detection network model to perform multi-target detection and obtain the real-time multi-target detection results of the factory.

[0055] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; The processor, when executing the program stored in the memory, implements the multi-target detection method based on image recognition of the present invention.

[0056] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EI) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM), or non-volatile memory, such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0057] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0058] Furthermore, to achieve the above objectives, embodiments of the present invention also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image recognition-based multi-target detection method of the embodiments of the present invention.

[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable hardware devices (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0064] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-target detection method based on image recognition, characterized in that, The method includes: Based on image recognition algorithms, a multi-target detection network model is constructed, which includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. Collect real-time raw images of the factory to be inspected, and preprocess the real-time raw images to obtain preprocessed real-time standard images; The preprocessed real-time standard image is input into the multi-object detection network model to perform multi-object detection, and the real-time multi-object detection results of the factory are obtained.

2. The multi-target detection method based on image recognition according to claim 1, characterized in that, Based on image recognition algorithms, a multi-object detection network model is constructed, including: Collect several historical raw images of the factory to be inspected, and preprocess the historical raw images to construct a training dataset; An initial multi-target detection network model is constructed using image recognition algorithms. The multi-target detection network model includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. A bidirectional guided feature fusion mechanism and a dynamic focusing attention mechanism are embedded in the enhanced feature fusion neck network, and an adaptive hard sample mining loss function is constructed. Using the training dataset, the initial multi-object detection network model, which is equipped with a bidirectional guided feature fusion mechanism, a dynamic focus attention mechanism, and an adaptive hard sample mining loss function, is iteratively trained to obtain the final multi-object detection network model.

3. The multi-target detection method based on image recognition according to claim 2, characterized in that, Collect several historical raw images of the factory to be inspected, and preprocess the historical raw images to construct a training dataset, including: Using surveillance cameras and industrial cameras, several historical raw images of the factory containing the target to be detected are collected, and the data of these historical raw images is cleaned to obtain several cleaned historical images. Image annotation is performed on several cleaned historical images to generate several annotated historical images containing target categories and bounding box coordinates; Data augmentation and preprocessing are performed on several labeled historical images to obtain a training dataset containing several preprocessed historical images.

4. The multi-target detection method based on image recognition according to claim 3, characterized in that, The enhanced feature fusion neck network is equipped with a semantic enhancement module, a spatial detail compensation module, and a feature fusion module built on a bidirectional guided feature fusion mechanism, as well as a feature focusing processing module built on a dynamic focusing attention mechanism.

5. The multi-target detection method based on image recognition according to claim 4, characterized in that, The formula for the adaptive hard sample mining loss function is: In the formula, The loss value for adaptive hard sample mining is the total loss value; This is the classification loss value; This represents the regression loss value; This represents the confidence loss value. For dynamic modulation factor; For loss weights; j For sample indication; J The total number of samples; In the formula, These are modulation parameters; It is a natural constant; This represents the IoU value between the predicted bounding box and the ground truth bounding box.

6. The multi-target detection method based on image recognition according to claim 5, characterized in that, Using the training dataset, an initial multi-object detection network model equipped with a bidirectional guided feature fusion mechanism, a dynamic focus attention mechanism, and an adaptive hard sample mining loss function is iteratively trained to obtain a final multi-object detection network model, including: The hyperparameters of an initial multi-object detection network model with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function are optimized using a multi-object simulated annealing algorithm to obtain the optimal hyperparameters. Based on the optimal hyperparameters, the hyperparameters of the initial multi-object detection network model are set to obtain the optimized multi-object detection network model. The training dataset is input into the optimized multi-object detection network model for iterative training. An adaptive hard sample mining loss function is used to calculate the total loss value for each training iteration until the total loss value is less than the total loss threshold, thus obtaining the final multi-object detection network model.

7. The multi-target detection method based on image recognition according to claim 6, characterized in that, The hyperparameters of an initial multi-object detection network model with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function are optimized using a multi-object simulated annealing algorithm to obtain the optimal hyperparameters, including: The hyperparameters of the initial multi-object detection network model, which is equipped with a bidirectional guided feature fusion mechanism, a dynamic focusing attention mechanism, and an adaptive hard sample mining loss function, are encoded into the state vector of the multi-object simulated annealing algorithm. A multi-objective evaluation function is constructed and used as the energy function of the multi-objective simulated annealing algorithm. The chaotic sequence is generated using the Logistic mapping, and then mapped to the solution space of the hyperparameters to obtain the initial state of the multi-objective simulated annealing algorithm. Based on the candidate hyperparameters corresponding to the initial state, candidate multi-target detection network models are constructed, and inference is run on the validation set. The corresponding energy value is calculated using the energy function. Generate a new state with random perturbation in the neighborhood of the initial state, and run inference on the validation set to calculate the energy value corresponding to the new state using the energy function; Calculate the energy difference between the new state and the initial state, and update the initial state based on the energy difference to obtain the updated state; Repeat the state update steps until the number of updates reaches the update number threshold or the energy value of the state increases less than the preset increase threshold after several consecutive updates. The state of the last update is then taken as the optimal state. Decoding the state vector of the optimal state yields the optimal hyperparameters of the multi-object detection network model.

8. The multi-target detection method based on image recognition according to claim 7, characterized in that, The formula for the multi-objective evaluation function is: In the formula, For state The corresponding multi-target evaluation value, i.e., energy value, of the multi-target detection network model; For state The accuracy value of the corresponding multi-target detection network model; For state The corresponding speed value of the multi-target detection network model; Weights for multi-objective evaluation; In the formula, For state The mean accuracy of the corresponding multi-target detection network model calculated on the validation set; In the formula, For state The number of frames transmitted per second for the corresponding multi-target detection network model; It is a local minimum.

9. The multi-target detection method based on image recognition according to claim 8, characterized in that, The preprocessed real-time standard image is input into a multi-object detection network model for multi-object detection, yielding real-time multi-object detection results for the factory, including: The preprocessed real-time standard image is converted into an input tensor, and the input tensor is then fed into the multi-object detection network model. The backbone feature extraction network of the multi-object detection network model is used to extract low-level, mid-level, and high-level features of the input tensor. The semantic enhancement module in the neck network of the enhanced feature fusion of the multi-object detection network model is used to semantically enhance the mid-level and high-level features to obtain semantic features; The spatial detail compensation module in the neck network of the enhanced feature fusion of the multi-object detection network model is used to perform spatial detail compensation on semantic features, low-level features and high-level features to obtain compensated deep features. The feature fusion module in the neck network of the enhanced feature fusion multi-object detection network model is used to fuse semantic features and compensated deep features to obtain a fused feature map. The enhanced feature fusion neck network of the multi-object detection network model uses a feature focusing processing module to generate an attention weight map of the fused features, and then dynamically focuses it with the fused feature map to obtain a dynamically focused feature map. Using the decoupled detection head of the multi-object detection network model, prediction is made based on the dynamic focusing feature map to obtain the initial real-time multi-object detection results, including the probability vector of each preset box belonging to each category, the bounding box regression offset of each preset box, and the corresponding confidence. The initial real-time multi-target detection results are sequentially post-processed, non-maximum suppression is performed, and coordinate mapping is applied to obtain the final real-time multi-target detection results for the factory.

10. A multi-target detection device based on image recognition, used to implement the multi-target detection method as described in any one of claims 1-9, characterized in that, The device includes: The model building unit is used to build a multi-target detection network model based on image recognition algorithms. The multi-target detection network model includes a backbone feature extraction network, an enhanced feature fusion neck network, and a decoupled detection head. The image acquisition unit is used to respond to the data fusion request from the application. It uses cross-source semantic mapping technology and ontology reasoning tree to perform semantic alignment on the multi-source heterogeneous data, identify and resolve deep semantic conflicts, and obtain standard data. The multi-target detection unit is used to input the preprocessed real-time standard image into the multi-target detection network model to perform multi-target detection and obtain the real-time multi-target detection results of the factory.

Images