Three-channel target detection method and device based on active random model
By introducing active random model and three-channel processing into the object detection algorithm, dynamically adjusting parameters and refining steps, the problem of insufficient robustness and adaptability in the existing technology is solved, and higher detection accuracy and stability are achieved , suitable for autonomous driving, intelligent monitoring and other fields.
Patent Information
- Application Number
- CN202411031785.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-12
AI Technical Summary
The existing object detection algorithms have shortcomings in robustness, generalization, flexibility, speed and performance, especially in complex backgrounds and different scenarios, and require improvement and optimization.
Using a three-channel object detection method based on active random model, by acquiring three-channel images, adding noise frames for feature extraction and encoding and decoding, generating dynamic boxes, dynamically adjusting model parameters, reducing prior dependencies, and flexibly adjusting the number of refinement steps. And introduce randomness in the training and inference stages to improve disturbance resistance and flexibility.
It significantly improves the accuracy and stability of object detection, improves the adaptability and generalization capabilities of the model in complex environments, achieves a good balance of speed and accuracy, and enhances the comprehensive performance of the detection algorithm.
Smart Images

Figure CN118941776A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a three-channel target detection method and device based on an active random model. Background Art
[0002] Target detection technology has been continuously iterating and optimizing along with the development of hardware and algorithms. It has gone through three stages, from classic traditional target detection algorithms to machine learning algorithms, and then to deep learning intelligent algorithms. The algorithms in each stage have unique advantages. After years of development, target detection technology has been widely used in many fields and has greatly promoted the development of artificial intelligence technology in recent years.
[0003] However, target detection algorithms still have many shortcomings in terms of robustness, generalization, flexibility, speed and performance, and need continuous improvement and optimization. Summary of the invention
[0004] In view of this, an embodiment of the present application provides a three-channel target detection method and device based on an active random model, which can improve the balance of detection speed and accuracy performance.
[0005] The technical solution of the embodiment of the present application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides a three-channel target detection method based on an active random model, the method comprising:
[0007] Acquire a first target image, and perform conversion processing on the first target image to obtain a second target image, wherein the second target image is a three-channel image;
[0008] Performing feature extraction on the second target image to obtain a feature map, and adding noise to the first bounding box based on the forward pass of the active random model to obtain a noise box;
[0009] Clipping ROI features from the feature map through the noise frame, performing encoding and decoding processing on the ROI features to obtain a prediction result of the first bounding box, and generating an initial target detection network model of the active random model based on the prediction result;
[0010] Performing denoising based on the reverse of the active random model to obtain a second bounding box, and generating a dynamic box for each of the three channels based on a Gaussian distribution, wherein the total number of dynamic boxes for each channel is the same and randomly distributed;
[0011] An actual target detection network model of the active random model is determined based on the dynamic box, and target detection is performed on the input image based on the actual target detection network model.
[0012] In a second aspect, an embodiment of the present application further provides a three-channel target detection device based on an active random model, the device comprising:
[0013] A conversion module, used for acquiring a first target image, and performing conversion processing on the first target image to obtain a second target image, wherein the second target image is a three-channel image;
[0014] An extraction module, configured to perform feature extraction on the second target image to obtain a feature map, and to add noise to the first bounding box based on the forward pass of the active random model to obtain a noise box;
[0015] A cropping module, used for cropping ROI features from the feature map through the noise box, performing encoding and decoding processing on the ROI features to obtain a prediction result of the first bounding box, and generating an initial target detection network model of the active random model based on the prediction result;
[0016] A generation module, configured to perform denoising processing based on the reverse of the active random model to obtain a second bounding box, and generate a dynamic box for each of the three channels based on a Gaussian distribution, wherein the total number of dynamic boxes for each channel is the same and randomly distributed;
[0017] A detection module is used to determine an actual target detection network model of the active random model based on the dynamic box, and perform target detection on the input image based on the actual target detection network model.
[0018] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the three-channel target detection method based on the active random model as described in any one of the first aspects.
[0019] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the three-channel target detection method based on the active random model described in any one of the first aspects is executed.
[0020] The embodiments of the present application have the following beneficial effects:
[0021] (1) Dynamic adjustment and reduction of prior reliance: By introducing an active random model, this method no longer relies on a fixed number of random boxes and filling strategies, and achieves dynamic adjustment of these parameters. This feature greatly reduces the reliance on prior knowledge of the target, making the model more adaptable in different scenarios and complex backgrounds. At the same time, it avoids the need for learnable queries, simplifies the way of obtaining candidate boxes, and improves the efficiency and flexibility of the detection process.
[0022] (2) Flexible adjustment of the number of refinement steps: This method allows users to flexibly adjust the number of refinement steps according to the specific requirements of detection performance in actual applications. This flexibility enables the model to be optimized for target scenes of different complexity and density, thereby significantly improving detection accuracy while maintaining detection speed. For bounding boxes that cannot reach the preset threshold, random boxes are used for re-updating, further improving detection accuracy and stability.
[0023] (3) Enhanced anti-disturbance performance: During the training process, this method effectively enhances the diversity of supervisory signals by adding random perturbations to the model input. This randomness not only affects the positive and negative definitions of samples, but also makes the model more resistant to disturbances during the training phase. This training strategy helps the model learn more robust feature representations and improves the generalization and stability of the model in complex environments.
[0024] (4) Flexibility and robustness in the reasoning phase: During the reasoning process, this method achieves random generation of bounding boxes by adjusting random dynamic parameters. This mechanism not only improves the flexibility of the algorithm in the reasoning phase, but also enables the model to better cope with uncertain factors in practical applications, such as lighting changes and occlusions. At the same time, through a refined detection processing flow, this method significantly improves detection accuracy while maintaining detection speed, achieving a good balance between speed and accuracy.
[0025] (5) Comprehensive performance improvement: In summary, this method comprehensively improves the comprehensive performance of the target detection algorithm by introducing a series of innovative strategies such as active random models, dynamic adjustment and reduction of prior dependence, flexible adjustment of refinement steps, enhanced anti-disturbance performance, and improved flexibility and robustness of the reasoning stage. This makes this method have broad application prospects and important practical value in many fields such as autonomous driving, intelligent monitoring, and medical image analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0027] Figure 1 is a flowchart of steps S101-S105 provided in an embodiment of the present application;
[0028] Figure 2 It is a flowchart of steps S201-S202 provided in an embodiment of the present application;
[0029] Figure 3 It is a flowchart of steps S301-S302 provided in an embodiment of the present application;
[0030] Figure 4 It is a flow chart of the training phase model provided in an embodiment of the present application;
[0031] Figure 5 This is a flow chart of an active random model in the training phase provided by an embodiment of the present application;
[0032] Figure 6 It is a schematic diagram of the structure of a three-channel implementation algorithm provided in an embodiment of the present application;
[0033] Figure 7 is a structural schematic diagram of a three-channel target detection device based on an active random model provided in an embodiment of the present application;
[0034] Figure 8 It is a schematic diagram of the composition structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0035] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.
[0036] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0037] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0038] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0039] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0041] See also Figure 1 , Figure 1 is a flow chart of steps S101-S105 of the three-channel target detection method based on the active random model provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S105 are shown for explanation.
[0042] In step S101, a first target image is acquired, and a conversion process is performed on the first target image to obtain a second target image, wherein the second target image is a three-channel image.
[0043] In step S102, feature extraction is performed on the second target image to obtain a feature map, and noise is added to the first bounding box based on the forward pass of the active random model to obtain a noise box.
[0044] In step S103, ROI features are cropped from the feature map through the noise box, the ROI features are encoded and decoded to obtain a prediction result of the first bounding box, and an initial target detection network model of the active random model is generated based on the prediction result.
[0045] In step S104, denoising is performed based on the reverse of the active random model to obtain a second bounding box, and a dynamic box is generated for each of the three channels based on Gaussian distribution, wherein the total number of dynamic boxes for each channel is the same and randomly distributed.
[0046] In step S105, an actual target detection network model of the active random model is determined based on the dynamic frame, and target detection is performed on the input image based on the actual target detection network model.
[0047] Here, the input first target image is converted into a three-channel image (such as an RGB image) suitable for subsequent processing. The original image is converted into an image with three color channels through image processing techniques, such as color space conversion, so as to better capture the color information and details in the image. Key features are extracted from the three-channel image using deep learning or traditional image processing techniques, which are crucial for subsequent target detection. Based on the active random model, noise is added to the first bounding box to generate a noise box. This step aims to simulate the uncertainty in the actual detection scene by introducing randomness, thereby improving the generalization ability of the model. According to the noise box, the features of the region of interest (ROI) are cropped from the feature map. These features will be used for subsequent target prediction. The cropped ROI features are encoded and decoded to extract deeper feature representations. This step usually involves deep learning models such as convolutional neural networks (CNNs) to optimize and enhance feature representations through encoding and decoding processes. Based on the prediction results after encoding and decoding processing, the initial target detection network model of the active random model is generated. This model will be used for subsequent target detection tasks. The prediction results are denoised using the reverse process of the active random model to reduce the impact of noise on the detection results, thereby obtaining a more accurate second bounding box. Dynamic boxes are generated for each of the three channels based on Gaussian distribution. These dynamic boxes are arbitrarily distributed on the image and have the same total number, aiming to capture feature changes in different channels and further improve the accuracy of target detection. Based on the dynamic boxes and the previous processing steps, the actual target detection network model of the active random model is determined. This model combines the advantages of multiple steps such as feature extraction, noise addition, encoding and decoding processing, denoising processing, and dynamic box generation, and has high detection accuracy and robustness. The actual target detection network model is used to perform target detection on the input image. The model can automatically identify the target object in the image and give its location and category information.
[0048] In some embodiments, the image parameters of the first target image are (x, b), where x represents the input image and b represents the bounding box; and the second target image is a three-channel image (xR, xG, xB).
[0049] Here, the image parameters of the first target image are specified as (x, b), where x represents the input image and b represents the bounding box associated with the image. Here, the bounding box b is usually used to indicate the approximate location of one or more target objects in the image. Next, you mentioned that the second target image is a three-channel image, represented as (xR, xG, xB), which usually means that the image is converted or decomposed into three color channels: red (R), green (G), and blue (B), which is the standard representation in the RGB color space.
[0050] In some embodiments, before converting the first target image, the method further includes:
[0051] Performing filtering and image enhancement on the first target image, wherein the filtering is used to remove noise, and the image enhancement is used to improve the signal-to-noise ratio;
[0052] Data augmentation of the target is performed according to the position of the true value frame, and the data augmentation includes at least one of target flipping, angle change, and illumination intensity adjustment.
[0053] Here, filtering is used to remove noise from the image. Noise can be caused by a variety of factors, such as sensor defects, transmission errors, or environmental interference. Through filtering (such as Gaussian filtering, median filtering, etc.), the image can be smoothed to reduce the impact of noise on subsequent processing. Image enhancement is used to improve the signal-to-noise ratio of the image, that is, to increase the proportion of useful information in the image. This can be achieved through a variety of methods, such as histogram equalization, contrast stretching, sharpening, etc. Image enhancement aims to make the target object in the image more prominent, which is convenient for subsequent feature extraction and target detection.
[0054] In some embodiments, the forward process based on the active random model adds noise to the first bounding box to obtain a noise box, including:
[0055] N noises are added to the corresponding first bounding box to obtain the noise box, where N is a positive integer greater than 1.
[0056] Here, a key step in the forward process is to add noise to the first bounding box to generate a noise box. This step aims to simulate the uncertainty in the actual detection scene by introducing randomness, thereby improving the generalization ability and robustness of the model.
[0057] First, N independent noise values are generated. These noise values can be randomly generated, and their distribution can be selected according to specific needs, such as Gaussian distribution, uniform distribution, etc. The amplitude of the noise (i.e., the difference between its maximum and minimum values) and the standard deviation (or degree of variation) also need to be appropriately adjusted according to the actual application scenario.
[0058] Then, the generated N noise values are added to the corresponding attributes of the first bounding box respectively. A bounding box is usually defined by its position (such as the x-coordinate and y-coordinate of the upper left corner) and size (such as width and height). Therefore, the N noises can be divided into groups (each group corresponds to the noise of one attribute. When allocating noise, it is necessary to pay attention to maintaining the rationality of the bounding box, that is, its position and size should not exceed the image boundary or become meaningless (such as width or height is negative). After the noise is added, the first bounding box becomes a noise box containing random perturbations. These noise boxes will be used in the subsequent target detection process as one of the inputs for model training and prediction.
[0059] In some embodiments, see Figure 2 , Figure 2 It is a flow chart of steps S201-S202 provided in an embodiment of the present application. The encoding and decoding processing of the ROI feature is performed to obtain the prediction result of the first bounding box, and the initial target detection network model of the active random model is generated based on the prediction result. It can be achieved through steps S201-S202, which will be explained in combination with each step.
[0060] In step S201, the ROI feature is encoded, and the encoded ROI feature is input into the decoder of the active random model to obtain the prediction result of the first bounding box, wherein the bounding box numbered i selects the top k predictions with the lowest cost matching the first bounding box through the optimal allocation method, and the prediction loss is accumulated and averaged, and i is a positive integer less than or equal to N.
[0061] In step S202, the bounding box is repeatedly detected at L time steps and trained into an initial target detection network model of the active random model, where L is a positive integer greater than or equal to 1.
[0062] Here, ROI features are first extracted from the input image. These features are usually obtained from the original image or three-channel image in some way (such as the feature extraction layer in a convolutional neural network), and they represent an abstract representation of a specific region (i.e., ROI) in the image. Next, these ROI features are encoded. The purpose of encoding is to convert high-dimensional, complex feature representations into low-dimensional, easy-to-process encoded forms for decoding and prediction in subsequent steps. The encoding process may involve multiple nonlinear transformations and dimensionality reduction operations.
[0063] The encoded ROI features are input into the decoder of the active random model. The role of the decoder is to restore the encoded features to representations in the original feature space and generate predictions for the first bounding box based on these representations. During the prediction process, for the bounding box numbered i, the top k predictions with the lowest matching cost with the bounding box are selected through the optimal allocation method (such as the Hungarian algorithm). Here, the cost may be based on some distance metric between the predicted bounding box and the true bounding box (such as IoU, i.e., intersection over union). The losses of these k predictions are accumulated and averaged to evaluate the accuracy of the current prediction and provide feedback for the subsequent training process.
[0064] To improve the stability and robustness of the model, the bounding boxes are repeatedly detected at L time steps. This means that for each bounding box, L prediction results are generated and their losses are calculated separately. Based on the results of these repeated detections and the calculated losses, the initial object detection network model of the active random model is trained. The training process may involve adjusting the parameters of the model (such as weights and biases) to minimize the prediction loss and improve the accuracy of the model. It should be noted that the value of L should be adjusted according to the specific application scenario and model performance. Larger values of L can provide more training data and feedback, but will also increase computational complexity and training time.
[0065] After the above encoding and decoding processing, prediction and training steps, the initial target detection network model of the active random model is finally generated. This model already has a certain target detection capability and can be further optimized and adjusted in subsequent practical applications.
[0066] In some embodiments, the prediction result of the first bounding box is predicted for each of the three channels respectively by a multi-stages detection algorithm, and set prediction is used for processing according to the random characteristics of the active random model, and the set prediction of the first step is performed to obtain the prediction result and set prediction loss of this stage, and the top k predictions with the minimum cost matching the first bounding box are selected by the optimal allocation method, and the prediction loss is accumulated and averaged, and the prediction result is (bpri, bpgi, bpbi), and the loss function is minimized and accumulated in the prediction process;
[0067] L Train =∑ T s=1 (bpr i -b i ) 2 +(bpg i -b i ) 2 +(bpbi -b i ) 2 , where s is the step size and T is the total step size.
[0068] Here, the prediction result of the first bounding box is predicted separately on each channel of the three-channel image by the multi-stages detection algorithm, and is processed by the set prediction method in combination with the random characteristics of the active random model. This method aims to improve the accuracy and robustness of target detection, which is achieved through multi-stage, multi-channel prediction and the use of randomness.
[0069] The Multi-Stages detection algorithm divides the object detection process into multiple stages, each of which performs certain feature extraction and prediction. In your scenario, these stages are applied to each channel of a three-channel image (red, green, blue). The image on each channel independently undergoes a series of processing steps such as convolution, pooling, feature extraction, etc. to generate a feature map unique to that channel. In each stage, the set prediction method is used for prediction. Setprediction is a prediction method that can handle the uncertainty of the number and order of elements in the output set. It is suitable for object detection tasks because the number and location of target objects in the image are unknown. In the first step (or stage) of setprediction, the model generates a series of predicted bounding boxes (bpri, bpgi, bpbi, etc., representing the prediction results on the red, green, and blue channels respectively), as well as the corresponding prediction confidence or score.
[0070] In order to evaluate the accuracy of the prediction results and select the top k predictions with the lowest matching cost with the first bounding box (the true bounding box), the optimal allocation method (such as the Hungarian algorithm) is used for matching. This method calculates a certain cost (such as IoU difference, distance, etc.) between each predicted bounding box and the first bounding box, and finds the matching solution with the lowest total cost. Loss function minimization and accumulation: For each matched predicted bounding box, calculate its prediction loss (such as L2 loss, cross entropy loss, etc.), and accumulate and average these losses to evaluate the overall prediction performance of this stage. Then, use optimization algorithms such as gradient descent to update the parameters of the model to minimize the loss function, thereby improving the prediction accuracy of the model.
[0071] Although the predictions on each channel are made independently, these predictions can be fused in subsequent processing (perhaps in the subsequent steps of the current stage or in other stages). The fusion method can be to weighted average, union or intersection of the predicted bounding boxes on different channels to comprehensively utilize multi-channel information and further improve detection performance.
[0072] As mentioned above, in order to enhance the stability and robustness of the model, the bounding box can be repeatedly detected at multiple time steps, and the loss function can be minimized and the parameters updated after each step. Through the above steps, using the multi-stages detection algorithm, set prediction method, optimal allocation method and multi-channel fusion technology, a target detection model that can handle complex images and has high accuracy can be constructed.
[0073] In some embodiments, the repeatedly detecting the bounding box at L time steps and training the initial target detection network model of the active random model includes:
[0074] The step size L is set, the loss function of the three channels is calculated multiple times, and a three-channel target detection network model based on the active random model is established.
[0075] Here, the bounding box is repeatedly detected at L time steps and trained into an initial target detection network model of an active random model. This is an important iterative learning process that aims to improve the accuracy and stability of the model through multiple calculations and optimizations.
[0076] L is a pre-set positive integer representing the number of time steps for repeated detection during training. The value of L can be determined based on the requirements of the specific task, the limitations of computing resources, and the room for improvement of model performance.
[0077] In each time step, the image data of the three channels (red, green, and blue) are processed separately, and the corresponding loss function is calculated. The loss function is an important indicator to measure the difference between the model's prediction results and the actual results. It reflects the performance of the model at the current time step. For each channel, the loss is calculated based on the difference between the predicted bounding box and the true bounding box on that channel. These losses may include position loss (such as the difference in bounding box coordinates), classification loss (such as the prediction accuracy of the target category), etc. It should be noted that due to the use of an active random model, the prediction results in each time step will be affected by randomness, so the calculation of the loss function will also be different.
[0078] Based on multiple calculations of the loss function, the model parameters (such as weights and biases) are continuously adjusted through optimization methods such as back propagation algorithms and gradient descent to minimize the value of the loss function. This process is repeated for L time steps, and each step updates the model parameters according to the current loss function value. As the training progresses, the prediction accuracy of the model will gradually improve, and the value of the loss function will gradually decrease. Finally, after L time steps of training, a three-channel target detection network model based on the active random model will be obtained. This model has learned how to predict the location and category of the target bounding box based on the input three-channel image. It should be noted that although the "three-channel" target detection network model is emphasized here, in fact, the input of the model is not limited to three-channel images. In practical applications, different numbers of channels can be selected as input according to the needs of specific tasks. In addition, the specific implementation of the active random model may also vary from task to task, but the basic principle is to use randomness to enhance the generalization ability and robustness of the model.
[0079] In some embodiments, the dynamic box corresponding to each channel includes an update strategy, which is used to remove prediction results whose sampling results are lower than a threshold α (0<α<1) and replace them with the update box. The update box and the output box that meets the threshold are combined into an update group box.
[0080] Here, the dynamic box corresponding to each channel adopts an update strategy that aims to optimize the prediction results and improve the accuracy and efficiency of target detection. Specifically, this update strategy includes removing the prediction results whose sampling results are lower than the threshold α and replacing these removed prediction results with the updated box.
[0081] Dynamic boxes are bounding boxes that are dynamically generated during object detection based on the current image and model predictions. These bounding boxes are used to mark and locate the target objects in the image. The update strategy is a series of optimization operations performed on these dynamic boxes to improve the quality of the prediction results. In this strategy, special attention is paid to the prediction results whose sampling results (which may refer to indicators such as prediction confidence, intersection with the true bounding box, etc.) are lower than a certain threshold α.
[0082] For the prediction results on each channel, the sampling results will be evaluated. If the sampling result of a prediction result is lower than the preset threshold α (where 0<α<1), the prediction result is considered unreliable and needs to be removed. Removing these low-quality prediction results can reduce the possibility of false detection and missed detection, thereby improving the accuracy of target detection.
[0083] After removing the predictions below the threshold, update boxes are needed to replace these removed predictions. Update boxes can be bounding boxes regenerated based on the current image and model state, or high-quality bounding boxes selected from other predictions. The generation and selection of update boxes should be based on certain strategies and rules to ensure that they can accurately mark the target objects in the image and minimize the difference with the true bounding box.
[0084] After the above update operation, the prediction results that meet the threshold (i.e. the prediction results that are not removed) and the update box will be combined into an update group box. The update group box is the final prediction result set of the target object in the image by the object detection network at the current time step. Each bounding box in the update group box has been strictly screened and optimized, and has high reliability and accuracy. They will be used in subsequent tasks such as target recognition and classification.
[0085] In some embodiments, see Figure 3 , Figure 3 It is a flow chart of steps S301-S302 provided in an embodiment of the present application. The target detection on the input image based on the actual target detection network model can be achieved through steps S301-S302, which will be explained in combination with each step.
[0086] In step S301, the three channels are iteratively sampled, and the ROI features are respectively cropped from the feature map through the update group box of each channel. After the ROI features are selected, the detection head is entered, and iteration is performed based on the inverse process of the active random model. Parameters are shared in the iteration. The sampling step size S is set according to the detection speed and accuracy requirements, and the sampling step size of each channel is the same.
[0087] In step S302, the detection results of each channel are fused with target information using feature fusion and matching rules to obtain the target detection result bs i .
[0088] Here, the input image usually contains three color channels (red, green, blue), and each channel is iteratively sampled independently. Iterative sampling means that the image of each channel is repeatedly processed in multiple time steps or iterations to capture richer features and context information. During the iterative sampling process of each channel, the corresponding ROI (region of interest) features are cropped from the feature map according to the update group box (i.e., the set of bounding boxes optimized in the previous step). These ROI features contain key information about the target object and are the basis for subsequent detection and processing. The cropped ROI features are sent to the detection head for processing. The detection head is a key part of the object detection network, responsible for tasks such as classification and localization of the target based on the ROI features. In the context of active random models, the detection head may also contain some special processing mechanisms to exploit the randomness of the model to improve the robustness and accuracy of the detection. During the detection head processing, the reverse process of the active random model is used for iteration. This usually means that during the iteration process, some parameters or states of the model are adjusted or updated based on the results of the previous iteration to better adapt to the current task and data. Reverse iteration helps the model gradually optimize its performance during the training process. In the three-channel iteration process, the parameters of the model are shared in the iteration. This means that no matter which channel or time step, the model uses the same set of parameters for prediction and processing. Parameter sharing helps reduce the number of parameters of the model, reduce the risk of overfitting, and improve the generalization ability of the model. According to the requirements of detection speed and accuracy, the same sampling step size S is set for each channel. The sampling step size determines the size of the interval between each time step during the iteration process. A smaller sampling step size can provide more iterations and more refined processing results, but it will also increase the computational complexity and time cost; a larger sampling step size is the opposite. Therefore, it is necessary to select a suitable sampling step size based on the requirements of the specific task and the limitations of computing resources. After the detection results are obtained in each channel, feature fusion and matching rules are used to fuse the target information. Feature fusion can be operations such as merging or weighted summing of feature maps of multiple channels to obtain a more comprehensive target representation; matching rules are used to determine the correspondence between the detection results of different channels so that they can be merged into a unified set of detection results. The final target detection results will contain information such as the category and location of all target objects in the image.
[0089] Below, the embodiments of the present application will be fully described.
[0090] The embodiment of the present application converts the image detection task into a task of generating the target position and size, and uses a three-channel active random model to process the target detection task. In the training phase, the backbone network obtains the image feature map, and then the noise is added to the truth box to obtain the noise box. The noise box cuts the features of the ROI from the feature map, and finally these ROI features are sent to the decoder. The decoder is responsible for predicting the truth box and forming an active random algorithm model through algorithm training. In the reasoning phase, the bounding box is generated by the active random model to predict the detection result of the target. The use of active random models for target detection in complex scenes improves the flexibility and robustness of the target detection algorithm. Through random perturbations, the training process is made random. The dynamic random mechanism and refined processing of the bounding box in the reasoning process propose a new detection method for the application of the target detection algorithm.
[0091] (1) Image data preprocessing and multi-channel image conversion.
[0092] The image captured by the camera is an RGB image with image parameters (x, b), where x is the input image and b is the bounding box. First, filtering and image enhancement are performed. The noise and other signals of the original image are removed by filtering. Image enhancement is performed according to the characteristics of the target to improve the overall signal-to-noise ratio of the image. The data of the target is augmented according to the position of the truth box, including target flipping, angle change, and light intensity adjustment, to increase the effective data of the data set. The image is an RGB three-channel image. The original image is extracted into a three-channel image (xR, xG, xB), and its features are extracted separately, so that the three-channel image information is fused and complementary information is extracted, thereby enhancing the richness of information features during subsequent processing.
[0093] (2) Obtain the target features through the backbone network and randomly add noise and active random model to encode and decode the ROI features to generate an algorithm model.
[0094] 1. Use the backbone network to extract features from the input three-channel image and output feature maps (ResNet or SwinTransformer, etc.).
[0095] 2. By adding noise in the forward direction of the active random model, the ROI feature map is encoded, and the result is predicted during decoding to complete the model establishment. The parameters are obtained through loss function adjustment to obtain the algorithm model.
[0096] In the forward process, by adding noise to the bounding box (the bounding box is a set, and the i-th box is represented by b i =(cx i ,cy i ,w i ,h i ), where (cx i ,cy i) is the center coordinate of the bounding box, (w i ,h i ) are the width and height of the bounding box respectively). By adding disturbance, the input data is gradually disturbed in multiple time steps, which affects the definition of positive and negative samples, making the training process have a certain degree of randomness and realizing the diversified supervision of the model. In the training phase, 1) first expand the true value box to make its total number a fixed value N. When filling, the labeled true value repeated box, dynamic box or imaging size box is used. These boxes can be fixed or randomly combined to ensure that the total number is fixed. 2) Use noise that tends to the prior distribution to superimpose on the expanded box (the noise box of the three channels is expressed as (bnr i ,bng i ,bnb i ), with a single channel bnr i (cxnr i ,cynr i ,wnr i ,hnr i ) is used to illustrate, where (cxnr i ,cynr i ) is the center coordinate of the noise box, (wnr i ,hnr i ) are the width and height of the noise box respectively. The noise scale is determined by the time parameter t. The distribution of parameter t is obtained according to the model step size. Since the signal-to-noise ratio has a significant impact on the algorithm model, the function of parameter t is processed using a monotonically decreasing function. The noise is controlled and the coordinates of the true value box are scaled accordingly, so that the target detection has a consistent signal-to-noise scaling value. 3) The noise box crops the features of the ROI from the feature map and sends it to the detection head.
[0097] In the decoding prediction process, by minimizing the loss function, reducing the difference between the prediction result and the true value frame, removing noise, and establishing a neural network model. 1) First, use the multi-stages optimized detection algorithm to perform predictions on three channels. According to the random characteristics of the active random model, a more global feature set prediction is used for processing. The first step of set prediction is performed to obtain the prediction result and set prediction loss of this stage. The optimal allocation method is used to select the top k predictions with the lowest cost that match the true value frame, and then the prediction loss is accumulated and averaged. The prediction result is (bpr i ,bpg i ,bpb i );2) Then the prediction process is performed by minimizing and accumulating the loss function.
[0098] L Train =∑ Ts=1 (bpr i -b i ) 2 +(bpg i -b i ) 2 +(bpb i -b i ) 2 , where s is the step size and T is the total step size.
[0099] 3. Set L according to the step size and repeat the above steps. The detection results of the three channels (bsr i ,bsg i ,bsb i ) Use feature fusion and matching strategy to fuse target information and get the target detection result bs i Since the entire model calculates the loss function multiple times, it brings better supervision and establishes the initial target detection network model of the active random model.
[0100] 4. Group the image data collected from the actual scene into batches and perform iteration training until the speed and accuracy balance of the task indicators is achieved.
[0101] (3) Target reasoning: Use an active random model to perform reverse denoising, decouple training and detection based on dynamic box updates and multi-step refinement, and provide target detection results.
[0102] The noise is removed by the reverse of the active random model to generate a bounding box, and M dynamic boxes are generated using Gaussian distribution. The ROI features are cropped from the feature map, and then the ROI features are sent to the tandem decoder. The box update strategy is adopted to gradually refine the three-channel prediction under S sampling steps, and the prediction information is fused to obtain the target detection result.
[0103] 1) Image preprocessing: The images collected by the camera are filtered and enhanced. The noise and other signals of the original image are removed through filtering. The image is enhanced according to the characteristics of the target to improve the overall signal-to-noise ratio of the image.
[0104] 2) The image is a three-channel RGB image. The original image is converted into a three-channel image, and its features are extracted separately, so that the three-channel image information can be fused and complementary information can be extracted, thereby enhancing the richness of information features in subsequent processing.
[0105] 3) Use the backbone network to extract features from the input three-channel image and output a feature map (ResNet or SwinTransformer, etc.).
[0106] 4) Generate dynamic boxes. Generate bounding boxes with random noise based on Gaussian distribution. Due to the characteristics of the active random model, the selection of the number of dynamic boxes will not cause changes in detection accuracy. The total number of boxes M is dynamically set according to the detection requirements; the three channels generate dynamic boxes respectively, and each channel image uses the same total number of boxes M with arbitrary distribution.
[0107] 5) The dynamic boxes of the three channels are respectively cropped from the feature map ROI features, and the ROI features are selected and entered into the detection head. The multi-stage series decoder is used as the main body for denoising and the detection box result is output. The dynamic box of the three channels is represented as (brr i ,brg i ,brb i ), with single channel brr i (cxrr i ,cyrr i ,wrr i ,hrr i ) is used to illustrate, where (cxrr i ,cyrr i ) is the center coordinate of the dynamic box, (wrr i ,hrr i ) are the width and height of the dynamic box respectively. The sampling results include two types, expected prediction and unexpected prediction. The expected prediction can correctly locate the target, while the unexpected prediction deviates from the target position.
[0108] 6) All three channels adopt a box update strategy. By setting the positioning matching degree threshold α (0<α<1), the result boxes below the threshold are removed and the unexpected prediction boxes are replaced by dynamic boxes. The update boxes are dynamically generated according to the Gaussian distribution and combined with the output boxes that meet the threshold into a new set of boxes. Each step is embedded in the active random process through the time parameter, so that the dynamic box of each input detection head can be updated.
[0109] 7) The three channels are sampled iteratively, and each uses the update group box of its own channel to crop the ROI features from the feature map. After selecting the ROI features, it enters the detection head and gradually refines it using the reverse process of the active random model. The detection head is reused in the iteration, and the parameters are shared in the iteration. The sampling step size S is set according to the detection speed and accuracy requirements, and the sampling step size of the three channels is the same.
[0110] 8) Repeat the previous steps according to the step size. After the iteration is completed, the detection results of the three channels are obtained (bsr i ,bsg i ,bsb i ).
[0111] 9) The detection results of the three channels are fused using feature fusion and matching strategy to obtain the target detection result bsi .
[0112] Take the operation of three-channel RGB data on multiple graphics cards as an example. During the training phase: Figure 5 This is the flow chart of the active random model in the training phase. 24 small batches are used for training, 200K iterations, 4 iterations of the active random model, data augmentation including target flipping, angle change, light intensity adjustment, etc. The data is normalized, and the resolution is set to 640 pixels on the long side and 480 pixels on the short side. Figure 4 This is the model flow chart for the training phase. The original data first uses a filtering algorithm to improve the signal-to-noise ratio; then the data is augmented to expand the sample data set; then the three-channel image data is extracted from the image and sent to the backbone network to extract features; the truth frame is superimposed with random noise, and the features of the cropped ROI are sent to the detection head. The total number of random noises is 300, and the total number of random noises for each channel is the same; the time step is set to 4, and 4 three-channel detection head processing is performed; the three-channel prediction results are obtained, the model is trained through the loss function, the model parameters are obtained, and the initial network model for target detection based on the active random model is established. After 200K iterations, a three-channel target detection model based on the active random frame is formed. Target detection phase: Figure 6 This is a schematic diagram of the algorithm structure for three channels. After the image data is preprocessed by filtering and target enhancement, the three-channel image data is extracted from the image and sent to the backbone network (ResNet or Swin Transformer, etc.) to extract features. The features are adjusted through the Neck layer, and the dynamic box is used to crop the features of the ROI. The number of dynamic boxes is set to 300, and the Gaussian function is used as the filling strategy. The three channels are configured with the same number of dynamic boxes. A detection head with 6 stages in series is used to process the three inputs separately to obtain three prediction results. The results include two types, expected predictions and unexpected predictions. The expected predictions can correctly locate the target, and the unexpected predictions deviate from the target position. The three prediction results are updated separately, and the threshold is set to 0.6. The result boxes below the threshold are removed, and the unexpected prediction boxes are replaced by the update boxes. The update boxes are based on Gaussian distribution is dynamically generated and combined with the output box that meets the threshold into a new group of boxes. Each step is embedded in the active random process through the time parameter, so that the dynamic box of each input detection head can be updated; iterative refinement, related parameters are shared in the iteration, and the detection head is reused in the iteration. The three channels each use the updated group box of their own channel to crop ROI features from the feature map respectively, and enter the detection head after selecting the ROI features. The reverse process of the active random model is used to gradually refine. After the iterative refinement is completed, the detection results of the three channels are obtained, and the entire generation framework is repeated for 2 cycles. Then, feature fusion and matching strategies are used to fuse target information to obtain target detection results.
[0113] In summary, the embodiments of the present application have the following beneficial effects:
[0114] (1) Dynamic adjustment and reduction of prior reliance: By introducing an active random model, this method no longer relies on a fixed number of random boxes and filling strategies, and achieves dynamic adjustment of these parameters. This feature greatly reduces the reliance on prior knowledge of the target, making the model more adaptable in different scenarios and complex backgrounds. At the same time, it avoids the need for learnable queries, simplifies the way of obtaining candidate boxes, and improves the efficiency and flexibility of the detection process.
[0115] (2) Flexible adjustment of the number of refinement steps: This method allows users to flexibly adjust the number of refinement steps according to the specific requirements of detection performance in actual applications. This flexibility enables the model to be optimized for target scenes of different complexity and density, thereby significantly improving detection accuracy while maintaining detection speed. For bounding boxes that cannot reach the preset threshold, random boxes are used for re-updating, further improving detection accuracy and stability.
[0116] (3) Enhanced anti-disturbance performance: During the training process, this method effectively enhances the diversity of supervisory signals by adding random perturbations to the model input. This randomness not only affects the positive and negative definitions of samples, but also makes the model more resistant to disturbances during the training phase. This training strategy helps the model learn more robust feature representations and improves the generalization and stability of the model in complex environments.
[0117] (4) Flexibility and robustness in the reasoning phase: During the reasoning process, this method achieves random generation of bounding boxes by adjusting random dynamic parameters. This mechanism not only improves the flexibility of the algorithm in the reasoning phase, but also enables the model to better cope with uncertain factors in practical applications, such as lighting changes and occlusions. At the same time, through a refined detection processing flow, this method significantly improves detection accuracy while maintaining detection speed, achieving a good balance between speed and accuracy.
[0118] (5) Comprehensive performance improvement: In summary, this method comprehensively improves the comprehensive performance of the target detection algorithm by introducing a series of innovative strategies such as active random models, dynamic adjustment and reduction of prior dependence, flexible adjustment of refinement steps, enhanced anti-disturbance performance, and improved flexibility and robustness of the reasoning stage. This makes this method have broad application prospects and important practical value in many fields such as autonomous driving, intelligent monitoring, and medical image analysis.
[0119] Based on the same inventive concept, the embodiment of the present application also provides a three-channel target detection device based on an active random model corresponding to the three-channel target detection method based on an active random model in the first embodiment. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned three-channel target detection method based on an active random model, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0120] like Figure 7 As shown, Figure 7 : is a schematic diagram of the structure of a three-channel target detection device 700 based on an active random model provided in an embodiment of the present application. The three-channel target detection device 700 based on an active random model includes:
[0121] The conversion module 701 is used to obtain a first target image, and perform conversion processing on the first target image to obtain a second target image, wherein the second target image is a three-channel image;
[0122] An extraction module 702 is used to extract features from the second target image to obtain a feature map, and to add noise to the first bounding box based on the forward pass of the active random model to obtain a noise box;
[0123] A cropping module 703 is used to crop ROI features from the feature map through the noise box, perform encoding and decoding processing on the ROI features to obtain a prediction result of the first bounding box, and generate an initial target detection network model of the active random model based on the prediction result;
[0124] A generating module 704 is used to perform denoising based on the reverse of the active random model to obtain a second bounding box, and generate a dynamic box for each of the three channels based on Gaussian distribution, wherein the total number of dynamic boxes for each channel is the same and randomly distributed;
[0125] The detection module 705 is used to determine the actual target detection network model of the active random model based on the dynamic frame, and perform target detection on the input image based on the actual target detection network model.
[0126] Those skilled in the art should understand that Figure 7 The implementation functions of each unit in the three-channel target detection device 700 based on the active random model can be understood by referring to the relevant description of the three-channel target detection method based on the active random model mentioned above. Figure 7 The functions of the various units in the three-channel target detection device 700 based on the active random model shown can be implemented by a program running on a processor, or can be implemented by a specific logic circuit.
[0127] In a possible implementation, the image parameters of the first target image are (x, b), where x represents the input image and b represents the bounding box; and the second target image is a three-channel image (xR, xG, xB).
[0128] In a possible implementation manner, before the conversion module 701 performs conversion processing on the first target image, the method further includes:
[0129] Performing filtering and image enhancement on the first target image, wherein the filtering is used to remove noise, and the image enhancement is used to improve the signal-to-noise ratio;
[0130] Data augmentation of the target is performed according to the position of the true value frame, and the data augmentation includes at least one of target flipping, angle change, and illumination intensity adjustment.
[0131] In a possible implementation, the extraction module 702 adds noise to the first bounding box based on the forward direction of the active random model to obtain a noise box, including:
[0132] N noises are added to the corresponding first bounding box to obtain the noise box, where N is a positive integer greater than 1.
[0133] In a possible implementation, the generation module 704 performs encoding and decoding processing on the ROI feature to obtain a prediction result of the first bounding box, and generates an initial target detection network model of the active random model based on the prediction result, including:
[0134] Encode the ROI feature, and input the encoded ROI feature into the decoder of the active random model to obtain a prediction result of the first bounding box, wherein the bounding box numbered i selects the top k predictions with the minimum cost matching the first bounding box through the optimal allocation method, and accumulates and averages the prediction losses, where i is a positive integer less than or equal to N;
[0135] The bounding box is repeatedly detected at L time steps and trained into an initial target detection network model of the active random model, where L is a positive integer greater than or equal to 1.
[0136] In a possible implementation, the prediction result of the first bounding box is predicted for each of the three channels respectively by a multi-stages detection algorithm, and setprediction is used for processing according to the random characteristics of the active random model, and set prediction of the first step length is performed to obtain the prediction result and setprediction loss of this stage, and the top k predictions with the minimum cost matching the first bounding box are selected by the optimal allocation method, and the prediction loss is accumulated and averaged, and the prediction result is (bpri, bpgi, bpbi), and the loss function is minimized and accumulated in the prediction process;
[0137] L Train =∑ T s=1 (bpr i -b i ) 2 +(bpg i -b i ) 2 +(bpb i -b i ) 2 , where s is the step size and T is the total step size.
[0138] In a possible implementation, the generating module 704 repeatedly detects the bounding box at L time steps and trains the initial target detection network model of the active random model, including:
[0139] The step size L is set, the loss function of the three channels is calculated multiple times, and a three-channel target detection network model based on the active random model is established.
[0140] In a possible implementation, the dynamic box corresponding to each channel includes an update strategy, which is used to remove prediction results whose sampling results are lower than a threshold α (0<α<1) and replace them with the update box, and the update box and the output box that meets the threshold are combined into an update group box.
[0141] In a possible implementation, the detection module 705 performs target detection on the input image based on the actual target detection network model, including:
[0142] The three channels are iteratively sampled, and the ROI features are respectively cut from the feature map through the update group box of each channel, and the ROI features are selected and then enter the detection head, and the iteration is performed based on the reverse process of the active random model. The parameters are shared in the iteration, and the sampling step size S is set according to the detection speed and accuracy requirements, and the sampling step size of each channel is the same;
[0143] The detection results of each channel are fused with target information using feature fusion and matching rules to obtain the target detection result bs i .
[0144] The above three-channel target detection device based on active random model has the following beneficial effects:
[0145] (1) Dynamic adjustment and reduction of prior reliance: By introducing an active random model, this method no longer relies on a fixed number of random boxes and filling strategies, and achieves dynamic adjustment of these parameters. This feature greatly reduces the reliance on prior knowledge of the target, making the model more adaptable in different scenarios and complex backgrounds. At the same time, it avoids the need for learnable queries, simplifies the way of obtaining candidate boxes, and improves the efficiency and flexibility of the detection process.
[0146] (2) Flexible adjustment of the number of refinement steps: This method allows users to flexibly adjust the number of refinement steps according to the specific requirements of detection performance in actual applications. This flexibility enables the model to be optimized for target scenes of different complexity and density, thereby significantly improving detection accuracy while maintaining detection speed. For bounding boxes that cannot reach the preset threshold, random boxes are used for re-updating, further improving detection accuracy and stability.
[0147] (3) Enhanced anti-disturbance performance: During the training process, this method effectively enhances the diversity of supervisory signals by adding random perturbations to the model input. This randomness not only affects the positive and negative definitions of samples, but also makes the model more resistant to disturbances during the training phase. This training strategy helps the model learn more robust feature representations and improves the generalization and stability of the model in complex environments.
[0148] (4) Flexibility and robustness in the reasoning phase: During the reasoning process, this method achieves random generation of bounding boxes by adjusting random dynamic parameters. This mechanism not only improves the flexibility of the algorithm in the reasoning phase, but also enables the model to better cope with uncertain factors in practical applications, such as lighting changes and occlusions. At the same time, through a refined detection processing flow, this method significantly improves detection accuracy while maintaining detection speed, achieving a good balance between speed and accuracy.
[0149] (5) Comprehensive performance improvement: In summary, this method comprehensively improves the comprehensive performance of the target detection algorithm by introducing a series of innovative strategies such as active random models, dynamic adjustment and reduction of prior dependence, flexible adjustment of refinement steps, enhanced anti-disturbance performance, and improved flexibility and robustness of the reasoning stage. This makes this method have broad application prospects and important practical value in many fields such as autonomous driving, intelligent monitoring, and medical image analysis.
[0150] like Figure 8 As shown, Figure 8 The present invention provides a schematic diagram of the structure of an electronic device 800 according to an embodiment of the present invention. The electronic device 800 includes:
[0151] A processor 801, a storage medium 802 and a bus 803, wherein the storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device 800 is running, the processor 801 communicates with the storage medium 802 via the bus 803, and the processor 801 executes the machine-readable instructions to perform the steps of the three-channel target detection method based on the active random model described in the embodiment of the present application.
[0152] In actual application, the components in the electronic device 800 are coupled together via bus 803. It is understood that bus 803 is used to realize the connection and communication between these components. Bus 803 includes not only a data bus, but also a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus 803.
[0153] The electronic device has the following beneficial effects:
[0154] (1) Dynamic adjustment and reduction of prior reliance: By introducing an active random model, this method no longer relies on a fixed number of random boxes and filling strategies, and achieves dynamic adjustment of these parameters. This feature greatly reduces the reliance on prior knowledge of the target, making the model more adaptable in different scenarios and complex backgrounds. At the same time, it avoids the need for learnable queries, simplifies the way of obtaining candidate boxes, and improves the efficiency and flexibility of the detection process.
[0155] (2) Flexible adjustment of the number of refinement steps: This method allows users to flexibly adjust the number of refinement steps according to the specific requirements of detection performance in actual applications. This flexibility enables the model to be optimized for target scenes of different complexity and density, thereby significantly improving detection accuracy while maintaining detection speed. For bounding boxes that cannot reach the preset threshold, random boxes are used for re-updating, further improving detection accuracy and stability.
[0156] (3) Enhanced anti-disturbance performance: During the training process, this method effectively enhances the diversity of supervisory signals by adding random perturbations to the model input. This randomness not only affects the positive and negative definitions of samples, but also makes the model more resistant to disturbances during the training phase. This training strategy helps the model learn more robust feature representations and improves the generalization and stability of the model in complex environments.
[0157] (4) Flexibility and robustness in the reasoning phase: During the reasoning process, this method achieves random generation of bounding boxes by adjusting random dynamic parameters. This mechanism not only improves the flexibility of the algorithm in the reasoning phase, but also enables the model to better cope with uncertain factors in practical applications, such as lighting changes and occlusions. At the same time, through a refined detection processing flow, this method significantly improves detection accuracy while maintaining detection speed, achieving a good balance between speed and accuracy.
[0158] (5) Comprehensive performance improvement: In summary, this method comprehensively improves the comprehensive performance of the target detection algorithm by introducing a series of innovative strategies such as active random models, dynamic adjustment and reduction of prior dependence, flexible adjustment of refinement steps, enhanced anti-disturbance performance, and improved flexibility and robustness of the reasoning stage. This makes this method have broad application prospects and important practical value in many fields such as autonomous driving, intelligent monitoring, and medical image analysis.
[0159] The embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by at least one processor 801, the three-channel target detection method based on the active random model described in the embodiment of the present application is implemented.
[0160] In some embodiments, the storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disk, or a compact disc read-only memory (CD ROM); it can also be various devices including one or any combination of the above memories.
[0161] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.
[0162] As an example, executable instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).
[0163] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.
[0164] The computer-readable storage medium has the following advantages:
[0165] (1) Dynamic adjustment and reduction of prior reliance: By introducing an active random model, this method no longer relies on a fixed number of random boxes and filling strategies, and achieves dynamic adjustment of these parameters. This feature greatly reduces the reliance on prior knowledge of the target, making the model more adaptable in different scenarios and complex backgrounds. At the same time, it avoids the need for learnable queries, simplifies the way of obtaining candidate boxes, and improves the efficiency and flexibility of the detection process.
[0166] (2) Flexible adjustment of the number of refinement steps: This method allows users to flexibly adjust the number of refinement steps according to the specific requirements of detection performance in actual applications. This flexibility enables the model to be optimized for target scenes of different complexity and density, thereby significantly improving detection accuracy while maintaining detection speed. For bounding boxes that cannot reach the preset threshold, random boxes are used for re-updating, further improving detection accuracy and stability.
[0167] (3) Enhanced anti-disturbance performance: During the training process, this method effectively enhances the diversity of supervisory signals by adding random perturbations to the model input. This randomness not only affects the positive and negative definitions of samples, but also makes the model more resistant to disturbances during the training phase. This training strategy helps the model learn more robust feature representations and improves the generalization and stability of the model in complex environments.
[0168] (4) Flexibility and robustness in the reasoning phase: During the reasoning process, this method achieves random generation of bounding boxes by adjusting random dynamic parameters. This mechanism not only improves the flexibility of the algorithm in the reasoning phase, but also enables the model to better cope with uncertain factors in practical applications, such as lighting changes and occlusions. At the same time, through a refined detection processing flow, this method significantly improves detection accuracy while maintaining detection speed, achieving a good balance between speed and accuracy.
[0169] (5) Comprehensive performance improvement: In summary, this method comprehensively improves the comprehensive performance of the target detection algorithm by introducing a series of innovative strategies such as active random models, dynamic adjustment and reduction of prior dependence, flexible adjustment of refinement steps, enhanced anti-disturbance performance, and improved flexibility and robustness of the reasoning stage. This makes this method have broad application prospects and important practical value in many fields such as autonomous driving, intelligent monitoring, and medical image analysis.
[0170] In the several embodiments provided in the present application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0171] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0172] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0173] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.
[0174] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A three-channel target detection method based on an active random model, characterized in that: The method comprises: Acquire a first target image, and perform conversion processing on the first target image to obtain a second target image, wherein the second target image is a three-channel image; Performing feature extraction on the second target image to obtain a feature map, and adding noise to the first bounding box based on the forward pass of the active random model to obtain a noise box; Clipping ROI features from the feature map through the noise frame, performing encoding and decoding processing on the ROI features to obtain a prediction result of the first bounding box, and generating an initial target detection network model of the active random model based on the prediction result; Performing denoising based on the reverse of the active random model to obtain a second bounding box, and generating a dynamic box for each of the three channels based on a Gaussian distribution, wherein the total number of dynamic boxes for each channel is the same and randomly distributed; An actual target detection network model of the active random model is determined based on the dynamic box, and target detection is performed on the input image based on the actual target detection network model.
2. The method according to claim 1, characterized in that The image parameters of the first target image are (x, b), where x represents the input image and b represents the bounding box; the second target image is a three-channel image (xR, xG, xB).
3. The method according to claim 1, characterized in that Before converting the first target image, the method further includes: Performing filtering and image enhancement on the first target image, wherein the filtering is used to remove noise, and the image enhancement is used to improve the signal-to-noise ratio; Data augmentation of the target is performed according to the position of the true value frame, and the data augmentation includes at least one of target flipping, angle change, and illumination intensity adjustment.
4. The method according to claim 1, characterized in that: The forward process based on the active random model adds noise to the first bounding box to obtain a noise box, including: N noises are added to the corresponding first bounding box to obtain the noise box, where N is a positive integer greater than 1.
5. The method according to claim 1, characterized in that The encoding and decoding process is performed on the ROI feature to obtain a prediction result of the first bounding box, and an initial target detection network model of the active random model is generated based on the prediction result, including: Encode the ROI feature, and input the encoded ROI feature into the decoder of the active random model to obtain a prediction result of the first bounding box, wherein the bounding box numbered i selects the top k predictions with the minimum cost matching the first bounding box through the optimal allocation method, and accumulates and averages the prediction losses, where i is a positive integer less than or equal to N; The bounding box is repeatedly detected at L time steps and trained into an initial target detection network model of the active random model, where L is a positive integer greater than or equal to 1.
6. The method according to claim 5, characterized in that The prediction result of the first bounding box is predicted for each of the three channels by the multi-stages detection algorithm. According to the random characteristics of the active random model, set prediction is used for processing, and the set prediction of the first step is performed to obtain the prediction result and set prediction loss of this stage. The optimal allocation method is used to select the top k predictions with the minimum cost matching the first bounding box, and the prediction loss is accumulated and averaged. The prediction result is (bpri, bpgi, bpbi), and the loss function is minimized and accumulated in the prediction process; L Train =∑ T s=1 (bpr i -b i ) 2 +(bpg i -b i ) 2 +(bpb i -b i ) 2 , where s is the step size and T is the total step size.
7. The method according to claim 5, characterized in that The step of repeatedly detecting the bounding box at L time steps and training the initial target detection network model of the active random model includes: The step size L is set, the loss function of the three channels is calculated multiple times, and a three-channel target detection network model based on the active random model is established.
8. The method according to claim 1, characterized in that The dynamic box corresponding to each channel includes an update strategy, which is used to remove the prediction results whose sampling results are lower than a threshold α (0<α<1) and replace them with the update box. The update box and the output box that meets the threshold are combined into an update group box.
9. The method according to claim 8, characterized in that The performing target detection on the input image based on the actual target detection network model comprises: The three channels are iteratively sampled, and the ROI features are respectively cut from the feature map through the update group box of each channel, and the ROI features are selected and then enter the detection head, and the iteration is performed based on the reverse process of the active random model. The parameters are shared in the iteration, and the sampling step size S is set according to the detection speed and accuracy requirements, and the sampling step size of each channel is the same; The detection results of each channel are fused with target information using feature fusion and matching rules to obtain the target detection result bs i .
10. A three-channel target detection device based on an active random model, characterized in that: The device comprises: A conversion module, used for acquiring a first target image, and performing conversion processing on the first target image to obtain a second target image, wherein the second target image is a three-channel image; An extraction module, configured to perform feature extraction on the second target image to obtain a feature map, and to add noise to the first bounding box based on the forward pass of the active random model to obtain a noise box; A cropping module, used for cropping ROI features from the feature map through the noise box, performing encoding and decoding processing on the ROI features to obtain a prediction result of the first bounding box, and generating an initial target detection network model of the active random model based on the prediction result; A generation module, configured to perform denoising based on the reverse of the active random model to obtain a second bounding box, and to generate a dynamic box for each of the three channels based on a Gaussian distribution, wherein the total number of dynamic boxes for each channel is the same and randomly distributed; A detection module is used to determine an actual target detection network model of the active random model based on the dynamic box, and perform target detection on the input image based on the actual target detection network model.
Citation Information
Cited By
Multivariable self-adaptive belt deviation rectification control method and system based on intelligent visual perception
CN119370519A
Road infrastructure monitoring method and system based on intelligent perception
CN120877091A
