Model training method, target detection method, device, equipment, medium and product

Abstract

The present disclosure provides a model training method, a target detection method, an apparatus, a device, a medium and a product. The method includes: obtaining multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; based on the first image to be detected and the first detection result image, a first model is trained, and the first model is used to detect abnormal targets in the first detection image. In this way, the trained first model can detect abnormal targets in the detection area, and the trained first model can reduce the amount of model parameters and the amount of calculation, so that the trained first model can be applied to edge devices.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and image processing, and in particular to a model training method, target detection method, device, equipment, medium and product. Background Art

[0002] With the development of artificial intelligence technology, image processing and target detection technologies have been widely used in intelligent monitoring, autonomous driving, industrial automation and other fields. Due to the limited processing power and storage resources of edge devices, the execution efficiency of target detection models and the lightweight design of models have become crucial.

[0003] The target detection models in related technologies are usually complex in design, have a large number of parameters, and have high requirements for computing resources, which makes it difficult for these models to run efficiently on edge devices. The deployment and execution of these models on edge devices are usually constrained by insufficient computing power, resulting in the inability to achieve real-time target detection, reducing detection speed and user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a model training method, a target detection method, an apparatus, a device, a medium and a product, so that the trained first model can detect abnormal targets in the detection area and reduce the model parameters and calculation amount.

[0005] According to a first aspect of an embodiment of the present disclosure, a model training method is provided, including: acquiring multiple groups of sample data under different detection scenarios, each group of sample data including a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both including a detection area and a non-detection area, and the detection area of ​​the first detection result image including marking information of abnormal targets; based on the first image to be detected and the first detection result image, training a first model, the first model being used to detect abnormal targets in the first detection image.

[0006] In some embodiments, the first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

[0007] In some embodiments, the first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; a convolution operation is performed on the input of the first module through the first convolutional layer to obtain a first sample feature map; a nonlinear transformation is performed on the first sample feature map through the first activation layer to obtain a second sample feature map; a third sample feature map is obtained by multiplying the first sample feature map with the second sample feature map; a segmentation operation is performed on the third sample feature map through the first segmentation unit to obtain a fourth sample feature map and a fifth sample feature map; a segmentation operation is performed on the fifth sample feature map through the second segmentation unit to obtain a sixth sample feature map and a seventh sample feature map; a convolution operation is performed on the seventh sample feature map through the second convolutional layer to obtain an eighth sample feature map; the eighth sample feature map and the sixth sample feature map are spliced ​​through the first splicing layer to obtain a ninth sample feature map ; Performing a segmentation operation on the ninth sample feature map through the third segmentation unit to obtain a tenth sample feature map and an eleventh sample feature map; Performing a convolution operation on the eleventh sample feature map through the third convolution layer to obtain a twelfth sample feature map; Splicing the twelfth sample feature map, the tenth sample feature map and the fourth sample feature map through the second splicing layer to obtain a thirteenth sample feature map; Performing a nonlinear transformation on the thirteenth sample feature map through the second activation layer to obtain a fourteenth sample feature map; By multiplying the thirteenth sample feature map with the fourteenth sample feature map, the sample feature map output by the first module is obtained; wherein, the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0008] In some embodiments, multiple groups of sample data under different detection scenarios are obtained, including: obtaining first images corresponding to different detection scenarios, the first image including detection areas and non-detection areas distinguished by different pixel values; obtaining a second image corresponding to the current video stream to be detected under each detection scenario; adding pixel values ​​of the second image and the first image under the same detection scenario to obtain a first image to be detected under a different detection scenario; marking abnormal targets included in the detection area of ​​the first image to be detected to obtain a first detection result image corresponding to the first image to be detected.

[0009] In some embodiments, obtaining first images corresponding to different detection scenarios includes: obtaining a third image based on video streams under different detection scenarios; and generating a first image corresponding to the third image based on a language segmentation model when the shape of the detection scene corresponding to the third image is a regular shape.

[0010] In some embodiments, the above method also includes: when the shape of the detection scene corresponding to the third image is an irregular shape, marking the non-detection area in the third image to obtain a first image corresponding to the third image, wherein the pixel value corresponding to the non-detection area in the first image obtained after marking is different from the pixel value of the detection area in the first image, and the detection area in the first image is the area where the third image is not marked.

[0011] In some embodiments, the first model is trained based on the first image to be detected and the first detection result image, including: inputting the first image to be detected into the first model to obtain a first output image; determining the loss value of the first output image and the first detection result image according to a preset loss function; and adjusting the parameters of the first model based on the loss value until the first model converges.

[0012] According to a second aspect of an embodiment of the present disclosure, there is provided a target detection method, comprising: acquiring a second image to be detected, the second image to be detected including a detection area and a non-detection area; inputting the second image to be detected into a trained first model to obtain a second detection result image corresponding to the second image to be detected; wherein the first model is trained based on multiple groups of sample data under different detection scenarios, each group of sample data including a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both including a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

[0013] In some embodiments, the first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

[0014] In some embodiments, the first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; the first convolutional layer performs a convolution operation on the input of the first module to obtain a first feature map; the first activation layer performs a nonlinear transformation on the first feature map to obtain a second feature map; the third feature map is obtained by multiplying the first feature map with the second feature map; the first segmentation unit performs a segmentation operation on the third feature map to obtain a fourth feature map and a fifth feature map; the second segmentation unit performs a segmentation operation on the fifth feature map to obtain a sixth feature map and a seventh feature map; the second convolutional layer performs a convolution operation on the seventh feature map to obtain an eighth feature map; the eighth feature map and the sixth feature map are spliced ​​through the first splicing layer to obtain a ninth feature map; The third segmentation unit performs a segmentation operation on the ninth feature map to obtain a tenth feature map and an eleventh feature map; the third convolutional layer performs a convolution operation on the eleventh feature map to obtain a twelfth feature map; the second splicing layer splices the twelfth feature map, the tenth feature map and the fourth feature map to obtain a thirteenth feature map; the second activation layer performs a nonlinear transformation on the thirteenth feature map to obtain a fourteenth feature map; the feature map output by the first module is obtained by multiplying the thirteenth feature map with the fourteenth feature map; wherein the convolution kernel of the first convolutional layer is smaller than the convolution kernel of the second convolutional layer, the convolution kernel of the second convolutional layer is the same as the convolution kernel of the third convolutional layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0015] In some embodiments, obtaining a second image to be detected includes: obtaining a first image corresponding to a first detection scene, the first image including a detection area and a non-detection area distinguished by different pixel values; obtaining a fourth image corresponding to a first video stream to be detected in the first detection scene; and performing a pixel value addition operation on the fourth image and the first image corresponding to the first detection scene to obtain a second image to be detected in the first detection scene.

[0016] In some embodiments, obtaining a first image corresponding to a first detection scene includes: determining a second detection scene that is the same as the first detection scene from different detection scenes; and determining the first image corresponding to the second detection scene in the different detection scenes as the first image corresponding to the first detection scene.

[0017] According to a third aspect of an embodiment of the present disclosure, a model training device is provided, comprising: a first acquisition module, configured to acquire multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; a training module, configured to train a first model based on the first image to be detected and the first detection result image, the first model being used to detect abnormal targets in the first detection image.

[0018] In some embodiments, the first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

[0019] In some embodiments, the first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; a convolution operation is performed on the input of the first module through the first convolutional layer to obtain a first sample feature map; a nonlinear transformation is performed on the first sample feature map through the first activation layer to obtain a second sample feature map; a third sample feature map is obtained by multiplying the first sample feature map with the second sample feature map; a segmentation operation is performed on the third sample feature map through the first segmentation unit to obtain a fourth sample feature map and a fifth sample feature map; a segmentation operation is performed on the fifth sample feature map through the second segmentation unit to obtain a sixth sample feature map and a seventh sample feature map; a convolution operation is performed on the seventh sample feature map through the second convolutional layer to obtain an eighth sample feature map; the eighth sample feature map and the sixth sample feature map are spliced ​​through the first splicing layer to obtain a ninth sample feature map ; Performing a segmentation operation on the ninth sample feature map through the third segmentation unit to obtain a tenth sample feature map and an eleventh sample feature map; Performing a convolution operation on the eleventh sample feature map through the third convolution layer to obtain a twelfth sample feature map; Splicing the twelfth sample feature map, the tenth sample feature map and the fourth sample feature map through the second splicing layer to obtain a thirteenth sample feature map; Performing a nonlinear transformation on the thirteenth sample feature map through the second activation layer to obtain a fourteenth sample feature map; By multiplying the thirteenth sample feature map with the fourteenth sample feature map, the sample feature map output by the first module is obtained; wherein, the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0020] In some embodiments, the first acquisition module includes: a first acquisition unit, configured to: acquire first images corresponding to different detection scenarios, the first image including detection areas and non-detection areas distinguished by different pixel values; a second acquisition unit, configured to: acquire the second image corresponding to the current video stream to be detected in each detection scenario; a first determination unit, configured to: add pixel values ​​of the second image and the first image in the same detection scenario to obtain the first image to be detected in different detection scenarios; a marking unit, configured to: mark abnormal targets included in the detection area in the first image to be detected, and obtain a first detection result image corresponding to the first image to be detected.

[0021] In some embodiments, the first acquisition unit is configured to perform the following steps: acquire a third image based on video streams under different detection scenarios; and when the shape of the detection scene corresponding to the third image is a regular shape, generate a first image corresponding to the third image based on a language segmentation model.

[0022] In some embodiments, the first acquisition unit is further configured to perform the following steps: when the shape of the detection scene corresponding to the third image is an irregular shape, annotate the non-detection area in the third image to obtain a first image corresponding to the third image, wherein the pixel value corresponding to the non-detection area in the first image obtained after annotation is different from the pixel value of the detection area in the first image, and the detection area in the first image is an area that is not annotated in the third image.

[0023] In some embodiments, the training module is further configured to perform the following steps: inputting the first image to be detected into the first model to obtain a first output image; determining the loss value of the first output image and the first detection result image according to a preset loss function; and adjusting the parameters of the first model based on the loss value until the first model converges.

[0024] According to a fourth aspect of an embodiment of the present disclosure, a target detection device is provided, comprising: a second acquisition module, configured to acquire a second image to be detected, the second image to be detected including a detection area and a non-detection area; a determination module, configured to input the second image to be detected into a trained first model to obtain a second detection result image corresponding to the second image to be detected; wherein the first model is trained based on multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, both the first image to be detected and the first detection result image include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

[0025] In some embodiments, the first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

[0026] In some embodiments, the first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; the first convolutional layer performs a convolution operation on the input of the first module to obtain a first feature map; the first activation layer performs a nonlinear transformation on the first feature map to obtain a second feature map; the third feature map is obtained by multiplying the first feature map with the second feature map; the first segmentation unit performs a segmentation operation on the third feature map to obtain a fourth feature map and a fifth feature map; the second segmentation unit performs a segmentation operation on the fifth feature map to obtain a sixth feature map and a seventh feature map; the second convolutional layer performs a convolution operation on the seventh feature map to obtain an eighth feature map; the eighth feature map and the sixth feature map are spliced ​​through the first splicing layer to obtain a ninth feature map; The third segmentation unit performs a segmentation operation on the ninth feature map to obtain a tenth feature map and an eleventh feature map; the third convolutional layer performs a convolution operation on the eleventh feature map to obtain a twelfth feature map; the second splicing layer splices the twelfth feature map, the tenth feature map and the fourth feature map to obtain a thirteenth feature map; the second activation layer performs a nonlinear transformation on the thirteenth feature map to obtain a fourteenth feature map; the feature map output by the first module is obtained by multiplying the thirteenth feature map with the fourteenth feature map; wherein the convolution kernel of the first convolutional layer is smaller than the convolution kernel of the second convolutional layer, the convolution kernel of the second convolutional layer is the same as the convolution kernel of the third convolutional layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0027] In some embodiments, the second acquisition module includes: a third acquisition unit, configured to: acquire a first image corresponding to the first detection scene, the first image including a detection area and a non-detection area distinguished by different pixel values; a fourth acquisition unit, configured to: acquire a fourth image corresponding to the first video stream to be detected in the first detection scene; a second determination unit, configured to: perform a pixel value addition operation on the fourth image and the first image corresponding to the first detection scene to obtain a second image to be detected in the first detection scene.

[0028] In some embodiments, the third acquisition unit is configured to perform the following steps: determine a second detection scene that is the same as the first detection scene from different detection scenes; and determine a first image corresponding to the second detection scene in the different detection scenes as the first image corresponding to the first detection scene.

[0029] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing computer programs or instructions; wherein the processor executes the computer program or instructions to implement the steps in the method of the above-mentioned first aspect, second aspect and possible implementations thereof.

[0030] According to the sixth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, wherein the storage medium stores a computer program or instructions. When the computer program or instructions in the storage medium are executed by a processor, the steps in the method in the above-mentioned first aspect, second aspect and possible implementations thereof are implemented.

[0031] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implements the steps of the method in the above-mentioned first aspect, second aspect and possible implementations thereof.

[0032] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0033] Acquire multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; based on the first image to be detected and the first detection result image, train a first model, and the first model is used to detect abnormal targets in the first detection image. In this way, the first model is trained based on multiple groups of sample data under different detection scenarios, so that the trained first model can detect abnormal targets in the detection area, and the trained first model can reduce the model parameter amount and calculation amount, so that the trained first model can be applied to edge devices, improve the throughput efficiency and accuracy of the first model on edge devices, reduce the calculation amount and power consumption, and enhance the robustness to the environment.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0036] Figure 1 is a flow chart of a model training method according to an exemplary embodiment;

[0037] Figure 2is a schematic diagram of a model training process according to an exemplary embodiment;

[0038] Figure 3 is a schematic diagram of a framework of a first model according to an exemplary embodiment;

[0039] Figure 4 is a structural schematic diagram of a first module according to an exemplary embodiment;

[0040] Figure 5 is a schematic diagram showing a second image and a first image to be detected according to an exemplary embodiment;

[0041] Figure 6 is a schematic diagram showing a process of determining a first image to be detected according to an exemplary embodiment;

[0042] Figure 7 is a flow chart of a target detection method according to an exemplary embodiment;

[0043] Figure 8 is a block diagram of a model training device according to an exemplary embodiment;

[0044] Fig. 9 is a block diagram of a target detection device according to an exemplary embodiment;

[0045] Fig.10 is a structural block diagram of an electronic device 1000 according to an exemplary embodiment;

[0046] Fig.11 is a block diagram of a server 1100 according to an exemplary embodiment. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0048] In recent years, with the development of artificial intelligence technology, image processing and target detection technologies have been widely used in many fields such as intelligent monitoring, autonomous driving, and industrial automation. Especially in the edge computing environment, the application of these technologies is particularly critical, because edge devices (such as mobile devices and embedded devices with limited processing power and storage resources) usually face the dual challenges of limited processing power and storage resources. In edge devices, the execution efficiency of algorithms and the lightweight design of models have become the focus of research, especially in the fields of abnormal target detection and intelligent monitoring, it is particularly important to develop lightweight and efficient target detection technology.

[0049] In order to meet the above needs, lightweight target detection technology has emerged, whose core goal is to significantly reduce computational complexity and resource consumption while maintaining efficient performance. This is particularly important for edge devices because they lack the ability to perform highly complex neural network operations. Lightweight target detection technology aims to achieve fast and accurate target detection by optimizing model structure and reducing computing requirements, thereby improving operational efficiency without reducing response speed. The following is an introduction to target detection models in some related technologies:

[0050] 1. MobileNets Series

[0051] The MobileNets series is one of the lightweight models. It significantly reduces the amount of computation and parameters of the model through depthwise separable convolution. Among them, the depthwise separable convolution decomposes the traditional convolution operation into two parts: depthwise convolution and point convolution. This design effectively reduces the computational cost and makes MobileNets suitable for running on mobile devices and embedded devices. With the iteration of versions, the MobileNets series further improves performance and efficiency by introducing new structures and optimization strategies, such as inverse residual structures and automated machine learning techniques.

[0052] 2. Squeeze the network

[0053] SqueezeNet achieves parameter compression through innovative network structure design. It introduces the "Fire" module, which contains a compression layer (squeeze layer) and an expansion layer (expand layer). The compression layer uses a 1x1 convolution kernel to reduce the depth of the input feature map, while the expansion layer expands the depth of the feature map through a combination of 1x1 and 3x3 convolution kernels. This design reduces the number of model parameters while maintaining good performance. The core of SqueezeNet's design is to achieve model lightweight through a sophisticated network structure design, rather than simply reducing the network depth or width.

[0054] 3. YOLO series

[0055] The YOLO series stands out in the field of target detection through its end-to-end detection method, which regards target detection as regression analysis and can accurately predict the category and location of the target in a single network forward propagation process. This design improves the detection speed, allowing YOLO to achieve real-time target detection while maintaining high accuracy. As technology evolves, the YOLO series further integrates lightweight network structures such as MobileNets and SqueezeNet to optimize performance. Faced with batch deployment with performance limitations of edge devices, YOLO can use lightweight convolution techniques such as partial convolution (PConv) for in-depth lightweight optimization to improve efficiency while improving detection results. However, the existing YOLO series has the problem of insufficient ability to adapt to changes in the detection area online when facing an operating environment with lens changes, which restricts its promotion in practical applications.

[0056] The target detection model in the above-mentioned related technologies has the following disadvantages:

[0057] 1. Large number of parameters

[0058] The above-mentioned target detection models are usually complex in design, with a large number of parameters and high demand for computing resources. This makes their deployment and execution on edge devices constrained by insufficient computing power, making it difficult to run efficiently, resulting in the inability to perform real-time target detection, affecting the response speed and user experience of the model when it is applied in the non-training stage (which may include the model verification stage and the model inference stage).

[0059] 2. It is difficult to balance real-time performance and accuracy

[0060] Achieving high-accuracy abnormal target detection and maintaining real-time processing speed on edge devices is a major challenge. In order to enhance the accuracy of detection, the above target detection models usually need to increase their complexity and number of parameters, which inevitably reduces the reasoning speed of the model. Especially in application scenarios where edge devices require fast response, such as traffic monitoring and autonomous driving, the real-time requirements are relatively strict. The above target detection models are difficult to meet the real-time requirements of these application scenarios without losing accuracy.

[0061] 3. Poor adaptability to scene changes

[0062] The above-mentioned object detection models usually use frames or lines to specify the area when dealing with dynamically changing detection environments, such as changes in the detection area caused by the movement of the surveillance camera. It is difficult to adapt to the detection area with complex shapes, which leads to missed detections and false detections. Therefore, this limitation has greatly restricted the popularity and effectiveness of these models in changing practical application scenarios.

[0063] In summary, although relevant technologies have made progress in the field of target detection, they still face some challenges and limitations in optimizing computing resource utilization, balancing real-time and accuracy, and coping with the complexity of actual application environments, which requires further research and improvement.

[0064] Based on this, the embodiment of the present disclosure obtains multiple groups of sample data in different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, both the first image to be detected and the first detection result image include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets. Based on the first image to be detected and the first detection result image included in each group of sample data, a first model is trained, and the first model is used to detect abnormal targets in the first detection image. In this way, the first model is trained based on multiple groups of sample data in different detection scenarios, so that the trained first model can detect abnormal targets in the detection area, and the trained first model can reduce the amount of model parameters and the amount of calculation, so that the trained first model can be applied to edge devices.

[0065] Figure 1 FIG. 1 is a flow chart of a model training method according to an exemplary embodiment. Figure 1 As shown, the model training method can be used in edge devices, and the method mainly includes the following steps:

[0066] In step 101, multiple groups of sample data under different detection scenarios are obtained.

[0067] In step 102, a first model is trained based on a first image to be detected and a first detection result image.

[0068] Here, the edge device can be understood as a device with limited processing power and storage resources, such as a computer with poor performance, an electronic device with limited storage, etc., which is not limited in the embodiments of the present disclosure. Each set of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected. Among them, both the first image to be detected and the first detection result image include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets, which can be text information, graphic box information, etc., which is not limited in the embodiments of the present disclosure. The first model is used to detect abnormal targets in the first detection image. Abnormal targets can be objects with abnormalities, for example, in traffic monitoring scenarios, vehicles with speeds higher than the maximum speed limit, pedestrians on highways, etc. Whether they are abnormal targets can be determined based on the detection scenario, which is not limited in the embodiments of the present disclosure.

[0069] It is understandable that the detection scene can be predetermined, or the corresponding detection scene can be increased or decreased according to the specific detection process, which is not limited in the embodiments of the present disclosure. The detection scene can be, for example, a traffic monitoring scene, an automatic driving scene, etc.

[0070] In some embodiments, for step 101, before training the first model, it is necessary to first collect multiple sets of sample data in different detection scenarios. In some embodiments, the process of collecting multiple sets of sample data may include the following steps: step a, collecting the first image to be detected in different detection scenarios; step b, marking the abnormal target included in the detection area of ​​the first image to be detected, and determining the first detection result image; step c, forming the first image to be detected and the first detection result image corresponding to the first image to be detected into a set of sample data, thereby obtaining multiple sets of sample data.

[0071] In some embodiments, the detection area and non-detection area included in the first image to be detected in the above step a can be distinguished by mask processing. For example, the non-detection area can be covered by a mask so that the non-detection area is invisible and the detection area is visible.

[0072] In some embodiments, for the above step b, when the non-detection area included in the first image to be detected is invisible and the detection area is visible, the first detection result image can be obtained by marking the abnormal targets included in the detection area visible in the first image to be detected.

[0073] In some embodiments, for step 102, after obtaining multiple groups of sample data in different detection scenarios, the first model can be trained through the multiple groups of sample data, and the training of the first model may include the following steps: step d, model parameter initialization, that is, setting the initial values ​​of the parameters in the first model, for example, some parameters can be set to 0 and other parameters can be set to 1; step e, training the first model, that is, based on the output image of the first model and the true value (that is, the first detection result image corresponding to the input image of the first model), determining the loss value according to a preset loss function, and adjusting the parameters in the first model based on the loss value until the first model converges; step f, saving the training results of the first model, which can be the values ​​of the parameters involved in the training process of the first model, for example, the model parameters when the first model converges; for example, the model parameters and loss values ​​after each training in the training process of the first model, the current number of iterations, etc., are not limited to this in the embodiments of the present disclosure.

[0074] In some embodiments, the above-mentioned preset loss function may include at least one of the loss functions such as cosine loss function, square loss function, cross entropy loss function, logarithmic loss function and mean absolute error loss function, and the embodiments of the present disclosure are not limited to this.

[0075] Figure 2 FIG. 1 is a schematic diagram showing a model training process according to an exemplary embodiment. Figure 2 As shown, the main Figure 1 Steps a to f involved in steps 101 to 102 are described below. The process mainly includes the following steps:

[0076] Step 201: collect first images to be detected in different detection scenarios.

[0077] Step 202: determine a first detection result image.

[0078] Step 203: determine multiple groups of sample data.

[0079] Step 204: Initialize model parameters.

[0080] Step 205: training the first model.

[0081] Step 206, saving the training result of the first model.

[0082] In some embodiments, for the above Figure 1The first model described in the embodiment may be a model obtained by improving the YOLOv10n model, wherein the YOLOv10n model includes a first backbone network, a first neck network and a head network, and the first model includes a second backbone network, a second neck network and a head network. The second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, and the second neck network is obtained by replacing the C2f module in the first neck network with the first module. The number of channels for convolution performed by the C2f module is greater than the number of channels for convolution performed by the first module.

[0083] It can be understood that since the number of channels for the C2f module to perform convolution is greater than the number of channels for the first module to perform convolution, the first module is lighter and more efficient. By replacing the C2f module in the first backbone network included in the YOLOv10n model with the first module, and replacing the C2f module in the first neck network included in the YOLOv10n model with the first module, while keeping the other modules in the first backbone network, the other modules in the first neck network, and the modules in the head network in the YOLOv10n model unchanged, the first model can be obtained without changing the overall architecture of the YOLOv10n model. In this way, the first module in the first model can reduce the number of channels for performing convolution, reduce channel calculations, model parameter amounts, calculation amounts, and memory access amounts, thereby optimizing the utilization of computing resources.

[0084] Figure 3 FIG. 1 is a schematic diagram of a framework of a first model according to an exemplary embodiment. Figure 3 As shown, the main Figure 1 The framework of the first model in is described as follows: the first model includes a second backbone network, a second neck network and a head network.

[0085] Among them, the second backbone network includes 11 modules in series, namely: 0. convolution layer (convolution layer, Conv), 1. Conv, 2. ARCM, 3. Conv, 4. anti-redundancy convolution module (anti-redundancy convolution module, ARCM), that is: Figure 1 The first module described in the embodiment, 5. spatial-channel decoupled downsampling (SCDown), 6. ARCM, 7. SCDown, 8. ARCM, 9. fast-spatial pyramid pooling (SPPF), 10. partial self-attention (PSA).

[0086] The second neck network includes 12 modules in series, namely: 11. concatenation (Concat), 12. Concat, 13. ARCM, 14. Upsample, 15. SPPF, 16. ARCM, 17. Conv, 18. Concat, 19. ARCM, 20. SCDown, 21. Concat, 22. C2fCIB (compact inverted block).

[0087] The head network includes 6 modules, namely: 3 one-to-one prediction heads (one-to-one head) and 3 one-to-many prediction heads (one-to-many head). Among them, each ARCM in the second neck network is connected to one one-to-one head and one one-to-many head in the head network respectively.

[0088] It can be understood that the above-mentioned second backbone network can extract features from the input image of the first model through multiple convolutional layers and activation functions, and obtain feature maps of different scales and depths containing information such as the position, shape, texture, etc. of the object in the input image. The second neck network is used to process and fuse the features extracted by the second backbone network. The convolutional layers, feature fusion modules, and fast-spatial pyramid pooling networks included in the second neck network can improve the representation ability and adaptability of the features. The head network can use the detection box regression layer and the category prediction layer to predict the position and category of abnormal targets in the input image.

[0089] In some embodiments, the head network of the above-mentioned first model is trained one-to-onehead and one-to-many head simultaneously during the model training process. In the non-training stage of the first model, only the one-to-onehead result can be output to reduce the use of parameters.

[0090] For the above Figure 3The modules included in the first model shown in the figure are: Conv is used to perform convolution operations, ARCM is used to optimize the computational efficiency in the traditional convolution process, SCDown is used to decouple space and channel, SPPF is used to process input of any size and generate output of fixed length, PSA is an attention mechanism module used in YOLOv10, Concat is used to perform splicing operations, Upsample is used to enlarge the size of the feature map, and the role of C2f in the C2fCIB module is to perform feature conversion, branch processing and feature fusion on the input data through two convolutional layers; the role of CIB is to use efficient deep convolution for spatial feature mixing, and use point convolution for channel feature mixing.

[0091] It should be noted that: Figure 3 The frames shown in the figure are for illustrative purposes only and are not intended to be limiting.

[0092] In some embodiments, for Figure 3 The framework of the first model shown in can be replaced by the first module Figure 3 The convolutional layers in are used to further optimize the first model and reduce its parameter number, computational complexity, and memory access frequency.

[0093] In some embodiments, for the above Figure 3The ARCM (i.e., the first module) in the example may include: a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer, and a second activation layer connected in series in sequence. The functions of the above layers are: performing a convolution operation on the input of the first module through the first convolutional layer to obtain a first sample feature map; performing a nonlinear transformation on the first sample feature map through the first activation layer to obtain a second sample feature map; performing a multiplication operation on the first sample feature map and the second sample feature map to obtain a third sample feature map; performing a segmentation operation on the third sample feature map through the first segmentation unit to obtain a fourth sample feature map and a fifth sample feature map; performing a segmentation operation on the fifth sample feature map through the second segmentation unit to obtain a sixth sample feature map and a seventh sample feature map; performing a convolution operation on the seventh sample feature map through the second convolutional layer to obtain an eighth sample feature map; performing a multiplication operation on the first sample feature map and the second sample feature map through the first splicing layer The eighth sample feature map and the sixth sample feature map are spliced ​​to obtain a ninth sample feature map; the ninth sample feature map is segmented by the third segmentation unit to obtain a tenth sample feature map and an eleventh sample feature map; the eleventh sample feature map is convolved by the third convolution layer to obtain a twelfth sample feature map; the twelfth sample feature map, the tenth sample feature map and the fourth sample feature map are spliced ​​by the second splicing layer to obtain a thirteenth sample feature map; the thirteenth sample feature map is nonlinearly transformed by the second activation layer to obtain a fourteenth sample feature map; the sample feature map output by the first module is obtained by multiplying the thirteenth sample feature map with the fourteenth sample feature map.

[0094] Here, the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, and the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer. In one example, the convolution kernel of the first convolution layer may be 1×1, and the convolution kernel of the second convolution layer may be 3×3.

[0095] Here, the activation functions adopted by the first activation layer and the second activation layer are the same or different. In some embodiments, the activation function can be a rectified linear unit (Leaky ReLU), Sigmoid, ReLU, GELU, PReLU, etc., which is not limited in the embodiments of the present disclosure.

[0096] Here, the first segmentation unit, the second segmentation unit, and the third segmentation unit have the same or different segmentation ratios, and the first splicing layer and the second splicing layer have the same or different structures. In one example, the segmentation ratio can be 0.5, that is, evenly divided, or can be other small values ​​greater than 0 and less than 1, which is not limited in the embodiments of the present disclosure.

[0097] It can be understood that the first module (i.e., the first convolution layer, the first activation layer, the first segmentation unit, the second segmentation unit, the second convolution layer, the first splicing layer, the third segmentation unit, the third convolution layer, the second splicing layer, and the second activation layer are connected in series. Figure 3 The input of each layer to the first module (such as, Figure 3 The output of module 3.Conv, module 5.SCDown, module 7.SCDown (not limited thereto) executes the steps of the corresponding layers to obtain the output of the first module. In this way, the input data is feature-enhanced through the activation functions corresponding to the first activation layer and the second activation layer, and the feature local channel convolution is realized through the first segmentation unit, the second segmentation unit, the second convolution layer, the third segmentation unit and the third convolution layer, which can optimize the computational efficiency in the traditional convolution process. Compared with the ordinary convolution, which fully convolves the input features, the feature channels contain a lot of redundant information, resulting in unnecessary calculation duplication. The first module selects some channels to perform convolution through segmentation technology, keeps the remaining channels in their original state and merges them with the channels after convolution to restore the original input feature structure, which not only reduces the computational burden, but also retains the integrity of the input features.

[0098] Figure 4 FIG. 1 is a schematic diagram showing a structure of a first module according to an exemplary embodiment. Figure 4 As shown, the main Figure 3 The structure of the first module in is described as follows: the first module includes the following connected in series: Conv k=1 (e.g., the first convolution layer with a convolution kernel of 1×1), Leaky ReLU (the activation function used in the first activation layer), a multiplication operation (Mul), a first segmentation unit (Split(2), which evenly segments the output of the first activation layer), a second segmentation unit (Split(2)), Convk=3 (e.g., the second convolution layer with a convolution kernel of 3×3), Concat (i.e., the first splicing layer), a third segmentation unit (Split(2)), Conv k=3 (e.g., the third convolution layer with a convolution kernel of 3×3), Concat (i.e., the second splicing layer), Leaky ReLU (the activation function used in the second activation layer), a multiplication operation (Mul), and an output.

[0099] like Figure 4As shown in the figure, the input is a feature map represented by H*W*C_in (denoted as A), h is the height, w is the width, and c is the number of channels. First, the first 1x1 convolutional layer is used to perform feature interaction between channels on the input A, and the number of output channels is adjusted to meet different feature dimension requirements to obtain the first sample feature map (denoted as A1, represented by H*W*C_Out); then, the output A1 of Conv k=1 is passed through the Leaky The ReLU activation function is used for nonlinear transformation to obtain the second sample feature map (denoted as A2), and A1 is multiplied by A2 to enhance the features and obtain the third sample feature map (denoted as A3); the first segmentation unit performs a segmentation operation on A3, and A3 is evenly divided into two parts along the channel dimension to obtain the fourth sample feature map (denoted as A4, represented by H*W*C_Out*0.5) and the fifth sample feature map (denoted as A5, represented by H*W*C_Out*0.5), and A4 is retained, and A5 is sent to the second segmentation unit. The second segmentation unit performs a segmentation operation on A5, and A5 is evenly divided into two parts along the channel dimension to obtain the sixth sample feature map (denoted as A6, represented by H*W*C_Out*0.25) and the seventh sample feature map (denoted as A7, represented by H*W*C_Out*0.25), and A6 is retained, and A7 is sent to Conv k=3 for deep feature extraction, and the eighth sample feature map (denoted as A8) is obtained; then, A8 and A6 are concatenated through the first concatenation layer to restore the channel dimension to 0.5, and the ninth sample feature map (denoted as A9) is obtained; A9 is segmented through the third segmentation unit to obtain the tenth sample feature map (denoted as A10, represented by H*W*C_Out*0.25) and the eleventh sample feature map (denoted as A11, represented by H*W*C_Out*0.25), and A11 is convolved through the third convolutional layer to obtain the twelfth sample feature map (denoted as A12); then, A12, A10 and A4 are concatenated through the second concatenation layer to restore the complete feature structure, and the thirteenth sample feature map (denoted as A13) is obtained; through the Leaky The ReLU activation function performs a nonlinear transformation on A13 to obtain the fourteenth sample feature map (denoted as A14); finally, by multiplying A13 and A14, the features are enhanced again to obtain the sample feature map output by the first module (represented by H*W*C_Out).

[0100] Thus, through the above Figure 4 The first module shown can achieve the following effects:

[0101] 1. Reduce computational complexity: The first module reduces the number of floating point operations (FLOPs) by reducing channel calculations. When split is set to 1 / 2, the channel calculation of the convolution in the first module is 3 / 16 of the normal case.

[0102] 2. Reduce the number of memory accesses: The first module reduces the number of memory accesses by reducing channel calculations, which is more conducive to the deployment of edge devices with limited input / output (I / O).

[0103] 3. Balance between real-time and accuracy: The first model constructed based on the first module has fewer parameters and can improve the detection effect.

[0104] In some embodiments, the following Table 1 compares the performance of the first model with the target detection model in the related art.

[0105] Table 1

[0106]

[0107]

[0108] Among them, YOLOv6-3.0-N, YOLOv8-N and YOLOv10-N (ie, YOLOv10n) in Table 1 are different versions of the YOLO series in the related art.

[0109] It can be seen from Table 1 that the performance of the first model is better than that of other models.

[0110] In some embodiments, in step 101, obtaining multiple groups of sample data under different detection scenarios may include the following steps: Step 1, obtaining first images corresponding to different detection scenarios, wherein the first image includes detection areas and non-detection areas distinguished by different pixel values; Step 2, obtaining a second image corresponding to the current video stream to be detected under each detection scenario, and adding pixel values ​​of the second image and the first image under the same detection scenario to obtain the first image to be detected under different detection scenarios; Step 3, marking abnormal targets included in the detection area of ​​the first image to be detected to obtain a first detection result image corresponding to the first image to be detected.

[0111] It can be understood that for step 1 and step 2, for different detection scenes, the first images (also called mask images) corresponding to different detection scenes can be obtained respectively by semantic segmentation method, manual smearing method or other methods. Since the first image includes detection areas and non-detection areas distinguished by different pixel values, the first image can clearly distinguish the detection areas and non-detection areas. Next, the second image corresponding to the current video stream to be detected in each detection scene is obtained. For example, the second image can be obtained by screenshot from the video stream to be detected; finally, the second image and the first image in the same detection scene are added by pixel value operation to obtain the first image to be detected in different detection scenes. In the process of adding pixel values, if a certain pixel value obtained after addition exceeds the maximum limit, it can be modified to the maximum limit to avoid errors. In this way, the first image to be detected obtained by the above method also includes detection areas and non-detection areas distinguished by different pixel values.

[0112] It can be understood that for step 3, after obtaining the first images to be detected in different detection scenarios, the abnormal targets included in the detection area of ​​each first image to be detected are marked. For example, the abnormal targets can be marked with a rectangular frame, or they can be marked in other ways. The embodiments of the present disclosure are not limited to this. In this way, the first detection result image corresponding to the first image to be detected can be obtained. Based on the first image to be detected in the same scene and the first detection result image corresponding to the first image to be detected, a set of sample data can be obtained for subsequent model training.

[0113] In one example, the pixel values ​​of the non-detection area (such as red, green, blue (RGB) values) can be set to (255, 255, 255), that is, white, and the pixel values ​​of the detection area can be set to (0, 0, 0), thereby obtaining a first image; in the process of adding pixel values ​​of the second image and the first image under the same detection scene, since the second image is the original image, the part of the image after addition with an RGB value greater than or equal to 255 is called the non-detection area, and the part with an RGB value greater than 255 is uniformly modified to 255 (the upper limit of RGB is 255), and the remaining image parts consistent with the second image are the detection areas, thereby obtaining the first image to be detected.

[0114] In some embodiments, for step 3, the abnormal target to be identified can be accurately marked, for example, the abnormal target is marked with a rectangular frame, and the upper left corner coordinates, lower right corner coordinates of the rectangular frame and the category to which the abnormal target belongs are recorded.

[0115] Figure 5 FIG. 1 is a schematic diagram showing a second image and a first image to be detected according to an exemplary embodiment. Figure 5As shown, the second image and the first image to be detected in the above step 2 are mainly described. Figure 5 As shown in (a) in FIG. 1 , it is the second image corresponding to the current video stream to be detected in a certain detection scene, such as Figure 5 As shown in (b), it is the first image to be detected in the detection scene, and the first image to be detected includes: a non-detection area (such as a mask covered area) and a detection area.

[0116] In some embodiments, for the above step 1, obtaining first images corresponding to different detection scenarios may include: obtaining a third image based on video streams under different detection scenarios; when the shape of the detection scene corresponding to the third image is a regular shape, generating a first image corresponding to the third image based on a language segmentation model.

[0117] Here, the language segmentation model may be a segmentation model (segment anything model, SAM) or other segmentation models, which are not limited in the embodiments of the present disclosure. The regular shape may be a shape that can be marked by a closed contour line, such as a rectangle, a square, or a circle.

[0118] It can be understood that for different detection scenes, based on any video stream under different detection scenes, a third image is obtained by cropping, screenshots, etc., and then, it is determined whether the shape of the detection scene corresponding to the third image is a regular shape. According to whether the shape of the detection scene is a regular shape, different methods are adopted to determine the first image. In the case where the shape of the detection scene corresponding to the third image is a regular shape, the first image corresponding to the third image is generated based on the language segmentation model. In this way, the first image can be generated efficiently and quickly. In the case where the first image is generated, subsequent image processing can be performed based on the first image to eliminate interference from non-detection areas, thereby simplifying the image processing process, improving the operating efficiency of the system, and allowing the first model to focus on the detection area for analysis.

[0119] In some embodiments, for the above-mentioned language segmentation model, the language segmentation model can be quantized by a quantization method, such as Int8 (8-bit integer) quantization, and the first image corresponding to the third image is generated by the quantized language segmentation model. In this way, the quantized language segmentation model can reduce the storage space, memory bandwidth, and computing resource consumption of the language segmentation model.

[0120] In one example, a first image corresponding to a third image can be generated based on the Int8 quantized SAM model, where the first image is a portable network graphics (PNG) mask image in which the color of the detection area (such as a sidewalk, road, etc.) is consistent with the third image and other areas (non-detection areas) are white.

[0121] In some embodiments, for the above-mentioned step 1, obtaining first images corresponding to different detection scenarios may also include: obtaining a third image based on the video stream under different detection scenarios; when the shape of the detection scene corresponding to the third image is an irregular shape, marking the non-detection area in the third image to obtain the first image corresponding to the third image.

[0122] Here, the pixel value corresponding to the non-detection area in the first image obtained after annotation is different from the pixel value of the detection area in the first image, and the detection area in the first image is the area where the third image is not annotated.

[0123] It is understandable that when the shape of the detection scene corresponding to the third image is irregular, the first image generated based on the language segmentation model may not be accurate enough, so the non-detection area in the third image can be annotated by manual painting, manual coloring, and manual marking to obtain the first image corresponding to the third image. In this way, a relatively accurate first image can be generated.

[0124] In some embodiments, for the above step 1, the first images corresponding to different detection scenes can be obtained by predetermining the first image corresponding to each detection scene; or the first image can be obtained in real time when the detection scene changes. In this way, the multiple sets of sample data obtained can involve multiple detection scenes, so that the first model obtained after subsequent training can adapt to changes in detection scenes of any shape.

[0125] In some embodiments, the first image corresponding to the third image is obtained by the above two methods, which can adapt to the needs of various detection scenarios. When the first image is determined, the first image to be detected can be quickly acquired without repeated manual intervention. The above method improves the flexibility and efficiency of operation and greatly simplifies user interaction, making it an ideal choice for quickly deploying and adjusting detection models in changing application environments. It has strong ease of use and reduces the requirements for professional knowledge.

[0126] Figure 6 FIG. 1 is a schematic diagram showing a process of determining a first image to be detected according to an exemplary embodiment. Figure 6 As shown, the steps 1 and 2 (i.e., obtaining the first image to be detected) included in the above step 101 are specifically described. The process mainly includes the following steps:

[0127] Step 601: Acquire a third image based on video streams in different detection scenarios.

[0128] Step 602: Determine whether the shape of the detection scene corresponding to the third image is a regular shape.

[0129] If yes, go to step 604; if no, go to step 603.

[0130] Step 603: annotate the non-detection area in the third image to obtain the first image corresponding to the third image.

[0131] Step 604: Generate a first image corresponding to the third image based on the language segmentation model.

[0132] Step 605: Acquire a second image corresponding to the current video stream to be detected in each detection scene.

[0133] Step 606 , performing a pixel value addition operation on the second image and the first image under the same detection scene to obtain a first image to be detected under a different detection scene.

[0134] It should be noted that: Figure 6 The steps shown in are for illustration only and are not intended to be limiting. For example, step 605 may be performed before step 604, or before step 603, or before step 601, or simultaneously with step 601, which is not limited in the present embodiment.

[0135] In some embodiments, in step 102, training the first model based on the first image to be detected and the first detection result image may include: step 1021, inputting the first image to be detected into the first model to obtain a first output image; step 1022, determining the loss value of the first output image and the first detection result image according to a preset loss function; step 1023, adjusting the parameters of the first model based on the loss value until the first model converges.

[0136] It can be understood that the first detection image is input into the first model, and the output of the first model is the first output image, which is the image obtained by detecting the first detection image. According to the preset loss function, the loss value of the first output image and the first detection result image is calculated; then, it is determined whether the loss value is less than the preset threshold value. When the loss value is greater than or equal to the preset threshold value, the parameters of the first model are adjusted, and the execution of steps 1021 to 1022 is returned until the loss value is less than the preset threshold value and the first model converges. When the loss value is less than the preset threshold value, it means that the first model converges. In this way, through the above process, the trained first model can be obtained.

[0137] In one example, after obtaining the first output image in the above step 1021, the distribution focal loss (DFL) can be calculated by the offset from the center point of the anchor point of the prediction box in the first output image to the upper left corner and the lower right corner of the detection area; then, the positioning loss (IoULoss) of the positive sample is calculated according to the intersection over union (IoU) between the prediction box and the marked box in the first detection result image; then, by comparing the predicted category of the abnormal target with the true category, the classification loss (binary cross-entropy, BCE Loss) of the positive sample is calculated. The above loss function can be implemented in Python language according to the definition of YOLOv10. Finally, the above DFL, IoU Loss and BCE Loss are weighted and summed to obtain the loss values ​​of the first output image and the first detection result image.

[0138] In some embodiments, for the above step 102, during the training process of the first model, the training of the first model can be stopped if at least one of the following conditions is met: the loss value is less than a preset threshold, the number of iterations reaches a predetermined threshold, or the detection performance of the first model meets the standard (such as mAP exceeds a threshold); otherwise, the training iteration will continue (i.e., step 1021 to step 1022).

[0139] In some embodiments, for step 1021, before inputting the first image to be detected into the first model and obtaining the first output image, the following steps may also be included: performing a preprocessing operation on the first image to be detected, and inputting the image obtained after the preprocessing operation into the first model.

[0140] Here, the preprocessing operation may include: performing an RGB channel conversion operation according to the stack function of numpy (Numerical Python), performing an affine transformation operation according to the warpAffine function of the open source computer vision library (opencv), performing a uniform image data dimension operation according to the transpose function of numpy, converting the image data into a trainable data format according to the from_numpy function of torch, and normalizing the image data RGB value to the range [0,1] according to the following formula (1). The preprocessing operation may also include other operations, which are not specifically limited in the embodiments of the present disclosure.

[0141]

[0142] Among them, R represents the red channel value of a pixel, R1 represents the normalized value of the red channel value of the pixel, G represents the green channel value of a pixel, G1 represents the normalized value of the green channel value of the pixel, B represents the blue channel value of a pixel, B1 represents the normalized value of the blue channel value of the pixel.

[0143] Figure 7 FIG. 1 is a flow chart of a target detection method according to an exemplary embodiment. Figure 7 As shown, the target detection method can be used in an edge device, and the method mainly includes the following steps:

[0144] In step 701, a second image to be detected is acquired.

[0145] In step 702, the second image to be detected is input into the trained first model to obtain a second detection result image corresponding to the second image to be detected.

[0146] Here, the second image to be detected includes a detection area and a non-detection area. The first model is trained based on multiple sets of sample data in different detection scenarios, each set of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

[0147] In some embodiments, for step 701, the method of acquiring the second image to be detected can refer to step 101 and step 201, which will not be repeated here.

[0148] In some embodiments, for step 702, the second image to be detected is input into the trained first model, the second image to be detected is detected by the first model, and the output of the second model is the second detection result image corresponding to the second image to be detected.

[0149] In some embodiments, the second detection result image in step 702 may include: a predicted category of an abnormal target, a predicted box of an abnormal target, etc., which is not limited in the embodiments of the present disclosure.

[0150] In the embodiment of the present application, through the above steps 701 and 702, abnormal targets can be identified through the trained first model to determine whether there are abnormal targets.

[0151] In some embodiments, in the non-training stage of the first model, before step 701, the trained first model can be loaded and a threshold for abnormal target detection can be set. Then, the second image to be detected is preprocessed and resized to adapt to the input requirements of the first model, and forward propagation calculation is performed through the first model. Finally, the output results of the first model are screened using the set threshold to identify and confirm the existence of abnormal targets.

[0152] In some embodiments, the trained first model can be deployed on the Suanneng SE7-32 edge processing device and applied to the tasks of abnormal parking detection of vehicles at subway entrances and abnormal stacking of items in key channels to maintain efficient and stable operation.

[0153] In some embodiments, for the first model in step 702, it can be a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the C2f module and the first module perform convolution on different channels.

[0154] It should be noted that the above process can refer to the relevant explanations of the first model during the model training process, which will not be repeated here.

[0155] In some embodiments, for the first model in step 702, the first module in the first model may include a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; the first convolutional layer performs a convolution operation on the input of the first module to obtain a first feature map; the first activation layer performs a nonlinear transformation on the first feature map to obtain a second feature map; the third feature map is obtained by multiplying the first feature map with the second feature map; the third feature map is segmented by the first segmentation unit to obtain a fourth feature map and a fifth feature map; the fifth feature map is segmented by the second segmentation unit to obtain a sixth feature map and a seventh feature map; the seventh feature map is convolved by the second convolutional layer to obtain an eighth feature map; the eighth feature map and the sixth feature map are spliced ​​by the first splicing layer. , obtaining a ninth feature map; performing a segmentation operation on the ninth feature map through the third segmentation unit to obtain a tenth feature map and an eleventh feature map; performing a convolution operation on the eleventh feature map through the third convolution layer to obtain a twelfth feature map; splicing the twelfth feature map, the tenth feature map and the fourth feature map through the second splicing layer to obtain a thirteenth feature map; performing a nonlinear transformation on the thirteenth feature map through the second activation layer to obtain a fourteenth feature map; obtaining a feature map output by the first module by multiplying the thirteenth feature map with the fourteenth feature map; wherein the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0156] It should be noted that the above process can refer to the relevant explanations of the first module in the model training process. The two only differ in the input image, and the rest of the implementation methods are the same, which will not be repeated here.

[0157] In some embodiments, obtaining a second image to be detected includes: obtaining a first image corresponding to a first detection scene, the first image including a detection area and a non-detection area distinguished by different pixel values; obtaining a fourth image corresponding to a first video stream to be detected in the first detection scene; and performing a pixel value addition operation on the fourth image and the first image corresponding to the first detection scene to obtain a second image to be detected in the first detection scene.

[0158] It can be understood that the first detection scene can be the current scene to be detected in the non-model training stage.

[0159] The above process can be referred to step 1 and step 2 in step 101, which will not be repeated here.

[0160] In some embodiments, obtaining a first image corresponding to a first detection scene includes: determining a second detection scene that is the same as the first detection scene from different detection scenes; and determining the first image corresponding to the second detection scene in the different detection scenes as the first image corresponding to the first detection scene.

[0161] It can be understood that the first images corresponding to different detection scenes have been determined during the model training process. Then, in the target detection process, when the first detection scene is known, the second detection scene that is the same as the first detection scene can be determined from different detection scenes, and then the first image corresponding to the second detection scene can be determined as the first image corresponding to the first detection scene, so that the first image corresponding to the first detection scene can be obtained. In this way, the first image corresponding to the first detection scene can be determined quickly and efficiently.

[0162] Based on the same inventive concept, an embodiment of the present disclosure provides a model training device to execute the above-mentioned model training method. Figure 8 FIG. 1 is a block diagram of a model training device according to an exemplary embodiment. Figure 8 As shown, the device mainly includes: a first acquisition module 801, configured to acquire multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; a training module 802, configured to train a first model based on the first image to be detected and the first detection result image, and the first model is used to detect abnormal targets in the first detection image.

[0163] In some embodiments, the first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

[0164] In some embodiments, the first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; a convolution operation is performed on the input of the first module through the first convolutional layer to obtain a first sample feature map; a nonlinear transformation is performed on the first sample feature map through the first activation layer to obtain a second sample feature map; a third sample feature map is obtained by multiplying the first sample feature map with the second sample feature map; a segmentation operation is performed on the third sample feature map through the first segmentation unit to obtain a fourth sample feature map and a fifth sample feature map; a segmentation operation is performed on the fifth sample feature map through the second segmentation unit to obtain a sixth sample feature map and a seventh sample feature map; a convolution operation is performed on the seventh sample feature map through the second convolutional layer to obtain an eighth sample feature map; the eighth sample feature map and the sixth sample feature map are spliced ​​through the first splicing layer to obtain a ninth sample feature map ; Performing a segmentation operation on the ninth sample feature map through the third segmentation unit to obtain a tenth sample feature map and an eleventh sample feature map; Performing a convolution operation on the eleventh sample feature map through the third convolution layer to obtain a twelfth sample feature map; Splicing the twelfth sample feature map, the tenth sample feature map and the fourth sample feature map through the second splicing layer to obtain a thirteenth sample feature map; Performing a nonlinear transformation on the thirteenth sample feature map through the second activation layer to obtain a fourteenth sample feature map; By multiplying the thirteenth sample feature map with the fourteenth sample feature map, the sample feature map output by the first module is obtained; wherein, the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0165] In some embodiments, the first acquisition module 801 includes: a first acquisition unit, configured to: acquire first images corresponding to different detection scenarios, the first image including detection areas and non-detection areas distinguished by different pixel values; a second acquisition unit, configured to: acquire a second image corresponding to the current video stream to be detected in each detection scenario; a first determination unit, configured to: add pixel values ​​of the second image and the first image in the same detection scenario to obtain the first image to be detected in different detection scenarios; a marking unit, configured to: mark abnormal targets included in the detection area in the first image to be detected, and obtain a first detection result image corresponding to the first image to be detected.

[0166] In some embodiments, the first acquisition unit is configured to perform the following steps: acquire a third image based on video streams under different detection scenarios; and when the shape of the detection scene corresponding to the third image is a regular shape, generate a first image corresponding to the third image based on a language segmentation model.

[0167] In some embodiments, the first acquisition unit is further configured to perform the following steps: when the shape of the detection scene corresponding to the third image is an irregular shape, annotate the non-detection area in the third image to obtain a first image corresponding to the third image, wherein the pixel value corresponding to the non-detection area in the first image obtained after annotation is different from the pixel value of the detection area in the first image, and the detection area in the first image is an area that is not annotated in the third image.

[0168] In some embodiments, the training module 802 is further configured to perform the following steps: inputting the first image to be detected into the first model to obtain a first output image; determining the loss value of the first output image and the first detection result image according to a preset loss function; and adjusting the parameters of the first model based on the loss value until the first model converges.

[0169] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0170] Based on the same inventive concept, an embodiment of the present disclosure provides a target detection device to execute the above target detection method. Fig. 9 FIG. 1 is a block diagram of a target detection device according to an exemplary embodiment. Fig. 9 As shown, the device mainly includes: a second acquisition module 901, configured to acquire a second image to be detected, the second image to be detected includes a detection area and a non-detection area; a determination module 902, configured to input the second image to be detected into the trained first model to obtain a second detection result image corresponding to the second image to be detected; wherein the first model is trained according to multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

[0171] In some embodiments, the first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and a head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

[0172] In some embodiments, the first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer connected in series in sequence; the first convolutional layer performs a convolution operation on the input of the first module to obtain a first feature map; the first activation layer performs a nonlinear transformation on the first feature map to obtain a second feature map; the third feature map is obtained by multiplying the first feature map with the second feature map; the first segmentation unit performs a segmentation operation on the third feature map to obtain a fourth feature map and a fifth feature map; the second segmentation unit performs a segmentation operation on the fifth feature map to obtain a sixth feature map and a seventh feature map; the second convolutional layer performs a convolution operation on the seventh feature map to obtain an eighth feature map; the eighth feature map and the sixth feature map are spliced ​​through the first splicing layer to obtain a ninth feature map; The third segmentation unit performs a segmentation operation on the ninth feature map to obtain a tenth feature map and an eleventh feature map; the third convolutional layer performs a convolution operation on the eleventh feature map to obtain a twelfth feature map; the second splicing layer splices the twelfth feature map, the tenth feature map and the fourth feature map to obtain a thirteenth feature map; the second activation layer performs a nonlinear transformation on the thirteenth feature map to obtain a fourteenth feature map; the feature map output by the first module is obtained by multiplying the thirteenth feature map with the fourteenth feature map; wherein the convolution kernel of the first convolutional layer is smaller than the convolution kernel of the second convolutional layer, the convolution kernel of the second convolutional layer is the same as the convolution kernel of the third convolutional layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the segmentation ratios corresponding to the first segmentation unit, the second segmentation unit and the third segmentation unit are the same or different, and the structures adopted by the first splicing layer and the second splicing layer are the same or different.

[0173] In some embodiments, the second acquisition module 902 includes: a third acquisition unit, configured to: acquire a first image corresponding to a first detection scene, the first image including a detection area and a non-detection area distinguished by different pixel values; a fourth acquisition unit, configured to: acquire a fourth image corresponding to a first video stream to be detected in the first detection scene; a second determination unit, configured to: perform a pixel value addition operation on the fourth image and the first image corresponding to the first detection scene to obtain a second image to be detected in the first detection scene.

[0174] In some embodiments, the third acquisition unit is configured to perform the following steps: determine a second detection scene that is the same as the first detection scene from different detection scenes; and determine a first image corresponding to the second detection scene in the different detection scenes as the first image corresponding to the first detection scene.

[0175] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0176] Based on the same inventive concept, an embodiment of the present disclosure provides an electronic device, which can be used to implement the above-mentioned model training method and target detection method. Fig.10 1 is a block diagram of an electronic device 1000 according to an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0177] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0178] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with at least one of display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0179] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include at least one of the following: instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, and videos. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0180] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1000.

[0181] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0182] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0183] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0184] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of the components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect the position change of the electronic device 1000 or a component in the electronic device 1000, the presence or absence of contact between the user and the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include, but is not limited to, at least one of the following: an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, and a temperature sensor.

[0185] The communication component 1016 is configured to facilitate communication between the electronic device 1000 and other devices in a wired or wireless manner. The electronic device 1000 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0186] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components.

[0187] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including executable instructions or a computer program, which can be executed by a processor 1020 of the electronic device 1000 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0188] In some embodiments, when the instructions in the non-temporary computer-readable storage medium are executed by a processor, any one of the above-mentioned model training methods of the embodiments of the present disclosure can be executed. For example, the method includes: obtaining multiple sets of sample data under different detection scenarios, each set of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; based on the first image to be detected and the first detection result image, training a first model, and the first model is used to detect abnormal targets in the first detection image.

[0189] In some embodiments, when the instructions in the non-temporary computer-readable storage medium are executed by a processor, any one of the above-mentioned target detection methods of the embodiments of the present disclosure can be executed. For example, the method includes: obtaining a second image to be detected, the second image to be detected includes a detection area and a non-detection area; inputting the second image to be detected into the trained first model to obtain a second detection result image corresponding to the second image to be detected; wherein the first model is trained according to multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, both the first image to be detected and the first detection result image include detection areas and non-detection areas, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

[0190] The embodiment of the present disclosure provides a computer program product, which includes: a computer program or executable instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer program or executable instructions from the computer-readable storage medium, and the processor executes the computer program or executable instructions, so that the computer device executes any one of the above-mentioned model training methods and target detection methods of the embodiment of the present disclosure.

[0191] Based on the same inventive concept, an embodiment of the present disclosure provides a server, which can be used to implement the above-mentioned model training method and target detection method. Fig.11 FIG. 1 is a block diagram of a server 1100 according to an exemplary embodiment. Fig.11, the server 1100 includes a processing component 1122, which further includes one or more processors, and a memory resource represented by a memory 1132, for storing instructions that can be executed by the processing component 1122, such as an application. The application stored in the memory 1132 may include one or more modules, each of which corresponds to a set of instructions. In addition, the processing component 1122 is configured to execute instructions to perform any of the above-mentioned model training methods, for example, the method includes: obtaining multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes labeling information of abnormal targets; based on the first image to be detected and the first detection result image, the first model is trained, and the first model is used to detect abnormal targets in the first detection image.

[0192] In some embodiments, the processing component 1122 is configured to execute instructions to perform any of the above-mentioned target detection methods, for example, the method includes: obtaining a second image to be detected, the second image to be detected including a detection area and a non-detection area; inputting the second image to be detected into the trained first model to obtain a second detection result image corresponding to the second image to be detected; wherein the first model is trained based on multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

[0193] The server 1100 may also include a power supply component 1126 configured to perform power management of the server 1100, a wired or wireless network interface 1150 configured to connect the server 1100 to a network, and an input / output (I / O) interface 1158. The server 1100 may operate an operating system stored in the memory 1132, such as Windows Server™, Mac OS X™, Unix™ Linux™, FreeBSD™, or the like.

[0194] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0195] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A model training method, characterized in that: include: Acquire multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; A first model is trained based on the first image to be detected and the first detection result image, and the first model is used to detect abnormal targets in the first detection image.

2. The method according to claim 1, characterized in that The first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and the head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

3. The method according to claim 2, characterized in that The first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer which are connected in series in sequence; Performing a convolution operation on the input of the first module through the first convolution layer to obtain a first sample feature map; Performing a nonlinear transformation on the first sample feature map through the first activation layer to obtain a second sample feature map; Obtaining a third sample feature map by multiplying the first sample feature map by the second sample feature map; Performing a segmentation operation on the third sample feature map by the first segmentation unit to obtain a fourth sample feature map and a fifth sample feature map; Performing a segmentation operation on the fifth sample feature map by the second segmentation unit to obtain a sixth sample feature map and a seventh sample feature map; Performing a convolution operation on the seventh sample feature map through the second convolution layer to obtain an eighth sample feature map; Splicing the eighth sample feature map and the sixth sample feature map through the first splicing layer to obtain a ninth sample feature map; Performing a segmentation operation on the ninth sample feature map by the third segmentation unit to obtain a tenth sample feature map and an eleventh sample feature map; Performing a convolution operation on the eleventh sample feature map through the third convolution layer to obtain a twelfth sample feature map; The twelfth sample feature map, the tenth sample feature map and the fourth sample feature map are spliced ​​through the second splicing layer to obtain a thirteenth sample feature map; Performing a nonlinear transformation on the thirteenth sample feature map through the second activation layer to obtain a fourteenth sample feature map; Obtaining a sample feature map output by the first module by performing a multiplication operation on the thirteenth sample feature map and the fourteenth sample feature map; Among them, the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the first segmentation unit, the second segmentation unit and the third segmentation unit have the same or different corresponding segmentation ratios, and the first splicing layer and the second splicing layer have the same or different structures.

4. The method according to claim 1, characterized in that: The obtaining of multiple sets of sample data in different detection scenarios includes: Acquire first images corresponding to the different detection scenarios, wherein the first images include detection areas and non-detection areas distinguished by different pixel values; Obtaining a second image corresponding to the current video stream to be detected in each detection scene; Performing pixel value addition operation on the second image and the first image under the same detection scene to obtain the first image to be detected under the different detection scene; Abnormal targets included in the detection area in the first image to be detected are marked to obtain a first detection result image corresponding to the first image to be detected.

5. The method according to claim 4, characterized in that The acquiring of first images corresponding to the different detection scenes respectively includes: Acquire a third image based on the video streams in the different detection scenarios; In a case where the shape of the detection scene corresponding to the third image is a regular shape, a first image corresponding to the third image is generated based on a language segmentation model.

6. The method according to claim 5, characterized in that The method further comprises: In the case where the shape of the detection scene corresponding to the third image is irregular, the non-detection area in the third image is marked to obtain a first image corresponding to the third image, wherein the pixel value corresponding to the non-detection area in the first image obtained after marking is different from the pixel value of the detection area in the first image, and the detection area in the first image is the area where the third image is not marked.

7. The method according to claim 1, characterized in that The step of training the first model based on the first image to be detected and the first detection result image includes: Inputting the first image to be detected into the first model to obtain a first output image; Determining loss values ​​of the first output image and the first detection result image according to a preset loss function; Parameters of the first model are adjusted based on the loss value until the first model converges.

8. A target detection method, characterized in that: include: Acquire a second image to be detected, wherein the second image to be detected includes a detection area and a non-detection area; Inputting the second image to be detected into the trained first model to obtain a second detection result image corresponding to the second image to be detected; Among them, the first model is trained based on multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include detection areas and non-detection areas, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

9. The method according to claim 8, characterized in that The first model is a model obtained by improving the YOLOv10n model, the YOLOv10n model includes a first backbone network, a first neck network and a head network, the first model includes a second backbone network, a second neck network and the head network, the second backbone network is obtained by replacing the C2f module in the first backbone network with the first module, the second neck network is obtained by replacing the C2f module in the first neck network with the first module, and the number of channels performed by the C2f module for convolution is greater than the number of channels performed by the first module for convolution.

10. The method according to claim 9, characterized in that The first module includes a first convolutional layer, a first activation layer, a first segmentation unit, a second segmentation unit, a second convolutional layer, a first splicing layer, a third segmentation unit, a third convolutional layer, a second splicing layer and a second activation layer which are connected in series in sequence; Performing a convolution operation on the input of the first module through the first convolution layer to obtain a first feature map; Performing a nonlinear transformation on the first feature map through the first activation layer to obtain a second feature map; Obtaining a third feature map by multiplying the first feature map by the second feature map; Performing a segmentation operation on the third feature map by the first segmentation unit to obtain a fourth feature map and a fifth feature map; Performing a segmentation operation on the fifth feature map by the second segmentation unit to obtain a sixth feature map and a seventh feature map; Performing a convolution operation on the seventh feature map through the second convolution layer to obtain an eighth feature map; Splicing the eighth feature map and the sixth feature map through the first splicing layer to obtain a ninth feature map; Performing a segmentation operation on the ninth feature map by the third segmentation unit to obtain a tenth feature map and an eleventh feature map; Performing a convolution operation on the eleventh feature map through the third convolution layer to obtain a twelfth feature map; The twelfth characteristic map, the tenth characteristic map and the fourth characteristic map are spliced ​​through the second splicing layer to obtain a thirteenth characteristic map; Performing a nonlinear transformation on the thirteenth characteristic map through the second activation layer to obtain a fourteenth characteristic map; Obtaining a feature map output by the first module by multiplying the thirteenth feature map by the fourteenth feature map; Among them, the convolution kernel of the first convolution layer is smaller than the convolution kernel of the second convolution layer, the convolution kernel of the second convolution layer is the same as the convolution kernel of the third convolution layer, the activation functions adopted by the first activation layer and the second activation layer are the same or different, the first segmentation unit, the second segmentation unit and the third segmentation unit have the same or different corresponding segmentation ratios, and the first splicing layer and the second splicing layer have the same or different structures.

11. The method according to claim 8, characterized in that The step of acquiring a second image to be detected comprises: Acquire a first image corresponding to a first detection scene, wherein the first image includes a detection area and a non-detection area distinguished by different pixel values; Acquire a fourth image corresponding to the first video stream to be detected in the first detection scenario; A pixel value addition operation is performed on the fourth image and the first image corresponding to the first detection scene to obtain the second image to be detected in the first detection scene.

12. The method according to claim 11, characterized in that The obtaining of a first image corresponding to a first detection scene includes: Determine a second detection scene that is the same as the first detection scene from different detection scenes; The first image corresponding to the second detection scene in the different detection scenes is determined as the first image corresponding to the first detection scene.

13. A model training device, characterized in that: include: a first acquisition module, configured to acquire multiple groups of sample data under different detection scenarios, each group of sample data including a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both including a detection area and a non-detection area, and the detection area of ​​the first detection result image includes marking information of abnormal targets; The training module is configured to train a first model based on the first image to be detected and the first detection result image, wherein the first model is used to detect abnormal targets in the first detection image.

14. A target detection device, characterized in that: include: A second acquisition module is configured to acquire a second image to be detected, wherein the second image to be detected includes a detection area and a non-detection area; a determination module configured to input the second image to be detected into the trained first model to obtain a second detection result image corresponding to the second image to be detected; Among them, the first model is trained based on multiple groups of sample data under different detection scenarios, each group of sample data includes a first image to be detected and a first detection result image corresponding to the first image to be detected, the first image to be detected and the first detection result image both include detection areas and non-detection areas, and the detection area of ​​the first detection result image includes marking information of abnormal targets.

15. An electronic device, characterized in that: include: processor; Memory for storing computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 12.

16. A non-transitory computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instructions in the storage medium are executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

17. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images