Image detection method, system, device and medium

Abstract

An image detection method, system, device and medium, which transform a target image to change the size and number of channels of the target image; then transform the target image after the size and number of channels are changed into a feature map, and obtain features of multiple scales from the feature map; generate a prediction frame of the target image based on the features of multiple scales, optimize the image detection algorithm according to the prediction frame of the target image, and generate an image detection model; use the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment. In the present invention, if the target object in the image to be detected is a human head, the safety protection equipment can be a helmet. The present invention designs a solution that can efficiently detect helmets and human heads of multiple scales in the same image, which can solve the problem that the image detection method in the prior art is difficult to detect human heads and helmets of multiple scales in the same image.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to an image detection method, system, equipment and medium. Background Art

[0002] At present, many construction site scenes have strict requirements on whether workers wear safety helmets. Therefore, it is very promising to detect whether people wear safety helmets in construction site scenes. In actual application scenarios, due to the distance between the image capture device and the person, there are safety helmets of different scales in the same image, that is, in the same image, the head and safety helmet close to the image capture device are relatively large, and the head and safety helmet far from the image capture device are relatively small; and the current image detection algorithm cannot fully detect the safety helmets and heads of all scales in the same image, and the current conventional one-stage real-time detection algorithm cannot detect smaller safety helmets and heads. On the other hand, the size of the image to be detected will have a great impact on the detection effect of the detection algorithm of targets such as safety helmets and heads. Although the larger the image size, the more conducive to the detection of the target, the too large image size will seriously affect the reasoning time of the detection algorithm. Therefore, a suitable image detection method is needed to solve the problems encountered in current actual application scenarios. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide an image detection method, system, device and medium for solving the technical problem that the image detection algorithm in the prior art cannot detect helmets and human heads of different sizes from an image.

[0004] To achieve the above object and other related objects, the present invention provides an image detection method, comprising the following steps:

[0005] Converting the target image to change the size and number of channels of the target image;

[0006] Converting the target image after changing the size and the number of channels into a feature map, and obtaining features of multiple scales from the feature map;

[0007] Generate a prediction box of the target image based on features of multiple different scales, optimize an image detection algorithm according to the prediction box of the target image, and generate an image detection model;

[0008] The image detection model is used to determine whether one or more target objects in the image to be detected have safety protection equipment.

[0009] Optionally, if the target image includes an image with completed frame annotation, the method further includes:

[0010] Comparing the predicted frame of the target image with the frame pre-labeled in the target image, and calculating the loss of one or more neural networks in the image detection algorithm according to the comparison result;

[0011] One or more neural networks in the image detection algorithm are iteratively optimized based on the loss to generate the image detection model.

[0012] Optionally, generating a prediction box of the target image based on features of multiple different scales includes:

[0013] Fuse some features of different scales so that features of one or more scales can perceive feature information of another one or more scales;

[0014] The neural network block is used to extract features from the fused features and the unfused features, and the feature extraction results are classified and regressed to generate a prediction frame of the target image; wherein the neural network block is derived from one or more neural networks.

[0015] Optionally, using the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment includes:

[0016] Using the image detection model to detect the image to be detected, and generating one or more prediction boxes;

[0017] Perform overlapping box filtering on the one or more prediction boxes so that each target object is associated with only one prediction box;

[0018] Obtain the category and confidence score of each prediction box after filtering, and draw the prediction box and the corresponding category on the image to be detected when the confidence score of the prediction box is greater than a preset score threshold, and determine whether the one or more target objects have safety protection equipment based on the drawing results.

[0019] Optionally, the target image after the size change has the same image quality as the original target image, and the size of the target image after the size change is smaller than the original target image;

[0020] And / or, the target image after the number of channels is changed has the same image quality as the original target image, and the number of channels of the changed target image is greater than that of the original target image.

[0021] Optionally, the target object includes a human head and / or a human body.

[0022] Optionally, before converting the target image, the target image is also preprocessed and enhanced;

[0023] The preprocessing includes at least one of the following: contrast adjustment, brightness adjustment, rotation, blur processing, and cropping;

[0024] The enhancement includes at least one of the following: online enhancement and offline enhancement.

[0025] The present invention also provides an image detection system, comprising:

[0026] An image conversion module, used to convert a target image and change the size and number of channels of the target image;

[0027] A feature extraction module, used to convert the target image after changing the size and the number of channels into a feature map, and obtain features of multiple different scales from the feature map;

[0028] An image detection model module, used to generate a prediction box of the target image based on features of multiple different scales, optimize an image detection algorithm according to the prediction box of the target image, and generate an image detection model;

[0029] The image detection module is used to use the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment.

[0030] Optionally, if the target image includes an image with completed frame annotation, the method further includes:

[0031] Comparing the predicted frame of the target image with the frame pre-labeled in the target image, and calculating the loss of one or more neural networks in the image detection algorithm according to the comparison result;

[0032] One or more neural networks in the image detection algorithm are iteratively optimized based on the loss to generate the image detection model.

[0033] Optionally, the image detection model module generates a prediction box of the target image based on features of multiple different scales, including:

[0034] Fuse some features of different scales so that features of one or more scales can perceive feature information of another one or more scales;

[0035] The neural network block is used to extract features from the fused features and the unfused features, and the feature extraction results are classified and regressed to generate a prediction frame of the target image; wherein the neural network block is derived from one or more neural networks.

[0036] Optionally, the image detection module uses the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment, including:

[0037] Using the image detection model to detect the image to be detected, and generating one or more prediction boxes;

[0038] Perform overlapping box filtering on the one or more prediction boxes so that each target object is associated with only one prediction box;

[0039] Obtain the category and confidence score of each prediction box after filtering, and draw the prediction box and the corresponding category on the image to be detected when the confidence score of the prediction box is greater than a preset score threshold, and determine whether the one or more target objects have safety protection equipment based on the drawing results.

[0040] Optionally, the target image after the size change has the same image quality as the original target image, and the size of the target image after the size change is smaller than the original target image;

[0041] And / or, the target image after the number of channels is changed has the same image quality as the original target image, and the number of channels of the changed target image is greater than that of the original target image.

[0042] The present invention also provides a computer device, comprising:

[0043] one or more processors; and

[0044] One or more machine-readable media storing instructions, when the one or more processors execute the instructions, enable the device to perform any of the methods described above.

[0045] The present invention also provides one or more machine-readable media, characterized in that instructions are stored thereon, and when the instructions are executed by one or more processors, the device executes any one of the methods described above.

[0046] As described above, the present invention provides an image detection method, system, device and medium, which have the following beneficial effects: the present invention converts the target image to change the size and number of channels of the target image; then converts the target image after the size and number of channels are changed into a feature map, and obtains multiple features of different scales from the feature map; generates a prediction frame of the target image based on the features of multiple scales, optimizes the image detection algorithm according to the prediction frame of the target image, and generates an image detection model; and uses the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment. In the present invention, if the target object in the image to be detected is a human head, the safety protection equipment may be a helmet. In view of the existing problems, the present invention designs a solution that can efficiently detect helmets and human heads of multiple scales (i.e., multiple sizes), which can solve the problem that the image detection method in the prior art is difficult to detect human heads and helmets of multiple scales in the same image, especially solves the problem of detecting small target human heads and helmets in the image. At the same time, the present invention can greatly reduce the detection time after converting the original image of larger size into an image of relatively small size, so that the present invention can ensure high detection efficiency in real time. The present invention also designs a detection unit of multiple scales, which can better detect human heads and helmets of multiple scales, and can well meet the detection requirements of human heads and helmets of different scales in actual construction site application scenarios. In addition, the present invention can also pre-process and enhance the image, eliminating the influence of interference factors such as lighting, occlusion and blur in the detection process of human heads and helmets. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a flow chart of an image detection method provided by an embodiment;

[0048] Figure 2 The image after box annotation;

[0049] Figure 3 For images with unchanged size and number of channels;

[0050] Figure 4 For images with changed size and number of channels;

[0051] Figure 5 A schematic flow chart of an image detection method provided by another embodiment;

[0052] Figure 6 for Figure 5 The framework diagram of the image detection method in ;

[0053] Figure 7 A schematic diagram of the hardware structure of an image detection system provided by an embodiment;

[0054] Figure 8 A schematic diagram of the hardware structure of a terminal device provided by an embodiment;

[0055] Fig. 9 A schematic diagram of the hardware structure of a terminal device provided in another embodiment.

[0056] Component number description

[0057] DETAILED DESCRIPTION

[0058] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0059] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0060] Please refer to 1 to Figure 4 As shown, the present invention provides an image detection method, comprising the following steps:

[0061] S100, converting the target image to change the size and number of channels of the target image;

[0062] S200, converting the target image after changing the size and the number of channels into a feature map, and obtaining features of multiple different scales from the feature map;

[0063] S300, generating a prediction frame of a target image based on features of multiple different scales, optimizing an image detection algorithm according to the prediction frame of the target image, and generating an image detection model;

[0064] S400, using an image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment. As an example, the target object in this method may be a human head and / or a human body. If the target object is a human head, the safety protection equipment may be a helmet; if the target object is a human body, the safety protection equipment may be a knee pad, a wrist guard, etc.

[0065] In response to the existing problems, this method designs a solution that can efficiently detect helmets and heads of multiple scales (i.e., multiple sizes), which can solve the problem that the image detection method in the prior art is difficult to detect heads and helmets of multiple scales in the same image, especially solves the problem of detecting small target heads and helmets in the image. At the same time, this method can greatly reduce the detection time after converting the original image of larger size into an image of relatively small size, which can enable this method to ensure high detection efficiency in real time. In addition, this method also designs detection units of multiple scales, which can better detect heads and helmets of multiple scales, and can well meet the detection needs of heads and helmets of different scales in actual construction site application scenarios.

[0066] In an exemplary embodiment, if the target image includes an image with completed frame annotation, it also includes: comparing the predicted frame of the target image with the frame pre-annotated by the target image, calculating the loss of one or more neural networks in the image detection algorithm according to the comparison result; iteratively optimizing the one or more neural networks in the image detection algorithm based on the loss to generate the image detection model. As an example, for example, the predicted frame of the target image is compared with the frame pre-annotated by the target image, determining the difference between the predicted frame of the target image and the pre-annotated frame, and calculating the center loss (i.e., focal loss) of the image detection algorithm according to the difference, iteratively optimizing one or more neural networks in the image detection algorithm according to the calculated center loss, and using the iteratively optimized image detection algorithm to obtain a new predicted frame, and then comparing the new predicted frame with the pre-annotated frame, terminating the optimization of one or more neural networks in the image detection algorithm when the comparison result meets the target condition, and generating the image detection model. Among them, the target condition can be that the overlap between the predicted frame and the pre-annotated frame meets the threshold in the actual application scenario. Or, after the optimization times reach a preset number, terminating the optimization of one or more neural networks in the image detection algorithm to generate the image detection model. In the embodiment of the present application, when the image is pre-annotated with a frame, if the person in the image is wearing a helmet, the corresponding frame category is annotated as wear; if the person is not wearing a helmet, the corresponding frame category is annotated as none. The annotated image is as follows: Figure 2 shown.

[0067] In an exemplary embodiment, a prediction frame of a target image is generated based on features of multiple different scales, including: fusing some features of different scales so that features of one or more scales can perceive feature information of another one or more scales; extracting features of the fused features and unfused features using a neural network block, and classifying and regressing the feature extraction results to generate a prediction frame of the target image; wherein the neural network block is derived from one or more neural networks. As an example, the neural network block in the embodiment of the present application includes but is not limited to a neural network block in a ResNet, VGG, MobileNet, or HRNet network.

[0068] In an exemplary embodiment, an image detection model is used to determine whether one or more target objects in an image to be detected have safety protection equipment, including: using an image detection model to detect the image to be detected and generating one or more prediction frames; using a non-maximum suppression algorithm to filter overlapping frames for one or more prediction frames so that each target object is associated with only one prediction frame; obtaining the category and confidence score of each prediction frame after filtering, and drawing the prediction frame and the corresponding category on the image to be detected when the confidence score of the prediction frame is greater than a preset score threshold, and judging whether one or more target objects have safety protection equipment according to the drawing result. As an example, for example, if the score threshold is set to 0.8 points in advance, a prediction frame greater than 0.8 points and after overlapping filtering is drawn on the image to be detected, and judging whether a person in the image is wearing a helmet on his head according to the category of the prediction frame. In the embodiment of the present application, the non-maximum suppression algorithm, referred to as the NMS algorithm, is Non-Maximum Suppression in English; it is used to search for local maximum values ​​and suppress maximum values. In the process of target object detection, a large number of candidate boxes will be generated at the position of the same target object, and these candidate boxes may overlap with each other. In this case, non-maximum suppression can be used to find the best target object bounding box and eliminate redundant bounding boxes. Among them, the best target object bounding box is to annotate the target object and its corresponding safety protection equipment with the minimum external bounding box, such as annotating the head and the helmet with the minimum external bounding box.

[0069] In an exemplary embodiment, the target image after the size change in the embodiment of the present application has the same image quality as the original target image, and the size of the target image after the size change is smaller than the original target image; and / or, the target image after the number of channels is changed has the same image quality as the original target image, and the number of channels of the target image after the size change is greater than the original target image. As an example, the width and height of a target image can be changed to half of the original, and the original three channels can be changed to 12 channels. For example, the RGB channel is changed to RGB RGB RGB RGB channel, and during the conversion process, one of every four upper and lower adjacent pixels is retained in turn for expansion. When the size change and the number of channels are changed, the image quality of the original image will not change, and the image information of the original image will be retained on the changed channel. In this method, large-size image input is conducive to the detection of helmets and head targets, but large-size image input will greatly increase the time of the entire image detection. This application converts the original large-size image into a small-size image and changes the number of channels of the original large-size image; it can not only reduce the detection time, but also store the image information of the large-size image on the channel, which is beneficial to the detection of helmets and head targets, so that the method can ensure high detection efficiency in real time. Figure 3 As shown, the image after the size and number of channels are changed is as follows Figure 4 shown.

[0070] In an exemplary embodiment, the target image may be preprocessed and enhanced before being converted. The preprocessing methods include but are not limited to: contrast adjustment, brightness adjustment, rotation, blur processing, and cropping; the enhancement methods include but are not limited to: online enhancement and offline enhancement. By preprocessing and enhancing the image, the embodiment of the present application can eliminate the influence of interference factors such as lighting, occlusion, and blur during the detection of human heads and helmets.

[0071] like Figures 2 to 6 As shown, the embodiment of the present application provides a method for detecting a human head and a helmet of multiple scales in the same image, comprising the following steps:

[0072] Step S101, firstly, the training image containing the human head and the helmet is preprocessed and enhanced by the image preprocessing and image enhancement module through the techniques of dynamic brightness adjustment, dynamic contrast adjustment, dynamic saturation adjustment, random cropping and blur processing, so that the detection of the human head and the helmet can avoid the influence of interference factors such as light, occlusion and blur as much as possible. The training image contains annotated images of the human head and the helmet with the minimum external bounding box. If the human head is wearing a helmet, the corresponding box category is marked as wear; if the human head is not wearing a helmet, the corresponding box category is marked as none. The annotated image is as follows: Figure 2shown.

[0073] Step S102, input the image processed in step S101 into the image lossless fast conversion module. After passing through this module, the width and height of the image size become half of the original input, and the number of channels of the image is changed from the original 3 channels to 12 channels, that is, the image size is reduced while the image information is retained on the channel. In the embodiment of the present application, large-size image input is beneficial to the detection of helmets and human heads, but large-size image input will greatly increase the time of the entire image detection. The present application converts the original large-size image into a small-size image, and changes the number of channels of the original large-size image; it can not only reduce the detection time, but also store the image information of the large-size image on the channel, which is beneficial to the detection of helmets and human heads, so that the method can guarantee a higher detection efficiency in real time. Among them, the image before the image size and the number of channels are transformed as shown in FIG. Figure 3 As shown, the image size and channel number after transformation are as follows Figure 4 shown.

[0074] Step S103, input the image after the size and number of channels are changed into a feature extraction module (backbone), and extract the image features using the neural network block in the feature extraction module. The feature extraction module in the embodiment of the present application is composed of a series of neural network blocks (blocks), and the input image is converted into a feature map during forward propagation in the feature module, and the size is gradually reduced. The neural network blocks (blocks) include but are not limited to the neural network blocks (blocks) in the ResNet, VGG, MobileNet, and HRNet networks.

[0075] Step S104, extracting features of four different scales from the feature extraction module, the four scales are respectively one quarter, one eighth, one sixteenth and one thirty-second of the original image, and are recorded as scale 1, scale 2, scale 3 and scale 4. The extracted features of the four scales are sequentially input into the neural network block in the feature fusion module, wherein the features corresponding to scale 3, scale 2 and scale 1 are fused with the features of the previous scale 4, scale 3 and scale 2 respectively before entering the neural network module, and after fusion, the features of scale 3 can perceive the feature information of scale 4, the features of scale 2 can perceive the feature information of scale 3, and the features of scale 1 can perceive the feature information of scale 2. As an example, an embodiment of the present application can determine whether a human head or a safety helmet is a large-scale target or a small-scale target in an image according to the pixels of the human head or the safety helmet; for example, if the size of the original image is 640 pixels * 640 pixels, then a human head or a safety helmet smaller than 8 pixels * 8 pixels will be determined as a small-scale target in the original image; if the size of the original image is 640 pixels * 640 pixels, then a human head or a safety helmet larger than 50 pixels * 50 pixels will be determined as a large-scale target in the original image.

[0076] Step S105, the feature maps of the four scales obtained by the feature fusion module are sequentially sent to the four detection heads in the detection head module for classification and regression, and the prediction box detected from the training image is output. The prediction box of the training image is compared with the pre-annotated box of the training image to determine the difference between the detected prediction box and the pre-annotated box, and the center loss focal loss of the image detection algorithm is calculated based on the difference. One or more neural networks in the image detection algorithm are iteratively optimized based on the calculated center loss, and a new prediction box is obtained by using the iteratively optimized image detection algorithm. The new prediction box is then compared with the pre-annotated box. When the comparison result meets the target condition, the optimization of one or more neural networks in the image detection algorithm is terminated to generate an image detection model. Among them, the target condition can be that the overlap between the prediction box and the pre-annotated box meets the threshold in the actual application scenario. Or after the optimization times reach a preset number, the optimization of one or more neural networks in the image detection algorithm is terminated to generate an image detection model.

[0077] Step S106, use the image detection model to detect the image to be detected, and obtain one or more prediction boxes output by the detection head; use the non-maximum suppression algorithm to filter the overlapping boxes of the prediction boxes output by the detection head, retain the prediction boxes of the head and the helmet whose confidence score is greater than the preset score threshold, and then draw the prediction boxes and the corresponding types on the image, and judge whether one or more target objects have safety protection equipment according to the drawing results. As an example, if the score threshold is set to 0.95 points in advance, the prediction boxes with a score greater than 0.95 and after overlapping filtering are drawn on the image to be detected, and judge whether the person in the image is wearing a helmet according to the category of the prediction box.

[0078] In view of the existing problems, this method designs a solution that can efficiently detect helmets and heads of multiple scales (i.e., multiple sizes), which can solve the problem that the image detection method in the prior art is difficult to detect heads and helmets of multiple scales in the same image, especially solves the problem of detecting small target heads and helmets in the image. At the same time, this method can greatly reduce the detection time after converting the original image of larger size into an image of relatively small size, which can enable this method to ensure high detection efficiency in real time. In addition, this method also designs detection units of multiple scales, which can better detect heads and helmets of multiple scales, and can well meet the detection requirements of heads and helmets of different scales in actual construction site application scenarios. In addition, this method can also preprocess and enhance the image, eliminating the influence of interference factors such as lighting, occlusion and blur in the detection process of heads and helmets.

[0079] The present invention also provides an image detection system, comprising:

[0080] The image conversion module M10 is used to convert the target image and change the size and number of channels of the target image;

[0081] A feature extraction module M20 is used to convert the target image after changing the size and the number of channels into a feature map, and obtain features of multiple different scales from the feature map;

[0082] The image detection model module M30 is used to generate a prediction frame of a target image based on features of multiple different scales, optimize an image detection algorithm according to the prediction frame of the target image, and generate an image detection model;

[0083] The image detection module M40 is used to use the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment.

[0084] In response to the current problems, this system has designed a solution that can efficiently detect helmets and heads of multiple scales (i.e., multiple sizes). This can solve the problem that the image detection system in the prior art is difficult to detect heads and helmets of multiple scales in the same image, especially the problem of detecting small target heads and helmets in the image. At the same time, this system can greatly reduce the detection time by converting the original image of larger size into an image of relatively small size, which can enable the system to ensure high detection efficiency in real time. In addition, this system has also designed detection units of multiple scales, which can better detect heads and helmets of multiple scales, and can well meet the detection needs of heads and helmets of different scales in actual construction site application scenarios.

[0085] In an exemplary embodiment, if the target image includes an image with completed frame annotation, it also includes: comparing the predicted frame of the target image with the frame pre-annotated by the target image, calculating the loss of one or more neural networks in the image detection algorithm according to the comparison result; iteratively optimizing the one or more neural networks in the image detection algorithm based on the loss to generate the image detection model. As an example, for example, the predicted frame of the target image is compared with the frame pre-annotated by the target image, determining the difference between the predicted frame of the target image and the pre-annotated frame, and calculating the center loss (i.e., focal loss) of the image detection algorithm according to the difference, iteratively optimizing one or more neural networks in the image detection algorithm according to the calculated center loss, and using the iteratively optimized image detection algorithm to obtain a new predicted frame, and then comparing the new predicted frame with the pre-annotated frame, terminating the optimization of one or more neural networks in the image detection algorithm when the comparison result meets the target condition, and generating the image detection model. Among them, the target condition can be that the overlap between the predicted frame and the pre-annotated frame meets the threshold in the actual application scenario. Or, after the optimization times reach a preset number, terminating the optimization of one or more neural networks in the image detection algorithm to generate the image detection model. In the embodiment of the present application, when the image is pre-annotated with a frame, if the person in the image is wearing a helmet, the corresponding frame category is annotated as wear; if the person is not wearing a helmet, the corresponding frame category is annotated as none. The annotated image is as follows: Figure 2 shown.

[0086] In an exemplary embodiment, the image detection model module generates a prediction frame of a target image based on features of multiple different scales, including: fusing some features of different scales so that features of one or more scales can perceive feature information of another one or more scales; extracting features of the fused features and unfused features using a neural network block, and classifying and regressing the feature extraction results to generate a prediction frame of the target image; wherein the neural network block is derived from one or more neural networks. As an example, the neural network block in the embodiment of the present application includes but is not limited to a neural network block in a ResNet, VGG, MobileNet, or HRNet network.

[0087] In an exemplary embodiment, the image detection module uses an image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment, including: using the image detection model to detect the image to be detected and generating one or more prediction boxes; using the non-maximum suppression algorithm to filter the overlapping boxes of one or more prediction boxes so that each target object is associated with only one prediction box; obtaining the category and confidence score of each prediction box after filtering, and when the confidence score of the prediction box is greater than a preset score threshold, the prediction box and the corresponding category are drawn on the image to be detected, and the one or more target objects are judged according to the drawing result. Whether the safety protection equipment is on the head of the person in the image. As an example, for example, if the score threshold is set to 0.8 points in advance, the prediction box with a score greater than 0.8 and after overlapping filtering is drawn on the image to be detected, and whether the person in the image is wearing a helmet is judged according to the category of the prediction box. In the embodiment of the present application, the non-maximum suppression algorithm, referred to as the NMS algorithm, is Non-Maximum Suppression in English; it is used to search for local maximum values ​​and suppress maximum values. In the process of target object detection, a large number of candidate boxes will be generated at the position of the same target object, and these candidate boxes may overlap with each other. In this case, non-maximum suppression can be used to find the best target object bounding box and eliminate redundant bounding boxes. Among them, the best target object bounding box is to annotate the target object and its corresponding safety protection equipment with the minimum external bounding box, such as annotating the head and the helmet with the minimum external bounding box.

[0088] In an exemplary embodiment, the target image after the size change in the embodiment of the present application has the same image quality as the original target image, and the size of the target image after the size change is smaller than the original target image; and / or, the target image after the number of channels is changed has the same image quality as the original target image, and the number of channels of the target image after the size change is greater than the original target image. As an example, the width and height of a target image can be changed to half of the original, and the original three channels can be changed to 12 channels. For example, the RGB channel is changed to RGB RGB RGB RGB channel, and during the conversion process, one of every four upper and lower adjacent pixels is retained in turn for expansion. When the size change and the number of channels are changed, the image quality of the original image will not change, and the image information of the original image will be retained on the changed channel. In this method, large-size image input is conducive to the detection of helmets and head targets, but large-size image input will greatly increase the time of the entire image detection. This application converts the original large-size image into a small-size image and changes the number of channels of the original large-size image; it can not only reduce the detection time, but also store the image information of the large-size image on the channel, which is beneficial to the detection of helmets and head targets, so that the method can ensure high detection efficiency in real time. Figure 3As shown, the image after the size and number of channels are changed is as follows Figure 4 shown.

[0089] In an exemplary embodiment, the target image may be preprocessed and enhanced before being converted. The preprocessing methods include but are not limited to: contrast adjustment, brightness adjustment, rotation, blur processing, and cropping; the enhancement methods include but are not limited to: online enhancement and offline enhancement. By preprocessing and enhancing the image, the embodiment of the present application can eliminate the influence of interference factors such as lighting, occlusion, and blur during the detection of human heads and helmets.

[0090] The present application also provides a system for detecting heads and helmets of multiple scales in the same image. Figures 2 to 6 As shown, the specific functions and technical effects of the system can be referred to the above embodiments, and will not be described in detail in this embodiment.

[0091] The present application also provides an image detection device, including:

[0092] Convert the target image to change the size and number of channels of the target image;

[0093] Convert the target image after changing the size and number of channels into a feature map, and obtain multiple features of different scales from the feature map;

[0094] Generate a prediction box of the target image based on features of multiple different scales, optimize the image detection algorithm according to the prediction box of the target image, and generate an image detection model;

[0095] The image detection model is used to determine whether one or more target objects in the image to be detected have safety protection equipment.

[0096] In this embodiment, the image detection device executes the above system or method. The specific functions and technical effects can be referred to the above embodiments and will not be repeated here.

[0097] The present application also provides a computer device, which may include: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the device to execute Figure 1In practical applications, the device can be used as a terminal device or a server. Examples of terminal devices may include: smart phones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, car computers, desktop computers, set-top boxes, smart TVs, wearable devices, etc. The embodiments of this application do not limit specific devices.

[0098] The present application also provides a non-volatile readable storage medium, in which one or more modules (programs) are stored. When the one or more modules are applied to a device, the device can execute the embodiment of the present application. Figure 1 Instructions for the steps involved in a data processing method.

[0099] Figure 8 A schematic diagram of the hardware structure of a terminal device provided for an embodiment of the present application. As shown in the figure, the terminal device may include: an input device 1100, a first processor 1101, an output device 1102, a first memory 1103 and at least one communication bus 1104. The communication bus 1104 is used to realize the communication connection between the components. The first memory 1103 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage. Various programs can be stored in the first memory 1103 to complete various processing functions and implement the method steps of this embodiment.

[0100] Optionally, the first processor 1101 may be implemented, for example, as a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and the processor 1101 is coupled to the input device 1100 and the output device 1102 via a wired or wireless connection.

[0101] Optionally, the input device 1100 may include multiple input devices, for example, at least one of a user interface for users, a device interface for devices, a software programmable interface, a camera, and a sensor. Optionally, the device interface for devices may be a wired interface for data transmission between devices, or a hardware insertion interface for data transmission between devices (such as a USB interface, a serial port, etc.); Optionally, the user interface for users may be, for example, a control button for users, a voice input device for receiving voice input, and a touch sensing device for users to receive user touch input (such as a touch screen with a touch sensing function, a touchpad, etc.); Optionally, the software programmable interface may be, for example, an entry for users to edit or modify programs, such as an input pin interface or an input interface of a chip; the output device 1102 may include output devices such as a display and a speaker.

[0102] In this embodiment, the processor of the terminal device includes functions for executing each module of the voice recognition device in each device. The specific functions and technical effects can be referred to the above embodiments and will not be repeated here.

[0103] Fig. 9 A schematic diagram of the hardware structure of a terminal device provided for another embodiment of the present application. Fig. 9 Yes Figure 8 A specific embodiment in the implementation process. As shown in the figure, the terminal device of this embodiment may include a second processor 1201 and a second memory 1202.

[0104] The second processor 1201 executes the computer program code stored in the second memory 1202 to implement the above embodiment. Figure 1 The method.

[0105] The second memory 1202 is configured to store various types of data to support operations on the terminal device. Examples of such data include instructions for any application or method used to operate on the terminal device, such as messages, pictures, videos, etc. The second memory 1202 may include a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage.

[0106] Optionally, the second processor 1201 is arranged in the processing component 1200. The terminal device may further include: a communication component 1203, a power component 1204, a multimedia component 1205, an audio component 1206, an input / output interface 1207 and / or a sensor component 1208. The specific components included in the terminal device are set according to actual needs, and this embodiment does not limit this.

[0107] The processing component 1200 generally controls the overall operation of the terminal device. The processing component 1200 may include one or more second processors 1201 to execute instructions to complete the above Figure 1 In addition, the processing component 1200 may include one or more modules to facilitate the interaction between the processing component 1200 and other components. For example, the processing component 1200 may include a multimedia module to facilitate the interaction between the multimedia component 1205 and the processing component 1200.

[0108] The power supply component 1204 provides power to various components of the terminal device. The power supply component 1204 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device.

[0109] The multimedia component 1205 includes a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a liquid crystal display (LCD) and a touch panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0110] The audio component 1206 is configured to output and / or input voice signals. For example, the audio component 1206 includes a microphone (MIC), and when the terminal device is in an operating mode, such as a voice recognition mode, the microphone is configured to receive an external voice signal. The received voice signal can be further stored in the second memory 1202 or sent via the communication component 1203. In some embodiments, the audio component 1206 also includes a speaker for outputting voice signals.

[0111] The input / output interface 1207 provides an interface between the processing component 1200 and the peripheral interface modules, which may be click wheels, buttons, etc. These buttons may include but are not limited to: volume buttons, start buttons, and lock buttons.

[0112] The sensor component 1208 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor component 1208 can detect the open / closed state of the terminal device, the relative positioning of the components, and the presence or absence of contact between the user and the terminal device. The sensor component 1208 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor component 1208 may also include a camera, etc.

[0113] The communication component 1203 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot for inserting a SIM card, so that the terminal device can log in to a GPRS network and establish communication with a server via the Internet.

[0114] From the above, we can see that Fig. 9 The communication component 1203, the audio component 1206, the input / output interface 1207, and the sensor component 1208 involved in the embodiment can all be used as Figure 8 Implementation method of the input device in the embodiment.

[0115] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An image detection method, characterized in that: The following steps are involved: Converting a target image to change the size and the number of channels of the target image; the target image after the size change has the same image quality as the original target image, and the size of the target image after the size change is smaller than the original target image; and / or the target image after the number of channels change has the same image quality as the original target image, and the number of channels of the target image after the size change is greater than the original target image; Converting the target image after changing the size and the number of channels into a feature map, and obtaining features of multiple scales from the feature map; Generate a prediction frame of the target image based on features of multiple different scales, optimize an image detection algorithm based on the prediction frame of the target image, and generate an image detection model; the process of generating a prediction frame of the target image based on features of multiple different scales includes: fusing some features of different scales so that features of one or more scales can perceive feature information of another one or more scales; extract features of the fused features and unfused features using a neural network block, and classify and regress the feature extraction results to generate a prediction frame of the target image; wherein the neural network block is derived from one or more neural networks; The image detection model is used to determine whether one or more target objects in the image to be detected have safety protection equipment; including: using the image detection model to detect the image to be detected and generate one or more prediction boxes; filtering the one or more prediction boxes for overlapping boxes so that each target object is associated with only one prediction box; obtaining the category and confidence score of each prediction box after filtering, and drawing the prediction box and the corresponding category onto the image to be detected when the confidence score of the prediction box is greater than a preset score threshold, and determining whether the one or more target objects have safety protection equipment based on the drawing result.

2. The image detection method according to claim 1, characterized in that: If the target image includes an image with completed frame annotation, it also includes: Comparing the predicted frame of the target image with the frame pre-labeled in the target image, and calculating the loss of one or more neural networks in the image detection algorithm according to the comparison result; One or more neural networks in the image detection algorithm are iteratively optimized based on the loss to generate the image detection model.

3. The image detection method according to claim 1, characterized in that: The target object includes a human head and / or a human body.

4. An image detection system, characterized in that: Included are: An image conversion module, used for converting a target image, changing the size and the number of channels of the target image; the target image after the size change has the same image quality as the original target image, and the size of the changed target image is smaller than the original target image; and / or the target image after the number of channels change has the same image quality as the original target image, and the number of channels of the changed target image is more than the original target image; A feature extraction module, used to convert the target image after changing the size and the number of channels into a feature map, and obtain features of multiple different scales from the feature map; An image detection model module is used to generate a prediction frame of the target image based on features of multiple different scales, optimize the image detection algorithm according to the prediction frame of the target image, and generate an image detection model; the process of generating the prediction frame of the target image based on features of multiple different scales includes: fusing some features of different scales so that features of one or more scales can perceive feature information of another one or more scales; extracting features of the fused features and unfused features using a neural network block, and classifying and regressing the feature extraction results to generate a prediction frame of the target image; wherein the neural network block is derived from one or more neural networks; An image detection module is used to use the image detection model to determine whether one or more target objects in the image to be detected have safety protection equipment; it includes: using the image detection model to detect the image to be detected and generate one or more prediction boxes; filtering the one or more prediction boxes for overlapping boxes so that each target object is associated with only one prediction box; obtaining the category and confidence score of each prediction box after filtering, and drawing the prediction box and the corresponding category onto the image to be detected when the confidence score of the prediction box is greater than a preset score threshold, and judging whether the one or more target objects have safety protection equipment based on the drawing result.

5. The image detection system according to claim 4, characterized in that: If the target image includes an image with completed frame annotation, it also includes: Comparing the predicted frame of the target image with the frame pre-labeled in the target image, and calculating the loss of one or more neural networks in the image detection algorithm according to the comparison result; One or more neural networks in the image detection algorithm are iteratively optimized based on the loss to generate the image detection model.

6. A computer device, characterized in that: include: one or more processors; and One or more machine-readable media storing instructions, when the one or more processors execute the instructions, cause the device to perform the method according to any one of claims 1 to 3.

7. One or more machine-readable media, characterized in that Instructions are stored thereon, and when the instructions are executed by one or more processors, the device performs the method according to any one of claims 1 to 3.

Images