Target detection method and device, storage medium, and processing equipment

Abstract

The present application relates to a target detection method and device, a storage medium, and a processing device, the method comprising: determining an object image region, the object image region being obtained based on a target object in an image to be detected; determining a reference image region from the object image region, the reference image region being a sub-image region of the object image region; and determining the number of target objects in the object image region based on the object image region and the reference image region. The present application can determine the number of target objects in the object image region based on the number of image regions in the object image region that match the reference image region, so that the determined number of target objects is more accurate.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target detection method and device, a storage medium, and a processing device. Background Art

[0002] Object Detection, as an important branch of image processing and computer vision, is widely used in many fields such as intelligent video surveillance, industrial inspection, aerospace, etc.

[0003] Generally, target detection technology can identify all the targets (objects) of interest in the target image and determine their categories and locations. However, due to the different shapes and postures of various targets, target detection technology cannot accurately determine the specific number of target objects in the image, resulting in low detection speed and low accuracy of target detection. Therefore, an efficient target detection method is urgently needed in the relevant technology. Summary of the invention

[0004] Based on this, in response to the above technical problems, the present application provides a target detection method and device, a storage medium, and a processing device to at least solve the problem of low detection speed and low accuracy of target detection in related technologies.

[0005] In a first aspect, an embodiment of the present application provides a target detection method, the method comprising:

[0006] Determine an object image region, wherein the object image region is obtained based on a target object in the image to be detected;

[0007] Determine a reference image region from the object image region, wherein the reference image region is a sub-image region of the object image region;

[0008] Based on the object image region and the reference image region, the number of the target objects in the object image region is determined.

[0009] The embodiment of the present application provides a target detection method, which can determine the object image area of ​​the image to be detected, and then determine the corresponding reference image area based on the object image area. Finally, the number of target objects in the object image area can be determined based on the reference image area and the object image area. Since the reference image area is a sub-image area of ​​the object image area, it contains more feature information of the object image area. Therefore, the number of target objects in the object image area can be determined based on the number of image areas in the object image area that match the reference image area, so that the determined number of target objects is more accurate.

[0010] Optionally, in an embodiment of the present application, determining the number of the target objects in the object image area based on the object image area and the reference image area includes:

[0011] Determine an image feature matching result between the object image region and the reference image region;

[0012] The number of the target objects in the object image region is determined according to the feature distribution in the image feature matching result.

[0013] Optionally, in one embodiment of the present application, determining the image feature matching result between the object image region and the reference image region includes:

[0014] Determining a feature map of the object image region and a feature map of the reference image region;

[0015] A cross-correlation process is performed on the feature map of the object image region and the feature map of the reference image region to generate a feature matching response map.

[0016] Optionally, in one embodiment of the present application, determining the number of the target objects in the object image area according to the feature distribution in the image feature matching result includes:

[0017] Determine the peak point in the feature matching response graph;

[0018] The number of the target objects in the object image area is determined according to the number of the peak points.

[0019] Optionally, in an embodiment of the present application, determining the number of target objects in the object image area according to the number of peak points includes:

[0020] When the number of the peak points is not greater than the number threshold, the number of the target objects in the object image region is determined to be the number of the peak points.

[0021] Optionally, in one embodiment of the present application, determining the number of target objects in the object image area according to the number of peak points includes:

[0022] In the case that the number of the peak points is greater than the number threshold, filtering the peak points according to a preset condition, the filtering process comprising: filtering the peak points whose response values ​​are less than the preset threshold from the peak points;

[0023] According to the filtering result of the filtering process, the number of the target objects in the object image area is determined.

[0024] Optionally, in one embodiment of the present application, determining a reference image region from the object image region includes:

[0025] Dividing the object image region into a plurality of image blocks;

[0026] Recognize feature points of the plurality of image blocks respectively to determine feature points of each of the image blocks;

[0027] An image block whose number of feature points meets a preset number requirement is used as a reference image area corresponding to the object image area.

[0028] Optionally, in one embodiment of the present application, before determining the object image area, the method further includes:

[0029] Determine at least one object image region obtained by detecting the target object in the image to be detected;

[0030] The determining the number of the target objects in the object image area based on the object image area and the reference image area further includes:

[0031] Based on the number of target objects contained in each of the at least one object image region, the total number of target objects contained in the image to be detected is determined.

[0032] Optionally, in one embodiment of the present application, the image to be detected is collected for the area to be detected, and the method further includes:

[0033] In response to the total number satisfying a first preset requirement, it is determined that the environmental state of the area to be detected is abnormal.

[0034] Optionally, in one embodiment of the present application, the method further includes:

[0035] In response to the number of target objects in the object image area meeting a second preset requirement, it is determined that the environmental state of the area to be detected is abnormal.

[0036] Optionally, in one embodiment of the present application, the image to be detected is collected for the area to be detected, and the method further includes:

[0037] Determining the type of each of the target objects;

[0038] Based on the type of each target object and the number of the target objects, it is determined whether the environmental state of the area to be detected is abnormal.

[0039] In a second aspect, an embodiment of the present application provides a target detection device, the device comprising:

[0040] A first determination module is used to determine an object image region, where the object image region is obtained based on a target object in an image to be detected;

[0041] A second determining module, configured to determine a reference image region from the object image region, wherein the reference image region is a sub-image region of the object image region;

[0042] The third determination module is used to determine the number of the target objects in the object image area based on the object image area and the reference image area.

[0043] In a third aspect, an embodiment of the present application provides a processing device, including a memory and a processor, wherein the memory stores computer program instructions, and the processor implements the steps of the target detection method described above when executing the computer program instructions.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the steps of the target detection method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A system architecture diagram of a target detection method provided in one embodiment of the present application;

[0046] Figure 2 A schematic diagram of a flow chart of a target detection method provided by an embodiment of the present application;

[0047] Figure 3 A schematic diagram of the input and output of a fire extinguisher detection model 300 provided in one embodiment of the present application;

[0048] Figure 4 A schematic diagram of the input and output of a fire extinguisher quantity detection model 400 provided in one embodiment of the present application;

[0049] Figure 5 A schematic diagram of the module structure of the target detection device 103 provided in one embodiment of the present application;

[0050] Figure 6 A schematic diagram of the module structure of a processing device 600 provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0052] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0053] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. The terms "first", "second", "third" and the like involved in this application are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0054] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, devices, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0055] In the embodiments of the present application, the fire extinguisher target detection can be used as an example for illustration without limitation. Figure 1 One exemplary scenario of an embodiment of the present application is described.

[0056] In an application scenario, fire extinguishers are important firefighting facilities for preventing and fighting fires. Usually, in areas where fires may occur in enterprises, factories or other public places, a certain number of fire extinguishers must be configured in accordance with the provisions of national fire protection technical specifications. Therefore, the target detection method provided in the embodiment of the present application can be used to determine the number of fire extinguishers in the image to be detected.

[0057] See also Figure 1 . Specifically, the acquisition device 101 can be used to collect images of the area to be detected. Among them, the acquisition device 101 can be an electronic device with data acquisition capabilities and data transmission and reception capabilities. For example, the acquisition device 101 can include an electronic device that can collect image information and / or video information of the area to be detected, for example, it can include a camera, a laser radar, etc. Among them, the camera can include a monocular camera, an infrared camera, a multi-eye camera, a depth camera, etc. The laser radar can include a single-line laser radar, a multi-line laser radar, etc., and the present application does not limit this. The acquisition device 101 can send the collected image to be detected of the area to be detected to the target detection device 103. The target detection device 103 can determine the object image area in the image to be detected based on the target object detection result of the image to be detected. Then, a reference image area can be determined from the object image area, and the reference image area is a sub-image area of ​​the object image area. Finally, the number of target objects contained in the object image area can be determined based on the object image area and the reference image area. It should be noted that the target detection device 103 can also be coupled to the inside of the acquisition device 101, so that the acquisition device 101 has the function of determining the number of target objects. For example, the acquisition device 101 is a camera, and the target detection device 103 can also be a server. The acquisition device 101 can communicate with the server, send the collected image to be detected to the server, and the server completes the target detection. In this way, the number of fire extinguishers in the fire extinguisher image area can be automatically determined, which saves manpower and is more efficient.

[0058] The target detection method described in this application is described in detail below with reference to the accompanying drawings. Figure 2It is a method flow diagram of an embodiment of the target detection method provided by the present application. Although the present application provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application. When the method is executed in the actual target detection process or device, it can be executed in sequence or in parallel according to the method shown in the embodiment or drawings (for example, a parallel processor or a multi-threaded processing environment).

[0059] Specifically, an implementation example of the target detection method provided in the present application is as follows: Figure 2 As shown, the method may include:

[0060] Step 201: Determine an object image region, where the object image region is obtained based on a target object in an image to be detected.

[0061] In the embodiment of the present application, the object image region may be an image region containing a target object. The target object may include various types of target objects, such as pedestrians, vehicles, fire extinguishers, and the like.

[0062] In the following content of this article, a fire extinguisher is used as an example of a target object for explanation. The fire extinguisher may include, for example, a push-type fire extinguisher, a portable fire extinguisher, and the like. The object image area may be obtained based on target object detection of the image to be detected. The image to be detected may include an image or video stream collected by the acquisition device 101 for the area to be detected. The image to be detected may include any format such as BMP, JPEG, PNG, SVG, etc. In addition, when the acquisition device 101 acquires the video stream, it may decode the video stream to obtain a plurality of frames of images, and then obtain the image to be detected from the plurality of frames of images.

[0063] After acquiring the image to be detected, target object detection can be performed on the image to be detected, and the object image area can be determined based on the detection result. In one embodiment of the present application, a target detection model can be used to detect the target object. The target detection model may include a model trained using machine learning. The model may include a convolutional neural network model (CNN), AlexNet, YOLO, ResNet, ResNet1001 (pre-activation), Hourglass, Inception, Xception, SENet, etc. Specifically, in one example, Figure 3As shown, the fire extinguisher target detection model 300 can be used to determine the fire extinguisher image area of ​​the image to be detected. The fire extinguisher target detection model 300 may include a feature extraction network layer, a fully connected layer, and the like. The feature extraction network layer may perform feature extraction on the target image to obtain a feature map of the target image. The feature map may include feature information such as grayscale, edge, texture, color, gradient histogram, etc. of the target object. Thereafter, the fully connected layer may classify and locate the feature map, and finally the fire extinguisher target detection model 300 may output the fire extinguisher image area in the target image.

[0064] Step 203: Determine a reference image region from the object image region, where the reference image region is a sub-image region of the object image region.

[0065] In an embodiment of the present application, the reference image may be a sub-image area of ​​the object image area. In one embodiment of the present application, the reference image area may be a sub-image area containing a target object in the object image area whose number of feature points meets a preset number requirement. The situation where the number of feature points of the target object in the object image area meets the preset number requirement may be that the number of feature points of the target object in the reference image area is the largest, or that the number of feature points of the target object in the reference image area is greater than a preset number threshold, and the present application does not impose any limitation on this. Specifically, in one embodiment of the present application, determining the reference image area from the object image area may include:

[0066] Step 301: Divide the object image area into a plurality of image blocks;

[0067] Step 303: performing feature point recognition on the plurality of image blocks respectively to determine a feature point of each image block;

[0068] Step 305: Use the image block whose number of feature points meets the preset number requirement as the reference image area corresponding to the object image area.

[0069] In an embodiment of the present application, the reference image area can be determined by dividing the image blocks obtained by dividing the object image area and performing feature point recognition on them. Specifically, the object image area can be subjected to image segmentation processing to cut the object image area into multiple image blocks. In an embodiment of the present application, the number of image blocks to be cut can be determined according to the size of the object image area, for example, it can be cut into 4 blocks, 6 blocks, 8 blocks, etc. In one example, when the size of the object image area is large, a larger number of cuts can be selected to obtain a larger number of cuts. The direction of the cuts can include horizontal cuts, vertical cuts, etc. After the cuts are completed, feature point recognition can be performed on the multiple image blocks, and the feature point recognition can be implemented based on a feature point recognition algorithm. Specifically, the feature point recognition algorithm can include a Scale Invariant Feature Transform (SIFT) algorithm, a Speeded Up Robust Features (SURF) algorithm, an Oriented Fast and Rotated Brief (ORB) algorithm, etc. After the feature points of the multiple image blocks are identified, the image block containing the largest number of feature points among the multiple image blocks can be determined as the reference image area corresponding to the object image area. Of course, in another embodiment of the present application, the image block containing the number of feature points higher than the average value among the multiple image blocks can also be selected as the reference image area, and the present application does not impose any restrictions on this. It should be noted that since the feature point recognition algorithm remains stable and invariant to rotation, scale transformation, brightness, etc., the direction of cutting will not affect the accuracy of selecting the reference image, so the present application does not impose any restrictions on the cutting direction.

[0070] Through the above embodiment, feature point recognition can be performed on the segmented image blocks, and the image blocks whose feature point quantity meets the preset quantity requirement can be used as the reference image area. Therefore, the determined reference image area can contain more feature points. In addition, since the feature point recognition algorithm has rotation invariance and has a certain degree of stability to perspective change and noise, the determined reference image area is more stable and accurate.

[0071] Step 203: Determine the number of the target objects in the object image area based on the object image area and the reference image area.

[0072] In an embodiment of the present application, since the object image area is an image obtained by performing target object detection on the image to be detected, the image often only includes the image of the target object. For example, the result output from the fire extinguisher target detection model 300 only includes the fire extinguisher image area. Then, the reference image area extracted from the object image area may contain more feature information of the target object. Therefore, the number of target objects in the object image area can be determined based on the number of image areas in the object image area that match the reference image area. Specifically, in one embodiment of the present application, determining the number of target objects in the object image area based on the object image area and the reference image area may include:

[0073] Step 401: Determine the image feature matching result between the object image region and the reference image region;

[0074] Step 403: Determine the number of the target objects in the object image region according to the feature distribution in the image feature matching result.

[0075] In the embodiment of the present application, the number of the target objects can be determined according to the image feature matching results of the object image area and the reference image area. Since there can be pixels with identification in an image, these pixels can be used to characterize the features of an image and become feature information. Therefore, the feature information of the object image area and the feature information of the reference image area can be extracted respectively, and the feature information of the object image area and the feature information of the reference image area are matched to determine the feature matching result. Finally, the image feature matching result can be determined according to the feature matching result. The image feature matching result can include the correspondence and matching degree of the feature information of the object image area and the feature information of the reference image area. The image feature matching result can include an image feature matching response map, an image feature matching response matrix, etc. In the embodiment of the present application, after determining the image feature matching result, the number of the target objects can be determined according to the feature distribution in the image feature matching result. The feature distribution can include the matching degree of the feature information of the object image area and the feature information of the reference image area. The matching degree can include similarity, correlation, etc. The feature distribution can also include the position information of the feature points in the reference image area and the feature points that match in the object image area. For example, the matched feature point is at the position (30, 40) of the object image region. It should be noted that the more feature points are included in the reference image region, the more accurate the feature matching result is.

[0076] Through the above embodiment, the reference image area of ​​the object image area can be feature matched with the object image area, and the obtained feature matching result can reflect the number of image areas in the object image area that match the reference image area. Finally, the number of target objects can be determined based on the number of image areas, thereby improving the accuracy of target detection.

[0077] Further, in one embodiment of the present application, the determining of the image feature matching result between the object image region and the reference image region includes:

[0078] Step 501: Determine a feature map of the object image region and a feature map of the reference image region;

[0079] Step 503: performing cross-correlation processing on the feature map of the object image region and the feature map of the reference image region to generate a feature matching response map.

[0080] In an embodiment of the present application, feature extraction can be performed on the reference image and the object image area respectively, and the feature map corresponding to the reference image area and the feature map corresponding to the object image area can be determined respectively. Then, the feature map of the object image area and the feature map of the reference image area can be cross-correlated. The cross-correlation processing can include using an image registration algorithm to perform a cross-correlation operation on the feature map of the object image area and the feature map of the reference image area to obtain a response value of each pixel. Among them, the larger the response value, the higher the matching degree. In some examples, the cross-correlation processing can be implemented using algorithms such as the Normalized Cross Correlation method (NCC) matching algorithm and the Sequential Similarity Detection Algorithm (SSDA). Finally, the feature matching response map can be determined based on the response value. The value of each pixel in the feature matching response map can be used to represent the response value of the pixel corresponding to the feature map of the object image area and the feature map of the reference image area.

[0081] In one embodiment of the present application, the feature matching response map can be determined using a target object quantity detection model. The target object quantity detection model may include convolutional neural networks (CNN) and various network modules based on CNN, such as AlexNet, ResNet, ResNet1001 (pre-activation), Hourglass, Inception, Xception, SENet, etc., which are not limited in the present application. Specifically, in one example, Figure 4 As shown, the fire extinguisher quantity detection model includes a reference image area determination module, a feature extraction module, and a feature matching module. Among them, the reference image area determination module can determine the reference image area corresponding to the fire extinguisher image area according to the method of determining the reference image area from the object image area as described above. The feature extraction module can be composed of multiple convolutional layers and pooling layers. The feature matching module can be composed of fully connected layers. In one embodiment of the present application, the feature extraction module may include a twin basic network with shared weights, and the twin basic network can be implemented using a variety of network structures that are helpful for expressing image area features, such as fully convolutional networks (FCN), U-type networks (U Net), deep lab version 2 (deeplabv2) or deeplab v3, etc. In one embodiment of the present application, the feature extraction module can perform feature extraction on the fire extinguisher image area I and the reference image area E respectively, and obtain the feature map Feat of the corresponding fire extinguisher image area. I And the feature map Feat of the reference image area E The feature extraction module may include five convolution layers. The first convolution layer may perform convolution calculations with a 1×1 convolution kernel to reduce the number of feature channels to 256, and then obtain a transformed feature map by convolution with a 3×3 convolution kernel. Then, the feature matching module may perform convolution calculations with a 1×1 convolution kernel to reduce the number of feature channels to 256. I And the feature map Feat of the reference image area E Perform feature matching, for example, the feature map Feat of the reference image area can be E As the convolution kernel and the feature map Feat of the fire extinguisher image area I A convolution operation is performed to generate a feature matching response map. Finally, the number of fire extinguishers corresponding to the fire extinguisher image area is determined according to the feature matching response map.

[0082] Through the above embodiment, the number of target objects corresponding to the object image region can be determined according to the feature matching response map. Since the number of target objects is determined according to the feature matching response map, the feature matching response map is not affected by factors such as the placement position of the target objects in the object image region and whether the target objects are adhered, that is, it is less affected by noise and has higher robustness. Therefore, the number of target objects determined is more accurate.

[0083] Further, in one embodiment of the present application, determining the number of the target objects in the object image region according to the feature distribution in the image feature matching result includes:

[0084] Step 601: determining the peak point in the feature matching response graph;

[0085] Step 603: Determine the number of the target objects in the object image area according to the number of the peak points.

[0086] In an embodiment of the present application, the quantity information of the target objects in the object image area can be determined according to the number of peak points of the feature matching response graph. Specifically, in one embodiment of the present application, when the number of peak points in the feature matching response graph is less than a quantity threshold, the number of peak points can be used as the number of target objects in the object image area. The quantity threshold can be set by the user according to actual needs, and can also be set based on the response characteristics of the feature matching response graph. Specifically, when there is only one peak point in the feature matching response graph, it can be determined that the template image and its own image area in the fire extinguisher image produce a peak response, so the quantity threshold can be set to 1.

[0087] Further, in one embodiment of the present application, determining the number of target objects in the object image area according to the number of peak points includes:

[0088] Step 701: When the number of the peak points is greater than the number threshold, filtering the peak points according to a preset condition, wherein the filtering includes: filtering the peak points whose response values ​​are less than the preset threshold from the plurality of peak points;

[0089] Step 703: Determine the number of the target objects in the object image area according to the filtering result of the filtering process.

[0090] In an embodiment of the present application, when the number of the peak points is greater than the number threshold, the plurality of peak points can be filtered based on a preset condition. The filtering process can include not counting the number of peak points that do not meet the preset condition in the plurality of peak points in the number of the peak points, that is, the number of peak points that meet the preset condition in the peak points can be determined as the number of the target objects. After determining the number of the peak points, the number of corresponding target objects in the object image area can be determined. Specifically, in an embodiment of the present application, the preset condition can include that the matching degree at the peak point is greater than the preset threshold. It should be noted that in order to ensure the accuracy of screening the peak points, the preset condition can also include that the matching degree of the response points adjacent to the peak point is less than the matching degree at the peak point, so as to prevent the occurrence of a matching situation where one target object corresponds to two peak points. Specifically, the peak points can be screened by adding a Hamming window function to the feature matching response graph. The Hamming window function can be set to a non-zero value at the peak point and a zero value in the remaining intervals.

[0091] Through the above embodiment, the peak points in the feature matching response graph can be filtered based on the preset conditions, so the number of peak points determined is more accurate, thereby improving the detection accuracy of the target detection method.

[0092] In practical applications, there may be multiple target objects in the image to be detected. For example, the target object may exist in the upper left corner, the upper right corner, etc. of the image to be detected. Therefore, it is necessary to count the number of multiple object image areas to determine the number of target objects in the image to be detected. Specifically, in one embodiment of the present application, before determining the object image area, it also includes: determining at least one of the object image areas obtained by detecting the target objects in the image to be detected; determining the number of the target objects in the object image area based on the object image area and the reference image area, and also includes: determining the total number of target objects contained in the image to be detected based on the number of the target objects contained in each object image area of ​​the at least one object image area.

[0093] In an embodiment of the present application, a detection result obtained by detecting a target object in an image to be detected can be obtained. The detection result may include at least one object image area. After determining at least the object image area according to the detection result, the number of target objects in each object image area can be determined according to the target detection method provided in each of the above embodiments. The total number of target objects contained in the image to be detected can be determined by counting the number of target objects in each object image area.

[0094] Furthermore, in one embodiment of the present application, the image to be detected is collected for the area to be detected, and the method further includes: in response to the total number satisfying a first preset requirement, determining that the environmental state of the scene to be detected is abnormal.

[0095] In an embodiment of the present application, the total number can be analyzed to determine whether the total number meets the first preset requirement. The situation that the total number of target objects contained in the image to be detected does not meet the first preset requirement may include that the total number of target objects in the current frame of the image to be detected does not match the total number of target objects in the previous frame of the image to be detected, and may also include that the total number of target objects in the image to be detected does not match the number threshold of target objects pre-set in the area to be detected, and the present application is not limited here. In the case where it is determined that the position information and / or quantity information of the target objects in the target image do not meet the preset requirements, it can be determined that the environmental state of the area to be detected is abnormal. In the case where it is determined that the environmental state of the area to be detected is abnormal, an abnormal reminder message can be sent to a client or other device. The client may include a host, a computer, a tablet computer, a digital assistant (PDA), a smart phone, a smart wearable device, etc., and the present application is not limited here. The abnormal reminder message may include continuously flashing an alarm indicator light, sounding an alarm bell, etc. It should be noted that, in the case where the total number of target objects in the image to be detected at the current moment does not meet the first preset requirement, it is determined that the environmental state of the area to be detected at the current moment is abnormal. If the total number of target objects in the image to be detected at the next moment meets the first preset requirement, the environmental state is restored to a normal state.

[0096] Through the above-mentioned embodiments, the total number of target objects in the image to be detected can be analyzed and processed in a timely manner. When the total number does not meet the first preset requirement, it is determined that the environmental state of the scene to be detected is abnormal, so that the user can obtain the abnormal environmental state of the area to be detected in real time for timely processing, thereby reducing the possibility of danger.

[0097] Furthermore, in another embodiment of the present application, the method further includes:

[0098] Step 801: Determine the type of each target object;

[0099] Step 803: Based on the type of each target object and the number of target objects, determine whether the environmental state of the area to be detected is abnormal.

[0100] In an embodiment of the present application, the type of each target object can be determined, for example, the type is determined to be a pedestrian, a fire extinguisher, a vehicle, etc. Specifically, in one embodiment of the present application, the number corresponding to each target object can be determined. Then, the minimum value of the number corresponding to each target object is determined. When the minimum value is less than a preset number threshold, the environmental state of the area to be detected is determined to be abnormal. For example, it can be determined that the number of hand-push fire extinguishers in the image to be detected is 3, and the number of portable fire extinguishers is 6. When it is determined that the number of hand-push fire extinguishers 3 is less than a preset number threshold 4, the environmental state of the area to be detected is determined to be abnormal. In another embodiment of the present application, the target object can have a preset alarm condition. The preset alarm condition can include the location information of the target object and / or the location information does not meet the preset condition. For example, within the scope of the factory assembly line area, according to regulations, two fire extinguishers should be placed at the position (10, 20) and the position (50, 100), so the preset location information is (10, 20) and (50, 100), and the preset number information is 4. When it is determined that the position information of the fire extinguisher in the image to be detected does not match the position information of (10, 20) and (50, 100) and / or the number of target objects in the target image is less than four, it is determined that the environmental state of the area to be detected is an abnormal state.

[0101] Through the above embodiments, the abnormal status of different target objects can be judged according to the type and number of target objects in the detection area, and abnormal reminder information can be sent in time, so that the user can obtain the abnormal status of different target objects in real time for timely processing, thereby reducing the possibility of danger.

[0102] On the other hand, the present application also provides a target detection device 103, such as Figure 5 As shown, the device 103 includes:

[0103] A first determination module 1031 is used to determine an object image region; the object image region is determined based on a result of target object detection on the image to be detected;

[0104] A second determining module 1033 is used to determine a reference image region from the object image region, where the reference image region is a sub-image region of the object image region;

[0105] The third determination module 1035 is configured to determine the number of the target objects in the object image region based on the object image region and the reference image region.

[0106] Optionally, in an embodiment of the present application, determining the number of the target objects in the object image area based on the object image area and the reference image area includes:

[0107] Determine an image feature matching result between the object image region and the reference image region;

[0108] The number of the target objects in the object image region is determined according to the feature distribution in the image feature matching result.

[0109] Optionally, in one embodiment of the present application, determining the image feature matching result between the object image region and the reference image region includes:

[0110] Determining a feature map of the object image region and a feature map of the reference image region;

[0111] A cross-correlation process is performed on the feature map of the object image region and the feature map of the reference image region to generate a feature matching response map.

[0112] Optionally, in one embodiment of the present application, determining the number of the target objects in the object image area according to the feature distribution in the image feature matching result includes:

[0113] Determine the peak point in the feature matching response graph;

[0114] The number of the target objects in the object image area is determined according to the number of the peak points.

[0115] Optionally, in an embodiment of the present application, determining the number of target objects in the object image area according to the number of peak points includes:

[0116] When the number of the peak points is not greater than the number threshold, the number of the target objects in the object image region is determined to be the number of the peak points.

[0117] Optionally, in one embodiment of the present application, determining the number of target objects in the object image area according to the number of peak points includes:

[0118] In the case that the number of the peak points is greater than the number threshold, filtering the peak points according to a preset condition, the filtering process comprising: filtering the peak points whose response values ​​are less than the preset threshold from the peak points;

[0119] According to the filtering result of the filtering process, the number of the target objects in the object image area is determined.

[0120] Optionally, in one embodiment of the present application, determining a reference image region from the object image region includes:

[0121] Dividing the object image region into a plurality of image blocks;

[0122] Recognize feature points of the plurality of image blocks respectively to determine feature points of each of the image blocks;

[0123] An image block whose number of feature points meets a preset number requirement is used as a reference image area corresponding to the object image area.

[0124] Optionally, in one embodiment of the present application, before determining the object image area, the method further includes:

[0125] Determine at least one object image region obtained by detecting the target object in the image to be detected;

[0126] The determining the number of the target objects in the object image area based on the object image area and the reference image area further includes:

[0127] Based on the number of target objects contained in each of the at least one object image region, the total number of target objects contained in the image to be detected is determined.

[0128] Optionally, in one embodiment of the present application, the image to be detected is collected for the area to be detected, and the method further includes:

[0129] In response to the total number satisfying a first preset requirement, it is determined that the environmental state of the area to be detected is abnormal.

[0130] Optionally, in one embodiment of the present application, the method further includes:

[0131] In response to the number of target objects in the object image area meeting a second preset requirement, it is determined that the environmental state of the area to be detected is abnormal.

[0132] Optionally, in one embodiment of the present application, the image to be detected is collected for the area to be detected, and the method further includes:

[0133] Determining the type of each of the target objects;

[0134] Based on the type of each target object and the number of target objects, determine whether the environmental state of the area to be detected is abnormal. On the other hand, the present application also provides a processing device, including a memory and a processor, wherein the memory stores computer program instructions, and the processor is configured to run the computer program instructions to execute the target detection method described in each of the above embodiments.

[0135] The processing device may be a physical device or a cluster of physical devices, or a virtualized cloud device, such as at least one cloud computing device in a cloud computing cluster. For ease of understanding, the present application uses the processing device as an independent physical device to illustrate the structure of the processing device.

[0136] like Figure 6 As shown, the processing device 600 includes: a processor and a memory for storing processor computer program instructions; wherein the processor is configured to implement the above-mentioned device when executing the computer program instructions. The processing device 600 includes a memory 601, a processor 603, a bus 605 and a communication interface 607. The memory 601, the processor 603 and the communication interface 607 communicate through the bus 605. The bus 605 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The communication interface 607 is used for communicating with the outside.

[0137] The processor 603 may be a central processing unit (CPU). The memory 801 may include a volatile memory, such as a random access memory (RAM). The memory 601 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD or a SSD, etc.

[0138] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] On the other hand, the present application further provides a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions implement the steps of the above-mentioned target detection method when executed by a processor.

[0140] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (Random Access Memory, RAM), a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), a static random access memory (Static Random-Access Memory, SRAM), a portable compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a digital versatile disc (Digital Video Disc, DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0141] The computer program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer program instructions from the network and forwards the computer program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0142] The computer program instructions for performing the operation of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Small talk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, by utilizing the status information of computer program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit can execute the computer program instructions to implement various aspects of the present application.

[0143] Various aspects of the present application are described herein with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions.

[0144] These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0145] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0146] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, system and method according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved.

[0147] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: The method comprises: Determine an object image region, wherein the object image region is obtained based on a target object in the image to be detected; Determine a reference image region from the object image region, the reference image region being a sub-image region of the object image region; the reference image region being a sub-image region containing feature points of a target object in the object image region and having a number that meets a preset number requirement; Determining the number of the target objects in the object image area based on the object image area and the reference image area; The method of determining the number of target objects in the object image area based on the object image area and the reference image area includes: determining an image feature matching result between the object image area and the reference image area; and determining the number of target objects in the object image area based on a feature distribution in the image feature matching result.

2. The method according to claim 1, characterized in that The determining of the image feature matching result between the object image region and the reference image region includes: Determining a feature map of the object image region and a feature map of the reference image region; A cross-correlation process is performed on the feature map of the object image region and the feature map of the reference image region to generate a feature matching response map.

3. The method according to claim 2, characterized in that Determining the number of the target objects in the object image area according to the feature distribution in the image feature matching result includes: Determine the peak point in the feature matching response graph; The number of the target objects in the object image area is determined according to the number of the peak points.

4. The method according to claim 3, characterized in that: The determining the number of the target objects in the object image area according to the number of the peak points includes: When the number of the peak points is not greater than the number threshold, the number of the target objects in the object image region is determined to be the number of the peak points.

5. The method according to claim 3, characterized in that: The determining the number of the target objects in the object image area according to the number of the peak points includes: In the case that the number of the peak points is greater than the number threshold, filtering the peak points according to a preset condition, the filtering process comprising: filtering the peak points whose response values ​​are less than the preset threshold from the peak points; According to the filtering result of the filtering process, the number of the target objects in the object image area is determined.

6. The method according to claim 1, characterized in that The step of determining a reference image region from the object image region comprises: Dividing the object image region into a plurality of image blocks; Recognize feature points of the plurality of image blocks respectively to determine feature points of each of the image blocks; An image block whose number of feature points meets a preset number requirement is used as a reference image area corresponding to the object image area.

7. The method according to any one of claims 1 to 6, characterized in that: Before determining the object image area, the method further includes: Determine at least one object image region obtained by detecting the target object in the image to be detected; The determining the number of the target objects in the object image area based on the object image area and the reference image area further includes: Based on the number of target objects contained in each of the at least one object image region, the total number of target objects contained in the image to be detected is determined.

8. The method according to claim 7, characterized in that The image to be detected is collected for the area to be detected, and the method further includes: In response to the total number satisfying a first preset requirement, it is determined that the environmental state of the area to be detected is abnormal.

9. The method according to any one of claims 1 to 6 and 8, characterized in that: The method further comprises: In response to the number of target objects in the object image area meeting a second preset requirement, it is determined that the environmental state of the area to be detected is abnormal.

10. The method according to claim 1, characterized in that The image to be detected is collected for the area to be detected, and the method further includes: Determining the type of each of the target objects; Based on the type of each target object and the number of the target objects, it is determined whether the environmental state of the area to be detected is abnormal.

11. A target detection device, characterized in that: The device comprises: A first determination module is used to determine an object image area; the object image area is determined based on a result of target object detection on the image to be detected; A second determination module is used to determine a reference image region from the object image region, wherein the reference image region is a sub-image region of the object image region; the reference image region is a sub-image region containing feature points of the target object in the object image region that meet a preset number requirement; A third determination module, configured to determine the number of the target objects in the object image area based on the object image area and the reference image area; The method of determining the number of target objects in the object image area based on the object image area and the reference image area includes: determining an image feature matching result between the object image area and the reference image area; and determining the number of target objects in the object image area based on a feature distribution in the image feature matching result.

12. A processing device, comprising a memory and a processor, wherein the memory stores computer program instructions, characterized in that: When the processor executes the computer program instructions, the steps of the target detection method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the target detection method according to any one of claims 1 to 10 are implemented.

Images