Target detection method and device, equipment and medium

Abstract

The embodiment of the invention discloses a target detection method and device, equipment and a medium, and the method comprises the steps: inputting a currently collected image of a warning region into a preset small model, and obtaining the position information of each target in the image outputted by the small model; and a preset multi-mode large model can identify the types of all the alarm targets in the image according to the image and the position information of each alarm target, so that subsequent target detection is realized. Compared with a small model, the multi-modal large model generally has stronger multi-modal data processing capability and can better capture fine features of a long-distance small target, so that the types of all alarm targets in the image are identified through the multi-modal large model, and the identification accuracy is improved. Therefore, the type identification accuracy of the long-distance small target by the perimeter precaution is improved, and the overall identification accuracy of the image by the perimeter precaution is improved.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a target detection method, apparatus, device and medium. Background Technology

[0002] Currently, network cameras (IPC cameras) that support perimeter security are widely deployed in residential areas, courtyards, farms, ports and other scenarios to monitor for suspicious targets, such as people, motor vehicles, animals and others, that may intrude into the security area.

[0003] Existing perimeter security systems employ small convolutional neural network (CNN) models for target detection, tracking, and type identification, with accuracy generally meeting requirements. However, a problem remains: low accuracy in identifying distant targets. For example, while these CNN models can detect and track targets, their accuracy in identifying distant targets is not high. To avoid excessive false alarms, existing technologies typically filter out such targets, resulting in current perimeter security systems failing to accurately identify distant target types, thus leading to low perimeter security accuracy. Summary of the Invention

[0004] This application provides a target detection method, apparatus, equipment, and medium for detecting targets in a warning area, thereby improving the accuracy of perimeter security identification.

[0005] In a first aspect, embodiments of this application provide a target detection method, the method comprising: Input the currently acquired image of the warning area into a preset small model, and obtain the location information of each target in the image output by the small model; The image and the location information of each target are input into a preset multimodal large model to identify the type of each target; Obtain the target type that triggers an alarm in the warning area; if the target type in the image contains the target type, then determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

[0006] In one possible implementation, the step of inputting the currently acquired image of the warning area into a preset small model and obtaining the location information of each target in the image output by the small model includes: Input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; After inputting the currently acquired image of the warning area into a preset small model to obtain the location information and first type of each target in the image output by the small model, and before inputting the image and the location information of each target into a preset multimodal large model to identify the type of each target, the method further includes: For each target in the image, if the first type is unknown, the target is identified as a target to be detected; otherwise, the accuracy of the first type of the target is determined based on the target's location information. If the accuracy is less than a preset threshold, the target is identified as a target to be detected. The step of inputting the image and the location information of each target into a preset multimodal large model includes: The image and the location information of each target to be detected are input into a preset multimodal large model.

[0007] In one possible implementation, the method further includes: If there is no target to be detected in the image, the target type that triggers an alarm for the warning area is obtained; if the first type of target identified in the image includes the target type, it is determined that there is an alarm target in the image, and the location information of the alarm target of the target type is marked and displayed in the image.

[0008] In one possible implementation, determining the accuracy of the first type of target based on the target's location information includes: Based on the location information, determine the target's width and height; The first sum value is determined based on the first product of the width and the preset first weight value, and the second product of the height and the preset second weight value; The second sum value is determined based on the first ratio of the width to the preset stable detection width and the second ratio of the height to the preset stable detection height; The accuracy of the first type of target is determined based on the first sum and the second sum.

[0009] In one possible implementation, before inputting the currently acquired image of the warning area into a preset small model, the method further includes: Based on the preset small model's suitable image size, the image of the currently acquired warning area is adjusted to obtain an adjusted first image; and the adjusted first image is used as the image of the currently acquired warning area.

[0010] In one possible implementation, before inputting the image and the location information of each target into a preset multimodal large model, the method further includes: Based on the preset image size for multimodal large model adaptation, the image and the position information of each target are adjusted to obtain the adjusted second image and the adjusted position information of each target; The step of inputting the image and the location information of each target into a preset multimodal large model includes: The second image and the adjusted position information of each target are input into a preset multimodal large model.

[0011] Secondly, embodiments of this application provide a target detection apparatus, the apparatus comprising: The processing module is used to input the currently acquired image of the warning area into a preset small model, obtain the location information of each target in the image output by the small model; input the image and the location information of each target into a preset multimodal large model to identify the type of each target; The acquisition module is used to acquire the target type that triggers an alarm for the warning area; The determination module is used to determine that an alarm target exists in the image if the type of the target in the image includes the target type, and to mark and display the location information of the alarm target of the target type in the image.

[0012] In one possible implementation, the processing module is specifically used to input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; The processing module is further configured to, for each target in the image, determine the accuracy of the first type of the target based on the target's location information if the first type is unknown, and if the accuracy is less than a preset threshold, determine the target as a target to be detected. The processing module is specifically used to input the image and the location information of each target to be detected into a preset multimodal large model.

[0013] In one possible implementation, the determining module is further configured to: if there is no target to be detected in the image, obtain the target type that triggers an alarm for the warning area; if the first type of target identified in the image includes the target type, determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

[0014] In one possible implementation, the processing module is specifically configured to: determine the width and height of the target based on location information; determine a first sum based on a first product of the width and a preset first weight value, and a second product of the height and a preset second weight value; determine a second sum based on a first ratio of the width to a preset stable detection width, and a second ratio of the height to a preset stable detection height; and determine the accuracy of a first type of the target based on the first sum and the second sum.

[0015] In one possible implementation, the processing module is further configured to adjust the image of the currently acquired warning area according to the preset small model-adapted image size to obtain an adjusted first image; and use the adjusted first image as the image of the currently acquired warning area.

[0016] In one possible implementation, the processing module is further configured to adjust the image and the position information of each target according to the preset image size adapted to the multimodal large model, to obtain an adjusted second image and the adjusted position information of each target; and input the second image and the adjusted position information of each target into the preset multimodal large model.

[0017] Thirdly, embodiments of this application also provide an electronic device, the electronic device including a processor, the processor being configured to execute a computer program stored in a memory to implement the steps of any of the methods described above.

[0018] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0019] In this embodiment, the currently acquired image of the warning area is input into a preset small model to obtain the location information of each target in the image output by the small model. This allows the preset multimodal large model to identify the type of all alarm targets in the image based on the image and the location information of each alarm target, thereby achieving subsequent target detection. Since the multimodal large model typically has stronger multimodal data processing capabilities than the CNN small model, it can better capture the subtle features of small targets at a distance. Therefore, by using the multimodal large model to identify the type of all alarm targets in the image, the accuracy of perimeter security in identifying the type of small targets at a distance is improved, thus enhancing the overall image recognition accuracy of perimeter security. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of a target detection process provided in an embodiment of this application; Figure 2 This is a simplified process diagram of a target detection method provided in an embodiment of this application; Figure 3 A schematic diagram illustrating a simplified process for re-identifying the type of each target output by a small model and determining whether it is a false alarm, provided for an embodiment of this application; Figure 4 A schematic diagram of a target detection device provided in an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0023] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0024] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0025] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0026] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0027] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0028] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0029] Example 1: Figure 1 This application provides a schematic diagram of a target detection process, which includes the following steps: S101: Input the currently acquired image of the warning area into a preset small model, and obtain the location information of each target in the image output by the small model.

[0030] The target detection method provided in this application is applied to electronic devices, such as network cameras, computers (PCs), servers, etc.

[0031] A warning zone refers to a specific area that requires special attention or where activities are restricted. The small model can be any type of neural network model, such as a convolutional neural network model, a multilayer feedback-forward neural network model, etc.

[0032] The electronic device acquires an image of the current surveillance area and inputs it into a pre-defined small model. The small model then performs target detection on the image to determine the location information of each target. The target's location information can be the coordinates of the top-left and bottom-right corners of the detection bounding box used to detect the target. There can be one or multiple targets in the image.

[0033] Small models are used to detect objects in images and determine the location information of each object in the image, which is an existing technology and will not be elaborated here.

[0034] S102: Input the image and the location information of each target into a preset multimodal large model to identify the type of each target.

[0035] Multimodal large models are any type of neural network model, such as convolutional neural network models, multilayer feedback neural network models, etc., and can process various types of data such as images, text, and sound.

[0036] The image from step 101 and the location information of each target in the image are input into a preset multimodal large model to identify the type of each target in the image. Specifically, the preset multimodal large model determines each target that needs type identification based on the image from step 101 and the location information of each target, and identifies the type of the target. The target types include people, motor vehicles, animals, and unknown types. Multimodal large model identification of target types in images is existing technology and will not be elaborated further here.

[0037] In this application, a small model and a multimodal large model can be deployed simultaneously on an electronic device to identify alarm targets in the warning area. The small model and the multimodal large model are two independent models and are not jointly trained. Therefore, maintenance personnel can optimize each model independently. This also avoids the problem of high latency caused by data transmission between multiple devices when deploying a small model on an electronic device and a multimodal large model in the cloud during target detection.

[0038] S103: Obtain the target type that triggers an alarm for the warning area; if the target type in the image contains the target type, then determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

[0039] The electronic device pre-sets the target types for alarming in each warning area. For example, warning area A has target types a and b that trigger alarms, while warning area B has target types a and c. The electronic device first acquires the target types for alarming in that warning area. Then, based on these target types and the types of each target identified in step 102, if the target type in the image contains that target type, it determines that an alarm target exists in the image, proving that an alarm target for that type exists in the acquired image of the warning area. The location information of the alarm target of that type is then marked and displayed in the image.

[0040] For example, if an electronic device acquires information indicating that the target types triggering an alarm in warning area A are a and b, and the target types in the captured image of warning area A are a, b, c, d, a, etc., then the electronic device determines that alarm targets exist in the image and marks the location information of the alarm target corresponding to target type a and the location information of the alarm target corresponding to target type b in the image. The method for marking the location information of the alarm target of each target type in the image can be to mark the coordinates of the upper left and lower right corners of the detection box of the alarm target.

[0041] If the electronic device does not identify the target type in the currently acquired image of the warning area, it is determined that there is no alarm target in the image.

[0042] In one possible implementation, the multimodal large model takes a long time to identify a target, while perimeter security requires high real-time performance for alarm events. Once a target triggers an alarm, video capture and key map acquisition need to be performed immediately. Therefore, while inputting the captured images and the location information of each target into the preset multimodal large model to identify the type of each target, the electronic device starts the application terminal of the electronic device to begin capturing video images and key maps for a preset time period of the alert area, and saves the video images and key maps corresponding to the currently captured images of the alert area in a temporary file. If the electronic device does not identify the target type that triggered the alarm in the currently captured images of the alert area, it is determined to be a false alarm event, and the video images and key maps of the alert area corresponding to the currently captured images of the alert area are deleted from the temporary file; if the electronic device identifies the target type that triggered the alarm in the currently captured images of the alert area, it is determined to be a non-false alarm event, and the video images and key maps of the alert area in the temporary file are displayed together with the currently captured images of the alert area.

[0043] In this embodiment, the currently acquired image of the warning area is input into a preset small model to obtain the location information of each target in the image output by the small model. This allows the preset multimodal large model to identify the type of all alarm targets in the image based on the image and the location information of each alarm target, thereby achieving subsequent target detection. Since the multimodal large model typically has stronger multimodal data processing capabilities than the CNN small model, it can better capture the subtle features of small targets at a distance. Therefore, by using the multimodal large model to identify the type of all alarm targets in the image, the accuracy of perimeter security in identifying the type of small targets at a distance is improved, thus enhancing the overall image recognition accuracy of perimeter security.

[0044] Example 2: To improve the efficiency of target detection, this embodiment further filters the identified targets before identifying the type of each target, thereby reducing the workload of the large multimodal model and improving the efficiency of target detection. Specifically: The step of inputting the currently acquired image of the warning area into a preset small model and obtaining the location information of each target in the image output by the small model includes: Input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; After inputting the currently acquired image of the warning area into a preset small model to obtain the location information and first type of each target in the image output by the small model, and before inputting the image and the location information of each target into a preset multimodal large model to identify the type of each target, the method further includes: For each target in the image, if the first type is unknown, the target is identified as a target to be detected; otherwise, the accuracy of the first type of the target is determined based on the target's location information. If the accuracy is less than a preset threshold, the target is identified as a target to be detected. The step of inputting the image and the location information of each target into a preset multimodal large model includes: The image and the location information of each target to be detected are input into a preset multimodal large model.

[0045] The electronic device inputs the currently acquired image of the warning area into a preset small model to perform target detection on the image. This not only determines the location information of each target in the image but also identifies the primary type of each target. The primary type includes people, motor vehicles, animals, and unknown types. The small model's target detection and primary type identification of each target in the image is existing technology and will not be elaborated upon here.

[0046] Accuracy refers to the degree to which a small model accurately identifies the type of target.

[0047] After acquiring the location information and first type of each target in the image output by the small model, the electronic device, for each target in the image, if the first type is an unknown type, it proves that the small model has not identified the type of the target, and the multimodal large model needs to identify the type of the target again. Specifically, the electronic device takes the target as the target to be detected, and uses it for the subsequent multimodal large model to identify the type of the target again. Optionally, the electronic device can also first directly determine the accuracy of the unknown type as zero, and then take the target of the unknown type as the target to be tested when making accuracy judgments later. This is not limited.

[0048] For each target in the image, if the first type is a non-unknown type, it proves that the small model has identified the type of the target. However, to ensure the accuracy of target recognition, the electronic device can also determine the accuracy of the type of the target identified by the small model. The specific operation is as follows: Based on the location information, determine the target's width and height; The first sum value is determined based on the first product of the width and the preset first weight value, and the second product of the height and the preset second weight value; The second sum value is determined based on the first ratio of the width to the preset stable detection width and the second ratio of the height to the preset stable detection height; The accuracy of the first type of target is determined based on the first sum and the second sum.

[0049] For each target identified in the image, the electronic device determines the target's width and height based on its location information. For example, if the target's location information is the coordinates of its top-left corner (X1, Y1) and bottom-right corner (X2, Y2) of its detection bounding box, then the target's width is... Height is Based on the target's width and the first product of a preset first weight value, and the target's height and the second product of a preset second weight value, a first sum of the first and second product values ​​is determined; based on the width and the first ratio of a preset stable detection width, and the height and the second ratio of a preset stable detection height, a second sum is determined; based on the third ratio of the first and second sums, the accuracy of the target's first type is determined, i.e. .

[0050] The stable detection width and stable detection height can be set by the user according to actual needs, or they can be determined based on a large number of real-world test cases for the same model. For example, if the model's type recognition is closest to reality when the stable detection width and stable detection height are both 60 pixels, then the user can preset these values ​​to 60 pixels. The preset first weight value and preset second weight value can also be set by the user according to actual needs. These preset first weight value and preset second weight value can be the same or different.

[0051] In one possible implementation, the preset first weight value and the preset second weight value can also be that the preset first weight value is a preset first value / stable detection width, and the preset second weight value is a preset second value / stable detection height. Where W1 is the preset first value and W2 is the preset second value. It should be noted that the values ​​of the first and second values ​​are determined based on a large number of actual test cases for the same small model. For example, when the first value is 0.7 and the second value is 0.3, the calculation of the accuracy of the small model is closest to the actual situation. Therefore, users can preset the first value to 0.7 and the second value to 0.3.

[0052] For each target in the image, if the first type is a non-unknown type, and the accuracy of the first type is determined based on the target's location information, then if the accuracy is less than a preset threshold, the target is identified as a target to be detected. In other words, if the accuracy of the target's first type is less than the preset threshold, it indicates that the small model's identification of the target's type is inaccurate. To ensure the accuracy of target identification, a multimodal large model is needed to re-identify the target's type. The electronic device inputs the image and the location information of each target to be detected into a preset multimodal large model for subsequent re-identification of the target's type.

[0053] The preset threshold can be set by the user according to actual needs, or it can be determined based on a large number of test cases for the same small model. For example, when the preset threshold is 0.9, the judgment of the accuracy of the small model is closest to the actual situation, so the user can preset the threshold to 0.9.

[0054] In one possible implementation, after the small model outputs the first type of each target in the image, for each target, the accuracy of the first type of that target is first calculated. If the accuracy of the first type of that target is less than a preset threshold, the image and the location information of the target to be detected are input into a preset multimodal large model to re-identify the type of the target to be detected, and the application is started to begin acquiring video images of the warning area and key maps for a preset time period. That is, this application adopts a method of simultaneously acquiring corresponding video images and key maps while identifying the type of the target to ensure the real-time performance of target detection and alarm.

[0055] In this embodiment, the type of each target in the currently acquired warning area image is directly identified by a small model. Then, the electronic device judges the accuracy of each type. When the accuracy is less than a preset threshold, the target with an accuracy less than the preset threshold is identified as the target to be detected. Subsequently, the location information of the target to be detected and the image are only input into the multimodal large model to identify the type of the target to be detected again, thereby greatly reducing the workload of the multimodal large model and improving the efficiency of target detection.

[0056] Example 3: To improve the efficiency of target detection, this application eliminates the need for re-identification of the target type using a multimodal large model when the target is not present in the acquired image of the warning area. Specifically: If there is no target to be detected in the image, the target type that triggers an alarm for the warning area is obtained; if the first type of target identified in the image includes the target type, it is determined that there is an alarm target in the image, and the location information of the alarm target of the target type is marked and displayed in the image.

[0057] When the electronic device determines that there is no target to be detected in the image of the acquired warning area, it proves that the accuracy of the first type of target output by the small model is not less than the threshold. At this time, it is no longer necessary to re-identify the type of the target to be detected through the multimodal large model. The electronic device directly obtains the target type that alarms for the warning area. If the first type of the target identified in the image contains the target type, it is determined that there is an alarm target in the image, and the location of the alarm target of the target type is marked in the image. That is, the specific implementation process described in step 103 will not be repeated here.

[0058] In this embodiment, the type of each target in the currently acquired warning area image is directly identified by the small model, and the accuracy of each type is judged. When the accuracy is not less than a preset threshold, it proves that the first type of target output by the small model is accurate. The electronic device directly uses the first type of target identified by the small model to identify subsequent alarm targets, thereby improving the efficiency of target detection.

[0059] Example 4: To ensure the accuracy of target detection, this embodiment of the application also preprocesses the currently acquired image of the warning area and the image input to the preset multimodal large model, specifically as follows: Before inputting the currently acquired image of the warning area into a preset small model, the method further includes: Based on the preset small model's suitable image size, the image of the currently acquired warning area is adjusted to obtain an adjusted first image; and the adjusted first image is used as the image of the currently acquired warning area.

[0060] For the currently acquired image of the warning area, the electronic device adjusts the image according to the preset small model's suitable image size to obtain an adjusted first image. Specifically, the image is scaled proportionally according to the size of the currently acquired warning area image and the preset small model's suitable image size to obtain the adjusted first image. This adjusted first image is then used as the currently acquired image of the warning area. The methods for adjusting the image include, but are not limited to, scaling and filling with preset pixel values.

[0061] For example, the image size of the currently acquired warning area is 640. 640 pixels; a preset small model is used for fast object detection, and this small model is suitable for images with a size of 1024. If the image is 1024 pixels, then the image of the currently acquired warning area is scaled proportionally to obtain an adjusted first image. This adjusted first image is then used as the image of the currently acquired warning area.

[0062] Before inputting the image and the location information of each target into a preset multimodal large model, the method further includes: Based on the preset image size for multimodal large model adaptation, the image and the position information of each target are adjusted to obtain the adjusted second image and the adjusted position information of each target; The step of inputting the image and the location information of each target into a preset multimodal large model includes: The second image and the adjusted position information of each target are input into a preset multimodal large model.

[0063] After acquiring the location information of each target output by the small model, before inputting the currently acquired image of the warning area and the location information of each target into the preset multimodal large model, the electronic device adjusts the currently acquired image of the warning area according to the image size adapted to the preset multimodal large model. The same scaling adjustment method is used to obtain the adjusted second image. For example, the preset image size adapted to the small model is 1024. If the image is 1024 pixels, then the size of the first image obtained by scaling the currently captured warning area proportionally will be 1024 pixels. 1024 pixels; while the multimodal large model has an image size of 1920. At 1080 pixels, the accuracy of target type recognition is the highest. Therefore, the size of the first image is adjusted using a scaling method to obtain the adjusted second image.

[0064] Since the image size output from the small model has been adjusted to the image size of the multimodal large model, the position information of each target also needs to be adjusted accordingly. Methods for adjusting the position information include: Calculate the scaling ratio of the image's width and height before and after adjustment, for example, , Then, adjust the position coordinates according to the scaling ratio to obtain the adjusted position information of each target. For example, the bounding box coordinates of a certain target are (X... min Y min (X) max Y max The adjusted bounding box coordinates are ( ). , (), , ).

[0065] The second image and the adjusted position information of each target are input into the multimodal large model. Based on the input second image and the adjusted position information of each target, the multimodal large model further identifies the type of each target.

[0066] In this embodiment, by adapting the size of the input image to different models and adjusting the input image before performing the corresponding type of recognition operation, the inference accuracy and efficiency of small models and multimodal large models are improved, thereby ensuring the accuracy of target detection.

[0067] Example 5: To clearly explain the object detection method of this application, the steps of the object detection method are described in detail. Figure 2 This application provides a simplified process diagram of a target detection method, which includes the following steps: S201: Acquire images of the current alert area.

[0068] Electronic devices capture images of the current surveillance area through pre-set network cameras.

[0069] S202: Input the currently acquired image of the warning area into a preset small model, and obtain the location information of each target in the image output by the small model.

[0070] The specific implementation process described in step 102 of the above embodiment 1 will not be repeated here.

[0071] S203: Select the target to be detected if the accuracy is less than the preset threshold.

[0072] The specific implementation process described in Example 2 above will not be repeated here.

[0073] S204: Input the image and the location information of each target to be detected into the preset multimodal large model to re-identify the type of the target to be detected.

[0074] The specific implementation process described in Example 2 above will not be repeated here.

[0075] S205: Determine the alarm target based on the results of re-identification in the preset multimodal large model.

[0076] The specific implementation process described in step 103 of the above embodiment 1 will not be repeated here.

[0077] In this embodiment, the perimeter protection range is significantly improved, with actual measurements showing that targets can be detected at approximately 65 meters in daytime scenarios. That is, in this technical field, perimeter protection algorithms can already detect targets at considerable distances, but they cannot identify the type of such targets. To ensure detection accuracy, perimeter algorithms typically do not report these unidentifiable target types, which also prevents an improvement in detection range. This application removes the aforementioned limitations of the perimeter algorithm. The algorithm reports unidentifiable target types and then uses parameters such as target type and location information to determine the accuracy of the target. Targets with an accuracy below a preset threshold are sent to a multimodal large-scale model for secondary analysis. These targets undergo secondary identification by the multimodal large-scale model, which identifies the types of targets that the perimeter algorithm cannot accurately identify, thereby improving the perimeter protection range while maintaining a high detection accuracy.

[0078] Figure 3 A simplified process diagram illustrating a method for re-identifying the type of each target output by a small model and determining whether it is a false alarm, provided in an embodiment of this application, includes the following steps: S301: Input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model.

[0079] S302: For each target in the image, if the first type is unknown, then the target is taken as the target to be detected; otherwise, the accuracy of the first type of the target is determined based on the target's location information.

[0080] S303: Determine whether the accuracy is less than a preset threshold. If yes, execute S304; otherwise, execute S305.

[0081] S304: This target has been identified as the target to be detected.

[0082] S305: Input the image and the location information of each target to be detected into a preset multimodal large model, and identify the type of each target to be detected again.

[0083] S306: Determine if there are false alarms in the multimodal large model. If so, execute S307; otherwise, execute S308.

[0084] S307: Determine that it is a false alarm event, and delete the video image and key map of the warning area corresponding to the currently acquired warning area image from the temporary file.

[0085] S308: If the event is determined to be non-false alarm, display the video image and key map of the alert area in the temporary file together with the currently acquired image of the alert area.

[0086] The above steps are all specific implementation processes described in Embodiment 2 above, and will not be repeated here.

[0087] Example 6: Based on the same concept, embodiments of this application provide a target detection apparatus. Figure 4 Please refer to the schematic diagram of a target detection device provided in this application embodiment. Figure 4 The device includes: Processing module 401 is used to input the currently acquired image of the warning area into a preset small model, obtain the location information of each target in the image output by the small model; input the image and the location information of each target into a preset multimodal large model, and identify the type of each target; The acquisition module 402 is used to acquire the target type that triggers an alarm for the warning area; The determination module 403 is used to determine that there is an alarm target in the image if the type of the target in the image includes the target type, and to mark and display the location information of the alarm target of the target type in the image.

[0088] In one possible implementation, the processing module 401 is specifically used to input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; The processing module 401 is further configured to, for each target in the image, determine the accuracy of the first type of the target if the first type is unknown, and if so, determine the target as a target to be detected; otherwise, determine the accuracy of the first type of the target based on the location information of the target, and determine the target as a target to be detected if the accuracy is less than a preset threshold. The processing module 401 is specifically used to input the image and the location information of each target to be detected into a preset multimodal large model.

[0089] In one possible implementation, the determining module 403 is further configured to: if there is no target to be detected in the image, obtain the target type that triggers an alarm for the warning area; if the first type of target identified in the image includes the target type, determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

[0090] In one possible implementation, the processing module 401 is specifically configured to: determine the width and height of the target based on the location information; determine a first sum based on the first product of the width and a preset first weight value, and the second product of the height and a preset second weight value; determine a second sum based on the first ratio of the width to a preset stable detection width, and the second ratio of the height to a preset stable detection height; and determine the accuracy of a first type of the target based on the first sum and the second sum.

[0091] In one possible implementation, the processing module 401 is further configured to adjust the image of the currently acquired warning area according to the preset small model-adapted image size to obtain an adjusted first image; and use the adjusted first image as the image of the currently acquired warning area.

[0092] In one possible implementation, the processing module 401 is further configured to adjust the image and the position information of each target according to the preset image size adapted to the multimodal large model, to obtain an adjusted second image and the adjusted position information of each target; and input the second image and the adjusted position information of each target into the preset multimodal large model.

[0093] Example 7: Based on the same concept, embodiments of this application provide an electronic device that can implement the steps of a target detection method described above. Figure 5 This application provides a schematic diagram of an electronic device structure, such as... Figure 5 As shown, it includes: processor 501, communication interface 502, memory 503 and communication bus 504, wherein processor 501, communication interface 502 and memory 503 communicate with each other through communication bus 504. The memory 503 stores a computer program. When the program is executed by the processor 501, the processor 501 performs the following steps: Input the currently acquired image of the warning area into a preset small model, and obtain the location information of each target in the image output by the small model; The image and the location information of each target are input into a preset multimodal large model to identify the type of each target; Obtain the target type that triggers an alarm in the warning area; if the target type in the image contains the target type, then determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

[0094] In one possible implementation, the processor 501 is specifically configured to input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; The processor 501 is further configured to, for each target in the image, determine the accuracy of the first type of the target based on the target's location information, and if the first type is an unknown type, then the target is designated as a target to be detected; otherwise, the processor determines the accuracy of the first type of the target based on the target's location information, and if the accuracy is less than a preset threshold, then the target is designated as a target to be detected. The processor 501 is specifically used to input the image and the location information of each target to be detected into a preset multimodal large model.

[0095] In one possible implementation, the processor 501 is further configured to: if there is no target to be detected in the image, obtain the target type that triggers an alarm for the warning area; if the first type of target identified in the image includes the target type, determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

[0096] In one possible implementation, the processor 501 is specifically configured to: determine the width and height of the target based on location information; determine a first sum based on a first product of the width and a preset first weight value, and a second product of the height and a preset second weight value; determine a second sum based on a first ratio of the width to a preset stable detection width, and a second ratio of the height to a preset stable detection height; and determine the accuracy of a first type of the target based on the first sum and the second sum.

[0097] In one possible implementation, the processor 501 is further configured to adjust the image of the currently acquired warning area according to the preset small model-adapted image size to obtain an adjusted first image; and use the adjusted first image as the image of the currently acquired warning area.

[0098] In one possible implementation, the processor 501 is further configured to adjust the image and the position information of each target according to the preset multimodal large model adapted image size, to obtain an adjusted second image and the adjusted position information of each target; and input the second image and the adjusted position information of each target into the preset multimodal large model.

[0099] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. Communication interface 502 is used for communication between the aforementioned electronic device and other devices. The memory can include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0100] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0101] Example 8: Based on the same concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a processor. When the program runs on the processor, it causes the processor to execute any of the target detection methods discussed above. Since the principle by which the above-described computer-readable storage medium solves the problem is similar to that of a target detection method, the implementation of the above-described computer-readable storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0102] Based on the same concept, this application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to execute any of the target detection methods described above. Since the principle by which the above computer program product solves the problem is similar to that of a target detection method, the implementation of the above computer program product can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0103] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] Computer programs used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing status information from the computer-readable program instructions to implement various aspects of this disclosure.

[0105] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0106] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0109] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0110] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A target detection method characterized by, The method includes: Input the currently acquired image of the warning area into a preset small model, and obtain the location information of each target in the image output by the small model; The image and the location information of each target are input into a preset multimodal large model to identify the type of each target; Obtain the target type that triggers an alarm in the warning area; if the target type in the image contains the target type, then determine that there is an alarm target in the image, and mark and display the location information of the alarm target of the target type in the image.

2. The method of claim 1, wherein, The step of inputting the currently acquired image of the warning area into a preset small model and obtaining the location information of each target in the image output by the small model includes: Input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; After inputting the currently acquired image of the warning area into a preset small model to obtain the location information and first type of each target in the image output by the small model, and before inputting the image and the location information of each target into a preset multimodal large model to identify the type of each target, the method further includes: For each target in the image, if the first type is unknown, the target is identified as a target to be detected; otherwise, the accuracy of the first type of the target is determined based on the target's location information. If the accuracy is less than a preset threshold, the target is identified as a target to be detected. The step of inputting the image and the location information of each target into a preset multimodal large model includes: The image and the location information of each target to be detected are input into a preset multimodal large model.

3. The method of claim 2, wherein, The method further includes: If there is no target to be detected in the image, the target type that triggers an alarm for the warning area is obtained; if the first type of target identified in the image includes the target type, it is determined that there is an alarm target in the image, and the location information of the alarm target of the target type is marked and displayed in the image.

4. The method of claim 2, wherein, The accuracy of determining the first type of target based on the target's location information includes: Based on the location information, determine the target's width and height; The first sum value is determined based on the first product of the width and the preset first weight value, and the second product of the height and the preset second weight value; The second sum value is determined based on the first ratio of the width to the preset stable detection width and the second ratio of the height to the preset stable detection height; The accuracy of the first type of target is determined based on the first sum and the second sum.

5. The method of claim 1, wherein, Before inputting the currently acquired image of the warning area into a preset small model, the method further includes: Based on the preset small model's suitable image size, the image of the currently acquired warning area is adjusted to obtain an adjusted first image; and the adjusted first image is used as the image of the currently acquired warning area.

6. The method according to claim 1 or 5, characterized in that, Before inputting the image and the location information of each target into the preset multimodal large model, the method further includes: Based on the preset image size for multimodal large model adaptation, the image and the position information of each target are adjusted to obtain the adjusted second image and the adjusted position information of each target; The step of inputting the image and the location information of each target into a preset multimodal large model includes: The second image and the adjusted position information of each target are input into a preset multimodal large model.

7. An apparatus for target detection, the apparatus comprising: The device includes: The processing module is used to input the currently acquired image of the warning area into a preset small model, obtain the location information of each target in the image output by the small model; input the image and the location information of each target into a preset multimodal large model to identify the type of each target; The acquisition module is used to acquire the target type that triggers an alarm for the warning area; The determination module is used to determine that an alarm target exists in the image if the type of the target in the image includes the target type, and to mark and display the location information of the alarm target of the target type in the image.

8. The apparatus of claim 7, wherein, The processing module is specifically used to input the currently acquired image of the warning area into a preset small model, and obtain the location information and first type of each target in the image output by the small model; The processing module is further configured to, for each target in the image, determine the accuracy of the first type of the target based on the target's location information if the first type is unknown, and determine the target as a target to be detected if the accuracy is less than a preset threshold. The processing module is specifically used to input the image and the location information of each target to be detected into a preset multimodal large model.

9. An electronic device, comprising: The electronic device includes a processor for implementing the method as described in any one of claims 1-6 when executing a computer program stored in a memory.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.

Images