Super large ai based object detection system

Abstract

The present invention is configured such that an image stream generated as an image capturing device captures an image is first received by an object detection module, thereby enabling the object detection module to attempt to detect a specific object in each image frame of the image stream. Subsequently, when a specific object is detected by the object detection module, the invention is configured to query a VLM supervisory module to determine whether the detected specific object is indeed a specific object and obtain an answer thereto. According to the present invention, even if the object detection module detects an object that should not be detected as a specific object, the VLM supervisory module having a vision language model generates answer information regarding whether the specific object detected by the object detection module is actually a specific object. Consequently, the phenomenon of an object that should not be detected being detected as a specific object is significantly suppressed, thereby improving the accuracy of object detection.

Description

Technology Field

[0001] The present invention relates to an object detection system based on a massive AI that detects specific objects in image frames of a video stream using a massive AI called a Vision Language Model (VLM). Background Technology

[0002] In a typical AI-based object detection system, when a video stream captured by a video recording device is input into an AI model unit, the AI ​​model unit attempts to detect specific objects (e.g., people, cars, etc.) in the image frames of the video stream using an embedded AI model. Subsequently, if there is an image frame in which the specific object is detected, the AI ​​model unit transmits the image frame in which the specific object is detected, along with information regarding the detected specific object, to an integrated control center.

[0003] However, such object detection systems may fail to detect a specific object even though it is present in an image frame of a video stream (this is called 'non-detection'), and may detect that a specific object is present even though it is not (this is called 'over-detection'). In order for an object detection system to improve the accuracy of object detection, it is necessary to minimize both non-detection and over-detection.

[0004] However, it is technically very difficult to create an object detection system that minimizes both missing and over-detection. In contrast, it is relatively easier to create an object detection system that minimizes missing or over-detection by focusing on either one side.

[0005] If an object detection system is created to minimize over-detection, the rate of non-detection becomes relatively high, which increases the likelihood of failing to detect specific objects in image frames of video streams. For example, assuming a fire occurs in a certain location, an object detection system that minimizes over-detection is highly likely to fail to properly detect the fire situation due to a relatively high rate of non-detection. Therefore, rather than creating an object detection system that minimizes over-detection, it is more desirable to create an object detection system that minimizes non-detection in terms of detecting objects more accurately.

[0006] However, since object detection systems that minimize non-detection have a relatively high over-detection rate, there is a high probability that they will detect the presence of a specific object in an image frame of a video stream even though the object is not present. For example, even if a fire has not occurred in a certain location, an object detection system that minimizes non-detection is highly likely to detect that a fire has occurred there due to its relatively high over-detection rate. Furthermore, an object detection system that minimizes non-detection may transmit information regarding such incorrectly detected fire situations to an integrated control center. In this case, if the integrated control center transmits information to people's terminals stating that a fire has occurred, it results only in confusion among people due to this incorrect information.

[0007] As such, object detection systems that minimize non-detection have a problem in that they detect objects that should not be detected as specific objects. Accordingly, there is a need to devise a method to improve the accuracy of object detection by minimizing the phenomenon of specific objects being not detected through an object detection system that minimizes non-detection, while suppressing the phenomenon of objects that should not be detected being detected as specific objects.

[0008] Meanwhile, Patent Document 1 below discloses an object detection method that captures an external image using a user terminal, identifies human objects using an object identification artificial intelligence algorithm, and detects users including a means of transportation and general users not using a means of transportation differently among the identified objects. Prior art literature

[0009] Published Patent Application No. 10-2024-0082460 (June 11, 2024) The problem to be solved

[0010] The purpose of the present invention is to provide a super-large AI-based object detection system capable of detecting specific objects in image frames of a video stream with high accuracy.

[0011] However, the technical problems that the present invention aims to solve are not limited to those described above, and other unmentioned technical problems will be clearly understood by a person skilled in the art from the description of the invention below. means of solving the problem

[0012] To achieve the above objectives, the super-large AI-based object detection system (1000) according to the present invention may include: an object detection module (100) that receives an image stream from an image capturing device (10), attempts to detect a specific object in each image frame of the image stream, and if there is an image frame in which the specific object is detected, outputs the image frame in which the specific object is detected and simultaneously outputs question information regarding whether the detected specific object is actually the specific object; and a VLM supervision module (200) that receives the image frame in which the specific object is detected and the question information from the object detection module (100), applies the image frame in which the specific object is detected and the question information to a Vision Language Model (VLM), and generates answer information regarding whether the specific object detected by the object detection module (100) in the image frame output by the object detection module (100) is actually the specific object.

[0013] The object detection module (100) may include: an artificial intelligence model unit (110) that is pre-trained to detect a specific object in an image frame and is capable of detecting a specific object in each image frame of the image stream when receiving an image stream from the image capturing device (10); and an artificial intelligence model output unit (120) that, when a specific object is detected in an image frame of the image stream by the artificial intelligence model unit (110), outputs the image frame in which the specific object is detected to the VLM supervision module (200) and simultaneously generates question information regarding whether the specific object detected by the artificial intelligence model unit (110) is actually the specific object and outputs it to the VLM supervision module (200).

[0014] When the artificial intelligence model unit (110) detects a specific object in an image frame of a video stream received from the video capturing device (10), it outputs the coordinates of the location where the specific object exists and the class of the specific object to the artificial intelligence model output unit (120), and the artificial intelligence model output unit (120)

[0015] The question information can be generated using the coordinates and the class of a specific object output by the artificial intelligence model unit (110).

[0016] Alternatively, the object detection module (100) may include: a zero-shot model unit (130) configured to receive an input prompt along with a video stream from the video capturing device (10) and to detect a specific object corresponding to the input prompt in each image frame of the video stream; and a zero-shot model output unit (140) which, when a specific object is detected in an image frame of the video stream by the zero-shot model unit (130), outputs the image frame in which the specific object is detected to the VLM supervision module (200) and simultaneously generates question information regarding whether the specific object detected by the zero-shot model unit (130) is actually the specific object and outputs it to the VLM supervision module (200).

[0017] When the zero-shot model unit (130) detects a specific object corresponding to the input prompt in an image frame of a video stream received from the video capturing device (10), it outputs the coordinates of the location where the specific object exists and the input prompt to the zero-shot model output unit (140), and the zero-shot model output unit (140) can generate the question information using the coordinates and input prompt output by the zero-shot model unit (130). Effects of the invention

[0018] The present invention is configured such that an image stream generated as an image capturing device captures an image is first received by an object detection module, thereby allowing the object detection module to attempt to detect a specific object in each image frame of the image stream (i.e., primary detection of a specific object). Subsequently, when a specific object is detected by the object detection module, the invention is configured to ask a VLM supervisory module whether the detected specific object is actually a specific object and obtain an answer thereto (i.e., secondary detection of a specific object). According to the present invention, even if the object detection module detects an object that should not be detected as a specific object, a VLM supervisory module having a vision language model generates answer information regarding whether the specific object detected by the object detection module is actually a specific object. Consequently, the phenomenon of an object that should not be detected being detected as a specific object is significantly suppressed, thereby improving the accuracy of object detection.

[0019] Furthermore, according to the present invention, a VLM supervision module having a vision language model is not configured to detect specific objects one by one in image frames of the video stream, but rather, the VLM supervision module is configured to generate an answer regarding whether the detected specific object is actually the specific object, on the condition that the object detection module detects a specific object in the image frames of the video stream. According to this invention, compared to the case where a VLM supervision module having a vision language model is configured to detect specific objects one by one in image frames of the video stream, the accuracy of object detection can be improved while reducing time and cost.

[0020] However, the effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0021] FIG. 1 is a diagram showing a super-large AI-based object detection system according to a first embodiment of the present invention, together with an image capturing device and an integrated control center server. FIG. 2 is a diagram exemplarily showing one image frame of a video stream received by the artificial intelligence model unit of FIG. 1 (or the zero-shot model unit of FIG. 3). FIG. 3 is a diagram showing a super-large AI-based object detection system according to a second embodiment of the present invention, together with an image capturing device and an integrated control center server. Specific details for implementing the invention

[0022] Hereinafter, a super-large AI-based object detection system according to the present invention will be described in detail with reference to the attached drawings. The attached drawings are provided merely as examples to ensure that the technical concept of the present invention is sufficiently conveyed to those skilled in the art, and the present invention is not limited to the drawings presented below and can be embodied in any other form. In this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0023] FIG. 1 is a drawing showing a super-large AI-based object detection system (hereinafter referred to as the 'object detection system') (1000) according to the first embodiment of the present invention, together with an image capturing device (10) and an integrated control center server (20).

[0024] The video recording device (10) serves the role of recording video, and a CCTV (closed-circuit television) may be an example. As the video recording device (10) records video, the video stream generated can be transmitted to an object detection system (1000).

[0025] The integrated control center server (20) is a server operated by the integrated control center and receives information about a specific object from the object detection system (1000). Here, the information about the specific object received by the integrated control center server (20) may include an image frame in which the specific object is detected, coordinates of the location where the specific object exists in the detected image frame, the class of the specific object, and answer information regarding whether the specific object in the image frame in which the specific object is detected is actually the specific object. The operator of the integrated control center server (20) can verify whether the specific object exists in the image frame of the video stream through the information about the specific object.

[0026] Meanwhile, the object detection system (1000) according to the first embodiment of the present invention may include an object detection module (100) and a VLM supervision module (200).

[0027] The object detection module (100) receives a video stream from a video capturing device (10) and attempts to detect a specific object in each image frame of the video stream. For example, the video stream received by the object detection module (100) from the video capturing device (10) may consist of 60 image frames per second, and in this case, the object detection module (100) may attempt to detect a specific object in each of the 60 image frames per second.

[0028] When the object detection module (100) detects a specific object in an image frame, it may immediately output the image frame in which the specific object was detected, the coordinates of the location where the specific object exists, and the class of the specific object, etc., to the integrated control center server (20). However, as explained in the background art above, the object detection module (100) of the object detection system (1000) that minimizes non-detection, in particular, has the problem of over-detection in which it detects targets that should not be detected as specific objects.

[0029] To solve these problems, the object detection module (100) is configured such that when there is an image frame in which a specific object is detected while attempting to detect a specific object, the image frame in which the specific object is detected is output to the VLM supervision module (200), and at the same time, question information regarding whether the detected specific object is actually the specific object is output to the VLM supervision module (200).

[0030] Here, the reason the object detection module (100) outputs the image frame in which the specific object is detected and the question information to the VLM supervision module (200) is to solve the problem of over-detection, in which an object that should not be detected is detected as a specific object, by asking the VLM supervision module (200) whether the specific object detected by the object detection module (100) is actually a specific object.

[0031] As shown in FIG. 1, the object detection module (100) may include an artificial intelligence model unit (110) and an artificial intelligence model output unit (120).

[0032] The artificial intelligence model unit (110) may be generated based on an artificial neural network such as Faster R-CNN (Region Based Convolutional Neural Networks), Mask R-CNN, SSD (Single Shot Detector), or YOLO (You Only Look Once), and may be pre-trained to detect a specific object in an image frame of the video stream. Accordingly, when the artificial intelligence model unit (110) receives a video stream from a video capturing device (10), it is possible to detect a specific object in each image frame of the video stream.

[0033] FIG. 2 is a diagram illustrating an exemplary image frame of a video stream received by an artificial intelligence model unit (110). For example, the artificial intelligence model unit (110) may be pre-trained to detect a "person" as a specific object in an image frame of a video stream. In this case, when the artificial intelligence model unit (110) receives a video stream from a video capturing device (10), it detects a "person" as a specific object in an image frame such as FIG. 2.

[0034] The fact that the artificial intelligence model unit (110) detected a specific object in an image frame of the video stream means that it detected the coordinates of the location of the specific object.

[0035] In this way, when the artificial intelligence model unit (110) detects a specific object in an image frame of a video stream received from a video recording device (10), the artificial intelligence model unit (110) can output the class of the specific object along with the coordinates of the location where the specific object exists to the artificial intelligence model output unit (120).

[0036] For example, when the artificial intelligence model unit (110) detects a person as a specific object in an image frame such as FIG. 2, the artificial intelligence model unit (110) can output "(x, y, width, height) = (7, 7, 4, 5)", which are coordinates for the location where the specific object exists, to the artificial intelligence model output unit (120), and can output "person," which is the class of the specific object, to the artificial intelligence model output unit (120). Here, the class of the specific object may correspond to a name that identifies the specific object. In addition, among the coordinates for the location where the specific object exists, x and y are coordinates corresponding to the top-left corner of the area where the specific object exists (i.e., the bounding box), width is the width of the bounding box, and height is the height of the bounding box.

[0037] The artificial intelligence model output unit (120) can output the image frame in which a specific object is detected to the VLM supervision module (200) when a specific object is detected in the image frame of the video stream by the artificial intelligence model unit (110). For example, when the artificial intelligence model unit (110) detects a person as a specific object in the image frame of FIG. 2, the artificial intelligence model output unit (120) can output the image frame of FIG. 2 (i.e., the image frame in which a specific object is detected) to the VLM supervision module (200).

[0038] At the same time, the artificial intelligence model output unit (120) can generate question information regarding whether the specific object detected by the artificial intelligence model unit (110) is actually the specific object by using the coordinates and the class of the specific object output by the artificial intelligence model unit (110), and output the generated question information to the VLM supervision module (200).

[0039] The artificial intelligence model output unit (120) may have a template pre-set for generating the above question information. Here, the template may be in a text format such as "Is the object at coordinates x: {a}, y: {b}, width: {c}, height: {d} in the given image {class}?"

[0040] Accordingly, when the artificial intelligence model output unit (120) outputs "(x, y, width, height) = (7, 7, 4, 5)", which is the coordinates of the location where the specific object exists, and "person", which is the class of the specific object, the artificial intelligence model unit (110) can apply "(x, y, width, height) = (7, 7, 4, 5)" and "person" to the template to generate question information in text format such as "Is the object at the coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" and output it to the VLM supervision module (200).

[0041] The VLM supervision module (200) may include a Vision Language Model (VLM). Here, the Vision Language Model is a combination of computer vision technology and natural language processing technology, and refers to a super-large AI model that has learned cross-modal representations for encoding and decoding through large-scale image and natural language data. As a result, when image frames and question information in text format are input into the Vision Language Model, the Vision Language Model extracts the meaning corresponding to the question information from the image frames as answer information, and outputs the extracted answer information by expressing it in natural language in text format.

[0042] The VLM supervision module (200) can receive from the artificial intelligence model output unit (120) of the object detection module (100) an image frame in which the specific object is detected (e.g., an image frame like Fig. 2) and question information (i.e., question information regarding whether the specific object detected by the artificial intelligence model unit (110) of the object detection module (100) is actually the specific object).

[0043] In this case, the VLM supervision module (200) can apply the image frame in which the specific object is detected and the question information to the vision language model, and the vision language model generates answer information regarding whether the specific object detected by the artificial intelligence model unit (110) of the object detection module (100) is actually the specific object in the image frame output by the artificial intelligence model output unit (120) of the object detection module (100).

[0044] To reiterate the previously mentioned example, the artificial intelligence model output unit (120) of the object detection module (100) can output question information such as "Is the object at coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" to the VLM supervision module (200).

[0045] In this regard, the VLM supervision module (200) can apply an image frame such as FIG. 2 and the question "Is the object at the coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" to the vision language model. At this time, the vision language model checks the coordinates included in the question information and the class of a specific object in the image frame such as FIG. 2, and then generates answer information regarding whether the specific object detected by the artificial intelligence model part (110) of the object detection module (100) is actually a specific object (e.g., a person).

[0046] In an image frame such as Figure 2 output by the artificial intelligence model output unit (120) of the object detection module (100), the vision language model can generate positive response information such as "It is a person" when it determines that a specific object detected by the artificial intelligence model unit (110) of the object detection module (100) is actually a specific object (e.g., a person).

[0047] In this way, when the vision language model generates positive response information, the VLM supervision module (200) can transmit information about a specific object (i.e., an image frame such as FIG. 2 in which a specific object is detected, coordinates of the location where the specific object exists in the image frame such as FIG. 2, the class of the specific object, response information regarding whether the specific object in the image frame such as FIG. 2 is actually the specific object, etc.) to the integrated control center server (20). In this case, the operator of the integrated control center server (20) can confirm that the specific object exists in the image frame of the video stream through the information about the specific object, and, depending on the case, can have the integrated control center server (20) transmit information to a user terminal (not shown) indicating that the specific object has been detected.

[0048] In contrast, the vision language model can generate negative response information such as "not a person" when it determines that a specific object detected by the artificial intelligence model unit (110) of the object detection module (100) in an image frame such as FIG. 2, which is output by the artificial intelligence model output unit (120) of the object detection module (100), is actually not a specific object (e.g., a person).

[0049] For example, if the artificial intelligence model unit (110) detects a car instead of a person even though it should have detected a person as a specific object, the artificial intelligence model output unit (120) can output question information such as "Is the object at coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" to the VLM supervision module (200), and in this case, the vision language model can generate negative answer information such as "not a person" for the specific object detected by the artificial intelligence model unit (110).

[0050] In this way, when the vision language model generates negative response information, the VLM supervision module (200) may not transmit any information to the integrated control center server (20). This is because it is desirable to transmit information about a specific object to the integrated control center server (20) only when the object detection system (1000) detects a specific object, thereby allowing the operator of the integrated control center server (20) to take action only when a specific object is detected.

[0051] The video stream generated as the video capturing device (10) captures the video may be configured to bypass the object detection module (100) and be received directly by the VLM supervision module (200), so that the VLM supervision module (200) detects a specific object in the image frame of the video stream.

[0052] However, since the vision language model included in the VLM supervision module (200) is a super-large AI model, even though the accuracy of object detection is high, the required GPU (graphic processing unit) resources are very high, so it can be said that it is inappropriate in terms of time and cost to detect objects in each of the 60 image frames per second, for example.

[0053] Accordingly, rather than the VLM supervision module (200) being configured to detect specific objects one by one in the image frames of the video stream, it is preferable for the VLM supervision module (200) to be configured to only verify whether the detected specific object is actually the specific object when the object detection module (100) detects a specific object in the image frames of the video stream, in order to reduce time and cost while improving the accuracy of object detection.

[0054] Meanwhile, FIG. 3 is a drawing showing an object detection system (1000) according to a second embodiment of the present invention together with an image capturing device (10) and an integrated control center server (20).

[0055] The object detection system (1000) according to the second embodiment of the present invention differs from the object detection system (1000) according to the first embodiment of the present invention only in that the object detection module (100) includes a zero-shot model unit (130) and a zero-shot model output unit (140) instead of an artificial intelligence model unit (110) and an artificial intelligence model output unit (120). Accordingly, only the differences will be explained below, and the description of the first embodiment of the present invention may be applied to the second embodiment of the present invention as is, unless it is in direct conflict with the description of the first embodiment of the present invention.

[0056] As shown in FIG. 3, in the second embodiment of the present invention, the object detection module (100) may include a zero-shot model unit (130) and a zero-shot model output unit (140).

[0057] The zero-shot model unit (130) refers to a model in which an artificial neural network or artificial intelligence model is trained based on zero shots. Here, zero-shot based learning refers to a learning method that enables an artificial neural network or artificial intelligence model to perform a specific task (e.g., detecting a specific object in an image frame) by inferring and predicting new concepts or categories without direct training data for the specific task.

[0058] This zero-shot model unit (130) cannot detect a specific object unless an input prompt for the object to be detected is provided. That is, the zero-shot model unit (130) can detect a specific object corresponding to the input prompt only when an input prompt for the object to be detected is provided. Accordingly, the video capturing device (10) is configured to transmit an input prompt commanding the detection of a specific object in addition to transmitting a video stream generated by capturing a video with the zero-shot model unit (130), and as a result, the zero-shot model unit (130) receives an input prompt along with the video stream from the video capturing device (10).

[0059] When the zero-shot model unit (130) receives the video stream and the input prompt, it can detect a specific object corresponding to the input prompt in each image frame of the video stream. For example, when the input prompt received by the zero-shot model unit (130) from the video capturing device (10) is "person," the zero-shot model unit (130) detects "person" as a specific object in an image frame such as FIG. 2.

[0060] The fact that the zero-shot model unit (130) detected a specific object in an image frame of the video stream means that it detected the coordinates of the location of the specific object.

[0061] In this way, when the zero-shot model unit (130) detects a specific object in an image frame of a video stream received from a video recording device (10), the zero-shot model unit (130) can output the input prompt to the zero-shot model output unit (140) along with the coordinates of the location where the specific object exists.

[0062] For example, when the zero-shot model unit (130) detects a "person" as a specific object in an image frame such as FIG. 2, the zero-shot model unit (130) can output "(x, y, width, height) = (7, 7, 4, 5)", which is the coordinates of the location where the specific object exists, to the zero-shot model output unit (140), and can output the input prompt "person" to the zero-shot model output unit (140).

[0063] The zero-shot model output unit (140) can output the image frame in which a specific object is detected to the VLM supervision module (200) when a specific object is detected in the image frame of the video stream by the zero-shot model unit (130). For example, when the zero-shot model unit (130) detects a person as a specific object in the image frame of FIG. 2, the zero-shot model output unit (140) can output the image frame of FIG. 2 (i.e., the image frame in which a specific object is detected) to the VLM supervision module (200).

[0064] At the same time, the zero-shot model output unit (140) can generate question information regarding whether a specific object detected by the zero-shot model unit (130) is actually a specific object by using the coordinates and input prompts output by the zero-shot model unit (130), and output the generated question information to the VLM supervision module (200).

[0065] A template for generating the above question information may be pre-set in the zero-shot model output unit (140). Here, the template may be in a text format such as "Is the object at coordinates x: {a}, y: {b}, width: {c}, height: {d} in the given image {input prompt} correct?"

[0066] Accordingly, when the zero-shot model output unit (140) outputs "(x, y, width, height) = (7, 7, 4, 5)", which is the coordinates of the location where a specific object exists, and "person", which is the input prompt, the zero-shot model output unit (140) can apply "(x, y, width, height) = (7, 7, 4, 5)" and "person" to the above template to generate question information in text format such as "Is the object at the coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" and output it to the VLM supervision module (200).

[0067] The VLM supervision module (200) can receive from the zero-shot model output unit (140) of the object detection module (100) an image frame in which the specific object is detected (e.g., an image frame like Fig. 2) and question information (i.e., question information regarding whether the specific object detected by the zero-shot model unit (130) of the object detection module (100) is actually the specific object).

[0068] In this case, the VLM supervision module (200) can apply the image frame in which the specific object is detected and the question information to the vision language model, and the vision language model generates answer information regarding whether the specific object detected by the zero-shot model unit (130) of the object detection module (100) is actually the specific object in the image frame output by the zero-shot model output unit (140) of the object detection module (100).

[0069] To reiterate the previously mentioned example, the zero-shot model output unit (140) of the object detection module (100) can output question information such as "Is the object at coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" to the VLM supervision module (200).

[0070] In this regard, the VLM supervision module (200) can apply an image frame such as FIG. 2 and the question "Is the object at the coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" to the vision language model. At this time, the vision language model checks the coordinates included in the question information and the input prompt in the image frame such as FIG. 2, and then generates answer information regarding whether the specific object detected by the zero-shot model part (130) of the object detection module (100) is actually a specific object (e.g., a person).

[0071] In an image frame such as FIG. 2 output by the zero-shot model output unit (140) of the object detection module (100), the vision language model can generate positive response information such as "It is a person" when it determines that a specific object detected by the zero-shot model unit (130) of the object detection module (100) is actually a specific object (e.g., a person).

[0072] In this way, when the vision language model generates positive response information, the VLM supervision module (200) can transmit information about a specific object (i.e., an image frame such as FIG. 2 in which a specific object is detected, coordinates of the location where the specific object exists in the image frame such as FIG. 2, the class of the specific object, response information regarding whether the specific object in the image frame such as FIG. 2 is actually the specific object, etc.) to the integrated control center server (20). In this case, the operator of the integrated control center server (20) can confirm that the specific object exists in the image frame of the video stream through the information about the specific object, and, depending on the case, can have the integrated control center server (20) transmit information to a user terminal (not shown) indicating that the specific object has been detected.

[0073] In contrast, the vision language model can generate negative response information such as "not a person" when it determines that a specific object detected by the zero-shot model unit (130) of the object detection module (100) in an image frame such as FIG. 2, which is output by the zero-shot model output unit (140) of the object detection module (100), is actually not a specific object (e.g., a person).

[0074] For example, if the zero-shot model unit (130) detects a car instead of a person even though it should have detected a person as a specific object, the zero-shot model output unit (140) can output question information such as "Is the object at coordinates x: {7}, y: {7}, width: {4}, height: {5} in the given image a {person}?" to the VLM supervision module (200), and in this case, the vision language model can generate negative answer information such as "not a person" for the specific object detected by the zero-shot model unit (130).

[0075] In this way, when the vision language model generates negative response information, the VLM supervision module (200) may not transmit any information to the integrated control center server (20). This is because it is desirable to transmit information about a specific object to the integrated control center server (20) only when the object detection system (1000) detects a specific object, thereby allowing the operator of the integrated control center server (20) to take action only when a specific object is detected.

[0076] As described above, although the present invention has been explained by limited embodiments and drawings, the present invention is not limited to the above embodiments, and various modifications and variations are possible from this description by those skilled in the art to which the present invention belongs. Accordingly, the technical concept of the present invention should be understood only by the claims, and all equivalent or analogous variations thereof shall be considered to fall within the scope of the technical concept of the present invention. Explanation of the symbols

[0077] 10: Video recording device 20: Integrated Control Center Server 100: Object detection module 110: Artificial Intelligence Model Department 120: Artificial intelligence model output section 130: Zero Shot Model Department 140: ZeroShot Model Output Section 200: VLM Supervision Module 1000: A massive AI-based object detection system

Claims

Claim 1 A super-large AI-based object detection system comprising an object detection module and a VLM supervision module, for solving the problem where the object detection module detects a target that should not be detected as a specific object, wherein the object detection module comprises an artificial intelligence model unit that is pre-trained to detect the specific object in an image frame, and detects the specific object in an image frame of the image stream when receiving a video stream from a video capturing device; A super-large AI-based object detection system comprising: an AI model output unit that, when a specific object is detected in an image frame of the video stream by the AI ​​model unit, outputs the image frame in which the specific object is detected to the VLM supervision module, and simultaneously generates question information regarding whether the specific object detected by the AI ​​model unit is actually the specific object, and outputs the generated question information to the VLM supervision module; wherein the VLM supervision module includes a vision language model that, when the image frame in which the specific object is detected and the question information are input, outputs the meaning corresponding to the question information in the image frame in which the specific object is detected as answer information; and wherein the VLM supervision module, when receiving the image frame in which the specific object is detected and the question information from the AI ​​model output unit of the object detection module, applies the image frame in which the specific object is detected and the question information to the vision language model to generate answer information regarding whether the specific object detected by the AI ​​model unit is actually the specific object in the image frame output by the AI ​​model output unit. Claim 2 delete Claim 3 A super-large AI-based object detection system according to claim 1, wherein the artificial intelligence model unit, when detecting a specific object in an image frame of a video stream received from the video capturing device, outputs the coordinates of the location where the specific object exists and the class of the specific object to the artificial intelligence model output unit, and the artificial intelligence model output unit generates the question information using the coordinates and the class of the specific object output by the artificial intelligence model unit. Claim 4 A super-large AI-based object detection system comprising an object detection module and a VLM supervision module, for solving the problem in which the object detection module detects a target that should not be detected as a specific object, wherein the object detection module comprises: a zero-shot model unit that receives an input prompt along with a video stream from a video capturing device and detects the specific object corresponding to the input prompt in an image frame of the video stream; A super-large AI-based object detection system comprising: a zero-shot model output unit that, when a specific object is detected in an image frame of the video stream by the zero-shot model unit, outputs the image frame in which the specific object is detected to the VLM supervisory module, and simultaneously generates question information regarding whether the specific object detected by the zero-shot model unit is actually the specific object, and outputs the generated question information to the VLM supervisory module; wherein the VLM supervisory module includes a vision language model that, when the image frame in which the specific object is detected and the question information are input, outputs the meaning corresponding to the question information in the image frame in which the specific object is detected as answer information; and wherein the VLM supervisory module, when receiving the image frame in which the specific object is detected and the question information from the zero-shot model output unit of the object detection module, applies the image frame in which the specific object is detected and the question information to the vision language model to generate answer information regarding whether the specific object detected by the zero-shot model unit is actually the specific object in the image frame output by the zero-shot model output unit. Claim 5 In claim 4, the zero-shot model unit outputs the coordinates of the location where the specific object exists and the input prompt to the zero-shot model output unit when detecting a specific object corresponding to the input prompt in an image frame of a video stream received from the video capturing device, and the zero-shot model output unit generates the question information using the coordinates and input prompt output by the zero-shot model unit, characterized by a super-large AI-based object detection system.

Images