Image management apparatus and method for generating and distributing quantized neural network model

Abstract

The present specification discloses a technology related to the distribution of an artificial intelligence model for object re-identification (Re-ID). A neural network model is trained in an image management apparatus, and the trained model is collectively quantized on a server side and transmitted to camera devices, so that the spatial distribution of feature vectors is consistently maintained even between heterogeneous camera devices equipped with different SoC chips. Accordingly, the accuracy of object re-identification is ensured, compatibility between camera devices of various manufacturers is secured, and the efficiency and reliability of the entire system can be improved.

Description

Image management device and method for generating and distributing a quantized neural network model

[0001] This specification relates to artificial intelligence-based image analysis technology, and more specifically, to an image management device and method capable of performing a consistent object re-identification function regardless of the camera manufacturer or SoC chip manufacturer.

[0002] Recently, in video surveillance systems, object re-identification (Re-ID) technology is being actively researched to recognize objects such as people or vehicles from cameras and to determine whether they are the same object.

[0003] In conventional technology, video data captured by a camera is transmitted to a Video Management System (VMS), where an artificial intelligence model is applied to extract feature vectors, which are then stored in a database. Subsequently, when a user requests a search, a query image is received as input, the same AI model is applied to generate query feature vectors, and it is determined whether the objects are the same by comparing them with previously stored feature vectors.

[0004] Recently, deep learning-based object recognition and object re-identification (Re-ID) technologies are being actively utilized in surveillance systems and smart city environments.

[0005] Conventionally, backend servers receiving video from cameras perform decoding, object detection, optimal frame selection, object cropping, and feature vector extraction. However, this method requires a large amount of computational resources, making it difficult to guarantee real-time processing.

[0006] To improve this, a structure can be proposed in which the feature vector extraction function is performed at the camera end (edge ​​device). However, the SoC chips applied to the camera vary by manufacturer, and the quantization and compilation processes differ depending on the toolchain software provided by each SoC manufacturer.

[0007] As a result, even with the same training model, different quantization results occur depending on the camera manufacturer, and consequently, the consistency of feature vectors collected from the backend is degraded, which may lead to a problem where Re-ID accuracy decreases.

[0008] To solve the aforementioned problems, the present specification aims to provide an image management device and method that can ensure consistent feature vector generation between cameras by quantizing a neural network model learned in an image management device so that the spatial distribution of feature vectors extracted from objects is uniform, and resolve problems arising from different quantization results by SoC manufacturers by simply performing NPU execution format conversion (compilation) through Toolchain software specific to SoC chip manufacturers at the camera end, improve the accuracy of object re-identification, and ensure the real-time performance of the surveillance system.

[0009] The technical problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the following detailed description of the invention.

[0010] An image management device according to one embodiment of the present specification includes: a communication unit; and a processor that learns a neural network model for object re-identification, quantizes the learned neural network model so that the spatial distribution of feature vectors extracted from an object is uniform to generate a quantized neural network model, and transmits the quantized neural network model to a plurality of camera devices through the communication unit.

[0011] The plurality of camera devices includes heterogeneous camera devices including different SoC chips, and the spatial distribution of feature vectors extracted from the object using the quantized neural network model can be maintained consistent among the plurality of camera devices.

[0012] The plurality of camera devices can be compiled using the manufacturer-specific toolchain software of the SoC chip, and the compiled neural network model can be controlled to be executed on an NPU mounted on each camera device.

[0013] The processor can receive metadata including attribute information of the object and a cropped image of the object, along with a feature vector for re-identifying the object from the plurality of camera devices, through the communication unit.

[0014] The processor can verify whether the accuracy of the object re-identification is within an allowable error range by comparing the quantized neural network model with the learned neural network model, and then transmit it to a plurality of camera devices through the communication unit.

[0015] In another embodiment of the present specification, a system for re-identifying another object comprises: at least one camera device that extracts a feature vector of an object based on a neural network model for re-identifying an object; and an image management device that performs object re-identification based on the feature vector of the object received from the camera device. The neural network model included in the at least one camera device may include a quantized neural network model that is learned by the image management device and then quantized so that the spatial distribution of the feature vector extracted from the object is uniform.

[0016] The above at least one camera includes different SoC chips and can convert and execute the quantized neural network model received from the image management device so that it can be applied to the SoC chip mounted on the camera device.

[0017] The above at least one camera device can transmit attribute information of the object or a cropped image along with the extracted feature vector to the image management device.

[0018] A control method for an image management device for object re-identification according to another embodiment of the present specification comprises: a step of learning a neural network model for object re-identification; a step of generating a quantized neural network model such that the spatial distribution of feature vectors extracted from an object is uniformly formed from the learned neural network model; and a step of transmitting the quantized neural network model to a plurality of camera devices.

[0019] The plurality of camera devices includes heterogeneous camera devices comprising different SoC chips, and the quantized neural network model can be consistently applied among the plurality of camera devices.

[0020] The control method of the image management device for object re-identification may further include a step of verifying whether the accuracy of object re-identification is within an allowable error range by comparing the quantized neural network model with the learned neural network model.

[0021] According to one embodiment of the present specification, since quantization is performed in an image management device, the same quantized neural network model is transmitted to a plurality of camera devices, thereby enabling server-centric quantization.

[0022] In addition, according to one embodiment of the present specification, the camera unit simply performs format conversion through Toolchain software specific to the SoC chip manufacturer, thereby enabling the generation of consistent feature vectors regardless of differences in NPUs by manufacturer.

[0023] In addition, according to one embodiment of the present specification, since the spatial distribution of feature vectors is maintained uniformly, the accuracy of distance comparison between cameras is guaranteed and object re-identification performance can be improved.

[0024] In addition, according to one embodiment of the present specification, by eliminating camera dependencies in the backend, the system architecture can be simplified and maintenance costs can be reduced.

[0025] The effects obtainable in this specification are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below.

[0026] The accompanying drawings, included as part of the detailed description to aid in understanding the present specification, provide embodiments of the present specification and, together with the detailed description, also explain the technical features of the present specification.

[0027] FIG. 1 is a drawing for illustrating a surveillance camera system for implementing a method for performing object re-identification according to one embodiment of the present specification.

[0028] FIG. 2 is an exemplary block diagram of an image management device according to one embodiment of the present specification.

[0029] FIG. 3 is an exemplary drawing for explaining the concept of performing object re-identification in an image management device according to one embodiment of the present specification.

[0030] FIG. 4 is an exemplary sequence diagram for deploying an AI model to an edge device environment according to one embodiment of the present specification.

[0031] FIG. 5a is a conventional method for reducing performance differences in AI models, and FIG. 5b is an exemplary diagram for explaining the process of generating and distributing a quantized neural network model according to one embodiment of the present specification.

[0032] FIG. 6 is a flowchart of a method for generating and distributing another quantized neural network model according to one embodiment of the present specification.

[0033] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present specification, provide embodiments of the present invention and explain the technical features of the present specification together with the detailed description.

[0034] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols will be assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the invention.

[0035] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0036] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0037] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0038] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0039] FIG. 1 is a drawing for illustrating a surveillance camera system for implementing a method for performing object re-identification according to one embodiment of the present specification.

[0040] Referring to FIG. 1, a surveillance camera system (10) according to one embodiment may include an image processing device (100) and an external device (300). In one embodiment, the surveillance camera system (10) may further include a network server (200) capable of performing wired and wireless communication with the image processing device (100) and the external device (300).

[0041] The image processing device (100) may be an electronic device for capturing images and audio of a surveillance area by capturing a surveillance area. The image processing device (100) may capture a surveillance area in real time for surveillance or security purposes and may be provided with one or more devices. The image processing device (100) may be a camera placed at a fixed position in a specific location, a camera that can move automatically or manually along a certain path, or a camera that can be moved by a person or a robot, etc. The image processing device (100) may be a PTZ camera having pan, tilt, and zoom functions. The image processing device (100) may be an analog camera, or an IP camera or network camera used by connecting to a wired or wireless internet. Depending on the function and use of the camera, the image processing device (100) may have various shapes and sizes, such as dome type, box type, bullet type, camouflage type, etc.

[0042] In one embodiment, the image processing device (100) is a visible light camera and can generate a visible image according to the luminance distribution of an object by acquiring image information by detecting light. In one embodiment, the image processing device (100) is an infrared light camera (or thermal camera) and can generate a thermal image that displays different colors depending on the intensity by detecting radiant energy (thermal energy) emitted by an object in the form of an infrared wavelength, which is a type of electromagnetic wave, and measuring the intensity of the thermal energy.

[0043] The image processing device (100) may have the function of recording the surveillance area or taking a photograph. The image processing device (100) may have the function of recording sounds occurring in the surveillance area. The image processing device (100) may have the function of generating an alert or performing recording or taking a photograph when a change, such as movement or sound, occurs in the surveillance area.

[0044] The image processing device (100) can analyze the captured image itself and / or an image obtained by editing the image. For example, the image processing device (100) can detect objects in the image by using an object detection algorithm. The object detection algorithm may be an AI-based algorithm, and may detect objects by applying a pre-trained artificial neural network model.

[0045] The image processing device (100) can analyze an image to generate metadata and index information for said metadata. The image processing device (100) can analyze the image and / or audio included in the image together or separately to generate metadata and index information for said metadata.

[0046] The network server (200) can provide various functions to the image processing device (100) and the external device (300) through the network (400). In FIG. 1, the network server (200) is provided independently from the image processing device (100), but the embodiments of the present invention are not limited thereto. For example, the network server (200) may be embedded in the image processing device (100). A description of the network server (200) will be provided later.

[0047] The external device (300) may include a server (3001) and a client (3002).

[0048] The server (3001) may be a device that includes a DVR (Digital Video Recorder), NVR (Network Video Recorder), VMS (Video Management System), etc., and provides video storage, search, and management functions. The server (3001) may receive and store video and / or audio from the video processing device (100) and index the video and / or audio for user search. The server (3001) may analyze the video and / or audio received from the video processing device (100). In one embodiment, a plurality of servers (3001) built at physically different locations may be federated with each other and managed and controlled by a single software. In one embodiment, the server (3001) may be built in a government office, police station, hospital, central control center, central control center, comprehensive situation room, etc.

[0049] The client (3002) is a terminal including a desktop PC, tablet PC, slate PC, notebook computer, mobile device such as a smartphone, etc., and may include software and / or applications for performing video monitoring and search, etc. by a user (operator). The client (3002) may be connected to the image processing unit (100) and / or server (3001) via a network. In one embodiment, the client (3002) may manage and operate the image processing unit (100) and server (3001) based on the cloud.

[0050] In one embodiment, the client (3002) transmits an information provision request signal to the image processing device (100) or server (3001) requesting the provision of all or part of the image and / or audio, and may receive all or part of the image and / or audio from the image processing device (100) or server (3001). The client (3002) transmits an information provision request signal to the image processing device (100) or server (3001) requesting metadata and / or index information for the metadata obtained by analyzing the image and / or audio, and may receive metadata and / or index information from the image processing device (100) or server (3001).

[0051] The image processing device (100), network server (200), and external device (300) can perform wired and wireless communication through the network (400). The network (400) may include, for example, wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs (Integrated Service Digital Networks), wireless internet such as 3G, 4G (LTE), 5G, WiFi, Wibro, Wimax, wireless LANs, CDMA, satellite communication, and wireless networks including short-range communication such as Bluetooth, RFID (Radio Frequency Identification), infrared communication (IrDA, infrared Data Association), UWB (Ultra Wideband), ZigBee, and NFC (Near Field Communication). In the case of wireless mobile communication, the network (400) may additionally include components such as a base station (BTS), a mobile switching center (MSC), a home location register (HLR), an access gateway that enables the transmission and reception of wireless packet data, and a Packet Data Serving Node (PDSN). The scope of this specification is not limited thereto.

[0052] Meanwhile, for the convenience of explanation, the image processing device (100) of FIG. 1 is referred to as a camera device (100).

[0053] FIG. 2 is an exemplary block diagram of an image management device according to one embodiment of the present specification. The image management device (20) shown in FIG. 2 may be an exemplary device (or server) capable of performing the same function as the server (3001) shown in FIG. 1.

[0054] Referring to FIG. 2, the image management device (20) may include a communication unit (21), an input unit (22), an interface (23), a display unit (24), an AI processor (25), a memory (26), and a database (27).

[0055] The image management device (20) can extract feature information of objects included in the image by analyzing metadata received from the image processing device (100, hereinafter referred to as the 'camera device'). Then, it constructs a database that a user can search by comparing it with previously stored feature information of objects. To this end, the image management device (20) includes a processor (28), a memory (26), an input unit (22), and a display unit (24). These components can be connected to each other and communicate via a bus.

[0056] The communication unit (21) can receive video data, voice data, still images, and / or metadata from the camera device (100). According to one embodiment, the communication unit (21) can receive video data, voice data, still images, and / or metadata from the camera device (10) in real time. The communication interface can perform at least one communication function among wired / wireless LAN (Local Area Network), Wi-Fi, ZigBee, Bluetooth, and Near Field Communication.

[0057] Input units (22) include a mouse, keyboard, joystick, remote control, etc. These input units can be connected to a bus via an input interface (23) including a serial port, a parallel port, a game port, a USB, etc. However, if the image management device (200) provides a touch function, the display unit (24) may include a touch sensor. In this case, there is no need to provide a separate input unit (22), and the user can directly input a touch signal through the display unit (24).

[0058] The interface (23) may include a network interface, a video interface, an input interface, etc. The network interface may include a network interface card, a modem, etc.

[0059] Even if the image management device (20) provides a touch function, if the display unit (24) does not include a touch sensor, a separate touch pad may be provided as an input unit (22). Various types of displays may be used for the display unit (24), such as LCD (Liquid Crystal Display), OLED (Organic Liquid Crystal Display), CRT (Cathode Ray Tube), and PDP (Plasma Display Panel). This display unit (24) is connected to a bus through a video interface (not shown), and data transmission between the display unit (24) and the bus can be controlled by a graphics controller (132).

[0060] The AI ​​processor (25) is for artificial intelligence image processing and applies a deep learning-based object detection algorithm learned as an object of interest in an image acquired through a surveillance camera system according to one embodiment of the present specification. The AI ​​processor (25) may be implemented as a single module with a processor (26) that controls the entire system, or as an independent module. In one embodiment of the present specification, the YOLO (You Only Lock Once) algorithm may be applied for object detection. YOLO is an AI algorithm suitable for surveillance cameras that process real-time video because it has a fast object detection speed. Unlike other object-based algorithms (Faster R-CNN, R_FCN, FPN-FRCN, etc.), the YOLO algorithm resizes a single input image and passes it through a single neural network only once, resulting in the output of a bounding box indicating the location of each object and a classification probability of what the object is. Finally, a single object is detected once through non-max suppression.

[0061] Meanwhile, it should be noted that the object recognition algorithm disclosed in this specification is not limited to the aforementioned YOLO and can be implemented with various deep learning algorithms.

[0062] Meanwhile, the learning model for object recognition applied in this specification may include a neural network model trained to extract the gender of a person when the object included in the aforementioned image is a person. As training data for training the neural network model, multiple images with different shooting location information and shooting time information obtained from multiple surveillance cameras separated by a certain distance or more are defined as input data, and a network may be trained to be designed to extract gender and image feature information from the multiple images.

[0063] The memory (26) stores various object information, and a database (27) is constructed by the processor (28). The memory (26) includes a non-volatile memory device and a volatile memory device. The non-volatile memory device is preferably a NAND flash memory that is small in volume, lightweight, and resistant to external shocks, and the volatile memory device is preferably a DDR SDRAM.

[0064] The database (27) is a storage for systematically managing feature information of an object stored in the memory (26), and stores feature information of an object transmitted from a camera device and maintains an indexing structure so that it can be compared with query feature information corresponding to an input query. Through this, similarity calculation results, comparison scores, and metadata can be managed to efficiently determine whether the objects are the same.

[0065] In one embodiment, the database (27) may include a storage structure separated by model version. For example, by independently storing and managing feature information extracted from the first artificial intelligence model and feature information extracted from the second artificial intelligence model, the existing feature information can be utilized as is without re-extracting it even if the artificial intelligence model is updated.

[0066] Additionally, the database (27) may include a vector search engine for high-speed searching of large-scale feature information and may apply Approximate Nearest Neighbor (ANN) based indexing techniques (e.g., FAISS, Annoy, HNSW, etc.). This enables real-time object re-identification even in a large-scale surveillance network environment.

[0067] In addition, the database (27) also stores management information such as user accounts, query history, and access logs, which can be used for security management and performance optimization of the system.

[0068] All components included in the processor (28) may be connected to the bus through at least one interface or adapter, or directly connected to the bus. Additionally, the bus may be connected to other subsystems in addition to the components described above. The bus may include a memory bus, a memory controller, a peripheral bus, and a local bus.

[0069] The processor (28) controls the overall operation of the image management device (20). For example, when metadata is received from a camera device (100) or a separate engine, feature information of an object included in the image can be extracted from the metadata and stored in a database (27). As an example of the metadata, the present specification may include the gender of an object (person) included in the image, feature information of the image, location information where the image capturing device is located, shooting time information, etc.

[0070] The processor (28) is preferably a CPU (Central Processing Unit), MCU (Micro Controller Unit), or DSP (Digital Signal Processor), but is not limited thereto and various logic operation processors may be used.

[0071] According to one embodiment, the processor (28) is a central control unit that performs overall control of the image management device (20) and is connected to the communication unit (21), input unit (22), interface (23), display unit (24), AI processor (25), memory (26), and database (27) to control data transmission and reception and operation among them.

[0072] The processor (28) manages metadata and feature information received from the camera device through the communication unit (21), and can generate query feature information by calling an operation module stored in the AI ​​processor (25) or memory (26) for an input query. In addition, it performs an operation to determine whether the object is the same by comparing the feature information of an object stored in the database (27) with the query feature information.

[0073] In one embodiment, the processor (28) determines whether to update the artificial intelligence model, and if the first model is updated to the second model in the camera device, it can synchronize the internal operation logic of the image management device in response. For example, it can additionally receive feature information generated from the updated second model and store it in the database (27), generate query feature information by applying both the first model and the second model to the input query, and then determine whether they are the same object by comparing the corresponding feature information of each model.

[0074] Additionally, the processor (28) can derive a final re-identification result by applying a reliability score or weight fusion to multiple comparison results, and this result can be provided to the user through the display unit (24).

[0075] FIG. 3 is an exemplary drawing for explaining the concept of performing object re-identification in an image management device according to one embodiment of the present specification.

[0076] A camera device (100) captures a surveillance space to acquire image data (31). The acquired image data (31) may detect objects of interest (32), such as people, by an object detection module. The detected objects may be analyzed by a feature extractor (33) to generate feature vectors (34) that reflect the external attributes of the objects. The feature vectors (34) may be represented, for example, as numerical vectors of hundreds of dimensions.

[0077] The generated feature vector (34) can be transmitted to the image management device (20) and stored in the image management device (20). The image management device (20) may include a VMS, NVR, and cloud device as described above. When a user requests a search for a specific object, a query (36) is entered.

[0078] The query (36) may be, for example, image data, and the image management device (200) may extract a feature vector for the query object and compare it with the feature vectors stored in the database (35). During the comparison process, the similarity between each feature vector is calculated, and the result may be provided as a search result (37) through a similarity search module. The search result (37) may include similarity scores such as 0.9, 0.8, 0.4, for example, and may present objects sorted in order of high similarity to the query object. Through this, the user can identify identical objects or similar objects.

[0079] In one embodiment of the present specification, the query may be generated not only by image data but also by still images, external image files uploaded by a user, or non-image data such as text and voice. In particular, text or voice may be converted into feature information reflecting the attributes of an object through a generative artificial intelligence model, and the user may request object re-identification with only the input of a simple description.

[0080] In addition, in one embodiment of the present specification, the image management device (20) may generate feature information by multiple artificial intelligence models, such as a convolutional neural network (CNN) or a Transformer network, for image-based queries. The generated feature information may be fused and used for final comparison. Additionally, voice or text queries may be converted into embedding vectors and compared in the same latent space as image feature vectors.

[0081] According to one embodiment of the present specification, the image management device (20) may calculate the similarity not only directly between the query feature information and the feature information received from the camera device (100) and stored in the database, but also after performing filtering based on specific attributes (e.g., color, pattern, motion, etc.). Additionally, comparison results generated from a plurality of models may be fused based on weights or selectively adopted according to reliability criteria.

[0082] In one embodiment of the present specification, a feature extractor (33) embedded in a camera device (100) is generated based on a neural network model learned in an image management device (20). That is, after the image management device (20) learns a neural network model for object re-identification, the learned model can be quantized so that the spatial distribution of feature vectors extracted from the object is uniform. The quantized neural network model is transmitted to a plurality of camera devices and can be compiled according to the manufacturer-specific toolchain software of the SoC chip mounted in the camera device and executed in the Neural Processing Unit (NPU) inside the camera. In this process, the plurality of camera devices may include different SoC chips, but the spatial distribution of feature vectors extracted from each camera device can be consistently maintained through the quantization process performed by the image management device (20). Accordingly, one embodiment of the present specification provides the effect of stably ensuring the accuracy of object re-identification even between heterogeneous cameras of various manufacturers.

[0083] FIG. 4 is an exemplary sequence diagram for deploying an AI model to an edge device environment according to one embodiment of the present specification.

[0084] Referring to FIG. 4, the image management device (20) learns a neural network model (AI model) for object re-identification (S400).

[0085] Here, training a neural network model for object re-identification can mean creating a neural network model capable of reliably extracting feature vectors of objects captured by camera devices using a large amount of training data. For example, in the case of Person Re-ID, training can be performed to map detailed attributes such as the upper body, clothing patterns, colors, and gait into a vector space. The feature space can be optimized so that the distance between feature vectors is close even when the same person is captured by different camera devices, while different people are clearly distinguishable. Additionally, in the case of Vehicle Re-ID, training can be performed by reflecting features such as vehicle type, color, external structure of the vehicle, or specific patterns attached to the vehicle (e.g., license plate, light shape, etc.). Through this, high similarity between feature vectors can be maintained even when the same vehicle is captured by different cameras.

[0086] In addition, a neural network model according to one embodiment of the present specification may be implemented as a convolutional neural network (CNN), a transformer network, or a hybrid network structure combining the two, and training data may be constructed by combining a public dataset and a closed dataset collected in a specific service environment.

[0087] Accordingly, the object re-identification neural network model according to one embodiment of the specification provides the ability to reliably distinguish and match objects captured in different environments in a feature space, and this can maintain consistency in object re-identification performance during the subsequent quantization step and the process of distributing the same model to multiple heterogeneous camera devices.

[0088] The image management device (20) quantizes the learned neural network model so that the spatial distribution of feature vectors extracted from the object is uniform, thereby generating a quantized neural network model (S410).

[0089] Quantizing so that the spatial distribution of feature vectors is uniform can mean the process of converting the parameters of a learned neural network model to low-bit precision so that relative distance information between feature vectors extracted in a high-dimensional space is preserved. For example, by converting 32-bit floating-point-based operations to 8-bit integer operations, the computational efficiency of the model is increased, and at the same time, the accuracy of the distance (similarity) comparison between feature vectors, which is important for re-identification performance, is not lost.

[0090] In this embodiment, quantization can be implemented through programs such as TensorFlow Lite, PyTorch QAT / PTQ, NVIDIA TensorRT, Intel OpenVINO, and ONNX / ONNC, and can be performed in a way that improves computational efficiency by converting floating-point operations into integer operations.

[0091] In this way, by performing the quantization process on the server side, that is, on the image management device (20), there is no need to perform quantization individually at each camera unit by relying on toolchain software specific to the SoC chip manufacturer. As a result, a neural network model with consistent standards can be provided for multiple heterogeneous camera devices.

[0092] Subsequently, the image management device (20) transmits the quantized neural network model to multiple camera devices (S420). This process can be performed via a wired or wireless network communication module, and since the model size is reduced, transmission delay (latency) and bandwidth burden can be minimized.

[0093] Multiple camera devices may have different computing environments depending on the SoC chip manufacturer, and each camera device may compile a received quantized neural network model through toolchain software provided by the SoC chip manufacturer and convert it into an execution format suitable for the data processing device of the camera device, such as an NPU (S430). For example, a camera device of company A may use a Tensor RT-based compiler, a camera device of company B may use an OpenVINO-based toolchain, and a camera device of company C may use its own proprietary toolchain.

[0094] The above camera device can be utilized to extract object feature vectors from image data by loading a compiled neural network model onto its own hardware (NPU, GPU, or CPU) (S440). Through this, the feature vectors extracted from the camera device are distributed according to the same quantization criteria regardless of the manufacturer and type of hardware, so that the object re-identification results performed by the image management device (20) can be maintained consistently.

[0095] FIG. 5a is a conventional method for reducing performance differences in AI models, and FIG. 5b is an exemplary diagram for explaining the process of generating and distributing a quantized neural network model according to one embodiment of the present specification.

[0096] Referring to FIG. 5a, an AI model (500) trained in an image management device (20) can be distributed to a plurality of camera devices (100). However, each camera device (100A, 100B, 100C, 100D) may include different SoC chips, and quantization and compilation may be performed differently depending on the toolchain software provided by the manufacturer of the SoC chip. As a result, the same AI model (500) is executed as a quantized model (500A, 500B, 500C, 500D) that is converted differently in each camera device, and consequently, inconsistencies may occur in the distribution of feature vectors, which may cause problems that reduce the accuracy of object re-identification.

[0097] In contrast, referring to FIG. 5B, the image management device (20) learns a neural network model (500) for object re-identification and quantizes the learned neural network model so that the spatial distribution of feature vectors extracted from the object becomes uniform, thereby generating a quantized neural network model (510).

[0098] The quantized neural network model (510) generated in this way can be transmitted to multiple camera devices (100) through a communication unit. Each camera device (100A, 100B, 100C, 100D) can execute the received quantized neural network model (510') by simply performing compilation using the toolchain software of the SoC chip equipped with it. Therefore, different quantization results do not occur for each camera device, and the same quantized neural network model can be consistently distributed and executed. As a result, the spatial distribution of feature vectors is maintained uniformly among multiple camera devices, thereby achieving the effect of improving object re-identification performance.

[0099] FIG. 6 is a flowchart of a method for generating and distributing another quantized neural network model according to one embodiment of the present specification. Hereinafter, each step is described assuming that it is performed by the processor (28) of the image management device (20) shown in FIG. 2.

[0100] Referring to FIG. 6, the processor (28) trains a neural network model for object re-identification (S600).

[0101] Here, training a neural network model for object re-identification refers to the process of optimizing the parameters of a neural network using multiple training datasets so that the external features of objects, such as people and vehicles, can be mapped into a unique vector space. For example, a model capable of reliably extracting feature vectors from object images can be trained using a Convolutional Neural Network (CNN) or a Transformer structure. The neural network model trained in this way is subsequently utilized in the camera device (100) to generate feature vectors for object re-identification.

[0102] The processor (28) quantizes the learned neural network model so that the spatial distribution of the feature vectors extracted from the object is uniform (S610).

[0103] In other words, quantization can be performed by converting weights and operations, originally represented as 32-bit floating-point numbers, into 8-bit integer operations to lighten the model, while simultaneously homogenizing the distribution to maintain the accuracy of distance calculations between feature vectors. For example, this can be implemented through programs and frameworks such as TensorFlow Lite, PyTorch QAT / PTQ, NVIDIA TensorRT, Intel OpenVINO, ONNX, and ONNC, and this process may involve verification to ensure that the discriminative power based on Euclidean distance or cosine similarity between feature vectors remains within an acceptable margin of error.

[0104] The processor (28) can transmit the quantized neural network model to a plurality of camera devices (100) through a communication unit (S620).

[0105] Multiple camera devices (100) may be heterogeneous devices including different SoC chips, and the quantized neural network model is compiled according to toolchain software provided by each SoC chip manufacturer and executed on an NPU mounted on the device. In this way, by performing quantization integrally at the image management device (20) on the server side and simply performing compilation at the camera side, the spatial distribution of feature vectors can be consistently maintained even among camera devices having different SoC chips.

[0106] One or more of the surveillance cameras, autonomous vehicles, user terminals, and servers may be linked with artificial intelligence modules, robots, augmented reality (AR) devices, virtual reality (VR) devices, devices related to 5G services, etc.

[0107] For the purpose of understanding the present invention, reference numerals have been used in the preferred embodiments illustrated in the drawings, and specific terms have been used to describe the embodiments; however, the present invention is not limited by these specific terms, and the present invention may include all components that are conventionally conceivable by those skilled in the art.

[0108] The present invention may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the present invention may employ direct circuit configurations such as memory, processing, logic, look-up tables, etc., which can execute various functions by the control of one or more microprocessors or other control devices. Similar to how the components of the present invention may be implemented as software programming or software elements, the present invention may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as combinations of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Additionally, the present invention may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations. The above terms may include the meaning of a series of software processes (routines) in conjunction with processors, etc.

[0109] The specific embodiments described in this invention are examples and do not limit the scope of the invention in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as "essential," "importantly," etc., a component may not be strictly necessary for the application of the invention.

[0110] In the specification of the present invention (particularly in the claims), the use of the term "above" and similar descriptive terms may be in both singular and plural. Furthermore, where a range is described in the present invention, it is implied to include an invention applying individual values ​​belonging to said range (unless otherwise stated), and is equivalent to describing each individual value constituting said range in the detailed description of the invention. Finally, regarding the steps constituting the method according to the present invention, unless explicitly stated or otherwise stated, said steps may be performed in a suitable order. The present invention is not necessarily limited by the order in which said steps are described.

Claims

1. Communications Department; A processor that trains a neural network model for object re-identification, quantizes the trained neural network model so that the spatial distribution of feature vectors extracted from an object is uniform to generate a quantized neural network model, and transmits the quantized neural network model to a plurality of camera devices through the communication unit; An image management device including 2. In Paragraph 1, An image management device characterized in that the plurality of camera devices include heterogeneous camera devices including different SoC chips, and the spatial distribution of feature vectors extracted from the object using the quantized neural network model is maintained consistent among the plurality of camera devices.

3. In Paragraph 2, An image management device characterized by compiling the plurality of camera devices using the manufacturer-specific toolchain software of the SoC chip and executing the compiled neural network model on an NPU mounted on each camera device.

4. In Paragraph 1, The above processor is, An image management device characterized by receiving metadata including attribute information of the object and a cropped image of the object, along with a feature vector for re-identifying the object from the plurality of camera devices, through the communication unit.

5. In Paragraph 1, The above processor is, An image management device characterized by verifying whether the accuracy of object re-identification is within an allowable error range by comparing the quantized neural network model with the learned neural network model, and then transmitting it to a plurality of camera devices through the communication unit.

6. At least one camera device that extracts feature vectors of an object based on a neural network model for object re-identification; An image management device that performs object re-identification based on a feature vector of the object received from at least one camera device; A system characterized in that the neural network model included in the above-mentioned at least one camera device comprises a quantized neural network model such that the spatial distribution of feature vectors extracted from the object is uniform after being learned in the image management device.

7. In Paragraph 1, The above at least one camera is, A system comprising different SoC chips, characterized by converting and executing the quantized neural network model received from the image management device so that it can be applied to the SoC chip mounted on the camera device.

8. In Paragraph 1, A system characterized by the above-mentioned at least one camera device transmitting attribute information of the object or a cropped image along with an extracted feature vector to the image management device.

9. A method for controlling an image management device for object re-identification, Step of training a neural network model for object re-identification; The step of generating a quantized neural network model such that the spatial distribution of feature vectors extracted from an object is uniformly formed using the above-mentioned learned neural network model; and A step of transmitting the above-mentioned quantized neural network model to a plurality of camera devices; A method including 10. In Paragraph 9, The above plurality of camera devices includes heterogeneous camera devices comprising different SoC chips, and A method characterized by the above-described quantized neural network model being consistently applied among the plurality of camera devices.

11. In Paragraph 9, A step of verifying whether the accuracy of object re-identification is within an allowable error range by comparing the quantized neural network model with the learned neural network model; A method characterized by further including

Images