Method of providing indoor positioning and navigation by use of AI image analysis of CCTV videos

Abstract

The present invention generally relates to a technology for measuring a user's location and guiding a movement path within a building. In particular, the present invention relates to an AI video analysis-based indoor positioning and navigation technology that provides indoor navigation services by analyzing CCTV footage generated by multiple CCTV cameras distributed indoors for video surveillance in real time using artificial intelligence (AI), performing object detection, object tracking, and real-time object Re-ID for a service requester, and aligning this with indoor map data for the corresponding space. According to the present invention, since indoor positioning and navigation services are provided by utilizing multiple CCTV cameras installed for video surveillance, there is an advantage of low system construction costs and maintenance burdens. In particular, according to the present invention, since route guidance and verification of route movement are performed based on waypoints of the movement path, there is an advantage of being able to overcome CCTV blind spots in indoor navigation.

Description

Technology Field

[0001] The present invention generally relates to a technology for measuring a user's location and guiding a movement path inside a building.

[0002] In particular, the present invention relates to an AI video analysis-based indoor positioning and navigation technology that provides an indoor navigation service by analyzing CCTV footage generated by a plurality of CCTV cameras distributed indoors for video surveillance in real time using artificial intelligence (AI), performing object detection, object tracking, and real-time object Re-ID for a service requester, and aligning it with indoor map data for the corresponding space. Background Technology

[0003] Recently, as the scale of buildings such as department stores, exhibition halls, and outlets has increased, there has been active development of indoor navigation technology that guides users to their respective destinations within the building.

[0004] Generally, navigation technology is developed based on the Global Positioning System (GPS), but alternative technology is required because receiving GPS signals is difficult in indoor spaces.

[0005] Conventionally, Bluetooth (BLE) and WiFi technologies were used for indoor navigation. With multiple BLE beacons and WiFi access points installed within a building, the Received Signal Strength Indicator (RSSI) values ​​of the beacon signals and WiFi signals are measured by a smartphone carried by a user inside the building. Then, triangulation and fingerprinting techniques are applied to these RSSI values ​​to determine the user's location, and indoor navigation is provided based on this. However, this method has the disadvantage of high system construction costs and maintenance burdens because it requires installing multiple BLE beacon devices and WiFi access point devices within the building. Prior art literature

[0006] Republic of Korea Registered Patent No. 10-2144431 "AI-based indoor positioning system using a mobile terminal" Republic of Korea Registered Patent No. 10-1947134 "Method and apparatus for providing indoor navigation service" Republic of Korea Published Patent No. 10-2024-0093155 "Indoor positioning device and method thereof" Republic of Korea Published Patent No. 10-2023-0049251 "UWB-based AI indoor location tracking system and method" Republic of Korea Published Patent No. 10-2022-0145056 "Indoor navigation service method and apparatus" Republic of Korea Published Patent No. 10-2017-0130234 "Electronic device providing indoor navigation and method thereof" US Registered Patent US 8,872,800 B "Optical tablet stylus and indoor navigation system" US Registered Patent US 10,422,645 B "Electronic apparatus providing indoor navigation and method thereof" Japanese published patent JP 2016-512342 A The problem to be solved

[0007] The objective of the present invention is to provide a technology that generally measures a user's location inside a building and guides their movement path.

[0008] In particular, the objective of the present invention is to provide an AI video analysis-based indoor positioning and navigation technology that analyzes CCTV footage generated by multiple CCTV cameras distributed indoors for video surveillance in real time using artificial intelligence (AI), performs object detection, object tracking, and real-time object Re-ID for a service requester, and aligns it with indoor map data for the corresponding space to provide an indoor navigation service.

[0009] The problems solved by the present invention are not limited to this, and other problems can be understood from the description in this specification. means of solving the problem

[0010] To achieve the above objective, the present invention discloses a method for providing indoor positioning and navigation based on AI video analysis of CCTV footage, wherein the method analyzes in real time by artificial intelligence (AI) footage generated from multiple CCTV cameras distributed and installed indoors in a specific space for video surveillance, thereby providing indoor positioning and navigation services.

[0011] A method for providing indoor positioning and navigation based on AI video analysis of CCTV footage according to the present invention comprises: a step in which a navigation UI unit (110) of a user smartphone (100) transmits a route guidance request for a specific destination point indoors to an indoor LBS server (200) in response to user operation; a step in which a recognition information processing unit (210) of the indoor LBS server (200) transmits a recognition information request for a route guidance requester to the user smartphone (100); a step in which a recognition information providing unit (120) of the user smartphone (100) provides basic identification information of the route guidance requester to the indoor LBS server (200); a step in which a video analysis processing unit (220) of the indoor LBS server (200) acquires a plurality of object images that match the basic identification information through AI video analysis of a plurality of CCTV footages; and a step in which a video analysis processing unit (220) of the indoor LBS server (200) transmits a plurality of object images to the user smartphone (100). A step in which the recognition information providing unit (120) of the user smartphone (100) identifies a user selection of an object image of the route guidance requester among a plurality of object images in response to user operation; a step in which the recognition information providing unit (120) of the user smartphone (100) provides user selection information regarding the object image of the route guidance requester to the indoor LBS server (200); a step in which the image analysis processing unit (220) of the indoor LBS server (200) acquires unique feature information for distinguishing and identifying the route guidance requester in the CCTV video based on the user selection information; a step in which the indoor positioning processing unit (230) of the indoor LBS server (200) identifies the initial location of the route guidance requester based on indoor map data and user selection information; a step in which the route guidance processing unit (240) of the indoor LBS server (200) acquires route guidance data by analyzing the initial location, the destination point, and the route based on indoor map data; a step in which the route guidance processing unit (240) of the indoor LBS server (200) transmits the acquired route guidance data to the user smartphone (100).The method may be configured to include: a step in which a navigation processing unit (130) of a user smartphone (100) provides indoor navigation guidance based on route guidance data; a step in which a video analysis processing unit (220) of an indoor LBS server (200) performs AI video analysis corresponding to unique feature information on a plurality of CCTV videos; a step in which a movement path recognition unit (250) of an indoor LBS server (200) identifies the actual movement path of a route guidance requester based on indoor map data and the result of AI video analysis corresponding to unique feature information; a step in which a route guidance update unit (260) of an indoor LBS server (200) updates route guidance data in real time by comparing the route guidance data with the actual movement path; a step in which a route guidance update unit (260) of an indoor LBS server (200) transmits the updated route guidance data to the user smartphone (100); and a step in which a navigation processing unit (130) of a user smartphone (100) updates indoor navigation guidance based on the updated route guidance data.

[0012] At this time, the step of acquiring multiple object images that match basic identification information may be configured to include: a step of acquiring multiple bounding boxes by AI object detection on multiple CCTV videos; a step of expanding multiple bounding box areas to the surroundings according to a preset ratio so as to include surrounding areas; and a step of acquiring multiple object images from multiple CCTV videos corresponding to the expanded bounding box areas.

[0013] Additionally, the method for providing indoor positioning and navigation based on AI video analysis of CCTV video according to the present invention comprises: a step in which a navigation UI unit (110) of a user smartphone (100) transmits a route guidance request for a specific destination point indoors to an indoor LBS server (200) in response to user operation; a step in which a recognition information processing unit (210) of the indoor LBS server (200) transmits a mobile flash flashing request to the user smartphone (100); a step in which a recognition information providing unit (120) of the user smartphone (100) controls the flash lamp of the smartphone to flash according to a preset flashing pattern; a step in which a recognition information processing unit (210) of the indoor LBS server (200) recognizes the flash lamp flashing in the CCTV video; and a step in which a recognition information processing unit (210) of the indoor LBS server (200) recognizes a route guidance requester corresponding to the recognized flash lamp flashing in the CCTV video. A step in which the video analysis processing unit (220) of the indoor LBS server (200) obtains unique feature information to distinguish and identify a route guidance requester in CCTV video; a step in which the indoor positioning processing unit (230) of the indoor LBS server (200) identifies the initial location of the route guidance requester based on indoor map data and the recognized flash lamp blinking; a step in which the route guidance processing unit (240) of the indoor LBS server (200) obtains route guidance data by route analysis based on the initial location, destination point, and indoor map data; a step in which the route guidance processing unit (240) of the indoor LBS server (200) transmits the obtained route guidance data to a user smartphone (100); a step in which the navigation processing unit (130) of the user smartphone (100) provides indoor navigation guidance based on the route guidance data; a step in which the video analysis processing unit (220) of the indoor LBS server (200) performs AI video analysis corresponding to the unique feature information on a plurality of CCTV videos;The indoor LBS server (200) may be configured to include the following steps: a movement path recognition unit (250) of the indoor LBS server (200) identifying the actual movement path of a route guidance requester based on an AI video analysis result corresponding to indoor map data and unique feature information; a route guidance update unit (260) of the indoor LBS server (200) updating route guidance data in real time by comparing the route guidance data with the actual movement path; a route guidance update unit (260) of the indoor LBS server (200) transmitting the updated route guidance data to a user smartphone (100); and a navigation processing unit (130) of the user smartphone (100) updating indoor navigation guidance based on the updated route guidance data.

[0014] In the present invention, AI video analysis corresponding to unique feature information may be configured to include object detection, object tracking, and real-time re-identification of the same person based on AI video analysis corresponding to the unique feature information of the route guidance requester.

[0015] Additionally, the step of acquiring route guidance data may be configured to include: a step of acquiring a recommended route from an initial location to a destination point by route analysis based on an initial location, a destination point, and indoor map data; a step of identifying CCTV coverage existing on the recommended route; a step of setting waypoints on the CCTV coverage existing on the recommended route; and a step of generating route guidance data by combining the data of the recommended route and the data of the waypoints.

[0016] Additionally, the step of updating route guidance data in real time may be configured to include: a step of checking whether the actual travel path corresponds sequentially to a series of waypoints for the recommended path; and a step of updating the route guidance data by including travel error data if, as a result of the above check, it is determined that the sequential correspondence fails.

[0017] Meanwhile, the computer program according to the present invention is stored on a computer-readable non-volatile storage medium to enable a computer to execute a method for providing indoor positioning and navigation based on AI video analysis of CCTV footage as described above. Effects of the invention

[0018] According to the present invention, since indoor positioning and navigation services are provided by utilizing multiple CCTV cameras installed for video surveillance, there is an advantage of low system construction costs and maintenance burden.

[0019] In particular, according to the present invention, since route guidance and verification of route movement are performed based on waypoints of the movement path, there is an advantage of being able to overcome CCTV blind spots in indoor navigation. Brief explanation of the drawing

[0020] [Fig. 1] is an overall system configuration diagram with indoor positioning and navigation technology applied for the present invention. [Fig. 2] is an example of indoor map data in the present invention. [Fig. 3] is a flowchart of a first embodiment of a method for providing indoor positioning and navigation based on AI video analysis of CCTV video according to the present invention. [Fig. 4] is a conceptual diagram of object detection by AI image analysis in the present invention. [Fig. 5] is an example of an indoor navigation screen displayed on a user's smartphone in the present invention. [Fig. 6] is a conceptual diagram of object detection, object tracking, and real-time re-identification of the same person by AI image analysis in the present invention. [Fig. 7] is a flowchart of a second embodiment of a method for providing indoor positioning and navigation based on AI video analysis of CCTV video according to the present invention. Specific details for implementing the invention

[0021] The present invention will be described in detail below with reference to the drawings.

[0022] In describing the present invention, detailed explanations of parts that overlap with the prior art may be omitted.

[0023] [Fig. 1] is an overall system configuration diagram of the AI ​​image analysis-based indoor positioning and navigation technology applied for the present invention.

[0024] The indoor positioning and navigation system of [Fig. 1] generally provides indoor positioning and navigation services by utilizing multiple CCTV cameras (10) that are distributed and installed indoors in a specific space for video surveillance. By utilizing the artificial intelligence (AI) video analysis function of the video surveillance system (300), the CCTV videos generated from multiple CCTV cameras (10) are analyzed in real-time using AI, and the indoor positioning and navigation services are provided using the results of the video analysis.

[0025] It is common practice to establish a CCTV-based video surveillance system for buildings to prevent crime and safety accidents, and to secure evidence after the fact. For this purpose, CCTV cameras (10) are installed at various points within the building and provide footage of those points in real time to the video surveillance system (300). The video surveillance system (300) generally analyzes CCTV footage using artificial intelligence (AI) technology, which is referred to as intelligent selective surveillance. In intelligent selective surveillance, CCTV footage is input into a pre-trained neural network model to perform object detection, abnormality detection, object tracking, and real-time re-identification of the same person.

[0026] Meanwhile, the user carries a smartphone (100) and receives an indoor navigation service by utilizing an app installed on the smartphone (100). At this time, the user's smartphone (100) may be a device owned by the user or a device provided by the building.

[0027] When the indoor LBS server (200) receives a request for location verification and navigation from a user, it operates in cooperation with the video control system (300) to receive real-time AI video analysis results for CCTV footage and combines them with its own indoor map data to provide indoor positioning and indoor navigation services for the user.

[0028] At this time, the indoor LBS server (200) utilizes an indoor map database (201) and a camera database (202). The indoor map database (201) is a database that stores and manages data on the indoor structure (indoor map data) for each building. [Fig. 2] is an example of indoor map data in the present invention. Additionally, the camera database (202) is a database that stores and manages data on the installation status of CCTV cameras for each building (camera map data).

[0029] [Fig. 3] is a flowchart of a first embodiment of a method for providing indoor positioning and navigation based on AI video analysis of CCTV video according to the present invention.

[0030] Referring to [Fig. 3], the user smartphone (100) is equipped with a navigation UI unit (110), a recognition information providing unit (120), and a navigation processing unit (130).

[0031] The navigation UI (user interface) section (110) is a component that provides a user interface (UI) for using indoor positioning and navigation services on a user smartphone (100).

[0032] The recognition information providing unit (120) is a component that provides external features (e.g., hat, clothing, hairstyle, gender, etc.) to the indoor LBS server (200) that enable the identification of a route guidance requester among numerous people captured in the CCTV video. Generally, these external features can be set through menu operations provided by the navigation UI unit (110).

[0033] The navigation processing unit (130) is a component that provides indoor navigation to the user using data provided by the indoor LBS server (200).

[0034] Additionally, referring to [Fig. 3], the indoor LBS server (200) is equipped with a recognition information processing unit (210), an image analysis processing unit (220), an indoor positioning processing unit (230), a route guidance processing unit (240), a movement path recognition unit (250), and a route guidance update unit (260).

[0035] The recognition information processing unit (210) is a component that requests recognition information about a route guidance requester via a user smartphone (100) and receives information on the appearance characteristics of the route guidance requester in response.

[0036] The video analysis processing unit (220) is a component that provides real-time AI video analysis for CCTV video. Preferably, the video analysis processing unit (220) can be implemented in a form that utilizes the AI ​​video analysis function that the video control system (300) has to perform smart selective control. Meanwhile, the video analysis processing unit (220) may be configured to have its own real-time AI video analysis function.

[0037] The indoor positioning processing unit (230) is a component that identifies where a route guidance requester is located indoors using CCTV video analysis results (multiple object images), user selection information regarding the same, indoor map data, and camera map data.

[0038] The route guidance processing unit (240) is a component that obtains route guidance data regarding the route that the route guidance requester must travel in the building by analyzing the route based on the initial location and destination point of the route guidance requester and the indoor map data.

[0039] The movement path recognition unit (250) is a component that identifies how the route guidance requester is actually moving at present by combining the result of real-time AI video analysis of the CCTV video using the route guidance requester's unique appearance feature information with the indoor map data and camera map data while the route guidance requester is moving toward the destination location after providing the first indoor navigation guidance.

[0040] The route guidance update unit (260) is a component that updates the route guidance data in real time based on the error between the route guidance data and the actual movement path when, as a result of comparing and analyzing the route guidance data and the actual movement path, it is determined that the route guidance requester is not moving in accordance with the original route guidance data. Then, the real-time updated route guidance data is transmitted to the user's smartphone (100) to guide the user to move along the correct path.

[0041] Hereinafter, a first embodiment of a method for providing indoor positioning and navigation based on AI video analysis of CCTV video according to the present invention is described.

[0042] Step (S110): First, the navigation UI unit (110) of the user smartphone (100) transmits a route guidance request for a specific destination point indoors to the indoor LBS server (200) in response to user operation.

[0043] Step (S120, S130): The recognition information processing unit (210) of the indoor LBS server (200) transmits a request for external features (e.g., hat, clothing, hairstyle, gender, etc.) that enable the route guidance requester to be distinguished and identified within the CCTV video to the user smartphone (100). This is referred to as a 'recognition information request'.

[0044] In response to this, the recognition information providing unit (120) of the user smartphone (100) provides basic identification information of the route guidance requester to the indoor LBS server (200). The user (route guidance requester) inputs appearance information about themselves by operating the menu displayed on the screen of the user smartphone (100), such as a beige hat, sunglasses, a white T-shirt, blue sneakers, etc. The recognition information providing unit (120) provides the information thus input to the indoor LBS server (200), which is called 'basic identification information'.

[0045] Step (S140, S150): Next, the video analysis processing unit (220) of the indoor LBS server (200) acquires multiple object images that match basic identification information through AI video analysis of multiple CCTV videos, and transmits these object images to the user smartphone (100). This is a process for the indoor LBS server (200) to analyze CCTV videos and search for a route guidance requester.

[0046] The indoor LBS server (200) searches for objects having the external features of the basic identification information in the CCTV video through AI video analysis. Generally, multiple objects having the external features of the basic identification information will be found. Since the route guidance requester cannot exist in multiple locations at the same time, these are multiple people with similar external features. Accordingly, the indoor LBS server (200) transmits images of multiple objects having the external features of the basic identification information to the user's smartphone (100) so that the route guidance requester can select which one they are among them.

[0047] In the present invention, the video analysis processing unit (220) of the indoor LBS server (200) detects objects that match basic identification information by AI video analysis of CCTV video. [Fig. 4] is a conceptual diagram of object detection by AI video analysis in the present invention. Each frame image constituting the CCTV video is input into an object detection neural network model to extract multiple objects of interest. The object detection neural network model outputs coordinate information and size information of the bounding box containing these objects of interest, and class information of the detected objects. In this way, the video analysis processing unit (220) extracts objects of interest by AI analyzing the CCTV video generated by multiple CCTV cameras (10) installed in the building, and compares the information of these objects of interest with basic identification information. Through this, multiple object images that match the basic identification information are obtained, and these object images are transmitted to the user smartphone (100).

[0048] The object detection neural network model extracts bounding boxes containing objects of interest from multiple CCTV videos. The video analysis processing unit (220) of the indoor LBS server (200) may be configured to obtain an object image by cropping the image of the bounding box area from the CCTV video. However, preferably, the bounding box area extracted by the object detection neural network model is expanded outward according to a preset ratio to include the surrounding area, and then the object image is obtained from the CCTV video corresponding to the expanded bounding box area. Preferably, the bounding box area is expanded horizontally and vertically by a preset specific ratio based on the center position of the bounding box. The horizontal expansion ratio and the vertical expansion ratio may be set to be the same or different.

[0049] By expanding the bounding box area in this way, the background area surrounding the object of interest in the CCTV footage is included within the expanded bounding box. Generally, the bounding box provided by object detection neural network models is formed to fit snugly along the boundary of the object of interest; however, this approach fails to properly capture information regarding the context in which the object is situated. By expanding the bounding box horizontally and vertically based on its center, the background area surrounding the object can be included. This enables the user requesting path guidance to accurately identify the object image.

[0050] Step (S160, S170): The navigation UI unit (110) of the user smartphone (100) displays multiple object images provided by the indoor LBS server (200) on the smartphone screen. On the screen, the route guidance requester performs an operation to select an object image corresponding to themselves among these object images. The recognition information providing unit (120) of the user smartphone (100) identifies the user selection operation and provides the corresponding user selection information to the indoor LBS server (200).

[0051] Step (S180): The image analysis processing unit (220) of the indoor LBS server (200) distinguishes between an object image corresponding to a route guidance requester and an object image not corresponding to a route guidance requester among a plurality of object images based on user selection information. Since the plurality of object images were extracted from the CCTV video by the basic identification information in (S140) above, they have similar external features. By distinguishing between the object image corresponding to a route guidance requester and the object image not corresponding to a route guidance requester among these similar object images, the image analysis processing unit (220) of the indoor LBS server (200) derives unique feature information for precisely distinguishing and identifying the route guidance requester from the CCTV video through additional image analysis using these two groups of object images.

[0052] Generally, in (S130), the route guidance requester is likely to roughly provide their own information (basic identification information) (e.g., beige hat, sunglasses, white T-shirt, blue sneakers, etc.). In places where many people are gathered, such as department stores or sports stadiums, many objects with such characteristics will be extracted from indoor CCTV footage. If the route guidance requester distinguishes themselves from these object images, the indoor LBS server (200) can extract the unique characteristics of the route guidance requester that distinguish them from other objects through additional video analysis.

[0053] Step (S190 ~ S210): The indoor positioning processing unit (230) of the indoor LBS server (200) identifies the initial location of the route guidance requester based on indoor map data, camera map data, and user selection information. Based on the user selection information, it is possible to determine which CCTV camera (10) the route guidance requester is located in. The shooting location or installation location of the corresponding CCTV camera (10) can be set as the initial location of the route guidance requester.

[0054] Then, the route guidance processing unit (240) of the indoor LBS server (200) obtains route guidance data by analyzing the route based on the initial location and destination point of the route guidance requester and the indoor map data of the building.

[0055] Since the path analysis algorithm is a technology used in general navigation services and is not a feature of the present invention, a detailed description of the algorithm itself for deriving the movement path is omitted in this specification. However, considering the field of indoor navigation utilizing CCTV footage, the configuration for acquiring data that guides the movement path (path guidance data) can be improved.

[0056] First, a route analysis algorithm is applied to the route guidance requester's initial location, destination point, and indoor map data to obtain a recommended route from the initial location to the destination point within the building.

[0057] Then, the shooting area (CCTV coverage) of the CCTV camera (10) on the recommended path is identified by referring to the camera map data, and waypoints are set on the CCTV coverage on the recommended path by referring to the indoor map data. Generally, multiple waypoints are set on the recommended path.

[0058] Since the CCTV camera (10) does not cover all areas of the building, there are CCTV blind spots. Due to these CCTV blind spots, it is often difficult to monitor whether the route guidance requester is moving correctly along the recommended route. Considering this problem, the present invention sets waypoints in the CCTV coverage area on the recommended route and checks whether the route guidance requester passes through these waypoints sequentially.

[0059] Based on these points, in a preferred embodiment of the present invention, waypoint data is combined with recommended route data to generate route guidance data.

[0060] Next, the route guidance processing unit (240) of the indoor LBS server (200) transmits the acquired route guidance data to the user smartphone (100).

[0061] When the navigation processing unit (130) of the user smartphone (100) receives route guidance data from the indoor LBS server (200), it provides indoor navigation guidance on the smartphone screen based on the route guidance data. [Fig. 5] is an example of an indoor navigation guidance screen displayed on the screen of the user smartphone (100) in the present invention.

[0062] Through the above process, an indoor navigation service (indoor navigation service) was launched to enable a route guidance requester to move from an initial location to a destination point. Below, we describe a configuration that determines whether a route guidance requester is moving normally along the recommended route by using AI video analysis of CCTV footage, and if the route guidance requester is going the wrong way, modifies and updates the route guidance.

[0063] Step (S220 ~ S250): First, the video analysis processing unit (220) of the indoor LBS server (200) performs AI video analysis corresponding to unique feature information on large-scale CCTV video generated by multiple CCTV cameras (10). In one embodiment, the video analysis processing unit (220) performs AI video analysis-based object detection, object tracking, and real-time re-identification of the same person corresponding to the unique feature information of the route guidance requester obtained in (S180).

[0064] [Fig. 6] is a conceptual diagram of object detection, object tracking, and real-time re-identification of the same person by AI video analysis in the present invention. The video analysis processing unit (220) utilizes a Neural Network Model (NNM) to perform object detection and object grouping to recognize objects of interest in multiple CCTV videos, attribute recognition and search by attribute to extract and utilize features of objects, action recognition to recognize human actions within the video, and re-identification of the same person in multiple CCTV videos. To this end, the video analysis processing unit (220) utilizes the AI ​​video analysis function provided by the video control system (300) or provides its own AI video analysis function.

[0065] As shown in [Fig. 4], a series of frame images obtained from CCTV footage are input into an object detection neural network model to perform an object detection process that extracts multiple objects of interest. Then, as shown in [Fig. 6], these objects of interest are input into an object tracker to perform an object grouping process that forms multiple groups of objects of interest by finding and grouping identical objects in multiple CCTV footages.

[0066] The object tracker identifies identical objects within a series of CCTV video frames related to multiple objects of interest and groups them into distinct groups. Subsequently, it performs object attribute recognition, behavior recognition, and identity re-identification on these groups of objects based on neural networks. These attribute recognition, behavior recognition, and identity re-identification processes are executed through inference using neural network models suited to their respective purposes (such as attribute recognition neural network models, pose estimation neural network models, behavior recognition neural network models, and Re-ID neural network models).

[0067] Object attribute recognition is a process designed to identify the detailed attributes of human and vehicle objects by extracting visual information that can identify the object, such as a person's gender, age, clothing, and personal belongings. The results of object attribute recognition can be effectively utilized when the unique characteristic information of a route guidance requester is based on specific attributes.

[0068] Action recognition is a process designed to identify what actions a human object is currently performing; specifically for video surveillance purposes, it identifies events such as falling or acts of violence. For action recognition, images of a group of human objects are input into a pose estimation neural network model to extract sequences of key points regarding human joint positions. These key point sequences are then input into an action recognition neural network model to classify the actions of each human object. The results of action recognition can be effectively utilized when the unique characteristic information of a route guidance requester is based on specific behaviors.

[0069] Real-time Person Re-IDentification refers to the process of finding a corresponding object (identical object) within the entire CCTV footage when a target is identified in the CCTV footage. In this invention, the location of a route guidance requester can be tracked by performing real-time Person Re-IDentification based on the unique feature information of the route guidance requester.

[0070] By combining the results of AI video analysis (object detection, object tracking, real-time re-identification of the same person) of the video analysis processing unit (220) as described above with indoor map data and camera map data, the movement path recognition unit (250) of the indoor LBS server (200) identifies the actual movement path of the route guidance requester. Then, the route guidance update unit (260) of the indoor LBS server (200) updates the route guidance data in real time by comparing the route guidance data with the actual movement path. If it is determined that the route guidance requester is moving along a path that does not correspond to the recommended path, the route guidance data is updated by notifying the error and, preferably, providing a new path.

[0071] In a preferred embodiment, the route guidance update unit (260) checks whether the actual movement path corresponds sequentially to a series of waypoints for the recommended path, and if it is determined that the sequential correspondence fails as a result of the check, it updates the route guidance data by including movement error data.

[0072] A series of waypoints is set along the recommended route, taking into account CCTV blind spots. If the route guidance requester is not sequentially identified in the CCTV footage of these waypoints by AI video analysis, it is determined that the route guidance requester is moving in the wrong direction and a movement error is reported.

[0073] Then, the route guidance update unit (260) of the indoor LBS server (200) transmits the updated route guidance data to the user smartphone (100).

[0074] When the navigation processing unit (130) of the user's smartphone (100) receives the updated route guidance data from the indoor LBS server (200), it updates the indoor navigation guidance accordingly. Through this, CCTV blind spots can be overcome, and the route guidance requester can verify whether they are moving accurately in a near real-time manner.

[0075] [Fig. 7] is a flowchart of a second embodiment of a method for providing indoor positioning and navigation based on AI video analysis of CCTV video according to the present invention.

[0076] The second embodiment of [Fig. 7] implements the process of identifying the route guidance requester in the first embodiment of [Fig. 3] in a different way. Detailed descriptions of parts that overlap with the first embodiment of [Fig. 3] may be omitted.

[0077] Step (S310): First, the navigation UI unit (110) of the user smartphone (100) transmits a route guidance request for a specific destination point indoors to the indoor LBS server (200) in response to user operation.

[0078] Step (S320 ~ S350): The recognition information processing unit (210) of the indoor LBS server (200) requests a mobile flash to the user smartphone (100) to distinguish and identify the route guidance requester within the CCTV video.

[0079] In response to this, the recognition information providing unit (120) of the user smartphone (100) controls the flash lamp (e.g., camera flash) of the smartphone to flash according to a preset flashing pattern.

[0080] The recognition information processing unit (210) of the indoor LBS server (200) recognizes the flash lamp blinking of the corresponding pattern in the CCTV video generated by multiple CCTV cameras (10). The recognition information processing unit (210) of the indoor LBS server (200) recognizes a person holding a smartphone (100) in which the flash lamp blinking is occurring as a route guidance requester.

[0081] Step (S360): Next, the video analysis processing unit (220) of the indoor LBS server (200) analyzes the CCTV video using AI to obtain unique feature information for distinguishing and identifying the route guidance requester. Preferably, a combination of object attribute recognition results (e.g., gender, age, clothing, belongings, etc.) for the route guidance requester from an attribute recognition neural network model can be set as the unique feature information of the route guidance requester.

[0082] Step (S370 ~ S390): The indoor positioning processing unit (230) of the indoor LBS server (200) identifies the initial location of the route guidance requester based on indoor map data and the recognized flash lamp blinking. Based on the recognition result of the flash lamp blinking, it is possible to determine which CCTV camera (10) the route guidance requester is in. The shooting location or installation location of the corresponding CCTV camera (10) can be set as the initial location of the route guidance requester.

[0083] Then, the route guidance processing unit (240) of the indoor LBS server (200) obtains route guidance data by analyzing the route based on the initial location and destination point of the route guidance requester and the indoor map data of the building.

[0084] Since the path analysis algorithm is a technology used in general navigation services and is not a feature of the present invention, a detailed description of the algorithm itself for deriving the movement path is omitted in this specification. However, considering the field of indoor navigation utilizing CCTV footage, the configuration for acquiring data that guides the movement path (path guidance data) can be improved.

[0085] First, a route analysis algorithm is applied to the route guidance requester's initial location, destination point, and indoor map data to obtain a recommended route from the initial location to the destination point within the building.

[0086] Then, the shooting area (CCTV coverage) of the CCTV camera (10) on the recommended path is identified by referring to the camera map data, and waypoints are set on the CCTV coverage on the recommended path by referring to the indoor map data. Generally, multiple waypoints are set on the recommended path.

[0087] Since the CCTV camera (10) does not cover all areas of the building, there are CCTV blind spots. Due to these CCTV blind spots, it is often difficult to monitor whether the route guidance requester is moving correctly along the recommended route. Considering this problem, the present invention sets waypoints in the CCTV coverage area on the recommended route and checks whether the route guidance requester passes through these waypoints sequentially.

[0088] Based on these points, in a preferred embodiment of the present invention, waypoint data is combined with recommended route data to generate route guidance data.

[0089] Next, the route guidance processing unit (240) of the indoor LBS server (200) transmits the acquired route guidance data to the user smartphone (100).

[0090] When the navigation processing unit (130) of the user's smartphone (100) receives route guidance data from the indoor LBS server (200), it provides indoor navigation guidance such as [Fig. 5] on the smartphone screen based on the route guidance data.

[0091] Through the above process, an indoor navigation service (indoor navigation service) was launched to enable a route guidance requester to move from an initial location to a destination point. Below, we describe a configuration that determines whether a route guidance requester is moving normally along the recommended route by using AI video analysis of CCTV footage, and if the route guidance requester is going the wrong way, modifies and updates the route guidance.

[0092] Step (S400 ~ S430): First, the video analysis processing unit (220) of the indoor LBS server (200) performs AI video analysis corresponding to unique feature information on a large-scale CCTV video generated by a plurality of CCTV cameras (10). In one embodiment, the video analysis processing unit (220) performs object detection, object tracking, and real-time re-identification of the same person based on AI video analysis, as described above with reference to [Fig. 6], corresponding to the unique feature information of the route guidance requester obtained in (S360).

[0093] By combining the results of AI video analysis (object detection, object tracking, real-time re-identification of the same person) of the video analysis processing unit (220) as described above with indoor map data and camera map data, the movement path recognition unit (250) of the indoor LBS server (200) identifies the actual movement path of the route guidance requester. Then, the route guidance update unit (260) of the indoor LBS server (200) updates the route guidance data in real time by comparing the route guidance data with the actual movement path. If it is determined that the route guidance requester is moving along a path that does not correspond to the recommended path, the route guidance data is updated by notifying the error and, preferably, providing a new path.

[0094] In a preferred embodiment, the route guidance update unit (260) checks whether the actual movement path corresponds sequentially to a series of waypoints for the recommended path, and if it is determined that the sequential correspondence fails as a result of the check, it updates the route guidance data by including movement error data.

[0095] A series of waypoints is set along the recommended route, taking into account CCTV blind spots. If the route guidance requester is not sequentially identified in the CCTV footage of these waypoints by AI video analysis, it is determined that the route guidance requester is moving in the wrong direction and a movement error is reported.

[0096] Then, the route guidance update unit (260) of the indoor LBS server (200) transmits the updated route guidance data to the user smartphone (100).

[0097] When the navigation processing unit (130) of the user's smartphone (100) receives the updated route guidance data from the indoor LBS server (200), it updates the indoor navigation guidance accordingly. Through this, CCTV blind spots can be overcome, and the route guidance requester can verify whether they are moving accurately in a near real-time manner.

[0098] Meanwhile, the present invention can be implemented in the form of computer-readable code on a computer-readable non-volatile recording medium. Various types of storage devices exist as such non-volatile recording media, such as hard disks, SSDs, CD-ROMs, NAS, magnetic tapes, web disks, and cloud disks. Additionally, the present invention may be implemented in the form of a computer program stored on a medium to execute a specific procedure in combination with hardware. Explanation of the symbols

[0099] 10: CCTV camera 100 : User smartphone 110 : Navigation UI section 120 : Recognition information providing unit 130 : Navigation processing unit 200 : Indoor LBS Server 201 : Indoor Map Database 202 : Camera Database 210 : Recognition Information Processing Unit 220 : Image analysis processing unit 230 : Indoor positioning processing unit 240 : Route guidance processing unit 250 : Movement path recognition unit 260 : Route guidance update section 300 : Video Control System

Claims

Claim 1 delete Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 A method for providing indoor positioning and navigation based on AI video analysis of CCTV footage, which provides indoor positioning and navigation services by analyzing in real-time by AI footage generated from multiple CCTV cameras distributed indoors in a specific space, comprising: a step in which a navigation UI unit (110) of a user smartphone (100) transmits a route guidance request for a specific destination point indoors to an indoor LBS server (200) in response to user operation; a step in which a recognition information processing unit (210) of the indoor LBS server (200) transmits a mobile flash flashing request to the user smartphone (100) to distinguish and identify the route guidance requester within the CCTV footage; a step in which a recognition information providing unit (120) of the user smartphone (100) controls the flashing of the smartphone's flash lamp according to a preset flashing pattern in response to the mobile flash flashing request; and a step in which the recognition information processing unit (210) of the indoor LBS server (200) recognizes the flashing of the flash lamp of the corresponding flashing pattern in the CCTV footage. A step in which the recognition information processing unit (210) of the indoor LBS server (200) recognizes a person holding a smartphone (100) in which the recognized flash lamp blinking is occurring in the CCTV video as a route guidance requester; a step in which the video analysis processing unit (220) of the indoor LBS server (200) obtains unique feature information for distinguishing and identifying the route guidance requester in the CCTV video; a step in which the indoor positioning processing unit (230) of the indoor LBS server (200) identifies the initial location of the route guidance requester based on the indoor map data of the building and the shooting location or installation location of the CCTV camera (10) that captured the recognized flash lamp blinking; a step in which the route guidance processing unit (240) of the indoor LBS server (200) obtains route guidance data by route analysis based on the initial location, the destination point, and the indoor map data of the building.A step in which the route guidance processing unit (240) of the indoor LBS server (200) transmits the acquired route guidance data to the user smartphone (100); a step in which the navigation processing unit (130) of the user smartphone (100) provides indoor navigation guidance based on the route guidance data; a step in which the video analysis processing unit (220) of the indoor LBS server (200) performs AI video analysis corresponding to the unique feature information on a plurality of CCTV videos; a step in which the movement path recognition unit (250) of the indoor LBS server (200) identifies the actual movement path of the route guidance requester based on indoor map data and the AI ​​video analysis result corresponding to the unique feature information; a step in which the route guidance update unit (260) of the indoor LBS server (200) updates the route guidance data in real time by comparing the route guidance data with the actual movement path; a step in which the route guidance update unit (260) of the indoor LBS server (200) transmits the updated route guidance data to the user smartphone (100). A method for providing indoor positioning and navigation based on AI video analysis for CCTV footage, comprising the step of the navigation processing unit (130) of the user's smartphone (100) updating indoor navigation guidance based on the updated route guidance data. Claim 6 A method for providing indoor positioning and navigation based on AI video analysis for CCTV footage, characterized in that, in claim 5, the AI ​​video analysis corresponding to the unique feature information comprises object detection, object tracking, and real-time re-identification of the same person based on AI video analysis corresponding to the unique feature information of the route guidance requester. Claim 7 The method for providing indoor positioning and navigation based on AI video analysis for CCTV footage according to claim 5, wherein the step of acquiring route guidance data comprises: acquiring a recommended route from the initial location to the destination point by route analysis based on the initial location, the destination point, and indoor map data; identifying CCTV coverage existing on the recommended route; setting waypoints on the CCTV coverage existing on the recommended route; and generating route guidance data by combining the data of the recommended route and the data of the waypoints; and wherein the step of updating the route guidance data in real time comprises: checking whether the actual movement path sequentially corresponds to a series of waypoints for the recommended route; and updating the route guidance data by including movement error data if, as a result of the check, it is determined that sequential correspondence fails. Claim 8 A computer program stored on a computer-readable storage medium to execute a method for providing indoor positioning and navigation based on AI video analysis of CCTV footage according to any one of claims 5 to 7 on a computer.

Images