Device, system, and method of articles management using image code

Abstract

A method for managing items using image codes is disclosed. The method for managing items using image codes comprises the steps of: setting a major installation location of items as a reference point and determining absolute location coordinate information, and then generating and attaching a reference point image code that includes one or more of location information such as the name of the installation location, floor number, room number, reference point identifier, reference point absolute coordinate, reference point photo, and precautions; attaching an item image code to an item that includes one or more of information such as item name, item identifier, initial installation location, initial installation date and time, manager name, manufacturer, model name, item photo, and remarks; a manager carrying a terminal and finding a surrounding reference point image code at a location close to the item and taking a photo; analyzing the reference point image code to extract absolute location coordinate information of the reference point; estimating the relative position between the reference point image code and the terminal using computer vision technology, and calculating the absolute coordinates of the terminal and the item by summing the relative position information and the absolute location coordinate information; and a manager inputting or updating the absolute coordinates of the item into an item management database.

Description

Technology Field

[0001] The following embodiments relate to a product management service technology that recognizes, stores, and updates information such as the absolute coordinate location and status of a product using an image code and an administrator terminal, and utilizes the same. Background Technology

[0003] Effective management is crucial when various items are stored in large spaces such as logistics warehouses, factories, workplaces, or stores. Generally, items are stored in various types of racks designed for segregation. A list is compiled containing identifiers for each space, along with the identifiers and management information of the items within, and item identifier labels are attached to the items to perform management tasks such as searching and updating as needed. However, if the number of items to be managed is high or the size of the segregated spaces is large, manual management becomes difficult for managers; therefore, RFID or barcodes are attached to the items, and the identifiers and management information are recognized using scanners. In large-scale logistics warehouses, mobile robots are sometimes utilized with modular racks, attached identifiers, and image recognition devices to locate and move items.

[0004] On the other hand, in cases where the management space is too large to be divided into modular racks of limited size, or where the rack configuration is inconsistent, it is difficult to estimate the location of items using the racks alone. Since separate location information is required, additional technologies are used to recognize item locations. Furthermore, with the rise of digital twin services, mapping the precise location of items to a location in virtual space has become a critical technological element. Prior art literature

[0006] KR1020140022308 Real-time product management system using QR codes KR102139620 Logistics management method and device using QR codes CN112045669A Asset management method and system US9805331B2 Smartphone-based asset management system US20200285977A1 ​​Asset tracking system and method US20220292451A1 Asset tracking, monitoring, and management system and method of use KR101356192 Method and system for determining smartphone position orientation based on video matching KR101475078 System and method for providing location information services using mobile codes KR102769386 Method and system for registering IoT devices in outdoor environments KR102167494 Method for providing storage organization services using cloud-based QR codes and storage boxes The problem to be solved

[0007] Existing item management systems lacked economic efficiency and practicality because the location information was inaccurate when estimating the absolute location coordinates of items, or the installation costs of related devices were high.

[0008] The above-mentioned patent document 1 relates to a system for managing products by attaching a QR code containing product information to a product, recognizing it on a mobile terminal, and transmitting this information to a server, but it lacks product location recognition and location management functions.

[0009] The above-mentioned patent document 2 relates to a system that authenticates genuine products by attaching a first QR code to a product and collects purchase location information, purchase date and time, subscriber information, etc., and collects movement information of the product at the logistics center node level using a second QR code attached to the product, but lacks detailed location recognition and location management functions for the product.

[0010] The above-mentioned patent document 3 is intended to inspect computer assets within a machine room and to recognize and manage the location of computers within cabinets by moving between cabinets containing multiple computers on a mobile robot, and by the cabinets identifying RFID tags attached to the computers and reporting this information to the robot. In this case, the function is limited to recognizing computers located at fixed positions inside the cabinets.

[0011] The above patent document 4 relates to a method for managing assets by taking a photo with a smartphone to recognize the location of the asset and storing this information in a database as one embodiment. There is a burden of having to install a camera at every location where an asset is located and continuously recognize the location of the asset using image processing technology, and there is a limitation in that it is difficult to distinguish when the location or asset photo is similar.

[0012] The above-mentioned patent document 5 relates to a method for attaching a tracking device to an asset and estimating the location of the asset within a building. The tracking device receives a signal from a communication network, distinguishes a network identifier, measures the strength of the received signal, transmits this to a backend server of a tracking application, and estimates the location of the asset. This method is a type of radio fingerprint estimation method, which requires measuring the signal strength of radio reception for all locations within the building in advance, and imposes a significant burden of continuously measuring signals and updating the database in response to changes in the radio environment. Additionally, there is a problem in that the cost of the tracking device attached to each asset is considerable.

[0013] In the above patent document 6, as an embodiment, a tracking and monitoring unit is attached to a mobile container, which includes various sensors, communication devices, GPS, QR codes, etc. Through this, status information of the mobile container is collected and provided to the user through an application connection platform. Since the QR code is used to recognize the container, it is not related to location recognition technology, is limited to the management of large items in container units, and can only be used outdoors.

[0014] The above patent document 7 relates to a method for estimating the location of a smartphone using an existing georeferencing reference image DB and aerial triangulation technology. However, there is a difficulty in constructing a large amount of reference image DB and a practical difficulty in continuously updating the image DB as the environment changes.

[0015] The above patent document 8 obtains location information by scanning a mobile code with a mobile terminal and displays it on a map, but it has a limitation in that it can only recognize the location at the point where the mobile code is attached, so it cannot recognize the location at other places.

[0016] The above patent document 9 relates to a method for registering and managing an IoT device using a QR code attached to the IoT device and GPS coordinates of a mobile terminal, but there is a problem that it cannot be applied because there is an error in the GPS estimated coordinates even in outdoor cases and GPS signals cannot be received indoors.

[0017] The above patent document 10 relates to a method of inputting and storing a QR code, an image of a stored item, a name, etc., of a storage box using a user terminal, and outputting an image of a stored item, etc., when the QR code is scanned by the user terminal, but there is no technical content for estimating the location of the storage box or the stored item.

[0018] The aforementioned patent documents are limited to product management service systems that lack a location verification function, and have problems such as being able to estimate the location information of a product only in limited cases or having inaccurate location information. means of solving the problem

[0020] A method for managing items using image codes according to one embodiment comprises the steps of: setting a main installation location of items as a reference point and determining absolute location coordinate information, and then generating and attaching a reference point image code that includes one or more of location information such as the name of the installation location, floor number, room number, reference point identifier, reference point absolute coordinate, reference point photo, and precautions; attaching an item image code to an item that includes one or more of information such as item name, item identifier, initial installation location, initial installation date and time, manager name, manufacturer, model name, item photo, and remarks; a manager carrying a terminal and finding a surrounding reference point image code at a location close to the item and taking a photo; analyzing the reference point image code to extract absolute location coordinate information of the reference point; estimating the relative position between the reference point image code and the terminal using computer vision technology, and calculating the absolute coordinates of the terminal and the item by summing the relative position information and the absolute location coordinate information; and the manager inputting or updating the absolute coordinates of the item into an item management database.

[0021] In the item management method using image codes according to the above features, the manager further includes the step of displaying a map on a terminal or an office computer and marking or searching for the installation location, reference point, and the location of the items.

[0022] In a method for managing items using image codes according to the above features, the method further includes the step of taking a photograph so that the item image code and the reference point image code are captured simultaneously, estimating the relative distance and rotation angle of the two image codes, and then estimating the absolute position of the item by adding the absolute coordinate information extracted from the reference point image code.

[0023] In a method for managing items using image codes according to the above features, the method further includes the step of first taking a picture of one of the image codes of an item or a reference point, rotating or moving the camera, and then taking a picture of another image code to estimate the relative distance and rotation angle between the two image codes, continuously taking pictures while the camera is rotating or moving or processing inertial sensor data embedded in the camera to estimate the camera's movement position information, and summing this information to calculate the relative position and rotation angle between the item image code and the reference point image code.

[0024] In a product management method using an image code according to the above features, when a photo of a product is taken with a terminal and terminal-related information is input, the photo and information are registered in a database, and the method further includes the step of generating and outputting a product image code that includes a portion of this information.

[0025] In a method for managing items using an image code according to the above features, when an administrator takes a picture of an item image code with a terminal, the method further includes a step of decoding the item image code from the terminal or a service server to search for the content included in the item image code or additional content stored in the service server.

[0026] In the item management method using an image code according to the above features, the coordinates of a reference point may be set by an administrator specifying a location on a map of a terminal or computer, or the method further includes the step of collecting the coordinate information using one or more of GPS, location beacons, radio signals, optical signals, acoustic signals, geomagnetic signals, distance measuring devices, drones, and mobile robots.

[0027] In a method for managing goods using an image code according to the above features, when a terminal connects to a wireless network or a mobile communication network, it further includes a step of connecting to the communication network using a reference point image code or a goods image code as authentication information.

[0028] In a method for managing goods using an image code according to the above features, the image code displayed or attached to the goods may be displayed or attached by a user or manager, or may be displayed or attached at the time of production or shipment of the goods, further comprising the step of

[0029] In a method for managing goods using an image code according to the above features, the method further includes the step of attaching a temporary goods identifier to a goods or marking it on the surface of the goods, taking a photograph of the goods to include the temporary goods identifier, and automatically recognizing the temporary goods identifier using image processing technology at a terminal or service server and linking it with a database.

[0030] In a method for managing goods using an image code according to the above features, the method further includes the step of displaying a temporary reference point identifier at a reference point, entering management information into a database, and then generating a reference point image code and attaching it to the reference point.

[0031] In a method for managing goods using image codes according to the above features, the method further includes the step of taking a photograph that includes both the goods and the goods image code after the goods image code is attached, and applying image processing and machine learning techniques to the photograph to store the morphological characteristics of the goods and the goods identifier together in a database.

[0032] In a method for managing goods using an image code according to the above features, instead of an image code displayed or attached to the goods, the method further includes a step in which a manager terminal verifies the identifier of the goods using a near-field communication module, such as NFC (Near Field Communications) or RFID, attached to or embedded in the goods.

[0033] In a method for managing items using image codes according to the above features, the method further includes the step of photographing an item image code of another item whose coordinate information is known in a database using a manager terminal, recognizing the identifier of the item and obtaining coordinate information, and then applying computer vision technology to the photograph to estimate the relative position and summing the coordinate information to estimate the coordinates of the terminal and other items near the terminal.

[0034] In a method for managing items using image codes according to the above features, the method further includes the step of a manager terminal or mobile robot moving and capturing an image code of an item or reference point, recognizing an object or reference point identifier in the image, searching a database with this identifier to collect coordinate information of the item or reference point, and applying visual odometry or SLAM technology to eliminate position coordinate estimation errors of the item and precisely estimate the position coordinates.

[0035] In a method for managing items using an image code according to the above features, when the subject of management is a person or animal or plant, the face or appearance of the person or animal or plant becomes an identifier of the subject of management, and further includes the step of photographing a reference point image code with a terminal at a location close to the person or animal or plant and estimating the location coordinates of the terminal and the person or animal or plant using computer vision technology.

[0036] In a method for managing items using an image code according to the above features, in order to estimate relative position coordinates by applying computer vision technology to an image code of an item or reference point captured by a terminal, the image code must have multiple mark indicators, the shape and size of the mark indicators and the distance between the marks must be predefined, and the method further includes the step of including a unique identifier of the item or reference point in text form.

[0037] In a method for managing items using an image code according to the above features, in addition to attaching the image code to an object or reference point, the method further includes the step of displaying the image code on the surface of the object or reference point using a display device.

[0038] In one embodiment, an item management system using image codes comprises: a reference point image code including one or more of location information such as the name of the installation location, floor number, room number, reference point identifier, reference point absolute coordinates, reference point photo, and precautions; an item image code including one or more of information such as item name, item identifier, initial installation location, initial installation date and time, manager name, item photo, and remarks; a terminal carried by a manager who stands next to an item to locate surrounding reference point image codes and take a photo, analyzes the reference point image code to extract absolute location coordinate information of the reference point, estimates the relative position between the reference point image code and the terminal using computer vision technology, calculates the absolute coordinates of the terminal and the item by summing the relative location information and the absolute location coordinate information, and inputs or updates the absolute coordinates of the item into an item management database; and a service server that operates a database to which the manager terminal connects via a communication network to search for or modify information related to the reference point or item. Effects of the invention

[0040] According to one embodiment of the present invention, through an item management method using image codes, there is a highly practical effect of recognizing the precise absolute location of items within an installation location and searching for and updating item management information using very inexpensive image codes and a standard manager's smartphone. In particular, when items are installed or stored at arbitrary locations within an installation site rather than being stored in small, modular racks, there is an effect of economically performing the task of setting a reference point, estimating the location of the items, and managing them.

[0041] In addition, since real-time location management of items is crucial for establishing a digital twin system, this invention can be a very economical and practical solution. Brief explanation of the drawing

[0043] FIG. 1 is a configuration diagram of an item management system using an image code according to one embodiment. FIG. 2 is a flowchart of an item management service using an image code according to one embodiment. FIG. 3 is an item management device using an image code according to one embodiment. FIG. 4 is an item management service server using an image code according to one embodiment. FIG. 5 is a new image code for an item management service according to one embodiment. Specific details for implementing the invention

[0044] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions that fall within the technical concept described by the embodiments.

[0045] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0046] When it is mentioned that a component is "connected" to another component, it should be understood that it may be directly connected to or connected to that other component, but there may also be other components in between.

[0047] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0048] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0049] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0050] An embodiment of the item management system using an image code according to the present invention is as shown in FIG. 1. FIG. 1 is a configuration diagram of an item management service system using an image code according to an embodiment. An installation location (100) where a large number of items are installed is a type of logistics warehouse, factory, facility, mobile object, store, etc., and various pillars, walls, racks, fixed objects, mobile objects, etc. inside the installation location serve as reference points. The manager obtains the absolute coordinates of these reference points from a map or by a separate method, includes them in a reference point image code (102) to generate the code, and attaches it to the corresponding reference point. This reference point image code includes management information such as absolute coordinate information, a reference point identifier, a warehouse name, a pillar identifier, and a floor number. Alternatively, if only the reference point identifier is included, other management information can be obtained from the service server (104) by searching for the reference point identifier on the manager terminal (103).

[0051] In addition, an item image code (101) is attached to various items in the warehouse. This item image code may include management information such as an item identifier and an item name, or the item image code may include only an item identifier, and other management information including the location coordinates of the item may be obtained from the administrator terminal through the service server (104).

[0052] The administrator terminal (103) captures and decodes a reference point image code at a location close to an arbitrary item to collect absolute coordinate information of the reference point, and applies computer vision technology to the photo to estimate the absolute position coordinates of the administrator terminal. This technology calculates the shape information of the mark of the image code known in advance and the feature point information of the corresponding mark extracted from the photo to obtain the relative position between the image code and the camera and the camera rotation angle. The above coordinates correspond to the absolute position coordinates of the item.

[0053] The administrator updates the database of the relevant item on the service server by inputting the absolute coordinate information of the item estimated through the terminal. The administrator or user can check the installation location, reference point, or the location of the items on a map on the terminal or computer. The administrator or user can select and search for the installation location, reference point, or item by touching it on the map on the terminal. The same applies to the administrator's computer. In one embodiment, the administrator has the authority to update the database of the service server, while the user may only search the database of the service server with permission.

[0054] FIG. 2 is a flowchart of an item management service using an image code according to one embodiment. Several key locations at the item installation site are set as reference points, and after determining absolute coordinate information, a reference point image code is generated and attached. When absolute coordinates are included in the image code, there is an advantage in that the accurate absolute coordinate information of the item can be recognized without the administrator terminal establishing a communication connection with the service server. In one embodiment based on the Earth coordinate system, absolute coordinates correspond to latitude and longitude and are displayed on a smartphone or computer screen, such as Google Maps. Relative coordinates refer to the 3D or 2D relative position and rotation angle of a positioning target relative to a reference point.

[0055] If relative coordinates between multiple reference points at the installation site are established and included in the reference point image code, the location of the item can also be recognized relatively. However, in this case, the location cannot be displayed on a map on any device, such as Google Maps. If a separate installation site map is constructed, the relative location of the item can be displayed.

[0056] An image code containing an item identifier is attached to the item. On the service server, an administrator inputs management information, including the location of the item, into the database. The administrator carries an administrator terminal—specifically, a smartphone in one embodiment—and locates a nearby reference point image code from a position close to the item. By capturing the reference point image code with the smartphone and using computer vision technology to estimate the relative position between the reference point image code and the smartphone, the administrator decodes and extracts the absolute position coordinate information contained in the reference point image code, and calculates the absolute coordinates of the smartphone by summing the two pieces of information. These coordinates correspond to the absolute coordinates of the item. The administrator accesses the service server to input or update this absolute coordinate information into the database. Additionally, the administrator can access the service server to search for or modify the item's management information. Furthermore, if necessary, the administrator can display a map on their smartphone or an office computer to search for the installation location, such as a logistics warehouse, and the location of items within the warehouse.

[0057] In Figure 2, if the manager does not see the reference point image code at the item's location, the manager can estimate the absolute position of the item using an additional method. That is, the manager moves to a location where both the item's image code and the reference point's image code are visible, and then takes a picture so that both image codes are captured simultaneously. After estimating the relative distance and rotation angle of the two image codes by applying computer vision technology, the manager extracts absolute coordinate information from the reference point's image code and can estimate the absolute position of the item by summing these two pieces of information.

[0058] Additionally, if the image codes of the item and the reference point cannot be simultaneously included in the photograph, the relative distance and rotation angle between the two image codes can be estimated by first photographing one of them, rotating or moving the camera, and then photographing the other image code. In this case, the camera's movement position information must be estimated by continuously taking photographs while the camera is rotating or moving, or by processing data from the inertial sensor embedded in the camera. That is, the relative position and rotation angle between the camera and the first captured image code are estimated, the camera's movement and rotation information is estimated, and finally, the relative position and rotation angle between the camera and the other captured image code are estimated; these three pieces of information are then summed to calculate the relative position and rotation angle between the item image code and the reference point image code.

[0059] FIG. 3 is an item management device (30) using an image code according to one embodiment. One embodiment corresponds to a smartphone of an administrator or user. Under the control of a control unit (31), a camera (32) captures an image code of an item or a reference point, and the captured photo or management information entered by the administrator is stored in a storage unit (33). The location estimation unit (34) obtains absolute coordinates by decoding the image code included in the captured photo, or estimates relative coordinates by applying computer vision technology. The display unit (35) displays the captured photo or item management information searched from a service server. The communication unit (36) is for connecting to a service server, and the power unit (37) supplies power to the terminal.

[0060] FIG. 4 is an item management service server (40) using an image code according to one embodiment. Under the control of the control unit (41), the user management unit (42) performs the role of authenticating and managing an administrator or user who connects through the communication unit (46). The storage unit (43) and the item DB (44) store installation locations, reference points, item management information, etc., and perform the role of a database, allowing the administrator to search for or modify information. Optionally, the calculation of the relative coordinates of the item may be performed by the location estimation unit (45) of the service server (40) rather than the administrator terminal.

[0061] The present invention relates to a device, system, and service for storing and retrieving management data, such as the location or history of an item, using an image code such as a QR code or a barcode and a user or manager terminal such as a smartphone.

[0062] An item image code, such as a QR code, barcode, or a new type of image code, is attached to the items under management. This item image code includes one or more pieces of information such as item name, item identifier, initial installation location, initial installation date and time, manager name, manufacturer, model name, item photo, and remarks. The item identifier is set by the manager or automatically generated by the manager terminal or service server. For example, if a photo of an item is taken with a smartphone and the above information is entered, the photo and information are registered on the service server, and an item image code containing a portion of this information can be printed using a printer. This item image code is attached to the corresponding item. Alternatively, as an example, it can be displayed at a reference point using a display device such as a laser indicator.

[0063] In one embodiment, when the status of the item changes, if an administrator takes a photo of the item image code with a smartphone, the smartphone or service server decodes the item image code, allowing the administrator to search not only for the content included in the item image code but also for additional content stored on the service server. Additionally, the administrator can access the service server via a smartphone or computer to modify the item information.

[0064] In one embodiment, a reference point image code is attached to any reference point at an installation location where the item is installed, including one or more of the following location information: the name of the installation location, coordinates, floor number, room number, reference point identifier, absolute coordinates of the reference point, a photo of the reference point, and precautions. The absolute coordinates of the reference point can be collected by an administrator by specifying the location of the reference point on a map on a smartphone or computer. The method for collecting the relative coordinates of the reference point is the same. Alternatively, the coordinate information of the reference point may be collected using GPS, location beacons, radio signals, optical signals, acoustic signals, geomagnetic signals, distance measuring devices, drones, mobile robots, etc. The coordinates are absolute coordinates of the Earth or relative coordinates based on a specific reference point.

[0065] The administrator can input information regarding the above reference point image code using a smartphone or computer, register it with the service server, and print it out. The printed reference point image code is attached to the reference point at the corresponding installation location.

[0066] In another embodiment, each image code can be displayed on the surface of an article or reference point using a display device such as a laser indicator.

[0067] The manager uses a smartphone to capture and decode the image code of a nearby reference point at the item's location to extract the reference point's coordinates. After applying computer vision technology to the reference point image code to estimate the relative position between the reference point code and the item's location, the two pieces of information are combined to calculate the item's location coordinates. In particular, if the reference point image code includes absolute coordinate information, the Earth's absolute coordinates are calculated.

[0068] By utilizing information registered on the service server, the installation location where the above reference point is registered and the registered item information can be displayed on a map on the administrator's or user's smartphone.

[0069] Using the present invention, services such as product status management, lifespan management, replacement orders, repair requests, and remote SW updates can be provided.

[0070] The service server according to the present invention can be linked with the databases of product manufacturers, sales companies, rental companies, etc.

[0071] For the function of the present invention, when connecting to wireless communication such as a wireless LAN or a mobile communication network, a reference point image code or an image code displayed on an item can be used as authentication information to be linked to the communication network.

[0072] Image codes marked or attached to the product may be marked or attached by a user or manager, or may be marked or attached at the time of production or shipment of the product.

[0073] To facilitate the generation of item image codes and the establishment of a database for item management information, a temporary item identifier sticker (e.g., a temporarily assigned serial number) is initially attached to the item or marked on the surface, and then removed when the item image code is subsequently attached. Additionally, when photographing the item, the temporary item identifier is captured to be included, and the terminal or service server can automatically recognize the temporary item identifier using image processing technology. Since both the item and the temporary item identifier are included in the same photograph, the terminal or service server can simultaneously recognize and link the item and the temporary item identifier using image processing technology. Information regarding the item can be entered into the terminal while the photograph is being taken on-site, or the temporary item identifier can be matched with the identifier assigned to the item at a separate location to link the item's management information stored in the database. In cases where the location for generating the item image code differs from the location where the item is installed, the necessary item information is entered into the database using the temporary item identifier, and after generating the item image code, the item image code is attached while comparing it with the temporary item identifier. This is an efficient method when there are many similar items subject to management.

[0074] The method of displaying a temporary reference point identifier at a reference point, entering management information into the service server's database, generating a reference point image code, and attaching it to the reference point can also be applied in the same way as the above-mentioned temporary item identifier method.

[0075] After an item image code is attached, a photograph can be taken that includes both the item and the item image code, and registered in the service server's database. By applying image processing and machine learning technologies to this photograph, the item's morphological characteristics and identifiers can be compiled into a database that can be utilized for purposes such as item classification.

[0076] Instead of image codes marked or attached to the item, a manager terminal can verify the item's identifier using a near-field communication module, such as NFC (Near Field Communications) or RFID, attached to or embedded in the item. Even in this case, the item's location is estimated by photographing the reference point image code attached to the reference point with the manager terminal and applying computer vision technology.

[0077] Instead of attaching a reference point image code to a reference point at the installation location, if a location beacon transmitting absolute location coordinates or a unique code is installed, a module for estimating absolute or relative coordinates can be attached to the item. Alternatively, a smartphone located near the item estimates absolute or relative coordinates using the location beacon signal and utilizes them as the item's location coordinates; the smartphone then captures an image of the item's image code to verify the item's identifier and searches for or modifies the item's information by linking the estimated location coordinate information. This process may also be linked with a service server's database.

[0078] After recognizing the identifier of a specific item using the item image code of another item, the service server obtains the item's coordinate information, and based on this, the coordinates of the administrator's smartphone can be estimated. Consequently, the coordinates of items close to the administrator's smartphone can also be estimated. In other words, since the absolute Earth coordinates cannot be determined when there is no reference point image code near the administrator, this method utilizes the absolute coordinate information of other nearby items.

[0079] The administrator uses a smartphone camera to capture and decode the item image code of another item visible near the item whose absolute coordinates are to be estimated, thereby determining the item's absolute coordinates through a service server. Then, computer vision technology is applied to the image code of the item to estimate the relative coordinates of the administrator's smartphone relative to the item, and these two pieces of information are combined to calculate the absolute coordinates of the administrator's terminal. These absolute coordinates represent the absolute coordinates of the item to be estimated. In this way, even if the reference point image code is not visible, the absolute coordinates of items can be estimated continuously using a cascade method.

[0080] In another embodiment, a dedicated terminal for item management by the manager may be used instead of the manager's smartphone.

[0081] In addition, as another embodiment, a mobile robot can perform the role of the manager by identifying the location of an item or reference point instead of a smartphone and capturing the item image code. In this case, the mobile robot can accurately estimate the absolute or relative position of the item by continuously capturing video of the installation location as well as the image code of the reference point and applying visual odometry or SLAM (Simultaneous Location And Mapping) technology. In this case as well, the absolute position coordinates collected from the reference point image code are used to eliminate the accumulated error of the above technology.

[0082] In another embodiment, the subject of management may be a person or animal or plant rather than an item. In such cases, instead of an item image code, a person's face or appearance serves as the identifier for the subject of management. The same applies to animals and plants. A reference point image code is photographed using a smartphone to estimate the absolute or relative position coordinates of a person or animal located near the smartphone, and this information is included in the management information. Other operating principles and methods are the same as those used when using the item image code. In particular, since people or animals have the attribute of movement, a device with a built-in camera can be attached to the person or animal to track their movement trajectory. In this case, coordinate information of other people or animals in the vicinity is collected, and computer vision technology is applied to the captured photograph to continuously estimate the coordinates of the device with the built-in camera; if the device is attached to the person or animal, the coordinates will match those of the device.

[0083] The image code of the present invention is, in one embodiment, a conventional QR code or barcode. In the case of a QR code, it can contain a large amount of text information and, since it is a two-dimensional code, applying computer vision technology using square mark information at the corners of the code has the advantage of being able to estimate the relative distance and rotation angle between the QR code and the camera that took the QR code photo. However, in the case of a QR code, there is the inconvenience of having to take a photo and analyze it using a decoding program of a separate device in order to extract text information. That is, an administrator or user cannot extract text information visually.

[0084] Therefore, in the present invention, a new image code can be used as an object image code or a reference point image code. That is, the purpose is to enable an administrator to visually recognize text information to determine approximate location information, and to allow the relative position with respect to the camera to be estimated by taking a picture of the corresponding image code with a camera.

[0085] FIG. 5 is a new image code (50) for use in an item management service according to one embodiment. In order to estimate the relative position by applying computer vision technology to an image captured by a camera, there must be a plurality of predefined and known mark indicators (51). The shape, size, and distance of the mark indicators must be predefined. In addition, they must not be perfectly symmetrical in order to distinguish the attachment direction of the image code.

[0086] The new image code (50) includes text (52) or image information. In the example of FIG. 5, text (52) information is included so that an administrator can visually recognize the information. This text is a type of reference point identifier that recognizes an image captured by a camera to extract text information, and is used to search and link with a database stored in an existing terminal or service server to search for or update location coordinates and management information of the reference point. That is, the administrator can visually recognize and identify the approximate location of the reference point, and the terminal or service server can also recognize and use it. For example, "2A03" is defined in advance as a reference point identifier according to a certain rule, and the administrator can recognize it as the 3rd rack of Module A on the 2nd floor of the logistics warehouse.

[0087] The absolute coordinate location information and other management information of the corresponding reference point identifier are stored in the administrator terminal or the service server's database. Furthermore, since the approximate location of the administrator terminal or reference point can be recognized by the administrator inputting it into the terminal or using GPS, additional identifiers are not required to distinguish logistics warehouses. In other words, once the administrator designates a logistics warehouse, reference points within the warehouse can be distinguished using text information within the image code. Since there is always a possibility of recognition errors in image recognition using computer vision, an image code that is as large and simple as possible is desirable.

[0088] In addition, since image codes are not aesthetically pleasing, it is best to attach them in a form that is as small and simple as possible. The image code can also be embedded in an aesthetically pleasing picture. In this case, if the position or size of the mark (51) and the position or size of the text (52) information are defined and standardized in advance, the possibility of errors in image recognition is reduced, and the aesthetics are not compromised.

[0089] The new image code (50) can be used as an item image code. In this case, there is an advantage that the manager can visually check information about the item, but the item location coordinates and management information must be obtained by searching the database of the terminal or service server using text (52).

[0090] Additionally, image codes can be printed on paper and attached, or they can be displayed on a display device. In this case, the content of the image code can be changed dynamically, or the image code can be displayed only when necessary, such as at regular time intervals, upon the occurrence of events like an administrator entering, or when an administrator instructs its display. Furthermore, if the image code is attached to a moving object, such as a mobile robot, and includes the absolute position coordinates of the object, the image code displayed on the display device may be changed.

[0091] A method according to one embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or those known and available to those skilled in the art of computer software. Embodiments of a computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. In particular, a smartphone may be one of a tablet, a laptop, a computer, a PDA, a dedicated terminal, etc.

[0092] Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0093] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0094] Although the embodiments described above have been explained with reference to limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, appropriate results can be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0095] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[0097] 100 emplacement 101 Item image code 102 Reference point image code 103 Administrator terminal 104 Service Server 50 Image code 51 mark 52 Text

Claims

Claim 1 A method for managing items using image codes comprises the steps of: setting a major installation location of items as a reference point and determining absolute location coordinate information, and then generating and attaching a reference point image code that includes one or more of location information such as the name of the installation location, floor number, room number, reference point identifier, reference point absolute coordinate, reference point photo, and precautions; attaching an item image code to an item that includes one or more of information such as item name, item identifier, initial installation location, initial installation date and time, manager name, manufacturer, model name, item photo, and remarks; a manager carrying a terminal and finding a surrounding reference point image code at a location close to the item and taking a photo; analyzing the reference point image code to extract absolute location coordinate information of the reference point; estimating the relative position between the reference point image code and the terminal using computer vision technology, and calculating the absolute coordinates of the terminal and the item by summing the relative position information and the absolute location coordinate information; and a manager inputting or updating the absolute coordinates of the item into an item management database. Claim 2 A method for managing items using image codes according to claim 1, further comprising the step of a manager displaying a map on a terminal or office computer and displaying or searching for the installation location, reference point, and location of items. Claim 3 A method for managing an item using an image code according to claim 1, further comprising the step of taking a photograph so that an item image code and a reference point image code are captured simultaneously, estimating the relative distance and rotation angle of the two image codes, and then estimating the absolute position of the item by adding the absolute coordinate information extracted from the reference point image code. Claim 4 A method for managing an item using an image code according to claim 1, further comprising the step of first taking a picture of one of the image codes of an item or a reference point, rotating or moving the camera, and then taking a picture of another image code to estimate the relative distance and rotation angle between the two image codes, continuously taking pictures while the camera is rotating or moving or processing inertial sensor data embedded in the camera to estimate the camera's movement position information, and summing this information to calculate the relative position and rotation angle between the item image code and the reference point image code. Claim 5 A method for managing goods using image codes according to claim 1, further comprising the step of taking a photo of a goods with a terminal and inputting terminal-related information, registering the photo and information in a database, and generating and outputting a goods image code including a part of the information. Claim 6 A method for managing goods using an image code according to claim 1, further comprising the step of decoding the goods image code from the terminal or service server when a manager takes a picture of the goods image code with a terminal, and searching for the content included in the goods image code or additional content stored in the service server. Claim 7 A method for managing goods using an image code according to claim 1, wherein the coordinates of a reference point may be set by an administrator specifying a location on a map of a terminal or computer, or further comprising the step of collecting the coordinate information using one or more of GPS, location beacons, radio signals, optical signals, acoustic signals, geomagnetic signals, distance measuring devices, drones, and mobile robots. Claim 8 A method for managing goods using an image code according to claim 1, further comprising the step of linking to a communication network using a reference point image code or a goods image code as authentication information when the terminal connects to a wireless LAN or a mobile communication network wirelessly. Claim 9 A method for managing goods using an image code according to claim 1, further comprising the step of the image code being displayed or attached to the goods being displayed or attached by a user or manager, or being displayed or attached at the time of production or shipment of the goods. Claim 10 A method for managing goods using an image code according to claim 1, further comprising the step of attaching a temporary goods identifier to a goods or marking the surface of a goods, taking a photograph of the goods to include the temporary goods identifier, and automatically recognizing the temporary goods identifier using image processing technology at a terminal or service server and linking it with a database. Claim 11 A method for managing goods using an image code according to claim 1, further comprising the step of marking a temporary reference point identifier at a reference point, entering management information into a database, and then generating a reference point image code and attaching it to the reference point. Claim 12 A method for managing goods using image codes according to claim 1, further comprising the step of taking a photograph that includes the goods and the goods image code together after the goods image code is attached, and applying image processing and machine learning techniques to the photograph to store the morphological characteristics of the goods and the goods identifier together in a database. Claim 13 A method for managing an item using an image code according to claim 1, further comprising the step of a manager terminal verifying the identifier of an item using a short-range wireless communication module attached to or embedded in the item, instead of an image code marked or attached to the item. Claim 14 A method for managing items using image codes according to claim 1, further comprising the step of photographing an item image code of another item whose coordinate information is known in a database using a manager terminal, recognizing the identifier of the item and obtaining coordinate information, and then applying computer vision technology to the photograph to estimate a relative position and summing the coordinate information to estimate the coordinates of the terminal and other items near the terminal. Claim 15 A method for managing items using image codes according to claim 1, further comprising the step of video recording an item or reference point image code while a manager terminal or mobile robot moves and recognizes an object or reference point identifier in the image, searches a database with this identifier to collect coordinate information of said item or reference point, and applies visual odometry or SLAM technology to eliminate position coordinate estimation errors of the item and precisely estimate the position coordinates. Claim 16 A method for managing goods using an image code according to claim 1, wherein, when the subject of management is a person or animal or plant, the face or appearance of the person or animal or plant becomes an identifier of the subject of management, and further comprising the step of photographing a reference point image code with a terminal at a location close to the person or animal or plant and estimating the location coordinates of the terminal and the person or animal or plant using computer vision technology. Claim 17 A method for managing an item using an image code according to claim 1, wherein, in order to estimate relative position coordinates by applying computer vision technology to an image code of an item or reference point captured by a terminal, the image code must have multiple mark indicators, the shape and size of the mark indicators and the distance between the marks must be predefined, and a unique identifier of the item or reference point is included in text form. Claim 18 A method for managing an item using an image code according to claim 1, further comprising the step of displaying the image code on the surface of an object or reference point using a display device, in addition to attaching the image code to an object or reference point. Claim 19 In an item management system using image codes, a reference point image code including one or more of location information such as the name of the installation location, floor number, room number, reference point identifier, reference point absolute coordinates, reference point photo, and precautions; an item image code including one or more of information such as item name, item identifier, initial installation location, initial installation date and time, manager name, item photo, and remarks; a terminal carried by a manager who stands next to an item to locate surrounding reference point image codes and take a photo, analyzes the reference point image code to extract absolute location coordinate information of the reference point, estimates the relative position between the reference point image code and the terminal using computer vision technology, calculates the absolute coordinates of the terminal and the item by summing the relative location information and the absolute location coordinate information, and inputs or updates the absolute coordinates of the item into an item management database; and A product management system using image codes, comprising a service server that operates a database in which an administrator terminal connects to a communication network to search for or modify information related to reference points or products.

Images