Real-time logistics control system and method using barcode camera

Abstract

Disclosed are a logistics control system and a method for controlling the operation of the logistics control system. The present specification recognizes barcodes of articles loaded in a predetermined space and simultaneously stores article image information, the barcodes being recognized through a camera device in which a barcode reader and an image capturing module are configured as a single housing. In addition, the present specification provides a user interface, on which location information of an article is mapped, through a display, thus making it possible to efficiently implement operations of the logistics control system such as the real-time loading state, movement state, etc. of the article.

Description

Real-time logistics control system and method using barcode cameras

[0001] This specification relates to a real-time logistics control system and method using a barcode camera.

[0002] To verify the distribution process of an item in a logistics management system, barcode information can be obtained. Furthermore, if an issue arises with an item, such as loss or damage, users can directly search for the item using barcode information to verify its distribution process.

[0003] Previously, logistics management involved manual management or the use of RFID and beacons, which involved attaching RFID tags or beacon transmitters to track the location of items. However, manual management methods presented challenges in remote, real-time monitoring of item status and location. Furthermore, RFID and beacon systems require significant maintenance costs, requiring RFID tags and beacon receivers for each cargo. Furthermore, because they use a specific frequency band, errors can be high, depending on the cargo's material, and accurate location tracking is often difficult.

[0004] This specification aims to solve the aforementioned problems and implement an RTLS system utilizing a camera capable of barcode recognition.

[0005] In addition, the purpose of this specification is to provide a user interface that facilitates real-time movement and status identification of goods based on barcode information and image information acquired from a barcode recognition camera installed at a logistics site.

[0006] The technical problems to be solved by this specification are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the detailed description of the invention below.

[0007] A logistics control system according to one embodiment of the present specification includes: a camera device for photographing a space for loading goods; and a computing device configured such that a first image sensor for obtaining barcode information of the goods and a second image sensor for obtaining an image including the goods are integrated into a single housing, and a structure of the space is mapped to provide a user interface for checking the loading status of the goods, and a location of the goods is mapped and provided on the user interface based on the barcode information and image received from the camera device.

[0008] The first image sensor and the second image sensor are triggered simultaneously with a single signal, so that the shooting points are synchronized.

[0009] The above space includes a first zone covered by the angle of view of the first camera device, a second zone covered by the angle of view of the second camera device, and each zone can be configured with at least one identifiable item loading space.

[0010] The computing device can store information of each camera device corresponding to each zone among a plurality of zones in memory by matching the information, and each camera device can store maximum product height information capable of recognizing the barcode information of the product in the memory according to lens performance.

[0011] The computing device can provide detailed area information of the item loading space, the barcode information, and the time information at which the item was loaded within each area to the user interface.

[0012] The computing device can provide the interface with the current location of a specific item, a movement path, and information about the item based on a predetermined input to the user interface.

[0013] The computing device may provide the user interface by outputting it to a display unit, and may display an indicator indicating a real-time recognition result of the barcode information on the user interface.

[0014] The above indicator may further include a stacking indicator that indicates a stacking state when multiple items are loaded at the same coordinates.

[0015] The computing device can provide product information corresponding to the barcode information and image information of the product taken together based on a predetermined input provided to the user interface.

[0016] The orientation of the camera device may be set so that the first image sensor and the second image sensor are oriented in a vertical direction with respect to the ground of the space.

[0017] The first image sensor may be a mono sensor, and the second image sensor may be a color sensor.

[0018] The computing device can calculate location information of the product based on the coordinate values ​​of the barcode information obtained through the mono sensor.

[0019] The above camera device may further include an LED module disposed on one side of the housing.

[0020] A method for controlling the operation of a logistics control system according to another embodiment of the present specification comprises the steps of: displaying a user interface on which a structure of a space for loading goods is mapped on a display unit; obtaining an image including barcode information of the goods and the goods through a camera device when the goods are received in the space; and confirming location information of the goods in the image and mapping and displaying the location of the goods on the interface; wherein the camera device is configured such that a first image sensor for obtaining barcode information of the goods and a second image sensor for obtaining an image of the goods are integrated into a single housing, and the first image sensor and the second image sensor are triggered simultaneously with a single signal so that the shooting time points are synchronized, and the user interface includes at least one area each covered by an angle of view of at least one surveillance camera for shooting the space, and coordinate information of the at least one area is linked to the acquired image.

[0021] This specification can implement a more efficient RTLS system by utilizing a camera capable of barcode recognition.

[0022] In addition, this specification facilitates real-time movement and status identification of goods based on barcode information and image information obtained from a barcode recognition camera installed at a logistics site.

[0023] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0024] The accompanying drawings, which are incorporated in and constitute a part of the detailed description to aid in understanding the present specification, provide embodiments of the present specification and, together with the detailed description, explain the technical features of the present specification.

[0025] FIG. 1 is a drawing for explaining a logistics control system according to one embodiment of the present specification.

[0026] FIG. 2 is a schematic block diagram of a camera device according to one embodiment of the present specification.

[0027] FIG. 3 is a drawing for explaining a computing device that controls the operation of a logistics control system according to one embodiment of the present specification.

[0028] Figure 4 is a conceptual diagram for explaining the operation of a logistics control system according to one embodiment of the present specification.

[0029] FIG. 5 is a drawing for explaining a process of recognizing a barcode spaced a predetermined distance apart in a logistics control system according to one embodiment of the present specification.

[0030] Figure 6 is a flowchart of a control method of a logistics control system according to one embodiment of the present specification.

[0031] Figure 7 illustrates a simulation environment for operating a logistics control system in an actual space where goods are loaded according to one embodiment of the present specification.

[0032] FIG. 8 is an example of a user interface for managing the status of loaded goods in a logistics control system according to one embodiment of the present specification.

[0033] FIG. 9 is an example of a list providing the loading status of goods monitored through a logistics control system according to one embodiment of the present specification.

[0034] FIG. 10 is an example of a user interface for checking the movement process of an item in real time according to one embodiment of the present specification.

[0035] The accompanying drawings, which are incorporated in and constitute a part of the detailed description to aid in understanding the present specification, provide embodiments of the present specification and, together with the detailed description, explain the technical features of the present specification.

[0036] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0037] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0038] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0039] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0040] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0041] FIG. 1 is a drawing for explaining a logistics control system according to one embodiment of the present specification.

[0042] Referring to FIG. 1, a logistics control system (10) according to one embodiment of the present specification may include a photographing device (100) and an image management server (200). The photographing device (100) may be an electronic photographing device placed at a fixed location in a specific location, an electronic photographing device that can be automatically or manually moved along a predetermined path, or an electronic photographing device that can be moved by a person or a robot. The photographing device (100) may be an IP camera that is connected to a wired or wireless Internet. The photographing device (100) may be a PTZ camera having pan, tilt, and zoom functions. The photographing device (100) may have a function of recording or taking pictures of a monitored area. The photographing device (100) may have a function of recording sounds generated in the monitored area. The photographing device (100) may have a function of generating a notification or performing recording or taking pictures when a change, such as movement or sound, occurs in the monitored area.

[0043] Meanwhile, the photographing device (100) may be configured as a single housing with a barcode information acquisition unit (160) that is separate from the image sensor (110) that acquires image information. The present specification includes a first image sensor that acquires barcode information of an article and a second image sensor that acquires an image including the article, and the barcode information acquisition unit (160) may correspond to the first image sensor.

[0044] Meanwhile, the first image sensor and the second image sensor can be triggered simultaneously with a single signal so that the shooting times can be synchronized and operated. Therefore, when an item is loaded in the item loading space and the shooting device (camera device, 100) recognizes the barcode attached to the top of the item, the shooting device (100) recognizes the barcode through the first image sensor and simultaneously acquires an image including the item through the second image sensor. If the first image sensor and the second image sensor are respectively provided in independent housings rather than in a single housing, even if the control operation for shooting time synchronization is implemented, there is bound to be a difference between the time when the actual item's barcode is recognized and the time when the item's image is captured due to a slight delay in the operation of each image sensor.

[0045] Meanwhile, the photographing device (100) may further include an LED module (101) arranged on one side of the housing. The location where the photographing device (100) is installed is a warehouse where cargo is loaded, and when cargo is loaded in a space where the warehouse height is at least 3 m, the photographing device (100) may be fixed so as to be directed vertically downward at the vertical top of the cargo. According to one embodiment, when the product is a refrigerator, in order to photograph a space where multiple 2 m refrigerators are loaded with one photographing device (100), the height at which the photographing device is installed may be fixed at a point higher than the height of the product.

[0046] Accordingly, the photographing device (100) may be further equipped with a predetermined LED device to increase the recognition rate of a barcode attached to the top of an item spaced a certain distance away.

[0047] The image management server (200) may be a device that receives and stores the image captured by the photographing device (100) and / or the image obtained by editing the image. The image management server (200) may analyze the received image to correspond to the intended use. For example, the image management server (200) may detect an object using an object detection algorithm to detect an object in the image. The object detection algorithm may be an AI-based algorithm, and may detect an object by applying a pre-trained artificial neural network model.

[0048] Meanwhile, the image management server (200) may store various learning models suited to the purpose of image analysis. In addition to the learning model for object detection described above, it may also store a model capable of obtaining the movement speed of a detected object.

[0049] Additionally, the video management server (200) can analyze the received video to generate metadata and index information for the metadata. The video management server (200) can analyze video information and / or audio information included in the received video together or separately to generate metadata and index information for the metadata.

[0050] The image management system (10) may further include an external device (300) capable of performing wired or wireless communication with the photographing device (100) and / or the image management server (200).

[0051] An external device (300) may transmit an information provision request signal requesting provision of all or part of an image to the image management server (200). The external device (300) may transmit an information provision request signal requesting metadata obtained by analyzing an image and / or index information for the metadata to the image management server (200).

[0052] The image management system (10) may further include a communication network (400) which is a wired or wireless communication path between the photographing device (100), the image management server (200), and / or the external device (300). The communication network (400) may include wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), ISDNs (Integrated Service Digital Networks), or wireless networks such as wireless LANs, CDMA, Bluetooth, and satellite communication, but the scope of the present specification is not limited thereto.

[0053] FIG. 2 is a schematic block diagram of a surveillance camera according to one embodiment of the present specification.

[0054] FIG. 2 is a block diagram showing the configuration of the camera illustrated in FIG. 1. Referring to FIG. 2, the camera device (100) is described as a network camera that performs an intelligent image analysis function to generate the image analysis signal, but the operation of the network surveillance camera system according to an embodiment of the present invention is not necessarily limited thereto.

[0055] The camera device 100 includes an image sensor (110), an encoder (120), a memory (130), a communication unit (140), an AI processor (150), a barcode information acquisition unit (160), an interface unit (170), and a first processor (180).

[0056] The image sensor (110) performs the function of capturing an image by photographing a surveillance area, and can be implemented as, for example, a CCD (Charge-Coupled Device) sensor, a CMOS (Complementary Metal-Oxide-Semiconductor) sensor, etc.

[0057] The encoder (120) performs an operation of encoding an image acquired through an image sensor (110) into a digital signal, which may follow, for example, H.264, H.265, MPEG (Moving Picture Experts Group), M-JPEG (Motion Joint Photographic Experts Group) standards, etc.

[0058] The memory (130) can store video data, audio data, still images, metadata, etc. The metadata may be data including object detection information (movement, sound, intrusion into a designated area, etc.) captured in the surveillance area, object identification information (person, car, face, hat, clothing, etc.), and detected location information (coordinates, size, etc.). In addition, the metadata may include various environmental factors including weather information when the video is captured. In addition, the metadata may include lighting conditions when the video is captured. The above-described metadata may be combined with the vehicle color detection method according to one embodiment of the present specification to perform more precise vehicle color detection.

[0059] In addition, the still image is generated together with the metadata and stored in the memory (130), and can be generated by capturing image information for a specific analysis area among the image analysis information. For example, the still image can be implemented as a JPEG image file.

[0060] For example, the still image may be generated by cropping a specific area of ​​image data determined to be an identifiable object among image data of the surveillance area detected in a specific area and for a specific period of time, and this may be transmitted in real time together with the metadata.

[0061] The communication unit (140) transmits the video data, audio data, still images, and / or metadata to the video receiving / searching device (300). According to one embodiment, the communication unit (140) can transmit the video data, audio data, still images, and / or metadata to the video receiving device (300) in real time. The communication unit (140) can perform at least one communication function among wired / wireless Local Area Network (LAN), Wi-Fi, ZigBee, Bluetooth, and Near Field Communication.

[0062] The AI ​​processor (150) is for artificial intelligence image processing and can apply a deep learning-based object detection algorithm learned from images acquired through a surveillance camera system. The AI ​​processor (150) can be implemented as a single module with a processor (160) that controls the entire system, or as an independent module.

[0063] The AI ​​processor (150) may include a learning data acquisition unit (151) and a model learning unit (152).

[0064] The AI ​​processor (150) may be mounted on an electronic device including an AI module capable of performing AI processing, a server including an AI module, or the like, or may be implemented as an independent device. AI processing may include all operations related to the control unit of a surveillance camera or video management server. For example, a surveillance camera or video management server may perform AI processing of an acquired video signal to perform processing / judgment and control signal generation operations.

[0065] The AI ​​processor (150) can be installed and operated in a client device that directly utilizes the AI ​​processing results, or in a device in a cloud environment that provides the AI ​​processing results to other devices. The AI ​​processor (150) can be included in a computing device capable of learning a neural network, and can be implemented in various electronic devices such as a server, desktop PC, laptop PC, tablet PC, etc.

[0066] The AI ​​processor (150) can learn a neural network for recognizing data related to surveillance cameras. Here, the neural network for recognizing data related to surveillance cameras can be designed to simulate the structure of the human brain on a computer and can include a plurality of network nodes having weights that simulate neurons of a human neural network. The plurality of network modes can exchange data according to their respective connection relationships so as to simulate the synaptic activity of neurons that exchange signals through synapses. Here, the neural network can include a deep learning model developed from a neural network model. In the deep learning model, the plurality of network nodes can be located in different layers and exchange data according to convolutional connection relationships. Examples of neural network models include various deep learning techniques such as deep neural networks (DNNs), convolutional deep neural networks (CNNs), recurrent Boltzmann machines (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), and deep Q-networks, which can be applied to fields such as computer vision, speech recognition, natural language processing, and speech / signal processing.

[0067] Meanwhile, the processor performing the functions described above may be a general-purpose processor (e.g., CPU), but may also be an AI-specific processor for artificial intelligence learning (e.g., GPU).

[0068] The memory (130) can store various programs and data required for the operation of the AI ​​processor (150). The memory (130) can be implemented as a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), a solid state drive (SDD), etc. The memory (130) is accessed by the AI ​​processor (150), and data can be read / written / modified / deleted / updated, etc. by the AI ​​processor (150). In addition, the memory (130) can store a neural network model (e.g., a deep learning model) generated through a learning algorithm for image data classification / recognition.

[0069] Meanwhile, the AI ​​processor (150) may include a data learning unit that learns a neural network for data classification / recognition. The data learning unit may learn criteria regarding which learning data to use to determine data classification / recognition and how to classify and recognize data using the learning data. The data learning unit may acquire learning data to be used for learning, and learn a deep learning model by applying the acquired learning data to the deep learning model. The data learning unit may be manufactured in the form of at least one hardware chip and mounted on the image processing device (100). For example, the data learning unit may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics processor (GPU) and mounted on the image processing device (100). In addition, the data learning unit may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored on a non-transitory computer-readable recording medium that can be read by a computer. In this case, at least one software module may be provided by an operating system (OS) or an application.

[0070] The data learning unit may include a learning data acquisition unit (151) and a model learning unit (152).

[0071] The learning data acquisition unit (151) can acquire learning data required for a neural network model for classifying and recognizing data. The model learning unit (152) can use the acquired learning data to train the neural network model to have judgment criteria regarding how to classify predetermined data. At this time, the model learning unit (152) can train the neural network model through supervised learning that uses at least some of the learning data as judgment criteria. Alternatively, the model learning unit (152) can train the neural network model through unsupervised learning that discovers judgment criteria by learning on its own using the learning data without guidance. In addition, the model learning unit (152) can train the neural network model through reinforcement learning using feedback on whether the results of situational judgment according to learning are correct. In addition, the model learning unit (152) can train the neural network model using a learning algorithm including error back-propagation or gradient descent.

[0072] When the neural network model is learned, the model learning unit (152) can store the learned neural network model in the memory (130). The model learning unit (152) can also store the learned neural network model in the memory of a server connected to the image processing device (100) via a wired or wireless network.

[0073] The data learning unit may further include a learning data preprocessing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or to save resources or time required for generating the recognition model.

[0074] The learning data preprocessing unit can preprocess the acquired data so that it can be used for learning to determine situations. For example, the learning data preprocessing unit can process the acquired data into a preset format so that the model learning unit (152) can utilize the acquired learning data for learning image recognition.

[0075] In addition, the learning data selection unit can select data required for learning from among the learning data acquired from the learning data acquisition unit (151) or the learning data preprocessed from the preprocessing unit. The selected learning data can be provided to the model learning unit (152).

[0076] Additionally, the data learning unit may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

[0077] The model evaluation unit inputs evaluation data into the neural network model, and if the analysis results output from the evaluation data do not satisfy a predetermined standard, it can cause the model learning unit (152) to relearn. In this case, the evaluation data may be predefined data for evaluating the recognition model. For example, the model evaluation unit can evaluate that the predefined standard is not satisfied if the number or ratio of evaluation data with inaccurate analysis results among the analysis results of the learned recognition model for the evaluation data exceeds a preset threshold.

[0078] The barcode information acquisition unit (160) is a module distinct from the image sensor (110) described above. In this specification, the barcode information acquisition unit (160) is defined as a first image sensor corresponding to a mono sensor, and the image sensor (110) used for image capturing is defined as a second image sensor. The barcode information acquisition unit (160) does not include a color filter and can absorb light of all wavelengths. In addition, the barcode information acquisition unit (160) is controlled so that the image sensor (110) for image capturing and the capturing time are synchronized.

[0079] The interface unit (170) may be provided on one side of the housing of the photographing device (100) to provide an interface for connection to an LED device. It goes without saying that, in addition to the LED device, other sub-devices for supporting the functions of a camera device equipped with a barcode recognition function may also be connected to the interface unit (170).

[0080] The first processor (180) controls the overall operation of the photographing device (100). The first processor (180) can control the operation of the photographing device (100) by processing signals, data, information, etc. input or output through the components discussed above, or by operating an application program stored in the memory (130). The processor (160) can operate at least two or more of the components included in the photographing device (100) in combination with each other to operate the application program.

[0081] This specification may be linked to one or more of a surveillance camera, an autonomous vehicle, a user terminal, and a server, including an artificial intelligence module, a robot, an augmented reality (AR) device, a virtual reality (VT) device, and a device related to a 5G service.

[0082] FIG. 3 is a diagram illustrating a computing device that controls the operation of a logistics control system according to one embodiment of the present specification. The computing device is a device capable of performing the functions of the image management server (200) illustrated in FIG. 2. For convenience of explanation, the computing device of FIG. 3 will be referred to as a logistics control device.

[0083] Referring to FIG. 3, the communication unit (210) may provide a function for communicating with an external device via a network. For example, a request generated by the second processor (240) of the logistics control device (200) according to a program code stored in a recording device such as a memory (230) may be transmitted to an external device via a network under the control of the communication unit (210). Conversely, control signals, commands, contents, files, etc. provided from an external device may be received by the logistics control device (200) via the communication unit (210) via a network. The logistics control device (200) receives barcode information of an item acquired from a camera device (100) and image information including the item via the communication unit (210).

[0084] The communication method is not limited, and may include not only a communication method utilizing a communication network that the network may include (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network), but also short-range wireless communication between devices. For example, the network may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. In addition, the network may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

[0085] In addition, the logistics control device (200) according to one embodiment of the present specification may include a user interface unit (220). The user interface unit (220) may be a means for interfacing with an input / output device. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the user interface unit (220) may be a means for interfacing with a device that integrates input and output functions, such as a touch screen. As a more specific example, the second processor (240) of the logistics control device (200) may use data provided by an external device to process commands of a computer program loaded into the memory (230), and a service screen or content may be displayed on the display through the user interface unit (220). For the convenience of explaining the operation control method of the goods control system according to one embodiment of the present specification, the user interface unit (220) may be understood as including both input / output means, and the logistics control device (200) may display a user interface that maps coordinate information of the goods loading space on the display unit, and the logistics manager may check the real-time movement status, location information, etc. of the goods based on the user interface displayed on the display. Here, the user interface may mean a screen configuration for visualizing the structure of the actual logistics loading space through the user interface unit (e.g., a display) and providing it on a screen. For the screen configuration, the logistics control system manager may mean a space model divided into a plurality of sections for loading a plurality of goods in an actual logistics loading space (e.g., a warehouse). The space model may be a result learned to be configured into a single screen through a combination of spatial images acquired through a plurality of camera devices installed on the actual logistics loading space.

[0086] Memory (230) is a computer-readable recording medium and may include a non-volatile memory (permanent mass storage device) such as a random access memory (RAM), a read-only memory (ROM), and a disk drive. In addition, program code for controlling a logistics control system may be temporarily or permanently stored in the memory (230).

[0087] The second processor (240) can acquire barcode information from a barcode information acquisition module equipped in a logistics control system that acquires barcode information of an item attached to the item. In addition, the second processor (240) can acquire image information from a camera device (100) that acquires image information about the item by synchronizing the shooting time with the barcode information acquisition module. In addition, the second processor (240) can monitor the item based on the barcode information and image information.

[0088] Figure 4 is a conceptual diagram for explaining the operation of a logistics control system according to one embodiment of the present specification.

[0089] Referring to FIG. 4, a logistics control system according to one embodiment of the present specification may have a field of view (FoV) of a camera device (100) that covers a rectangular space corresponding to a predetermined size (e.g., horizontal length (V) and vertical length (H)). In addition, the camera device (100) is fixedly installed at a point spaced apart from the ground by a predetermined distance (D), so that a first image sensor of the camera device (100) recognizes barcodes (BC1, BC2, BC3, BC4) attached to the top of an item, and a second image sensor captures an image so that all four items captured within the field of view range are included in one image frame, and the image information as a result of the capture may be stored together with each barcode information.

[0090] Meanwhile, the present specification may vary the field of view of the camera device as shown in the table below depending on the distance (D) from the ground of the goods loading space and the performance of the lens equipped in the camera device. For example, when a surveillance camera device (100) equipped with a 16 mm lens is installed at a height of 5 m from the ground, it can recognize all goods loaded in a space with a field of view of 239 (cm) * 440 (cm) wide and the barcode information of the corresponding goods. In addition, when a surveillance camera device (100) equipped with a 12 mm lens is installed at a height of 6 m from the ground, it can recognize all goods loaded in a space with a field of view of 361 (cm) * 691 (cm) wide and the barcode information of the corresponding goods.

[0091] Table 1 shows the optimal field of view range for which barcode and image recognition can be performed smoothly when the distance between the camera device (100) and the logistics loading surface is in the range of 3 m to 8 m.

[0092] Distance D(m)FOV when 16mm lens is applied(cm)FOV when 12mm lens is applied(cm)HVHV326414434618143521914612415440239576301652828769136176163358074218704383922482

[0093] That is, one camera device can perform barcode recognition and image capture simultaneously according to lens performance and installed height, and the area information of the logistics loading space can be set in advance to check the loading status of the logistics.

[0094] FIG. 4 describes a photographing device (100) corresponding to one zone among the entire space where the goods are loaded, as shown in FIG. 7. However, according to one embodiment of the present specification, the entire space where the goods are loaded, as shown in FIG. 7, may be divided into Zone A, Zone B, Zone F, and Zone F, as shown in FIG. 7, and there may be a plurality of photographing devices (100) corresponding to each zone and photographing each zone. Here, the photographing devices corresponding to each zone may have a field of view range to cover the space of the corresponding zone. Meanwhile, according to one embodiment, the photographing devices corresponding to each zone may have a field of view range that can photograph a passageway and a neighboring zone outside the corresponding zone. That is, the photographing devices according to one embodiment of the present specification may have field of view ranges that are at least partially identical between the photographing devices depending on the purpose.

[0095] In one embodiment, the photographing device covering Zone A may have a field of view that covers Zone A and the passage between Zone A and Zone D. In addition, the photographing device covering Zone A may have a field of view that covers Zone A and the neighboring Zone D. In another embodiment, the photographing device may have a field of view that can cover the entire space where the goods illustrated in FIG. 7 are loaded.

[0096] FIG. 5 is a diagram illustrating a process for recognizing a barcode spaced a predetermined distance apart in a logistics control system according to one embodiment of the present disclosure. Referring to FIG. 5 , the result of recognizing a barcode located 6 meters away from a camera device (100) is shown. Therefore, even in large-scale logistics warehouses, the present disclosure can efficiently acquire barcode information and image information attached to items by taking into account the performance of the camera device.

[0097] Accordingly, the logistics control system according to one embodiment of the present specification may automatically calculate and provide the optimal number and installation locations of camera devices based on the entire area of ​​the logistics loading space by considering the angle of view that can be covered by one camera device when the area of ​​the logistics loading space is large.

[0098] FIG. 6 is a flowchart of a control method of a logistics control system according to one embodiment of the present disclosure. The control method illustrated in FIG. 6 may be implemented by a combination of the first processor (180) of the camera device (100) illustrated in FIG. 2 and / or the second processor (240) of the logistics control device (200) illustrated in FIG. 3. FIG. 7 illustrates a simulation environment for operating a logistics control system in an actual space where goods are loaded according to one embodiment of the present disclosure. In addition, FIG. 8 is an example of a user interface for managing the status of goods loaded in the logistics control system according to one embodiment of the present disclosure. Hereinafter, the operation of the logistics control system according to one embodiment of the present disclosure will be described with reference to FIGS. 6 to 8 together.

[0099] Referring to FIG. 6, the logistics control device (200) can output a user interface in which the structure of the entire space where goods are loaded is mapped to the display unit (S600).

[0100] Referring to FIG. 7, the entire space where the goods are actually loaded can be divided into Zone A, Zone B, Zone F, and Zone F as illustrated in FIG. 7. Here, camera information capable of acquiring barcodes and image information of goods loaded in each Zone, such as a first camera device covering Zone A and a second camera device covering Zone B, may be matched in advance and stored in the memory of the logistics control device (200). Meanwhile, when the size of one Zone is large, each Zone may be configured with a predetermined number of sub-Zones. According to one embodiment, the logistics control device (200) may output a user interface (an example illustrated in FIG. 8) on the display unit so that the spatial structure can be mapped so that the Bin A-1, Bin A-2, Bin A-3, and Bin A-4 zones constituting the actual space Zone A and Zone A can be identified through the display unit, thereby enabling the loading status of the goods to be checked.

[0101] For example, the logistics control device (200) can store the 2D coordinate system of the actual space illustrated in FIG. 7 by matching it with the coordinate system of the camera device that photographs the actual space. Preferably, the 2D coordinate system of the rectangular Zone A can be stored by matching it with the coordinate system within the image acquired through the first camera that photographs Zone A.

[0102] Accordingly, referring to FIG. 8, the user interface to which the actual space structure is mapped is configured by converting the size of the actual warehouse of FIG. 7 into the size of W1*W2. In addition, the actual space Zone A of FIG. 7 can be configured by converting the size of A1*A2, and the actual space Bin A-3 can be configured by converting the size of A31*A32, respectively, on the user interface. Accordingly, the actual goods loading space of FIG. 7 can be provided through the display unit of the logistics control device (200) as a user interface to which the structure of the actual space is mapped, as illustrated in FIG. 8.

[0103] Accordingly, if the first camera device is a device covering Zone A, the first camera device can photograph the first cargo (F) loaded in Zone A and simultaneously obtain barcode information (BC) provided on the first cargo (F) (S610), and transmit the barcode information and image information to the logistics control device (200).

[0104] The logistics control device (200) can extract coordinate information of an item within an image and map it onto the user interface (S620). For example, the logistics control device (200) can extract coordinate information of the first cargo (F) from an image including the first cargo (F1) and display an object (D015) corresponding to the first cargo (F1) in the Bin A-2 area. The object displayed on the user interface can be displayed as a predetermined graphic object, and barcode information can also be displayed together with the graphic object.

[0105] According to one embodiment, the logistics control device (200) may calculate coordinate information of an item based on the position coordinate values ​​of a barcode acquired through a first image sensor (mono sensor) and provide this information on a user interface mapped with an actual spatial structure. At this time, the detailed bin or zone in which the item is located may be determined based on the recognition center point of the barcode.

[0106] Meanwhile, referring to FIG. 8, the logistics control device (200) may recognize the result from a camera device covering each zone when the cargo object (D015, C011, A003, C012) is detected in any one of the preset Zones A to F. However, in the case of cargo objects B008 and D014, although they are located outside the preset Zones A to F, at least a portion of the cargo object may be included in the field of view of the camera device, allowing the barcode to be recognized. In this case, the logistics control device (200) may display the object in a different color on the user interface.

[0107] According to one embodiment of the present specification, there may be a case where multiple items are stacked in two or more layers because the height of the items is relatively low. In Fig. 7, C012 cargo (F21) is primarily loaded in Bin F-3 in Zone F, and the photographing device corresponding to Zone F can obtain barcode information (B21) of the C012 cargo (F21). In a state where the C012 cargo (F21) is loaded, C017 cargo (F22) can be additionally loaded on top of the C012 cargo (F21). In this case, the photographing device can additionally obtain barcode information (BC22) attached to the C017 cargo (F22).

[0108] However, in this case, as illustrated in FIG. 8, only the additionally loaded C017 cargo (F22) instead of the primarily loaded C012 cargo (F21) may be displayed in the Bin F-3 area on the interface provided on the display unit of the logistics control device (200). However, the logistics control device (200) determines whether the primarily loaded C012 cargo (F21) has a history of moving out of the Bin F-3 area of ​​Zone F, and if no history of movement exists, the interface screen displayed on the display unit may display a stacking indicator (SI) to confirm that the C012 cargo (F21) and the C017 cargo (F22) are in a multi-layer stacking state. The stacking indicator (SI) may be provided at the upper right of the cargo graphic object, and may indicate the number of loads at the corresponding location together with a symbol such as x2.

[0109] Meanwhile, referring to FIG. 9, the logistics control device (200) can display barcode information and image information obtained as a result of barcode recognition and image capture when an item is loaded into a specific space by mapping them to the user interface described above, and at the same time, display a list including detailed area information (Bin A-1, Bin A-2, etc.) of the item loading space within each area, the barcode information (D015, C011, A003, C012, C017), and information on the time at which the item was loaded.

[0110] According to one embodiment, as described above in the Bin F-3 zone of FIGS. 7 to 8, it is determined whether there is a history of C012 cargo (F21) moving out of the Bin F-3 zone of Zone F, and if there is no history of movement, C017 (F22) may be additionally displayed under C012 cargo (F21), and the loading time of the multi-layered cargo may be provided together with the list items.

[0111] According to one embodiment of the present specification, a logistics control system comprises an image sensor for barcode recognition and an image sensor for video recording, mounted on a single body. Since the recording times are synchronized, the captured video information can be stored together with the recognized barcode information (S630). This facilitates easier verification of the loading status of items and whether any damage occurred during transport.

[0112] FIG. 10 is an example of a user interface for checking the movement process of an item in real time according to one embodiment of the present specification.

[0113] The logistics control device (200) may provide the current location, movement path, and information about a specific item to the interface based on a predetermined input to the user interface. For example, referring to FIG. 10, when an input for selecting a cargo object C011 is received while the current location of the cargo object C011 is indicated as being located in Bin F-2 of Zone F, movement path information about the cargo object C011 may be displayed together. The movement path information may display the cargo object C011 along with the movement direction at the initial location, intermediate location, and current location.

[0114] The above-described specification can be implemented as computer-readable code on a program-recorded medium. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. Therefore, the above detailed description should not be construed as limiting in any respect but should be considered as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are intended to be included in the scope of the present invention.

Claims

1. A camera device for photographing a space for loading goods; and The camera device is configured such that a first image sensor for obtaining barcode information of the product and a second image sensor for obtaining an image including the product are integrated into one housing. A computing device that provides a user interface for checking the loading status of the goods by mapping the structure of the space, and provides the location of the goods by mapping it to the user interface based on the barcode information and image received from the camera device; Logistics control system including.

2. In paragraph 1, A logistics control system characterized in that the first image sensor and the second image sensor are simultaneously triggered by a single signal so that the shooting points are synchronized.

3. In paragraph 1, A logistics control system characterized in that the space includes a first zone covered by the angle of view of a first camera device, a second zone covered by the angle of view of a second camera device, and each zone is composed of at least one identifiable item loading space.

4. In paragraph 3, The above computing device, Among multiple zones, the information of each camera device corresponding to each zone is matched and stored in memory, and A logistics control system characterized in that each of the above camera devices stores in the memory the maximum product height information that can recognize the barcode information of the product according to the lens performance.

5. In paragraph 3, The above computing device, A logistics control system characterized in that it provides detailed area information of the goods loading space, the barcode information, and the time information at which the goods were loaded within each of the above areas to the user interface.

6. In paragraph 5, The above computing device, A logistics control system characterized in that it provides the current location of a specific item, the movement path, and information about the item to the user interface based on a predetermined input to the user interface.

7. In paragraph 1, The above computing device, A logistics control system characterized in that the user interface is provided by outputting it to a display unit, and an indicator indicating the real-time recognition result of the barcode information is displayed on the user interface.

8. In paragraph 7, The above indicator further includes a stacking indicator that indicates the stacking state when multiple items are loaded at the same coordinates, in a logistics control system.

9. In paragraph 1, The above computing device, A logistics control system characterized in that, based on a predetermined input provided to the user interface, product information corresponding to the barcode information and image information of the product are provided together.

10. In paragraph 1, The logistics control system, characterized in that the first image sensor and the second image sensor are oriented so that the direction of the camera device is set to be perpendicular to the ground of the space.

11. In paragraph 1, A logistics control system characterized in that the first image sensor is a mono sensor and the second image sensor is a color sensor.

12. In paragraph 11, A logistics control system characterized in that the computing device calculates location information of the product based on the coordinate values ​​of the barcode information acquired through the mono sensor.

13. In paragraph 1, A logistics control system characterized by the above camera device further including an LED module disposed on one side of the housing.

14. A method for controlling the operation of a logistics control system, A step of displaying a user interface mapped to the structure of the space for storing goods on a display unit; A step of acquiring barcode information of the item and an image including the item through a camera device when the item is entered into the space above; and The method includes the step of verifying the location information of the item within the above video and mapping the location of the item to the interface to display it. The camera device is configured such that a first image sensor for acquiring barcode information of the product and a second image sensor for acquiring an image of the product are integrated into a single housing, and the first image sensor and the second image sensor are triggered simultaneously with a single signal so that the shooting time is synchronized. The above user interface is, A control method for a logistics control system, characterized in that the angle of view of at least one surveillance camera that photographs the space includes at least one area covered by each of the angles, and the at least one area is linked to the acquired image and coordinate information.

Images