Method for performing article control and apparatus for supporting same

Abstract

The present specification provides a photographing device for performing article control. More specifically, the photographing device comprises: a camera module for photographing, in a common view angle area, one or more articles moving on a conveyor belt, wherein the camera module includes a first image sensor for acquiring barcode information of an article and a second image sensor for acquiring color information of the article, and the first image sensor and the second image sensor are configured as a single housing in the photographing device, and are simultaneously triggered by a single signal so that photographing time points are synchronized; and a processor for controlling the overall operation of the photographing device, wherein the processor controls the first image sensor to acquire barcode information of the article from a photographed image, controls the second image sensor to acquire color information of the article from the photographed image, and identifies the state or position of the article on the basis of the acquired barcode information of the article and the acquired color information of the article.

Description

Method for performing goods control and device supporting same

[0001] This specification relates to a method for performing item control, and more specifically, to a method for performing item control using two image sensors and a device for supporting the same.

[0002] Previously, there were no image-based barcode recognition cameras that provided color images, and image-based barcode readers had the problem of not being able to distinguish the colors of items or cargo because they used monochrome image sensors for recognition accuracy.

[0003] Korean Patent No. 10-2382742 proposes a method for identifying an object by acquiring two image information using two cameras.

[0004] However, Korean Patent No. 10-2382742 does not disclose a method for obtaining barcode information and color information of an item by equipping one camera with two image sensors.

[0005] Therefore, research is needed on methods to obtain barcode information and color information of an item to confirm the condition or location of the item.

[0006] Accordingly, the purpose of this specification is to provide a method for obtaining barcode information and color information of an item using two synchronized image sensors.

[0007] In addition, the purpose of this specification is to provide a method for confirming the condition or location of an article by using the article's barcode information and the article's color information.

[0008] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0009] The present specification relates to a photographing device for performing item control, comprising: a camera module for photographing one or more items moving on a conveyor belt in a common field of view (FOV) area; the camera module includes a first image sensor for obtaining barcode information of the items and a second image sensor for obtaining color information of the items, wherein the first image sensor and the second image sensor are configured as one housing within the photographing device and are simultaneously triggered by one signal so that the photographing points of view are synchronized; and a processor for controlling the overall operation of the photographing device, wherein the processor controls the first image sensor to obtain barcode information of the items from a photographed image, controls the second image sensor to obtain color information of the items from the photographed image, and determines the state of the items or the location of the items based on the obtained barcode information of the items and the color information of the items.

[0010] In addition, in the present specification, the common field of view area is characterized in that it is formed as a common area of ​​a first shooting area captured by the first image sensor and a second shooting area captured by the second image sensor.

[0011] In addition, the processor in this specification is characterized by mapping barcode information of the product and color information of the product.

[0012] In addition, in the present specification, the processor is characterized in that it generates product status information by inserting barcode information of the product into a metadata area included in color information of the product, thereby mapping the barcode information of the product and the color information of the product.

[0013] In addition, the present specification further includes a communication unit for transmitting and receiving information with a server, and the processor is characterized in that it controls the communication unit to transmit the generated product status information to the server.

[0014] In addition, in the present specification, the processor is characterized in that it checks whether the first shooting area and the second shooting area match, and if the first shooting area and the second shooting area do not match, it synchronizes the trigger points of the first image sensor and the second image sensor.

[0015] In addition, the present specification is characterized in that the first image sensor is a mono sensor and the second image sensor is a color sensor.

[0016] In addition, the photographing device in this specification is characterized in that it further includes an LED module arranged on one side of the housing.

[0017] In addition, the present specification is characterized by including a step of photographing one or more articles moving through a conveyor belt in a common field of view (FOV) area formed by a first image sensor and a second image sensor whose photographing time points are synchronized by being simultaneously triggered by a single signal; a step of obtaining barcode information of the article by the first image sensor from the photographed image; a step of obtaining color information of the article by the second image sensor from the photographed image; and a step of confirming the state of the article or the position of the article based on the obtained barcode information of the article and the color information of the article.

[0018] This specification has the effect of enabling the location of an item to be identified, such as the condition of the item, whether it is damaged or stolen, by using two synchronized image sensors to obtain the barcode information and color information of the item, respectively.

[0019] In addition, this specification has the effect of being able to effectively use the size, volume, etc. of an item for logistics delivery by calculating the item's barcode information, item's color information, stereo function, etc.

[0020] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0021] The accompanying drawings, which are incorporated in and constitute a part of the detailed description to aid in the understanding of the present invention, provide embodiments of the present invention and, together with the detailed description, explain the technical features of the present invention.

[0022] Figure 1 is a diagram showing an example of a conceptual diagram of a logistics control system proposed in this specification.

[0023] Figure 2 shows an example of an internal block diagram of a photographing device proposed in this specification.

[0024] Figure 3 is a diagram showing an example of an internal block diagram of a camera module proposed in this specification.

[0025] Fig. 4 shows an example of a focus feedback screen of the photographing device proposed in this specification.

[0026] Fig. 5 shows an example of an LED setting screen of a photographing device proposed in this specification.

[0027] Fig. 6 shows an example of a sensor mode screen of the photographing device proposed in this specification.

[0028] Figure 7 shows an example of an internal block diagram of a server proposed in this specification.

[0029] Figure 8 is a block diagram of an AI device to which the method proposed in this specification can be applied.

[0030] FIG. 9 is a diagram showing an example of an operation method of a BCR open application for performing the method proposed in this specification.

[0031] Figure 10 shows an example of the BCR platform structure proposed in this specification.

[0032] Figure 11 is a flowchart showing an example of a method for performing logistics control proposed in this specification.

[0033] Fig. 12 is a diagram showing an example of a common field of view area of ​​a photographing device proposed in this specification.

[0034] Figure 13 is a diagram showing an example of barcode information of an article proposed in this specification.

[0035] Figure 14 is a flowchart showing another example of a method for performing logistics control proposed in this specification.

[0036] It should be noted that the technical terms used in this specification are merely used to describe specific embodiments and are not intended to limit the scope of the technology disclosed herein. Furthermore, unless specifically defined otherwise herein, the technical terms used herein should be interpreted as having a meaning generally understood by a person of ordinary skill in the art to which the technology disclosed herein pertains, and should not be interpreted in an excessively broad or narrow sense. Furthermore, if a technical term used herein is an incorrect technical term that does not accurately express the scope of the technology disclosed herein, it should be replaced with a technical term that can be correctly understood by a person of ordinary skill in the art to which the technology disclosed herein pertains. Furthermore, general terms used herein should be interpreted according to their dictionary definitions or according to the context, and should not be interpreted in an excessively narrow sense.

[0037] While terms including ordinal numbers, such as "first" and "second," used herein may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0038] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components are given the same reference numbers and redundant descriptions thereof will be omitted.

[0039] Additionally, when describing the technology disclosed in this specification, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the technology disclosed in this specification. Furthermore, it should be noted that the attached drawings are intended solely to facilitate understanding of the concepts of the technology disclosed in this specification and should not be construed as limiting the scope of the technology.

[0040] An image according to an embodiment of the present invention includes both still images and moving images unless there is a special limitation.

[0041]

[0042] *

[0043] Figure 1 is a diagram showing an example of a conceptual diagram of a logistics control system proposed in this specification.

[0044] The components illustrated in FIG. 1 are not essential for implementing a logistics control system (or logistics monitoring system), and thus the logistics control system described in this specification may have more or fewer components than the components listed above.

[0045] Referring to FIG. 1, the logistics control system (10) may include a photographing device (100) and a server (200).

[0046] The above logistics control system photographs an article moving through a logistics conveyor belt with a photographing device including a mono sensor and a color sensor, obtains barcode information of the article with the mono sensor, obtains color information of the article with the color sensor, analyzes the obtained barcode information and color information of the article to check the condition of the article, and transmits the analyzed result to a server to store and manage it on the server, thereby being able to check whether the article is damaged or stolen, and obtains metadata about features such as the size, volume, and pattern of the article.

[0047] The above-described photographing device and server can transmit and receive signals or information wired and / or wirelessly over a network.

[0048] The above-mentioned photographing device is a device that can implement a method for performing logistics control or logistics monitoring proposed in this specification, and may be a BCR (Barcode Reader) camera, a surveillance camera (or CCTV), an edge device, an AI camera, a network camera, etc., but this specification will explain using a BCR camera as an example, but is not limited thereto.

[0049]

[0050] Fig. 2 shows an example of an internal block diagram of a photographing device proposed in this specification.

[0051] The photographing device (100) may include a camera module (110), a wireless communication unit (120), a processor (130), a memory (140), an input unit (150), and an output unit (160).

[0052] The above camera module may be expressed as an image sensor and may be configured to include a mono sensor for mono photography and a color sensor for color photography. The mono sensor may be expressed as a first image sensor, and the color sensor may be expressed as a second image sensor.

[0053]

[0054] *Figure 3 is a diagram showing an example of an internal block diagram of a camera module proposed in this specification. Referring to Figure 3, it can be seen that the camera module (110) includes a mono sensor (111) and a color sensor (112) implemented as a single SoC.

[0055] The above mono sensor does not include a color filter and thus absorbs light of all wavelengths, while the above color sensor includes a color filter and thus absorbs light of a specific wavelength.

[0056] The wireless communication unit (120) can perform data communication with external devices (portable devices, servers, etc.) via a network. The wireless communication unit (120) can transmit captured images to an external camera system server or receive control commands from a user.

[0057] The wireless communication unit (120) may include wireless communication modules such as Bluetooth, Wireless Fidelity (Wi-Fi), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), Ultra-Wide Band Communication, Sub-1G, ZigBee, LoRa, etc.

[0058] The processor (130) typically controls the overall operation of the photographing device. The processor processes signals, data, information, etc. input or output through the components discussed above, or runs application programs stored in memory, thereby providing or processing appropriate information or functions to the user.

[0059] Additionally, the processor may control at least some of the components discussed with reference to FIG. 2 to drive an application program stored in memory. Furthermore, the processor may operate at least two or more of the components included in the photographing device in combination to drive the application program.

[0060] The memory (140) stores data supporting various functions of the photographing device. The memory can store a number of application programs (or applications) running on the photographing device, data for the operation of the photographing device, and commands. At least some of these application programs can be downloaded from an external server via wireless communication. Meanwhile, the application programs can be stored in the memory, installed on the photographing device, and driven by a processor to perform the operation (or function) of the photographing device.

[0061] The input unit (150) may include a user input unit (e.g., a touch key, a mechanical key, etc.) for receiving information from a user. The input unit may additionally include a camera or a video input unit for inputting a video signal, a microphone for inputting an audio signal, or an audio input unit. Voice data or image data collected by the input unit may be analyzed and processed into a user's control command.

[0062] The output unit (160) is for generating output related to visual, auditory, or tactile sensations, and may include a display unit, an audio output unit, etc.

[0063] At least some of the above components may operate cooperatively with each other to implement the operation, control, or control method of the photographing device according to various embodiments described below. In addition, the operation, control, or control method of the photographing device may be implemented on the photographing device by driving at least one application program stored in the memory.

[0064] In addition, the photographing device (100) (e.g., BCR camera) proposed in this specification has functions such as focus feedback, LED setup, and sensor mode, and can implement the method proposed in this specification through the Wise BCR OpenApp structure.

[0065] First, let's look at the focus feedback function. Figure 4 shows an example of the focus feedback screen of the photographing device proposed in this specification.

[0066] The above focus feedback function is a function that helps the BCR camera find the best focus by showing the current focus peak value with manual focus.

[0067] Referring to Fig. 4, when the Focus feedback tab displayed on the screen of the shooting device is activated by the user, the focus feedback function is executed, and when the refresh button is input or activated by the user, the MAX value can be initialized.

[0068] Next, we will examine the LED setup function. Fig. 5 shows an example of the LED setup screen of the photographing device proposed in this specification.

[0069] That is, the photographing device proposed in this specification further includes an LED module, and the LED module can be placed on one side of a housing that simultaneously includes a first image sensor and a second image sensor.

[0070] The above LED setup supports the LED lighting settings of the shooting device. Specifically, the LED settings support Enable, brightness, pre-charge time, and anti-blinking mode.

[0071] The above Enable function sets the LED turn on / off, the Brightness function sets the current PWM Duty of the LED, and the Pre-charge time function sets the Duty of the LED signal as a margin of several microseconds before / after the shutter cycle.

[0072] The above anti-blinking mode function, when enabled or activated, activates high frequency lighting to prevent the lighting from flickering.

[0073] Next, we will look at the sensor mode. Fig. 6 shows an example of the sensor mode screen of the photographing device proposed in this specification.

[0074] The above sensor mode is a mode for securing AI performance, and supports 15, 18, and 20 fps. When changing the sensor mode, reboot the shooting device to reset the input pipeline and re-run the WiseBCR App.

[0075]

[0076] Figure 7 shows an example of an internal block diagram of a server proposed in this specification.

[0077] The server (200) may be a device that controls the operation of the logistics control system proposed in this specification, and may include a communication unit (210), a memory (220), and a processor (230).

[0078] The above communication unit (210) may provide a function for communicating with an external device, such as a photographing device, via a network. For example, a request generated by the processor (230) of the server (200) according to a program code stored in a recording device, such as a memory (220), may be transmitted to an external device via a network under the control of the communication unit (210).

[0079] Conversely, control signals, commands, contents, files, etc. provided from an external device may be received by the server (200) through the communication unit (210) via a network. For example, control signals or commands from an external device received through the communication unit (210) may be transmitted to the processor (230) or memory (220).

[0080] The communication method is not limited, and may include not only a communication method that utilizes a communication network that the network may include (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network), but also short-range wireless communication between devices. For example, the network may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. In addition, the network may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

[0081] In addition, the server (200) may further include a user interface unit, although not shown. The user interface unit may be a means for interfacing with an input / output device. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the user interface unit may be a means for interfacing with a device that integrates input and output functions, such as a touch screen. As a more specific example, the processor (230) of the server (200) may display a service screen or content configured using data provided by an external device on a display through the user interface unit when processing commands of a computer program loaded in the memory (220).

[0082] The memory (220) is a computer-readable storage medium and may include a non-volatile memory (permanent mass storage device) such as a random access memory (RAM), a read-only memory (ROM), and a disk drive. In addition, program code for controlling a logistics control system may be temporarily or permanently stored in the memory (220).

[0083]

[0084] Figure 8 is a block diagram of an AI device to which the method proposed in this specification can be applied.

[0085] The AI ​​device (20) may be included as a component of at least a portion of the photographing device (100) or server (200) illustrated in FIG. 1 and may be provided to perform at least a portion of the AI ​​processing.

[0086] The AI ​​device (20) may include an AI processor (21), a memory (25), and / or a communication unit (27). If the AI ​​device is included in a photographing device, the communication unit (27) may be omitted.

[0087] The above AI device (20) is a computing device capable of learning a neural network, and can be implemented as various electronic devices such as a camera, server, desktop PC, notebook PC, tablet PC, etc.

[0088] The AI ​​processor (21) can learn a neural network using a program stored in the memory (25). In particular, the AI ​​processor (21) can learn a neural network for recognizing image-related data. Here, the neural network for recognizing image-related data can be designed to simulate the structure of the human brain on a computer, and can include a plurality of network nodes having weights that simulate neurons of the human neural network. The plurality of network modes can each exchange data according to a connection relationship so as to simulate the synaptic activity of neurons that exchange signals through synapses. Here, the neural network can include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes are located in different layers and can exchange data according to a convolution connection relationship. Examples of neural network models include various deep learning techniques such as deep neural networks (DNNs), convolutional deep neural networks (CNNs), recurrent Boltzmann machines (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), and deep Q-networks, which can be applied to fields such as computer vision, speech recognition, natural language processing, and speech / signal processing.

[0089] Meanwhile, the processor performing the functions described above may be a general-purpose processor (e.g., CPU), but may also be an AI-specific processor for artificial intelligence learning (e.g., GPU).

[0090] The memory (25) can store various programs and data required for the operation of the AI ​​device (20). The memory (25) can be implemented as a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), a solid state drive (SDD), etc. The memory (25) is accessed by the AI ​​processor (21), and data reading / recording / modifying / deleting / updating, etc. can be performed by the AI ​​processor (21). In addition, the memory (25) can store a neural network model (e.g., a deep learning model (26), a Re-ID model (28)) generated through a learning algorithm for image analysis according to one embodiment of the present invention.

[0091] Meanwhile, the AI ​​processor (21) may include a data learning unit (22) that learns a neural network for image analysis. The data learning unit (22) may learn criteria regarding which learning data to use for determining image analysis and how to classify and recognize data using the learning data. The data learning unit (22) may acquire learning data to be used for learning and apply the acquired learning data to the deep learning model, thereby learning the deep learning model.

[0092] The data learning unit (22) may be manufactured in the form of at least one hardware chip and mounted on the AI ​​device (20). For example, the data learning unit (22) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics processor (GPU) and mounted on the AI ​​device (20). In addition, the data learning unit (22) may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable medium that can be read by a computer. In this case, at least one software module may be provided by an operating system (OS) or an application.

[0093] The data learning unit (22) may include a learning data acquisition unit (23) and a model learning unit (24).

[0094] The learning data acquisition unit (23) can acquire learning data required for a neural network model for image analysis. For example, the learning data acquisition unit (23) can acquire image data and / or sample data for input into a neural network model as learning data.

[0095] The model learning unit (24) can use the acquired learning data to learn the neural network model to have a judgment criterion on how to classify a given data. At this time, the model learning unit (24) can train the neural network model through supervised learning that uses at least some of the learning data as a judgment criterion. Alternatively, the model learning unit (24) can train the neural network model through unsupervised learning that discovers a judgment criterion by learning on its own using the learning data without guidance. In addition, the model learning unit (24) can train the neural network model through reinforcement learning using feedback on whether the result of the situation judgment according to the learning is correct. In addition, the model learning unit (24) can train the neural network model using a learning algorithm including error back-propagation or gradient descent.

[0096] Once the neural network model is trained, the model training unit (24) can store the trained neural network model in memory. The model training unit (24) can also store the trained neural network model in the memory of a server connected to the AI ​​device (20) via a wired or wireless network.

[0097] The data learning unit (22) may further include a learning data preprocessing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or to save resources or time required for creating the recognition model.

[0098] The learning data preprocessing unit can preprocess the acquired data so that it can be used for learning to determine situations. For example, the learning data preprocessing unit can process the acquired data into a preset format so that the model learning unit (24) can utilize the acquired learning data for learning image recognition.

[0099] In addition, the learning data selection unit can select data required for learning from among the learning data acquired by the learning data acquisition unit (23) or the learning data preprocessed by the preprocessing unit. The selected learning data can be provided to the model learning unit (24). For example, the learning data selection unit can select only data for objects included in a specific area as learning data by detecting a specific area among images acquired through a camera.

[0100] Additionally, the data learning unit (22) may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

[0101] The model evaluation unit inputs evaluation data into the neural network model, and if the analysis results output from the evaluation data do not satisfy a predetermined standard, it can cause the model learning unit (22) to relearn. In this case, the evaluation data may be predefined data for evaluating the recognition model. For example, the model evaluation unit can evaluate that the predetermined standard is not satisfied if the number or ratio of evaluation data with inaccurate analysis results among the analysis results of the learned recognition model for the evaluation data exceeds a preset threshold.

[0102] The communication unit (27) can transmit the AI ​​processing result by the AI ​​processor (21) to an external device.

[0103] The AI ​​device (20) illustrated in Fig. 8 is functionally divided into an AI processor (21), a memory (25), a communication unit (27), etc., but it should be noted that the aforementioned components may be integrated into one module and referred to as an AI module.

[0104]

[0105] FIG. 9 is a diagram showing an example of an operation method of a BCR open application for performing the method proposed in this specification.

[0106] Referring to FIG. 9, a photographing device (e.g., a BCR camera) can communicate with a processor (or main server) of the photographing device, through a BCR open application that performs a method of checking the logistics control or goods status proposed in this specification, which is stored in a memory and called by a processor (or main server, 910).

[0107] Let's take a closer look at how the BCR Open application works.

[0108] The main server of the shooting device may refer to the processor of the camera, and the main server communicates with the open application (920) stored in the memory via OpenSDK (940).

[0109]

[0110] *And, the WNVA (Wise Net Video Analytics) library (921) of the Open application may be a core library for video (or video or image) analysis.

[0111] That is, WNVA analyzes the raw frame of the video to determine whether an event occurred and transmits the analysis results for the object (or objects) to the outside.

[0112] That is, WNVA can confirm and analyze whether barcode information and color information of an item have been obtained from a captured image through the first image sensor and the second image sensor, and transmit the analysis results to an external device.

[0113] The above Open application transmits settings and raw video frames to WNVA.

[0114] There are two types of metadata related to Open application operations in XML format: frame metadata and event metadata.

[0115] The above frame metadata and event metadata are generated based on the analysis results of each video raw frame from WNVA.

[0116] The above metadata is transmitted to the main server via Open SDK, and the main server transmits the metadata to an external device via RTSP (Real Time Streaming Protocol).

[0117] The web viewer uses frame metadata to overlay objects in the video (or image), and uses event metadata to display edges in the video when objects are detected.

[0118] Additionally, Event Status refers to information transmitted from WNVA when an object set by each AI function is detected.

[0119] The above Open application configures events received by WNVA into event metadata and transmits them to the main server of the shooting device using the OpenSDK API.

[0120] The web service (930) transmits and receives information between the open application and an external device (backend product) (300) such as a web viewer, NVR, or SSM (Smart Security Manager), and the main process handles GET / SET settings requested by the backend product.

[0121]

[0122] Next, we examine the operation of the OpenPlatform Manager. Figure 10 illustrates an example of the BCR platform structure proposed in this specification.

[0123] Referring to Figure 10, the WiseBCR platform can operate in the following ways: install / uninstall, run, and resource control.

[0124] First, install / uninstall is to install the application in the permitted part, run is to execute the application, and resource control is to check the usage of CPU and memory and determine whether the resource limit has been exceeded.

[0125]

[0126] Figure 11 is a flowchart showing an example of a method for performing logistics control proposed in this specification.

[0127] First, the photographing device photographs one or more items moving along a conveyor belt (S1110).

[0128] And, the photographing device obtains barcode information of the product through the first image sensor in a common field of view (FOV) area of ​​the first image sensor and the second image sensor included in the camera module of the photographing device, and obtains color information of the product through the second image sensor (S1120).

[0129] Depending on the arrangement of the first and second image sensors implemented in the above camera module, the common field of view area may be set differently. Fig. 12 is a diagram illustrating an example of the common field of view area of ​​the photographing device proposed in this specification.

[0130] FIG. 12a illustrates a common field of view area (1210) when the first image sensor and the second image sensor are arranged horizontally, and FIG. 12b illustrates a common field of view area (1210) when the first image sensor and the second image sensor are arranged vertically.

[0131] Figure 13 is a diagram showing an example of barcode information of an article proposed in this specification.

[0132] Referring to Figure 13, you can see that barcode information is recorded for each barcode.

[0133] The first image sensor may be a mono sensor that performs mono shooting, the second image sensor may be a color sensor that performs color shooting, and the shooting device may be a BCR (Barcode Reader) camera.

[0134] The first image sensor and the second image sensor are triggered simultaneously with a single signal, so that the shooting points are synchronized.

[0135] Here, the reason why color information of the product must be acquired through the second image sensor is because the condition of the product, such as damage to the product, cannot be properly identified using only the barcode information acquired through the first image sensor.

[0136]

[0137] *That is, if the color of the product appears blurry or is blurred in the color information of the product, it can be assumed that the product is damaged or destroyed.

[0138] In addition, the logistics control system proposed in this specification can also check information on the size and volume of goods (or cargo) by adding barcode information of goods, color information of goods, and the function of a stereo camera.

[0139] In addition, the logistics control system proposed in this specification can track the location of an item, etc., by additionally acquiring color information of the item, even for items whose barcode information is not recognized.

[0140] In addition, the photographing device proposed in this specification solves the problem of the angle of view captured by each sensor not matching when photographing an item moving on a conveyor belt by implementing the first image sensor and the second image sensor as one housing and one SoC (System on Chip) within the photographing device. That is, in the conventional case, by using two cameras with one sensor for each camera to photograph an item, a gap in the frames of the images captured by each camera occurred depending on the difference in the speed of the conveyor belt transporting the item, resulting in a problem of the barcode of the item not being properly recognized, and a problem of the image (color image) of the second image sensor corresponding to the recognized barcode not being properly matched even if the barcode of the item was recognized by the first image sensor. However, the logistics control system proposed in this specification solves the problem of the gap in frames occurring due to the speed of the conveyor belt by horizontally or vertically arranging two image sensors implemented as one SoC in one photographing device.

[0141] Here, the photographing device can analyze the barcode information and color information of the acquired item through the AI ​​device (or AI module) illustrated in FIG. 8 and the BCR Open Application operation procedure illustrated in FIG. 9, and provide the analysis results to an external device.

[0142] And, the photographing device maps the barcode information of the acquired item and the color information of the item (S1130).

[0143] The above photographing device generates product status information by inserting barcode information of the product acquired by the first image sensor into a metadata area included in (or related to) color information of the product acquired by the second image sensor.

[0144] Through this, the logistics control system can track the color image of the product as a timeline based on the product's barcode information (or barcode data).

[0145] And, the photographing device transmits the generated product status information to a server connected to the photographing device through a network (S1140).

[0146] The above server stores and manages the received item status information in a database (S1150).

[0147] That is, the photographing device or the server can check the condition of the article or track the location of the article based on the barcode information of the article and the color information of the article obtained.

[0148] Additionally, the photographing device confirms whether the first photographing area photographed by the first image sensor and the second photographing area photographed by the second image sensor match.

[0149] As a result of the above verification, if the first shooting area and the second shooting area do not match, the shooting device synchronizes the trigger points of the first image sensor and the second image sensor so that the first shooting area and the second shooting area match.

[0150] The shooting area used in this specification may also be referred to as Field of View (FOV).

[0151]

[0152] Figure 14 is a flowchart showing another example of a method for performing logistics control proposed in this specification.

[0153] First, the photographing device simultaneously triggers one or more items moving through a conveyor belt with a single signal and photographs them in a common field of view (FOV) area formed by a first image sensor and a second image sensor whose photographing points are synchronized (S1410).

[0154] And, the photographing device obtains barcode information of the product from the photographed image by the first image sensor (S1420).

[0155] And, the photographing device obtains color information of the product from the photographed image by the second image sensor (S1430).

[0156] Here, it is desirable that steps S1420 and S1430 be performed simultaneously, and each step is described separately for convenience of understanding and explanation.

[0157] And, the photographing device confirms the condition of the product or the location of the product based on the obtained barcode information of the product and the color information of the product (S1440).

[0158] Additionally, the photographing device can map barcode information of the article and color information of the article.

[0159] More specifically, the photographing device can map the barcode information of the article and the color information of the article by generating article status information by inserting the barcode information of the article into a metadata area included in the color information of the article.

[0160] And, the above-mentioned photographing device can transmit the generated item status information to the server.

[0161] In addition, the photographing device can check whether a first photographing area photographed by the first image sensor and a second photographing area photographed by the second image sensor match, and if the first photographing area and the second photographing area do not match, can synchronize the trigger points of the first image sensor and the second image sensor.

[0162]

[0163] The embodiments described above are combinations of components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, it is also possible to form an embodiment of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is self-evident that claims that do not have an explicit citation relationship in the patent claims may be combined to form an embodiment or may be incorporated as a new claim through a post-application amendment.

[0164] Embodiments of the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, an embodiment of the present invention may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0165] When implemented via firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, or the like that performs the functions or operations described above. The software code may be stored in memory and executed by a processor. The memory may be located within or external to the processor and may exchange data with the processor via various known means.

[0166] It will be apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential characteristics thereof. Therefore, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the scope of equivalents of the present invention are intended to be included within the scope of the present invention.

[0167] The method for performing product control of the present invention has been described with a focus on an example applied to a video camera system, but it can also be applied to various other video systems.

Claims

1. In a photographing device that performs goods control, A camera module that photographs one or more items moving on a conveyor belt in a common field of view (FOV) area. The above camera module includes a first image sensor that obtains barcode information of an item and a second image sensor that obtains color information of the item. The first image sensor and the second image sensor are configured as one housing within the photographing device, and are simultaneously triggered by one signal so that the photographing time is synchronized; and A processor for controlling the overall operation of the above photographing device, wherein the processor comprises: A photographing device characterized in that it controls the first image sensor to obtain barcode information of the product from a photographed image, controls the second image sensor to obtain color information of the product from the photographed image, and confirms the state of the product or the location of the product based on the obtained barcode information of the product and the color information of the product.

2. In paragraph 1, A photographing device characterized in that the common field of view area is formed as a common area of ​​a first photographing area photographed by the first image sensor and a second photographing area photographed by the second image sensor.

3. In the second paragraph, the processor, A photographing device characterized by mapping barcode information of the above product and color information of the above product.

4. In the third paragraph, the processor, A photographing device characterized in that it generates product status information by inserting barcode information of the product into a metadata area included in color information of the product, thereby mapping the barcode information of the product and the color information of the product.

5. In paragraph 3, It further includes a communication unit for transmitting and receiving information with the server, and the processor, A photographing device characterized in that it controls the communication unit to transmit the generated product status information to a server.

6. In the second paragraph, the processor, A photographing device characterized in that it checks whether the first photographing area and the second photographing area match, and if the first photographing area and the second photographing area do not match, it synchronizes the trigger points of the first image sensor and the second image sensor.

7. In paragraph 1, A photographing device, characterized in that the first image sensor is a mono sensor and the second image sensor is a color sensor.

8. In paragraph 1, A photographing device characterized in that it further includes an LED module arranged on one side of the housing.

9. In the method of performing goods control, A step of photographing one or more items moving through a conveyor belt in a common field of view (FOV) area formed by a first image sensor and a second image sensor whose photographing points are synchronized by being simultaneously triggered by a single signal; A step of acquiring barcode information of an item from a captured image by the first image sensor; A step of acquiring color information of an object from the above-described captured image by the second image sensor; and A method characterized by comprising a step of confirming the condition of the article or the location of the article based on the barcode information of the article obtained and the color information of the article.

Images