System and method for supervising events in an environment

Abstract

The present invention relates to a system and method for supervising events in an environment. More particularly, said system comprises a plurality of image capture devices (15) associated with a data acquisition subsystem (11), said image capture devices (15) being capable of capturing, over a period of time, respective images comprising respective metadata, wherein, from the processing of said metadata, a data analysis subsystem (13), by means of a processing algorithm, recognizes and identifies objects of interest and tracks them within said environment, so as to generate at least one indicator (140), in a user interface subsystem (14), representing the distribution and / or intensity of movement of said objects of interest within the environment. The method executed by said system is also claimed.

Description

SYSTEM AND METHOD FOR EVENT SUPERVISION IN AN ENVIRONMENT Field of Invention

[0001] The present invention relates to a system and method for monitoring events in an environment. More particularly, the present invention relates to a system and method that allows the monitoring of events, activities, occurrences, or, in other words, the distribution and / or intensity of movements in a real environment, such as a logistics warehouse. Fundamentals of the Invention

[0002] Logistics warehouses are environments with intense movement of people and equipment transporting cargo, essential for maintaining the flow of delivery and receipt of materials within the established times and according to the production or storage order.

[0003] The growing demand for a wide variety of products and the need to meet those demands with greater speed and efficiency is making the supply chain increasingly complex. One of the stages in the supply chain is the warehousing of inventory items. This stage presents numerous challenges, such as optimizing physical storage space and ensuring the speed in locating and dispatching stored items.

[0004] To address these challenges, warehouse managers need accurate information about stored items, such as location, quantity, frequency of item requests, and storage time, as well as the distribution and intensity of movements and / or processes. This information is used for inventory control and optimization of warehouse physical space.

[0005] This is because, when an item is requested, employees or equipment are activated to retrieve it. If a stored item is not located in the designated position, the entire inventory needs to be reviewed. Furthermore, if a frequently requested item is located far from the dispatch location and a rarely requested item is located near the dispatch location, the efficiency of the supply chain can be considerably impaired.

[0006] In view of this and the need for digitalization of solutions for the logistics chain, some warehouse management systems ("Warehouse Management System" or "WMS") are already available. These systems aim to monitor the flow of stored items to allow for their relocation within the warehouse. Such systems use technologies like barcodes or radio frequency identification (RFID) to collect data. This data is stored in databases and subsequently used to generate reports or maps containing information such as position and handling frequency, which assist in the relocation of items. This data collection can be highly dependent on human interaction, which can also lead to errors.

[0007] To correctly relocate items, users of warehouse management systems utilize these reports and verify information such as handling frequency and the most suitable location for stored items. If the relocation of stored items is done efficiently, the time spent retrieving the requested items can be significantly reduced in the long term, generating cost reductions and increasing the speed of receiving and dispatching items. However, the reports generated by state-of-the-art warehouse management systems lack dynamism and, eventually, precision regarding the information collected.

[0008] Due to the high number of items stored and the constant variation in the quantity of items, the lack of dynamism can result in the provision of outdated values, due to variations during the review of the positions of the stored items. An example of a prior art solution that already aimed to solve this type of problem is described in document BR102019013587.

[0009] Although the solution in that document already includes camera monitoring, it has not yet reached the full potential of vision systems. This is because camera monitoring systems, such as the one in question, generate a lot of data, and some of this data is not used for deeper mathematical analysis.

[0010] More specifically, the present invention aims to provide an agile, dynamic, precise, and automated solution that requires relatively little computational processing power and is also capable of monitoring events, such as movements that occur in the real environment, i.e., in the warehouse itself, for comparison with information contained in the virtual environment, in the warehouse management system, for example. This makes it possible to check whether access to certain areas or aisles of the logistics warehouse is actually necessary, or whether aisles further from the packing station are being accessed too frequently, causing a loss of productivity due to travel time and, therefore, requiring a reorganization of stored items.

[0011] In short, the current state of the art lacks a solution that allows for the monitoring of events in a real-world environment, such as a logistics warehouse, in order to enable its management in an agile, dynamic, precise, and automated manner, and therefore with less dependence on human interaction.

[0012] It is based on this scenario that the present invention arises. Objectives of the Invention

[0013] Thus, the fundamental objective of the invention in question is to disclose a system and a method for managing a warehouse;

[0014] The objective of the present invention is to provide a solution that allows for the monitoring of activities, occurrences, or movements in the real warehouse environment;

[0015] More specifically, the objective of the present invention is to provide a method and system that enable the management of a logistics warehouse in an agile, dynamic, precise, and automated manner, and therefore with little dependence on human interaction.

[0016] Another objective of the present invention is to provide a method and system that facilitates access to relevant information regarding warehouse activities in a dynamic way and, preferably, in real time. Summary of the Invention

[0017] The objectives summarized above are achieved through a system for monitoring events in an environment, said system comprising a plurality of image capture devices associated with a data acquisition subsystem, said image capture devices being capable of capturing respective images and metadata over a period of time, and from the processing of said metadata, a data analysis subsystem, through a processing algorithm, recognizes and identifies objects of interest and tracks them within said environment, in order to generate at least one indicator, in a user interface subsystem, that represents the distribution and / or intensity of movement of these objects of interest within the environment.

[0018] The present invention also relates to a method for monitoring events in an environment, said method including, among other possible steps: a first step consisting of capturing, by means of a plurality of image capture devices, a plurality of respective images of respective regions of the environment, said images comprising respective metadata; a second step consisting of, by means of a data analysis subsystem, processing, that is, recognizing, identifying and tracking objects of interest by means of the respective metadata of the images; and a third step consisting of, from the processing of the metadata, generating at least one indicator and displaying it in a user interface subsystem, said at least one indicator representing the distribution and / or intensity of movement of these objects of interest within the environment. Brief Description of the Drawings

[0019] The invention in question will now be described in detail, by way of example, based on the illustrative figures listed below:

[0020] [Fig. 1] schematically illustrates the system of the present invention and its subsystems;

[0021] Figure 2 illustrates an indicator, which can be generated according to the present invention, in the form of a heat map;

[0022] [Fig. 3] illustrates an indicator, which can be generated according to the present invention, in the form of a spaghetti map;

[0023] Figure 4 illustrates an indicator, which can be generated according to the present invention, in the form of an address access map;

[0024] Figure 5 illustrates a flowchart of the method of the present invention. Detailed Description of the Invention

[0025] Thus, focusing on achieving the aforementioned objectives, and describing in an exemplary, but not limiting, manner, the present invention relates to a system for monitoring events in an environment.

[0026] In the context of the present invention, the term "environment" refers to any and all environments capable of being used as a storage facility for items, including, but not limited to, warehouses, sheds, distribution centers, self-storage facilities, bonded warehouses, industrial depots, and backup stock.

[0027] The system10 in question comprises at least one data acquisition subsystem11 associated with, i.e., in data communication with, at least one data storage subsystem12 associated with, i.e., in data communication with, at least one data analysis subsystem13 and at least one user interface subsystem14 that is also associated with, i.e., is also in data communication with, the data storage subsystem12, the architecture in question being detailed below.

[0028] The data acquisition subsystem11 is associated with, or rather, communicates with, image capture devices15, such as cameras, installed in the environment in question, for example, a logistics warehouse, and may also include a communication module with other systems, for example, a warehouse management system (WMS). This data acquisition subsystem11 is responsible for integrating the data sources (i.e., image capture devices15, for example) and the data storage subsystem12.

[0029] The data storage subsystem12 comprises the formatting and provision of data to points of consumption, such as the data analysis subsystem13 and the machine learning module131, for example. The data storage subsystem12 comprises: a first data storage module, namely a Data Lake121 that stores the unstructured raw data sent by the data acquisition subsystem11, which is filtered and transformed and sent to a second data storage module, namely a Data Warehouse122 that then stores structured data to be read and processed by the data analysis subsystem13.Structured data, that is, metadata, stored in the Data Warehouse122 contains information about the image capture device15 that originated the images containing the respective metadata, about the identification of the object of interest, including characteristics about the generated contour, such as: area, center of gravity, bounding box limits, vector of points that originated the contour, among other possibilities. Finally, the data storage subsystem12 also comprises a third data storage module, namely, a Data Mart123 that stores processed data, that is, data that has already been manipulated and / or treated and / or interpreted by the data analysis subsystem13. The Data Mart123 can be accessed by the user interface subsystem14 for data consumption and display of at least one indicator140.

[0030] Furthermore, the structured data stored in the Data Warehouse122 can be consumed, that is, used, for the purpose of training processing algorithms that preferably include machine learning by a machine learning module131.

[0031] The data analysis subsystem13 comprises the analysis of structured data for information discovery and the processing of this structured data to generate at least one indicator140, using at least one processing algorithm. More specifically, the data analysis subsystem13 reads the structured data stored in the Data Warehouse122, executes at least one processing algorithm, preferably comprising artificial intelligence, and generates at least one indicator140 and writes it to the Data Mart123.

[0032] The user interface subsystem14 comprises the provision of the generated data, by means of at least one indicator140 such as graph(s), for example, heat map(s)141, spaghetti map(s)142 or address access map(s)143. More specifically, said user interface subsystem14 can be, for example, a web interface including a back-end and a front-end, wherein the back-end receives requests from the front-end, queries the requested data in the Data Mart123 and returns it to the front-end which displays information to the user in the form of indicators140, as mentioned above.

[0033] This user interface subsystem14 may comprise one or a plurality of indicator screens140, such as: heat map screen141, spaghetti map screen142, address access map screen143, among others. Alternatively, the screens and the information contained in them could be combined with each other so as to be displayed on a single screen, for example. Other combinations are also possible.

[0034] In general terms, these indicators140 represent the distribution and / or intensity of movement of objects of interest within the environment in question.

[0035] In other words, the heat map141 represents the frequency or concentration of events, for example, movements or processes occurring in the environment in question, for example, in the streets and corridors of a logistics warehouse, using a color scale to distinguish the different magnitudes.

[0036] The spaghetti map142, like the heat map141, also represents events, for example, movements occurring in the environment in question; however, in said spaghetti map142, lines are provided to illustrate the paths taken by respective objects.

[0037] Preferably, in both, it is possible to adjust the zoom140a, apply filters by period140c, filter by location140d, filter by cameras140e, filter by process140f, filter by object140g and filter by group140h (street, corridor, etc.). The process and object filters are data derived from the inference of a metadata classification algorithm, which will be described below, so they are susceptible to the inherent error of the algorithm. In the heat map141 it is also possible to apply filters related to the frequency of events140b.

[0038] The address access map143 represents or illustrates the number of accesses to specific locations in the environment, for example, in the case of a logistics warehouse, a specific aisle, shelf, and shelf position. In this case, it is also possible to adjust the zoom140ae, and, preferably, filters by period140c, resource140i, process140f, and shelf level140j are provided. Examples of resources140i can be the machines or equipment used to perform an action / event within the environment in question, such as a pallet jack or forklift, for example, in the case of a logistics warehouse.

[0039] Having described the system of the present invention, the computer-implemented method for monitoring events in an environment, executed by the system in question, is detailed below.

[0040] In this sense, the present invention also relates to a method for monitoring events in an environment, including the following steps:

[0041] A first stageS1 which consists of capturing, by means of a plurality of image capture devices15 arranged in respective geographic coordinates of the environment, a plurality of respective images of respective regions of the environment, said images comprising respective metadata;

[0042] A second stageS2 which consists of, through a data analysis subsystem13, processing, that is, recognizing, identifying and tracking objects of interest through the respective metadata of the images; and

[0043] A third stepS3 which consists of, from the processing of the metadata, generating at least one indicator140 and displaying it in a user interface subsystem14, said at least one indicator140 representing the distribution and / or intensity of movement of these objects of interest within the environment.

[0044] The method of the present invention may also preferably include a first sub-step S11, subsequent to the first step and prior to the second step, which consists of pre-processing said captured images in a data acquisition subsystem 11; as well as a second sub-step S12, subsequent to the first sub-step S11 and prior to the second step S2, which consists of storing respective pre-processed images in a data storage subsystem 12, more specifically, in the Data Warehouse 122.

[0045] More specifically, the preprocessing of the first sub-stageS11 consists of correlating geographic coordinates of each image capture device15 with an absolute reference point of a floor plan map of the environment. In this sense, it is possible to use the information about the center of the metadata to represent them in the environment's reference point, in order to represent the position of the object's center in relation to the image capture device15 that generated it.

[0046] Furthermore, the preprocessing of the first sub-stageS11 also consists of adjusting all the images to the same orientation and flattening them, using an image preprocessing algorithm embedded in a data acquisition subsystem11.

[0047] More specifically, it should be noted that adjusting the images to the same orientation involves rotating said images around their center. It is also important to highlight that flattening is done to remove distortions that may be generated by the lenses of the respective image capture devices.15 This pre-processing is fundamental for the captured images to correspond to the scale of the floor plan of the environment in question. Thus, it becomes possible, in the flattened image, to measure the distance between the geographic coordinates of the respective image capture device15 and any object or point of interest captured in said image.

[0048] Furthermore, said second stageS2 consists, more specifically, through a processing algorithm, in the data analysis subsystem13, of processing the metadata of the pre-processed images in order to identify and classify respective objects of interest, such as forklifts, pallet trucks, people, pallets, among others, by analyzing the variation of metadata of groups of pre-processed images, said groups being formed by a sequence of pre-processed images, said sequence representing a predetermined period of time, and said metadata being related to the area, speed and quantity of points that form the outline of respective objects of interest.

[0049] It is worth noting that, since the number of occurrences of each object of interest is variable, the time period predetermined by the user defines a fixed-size window of time instants to be considered by the neural network. In this way, a mask is created to handle objects that have a smaller number of occurrences than the defined size, which allows the artificial intelligence model to ignore time instants in which the object in question did not exist.

[0050] In other words, the time window size can be defined by analyzing the distribution of the number of frames per object, where all objects present in a frame are identified over a period of time and, after analyzing where the distribution of these occurrences is concentrated, the time window size is defined based on the time range in which the highest percentage of objects per frame are concentrated.

[0051] Preferably, this method also includes a third sub-step S23, subsequent to the second step S2 and prior to the third step S3, which consists of storing the processed metadata in the data storage subsystem12, more specifically, in the Data Mart123.

[0052] Finally, it should be noted that said at least one indicator140 is at least one of the following: heat map141, spaghetti map142, address access map143.

[0053] It is important to emphasize that the description above serves solely to illustrate a particular embodiment of the invention in question. Therefore, it is clear that modifications, variations, and constructive combinations of the elements that perform the same function in substantially the same way to achieve the same results remain within the scope of protection delimited by the appended claims.

Claims

System for monitoring events in an environment, said system comprising a plurality of image capture devices (15) and associated with a data acquisition subsystem (11), said image capture devices (15) being capable of, over a period of time, capturing respective images comprising respective metadata, said system being CHARACTERIZED by the fact that, from the processing of said metadata, a data analysis subsystem (13), by means of a processing algorithm, recognizes and identifies objects of interest and tracks them within said environment, in order to generate at least one indicator (140), in a user interface subsystem (14), which represents the distribution and / or intensity of movement of these objects of interest within the environment. System, according to claim 1, CHARACTERIZED by the fact that said metadata relates to the area and / or speed and / or quantity of points that form the outline of respective objects of interest. System, according to claim 1, CHARACTERIZED in that it further includes a data storage subsystem (12) that is associated with the data acquisition subsystem (11) and the data analysis subsystem (13) and the user interface subsystem (14). System, according to claim 3, CHARACTERIZED in that said data storage subsystem (12) comprises a Data Lake (121), a Data Warehouse (122) and a Data Mart (123), wherein said Data Lake (121) stores unstructured raw data received by the data acquisition subsystem (11), said Data Warehouse (122) stores structured data to be read and processed by the data acquisition subsystem (13), while said Data Mart (123) stores the data processed by the data acquisition subsystem (13). System according to claim 1, CHARACTERIZED in that the data analysis subsystem (13) comprises a machine learning module (131). System, according to claim 1, CHARACTERIZED in that the processing algorithm comprises artificial intelligence. System, according to claim 1, CHARACTERIZED in that said at least one indicator (140) is at least one of: heat map (141), spaghetti map (142), address access map (143). Method for monitoring events in an environment, said method including: a first stage (S1) which consists of capturing, by means of a plurality of image capture devices (15), a plurality of respective images of respective regions of the environment, said images comprising respective metadata, said method being CHARACTERIZED by the fact that it also includes: a second stage (S2) which consists of, by means of a data analysis subsystem (13), processing, i.e., recognizing, identifying and tracking objects of interest by means of the respective metadata of the images; and a third stage (S3) which consists of, from the processing of the metadata, generating at least one indicator (140) and displaying it in a user interface subsystem (14), said at least one indicator (140) representing the distribution and / or intensity of movement of these objects of interest within the environment. Method, according to claim 8, CHARACTERIZED by the fact that it further includes: a first sub-step (S11), subsequent to the first step and prior to the second step, which consists of pre-processing said captured images; a second sub-step (S12), subsequent to the first sub-step (S11) and prior to the second step (S2), which consists of storing respective pre-processed images in a data storage subsystem (12). Method, according to claim 9, CHARACTERIZED in that the preprocessing of the first sub-step (S11) consists of correlating geographic coordinates of each image capture device (15) with an absolute reference of a floor plan map of the environment. Method, according to claim 9, CHARACTERIZED in that the preprocessing of the first sub-step (S11) consists of adjusting all images to the same orientation and flattening them, by means of an image preprocessing algorithm embedded in a data acquisition subsystem11. Inventors, please confirm. A method according to claim 10, characterized in that adjusting the images to the same orientation consists of rotating said images around their center. The method, according to claim 9, is characterized in that: the second step (S2) consists, more specifically, by means of a processing algorithm, of processing the metadata of pre-processed images in order to identify and classify respective objects of interest, by means of the analysis of the metadata variation of groups of pre-processed images, said groups being formed by a sequence of pre-processed images, said sequence representing a predetermined period of time, and said metadata being related to the area, speed and quantity of points that form the contour of respective objects of interest. Method, according to claim 10, CHARACTERIZED in that it further includes: a third sub-step (S23), subsequent to the second step (S2) and prior to the third step (S3), which consists of storing the processed metadata in the data storage subsystem (12). Method, according to claim 8, CHARACTERIZED in that said at least one indicator (140) is at least one of: heat map (141), spaghetti map (142), address access map (143).

Images