Method for training a neural network, method and system for management and prediction of operation times in an environment, and corresponding computer-readable memory

Abstract

The present invention relates to a method for training a neural network (420) capable of managing and predicting operation times in an environment, comprising the steps of: I. Importing data from a WMS (220) relating to a time period predetermined by a user; II. Importing metadata (211) from a plurality of cameras (210) relating to the time period predetermined by the user; III. Labelling the metadata (211) imported from the cameras (220) of the previous step; IV. Repeating the preceding steps for a given period of time, in order to create a prediction of the execution time of each of the processes of interest; optionally, V. Monitoring the processes of interest in real time and comparing the predicted execution times of each of the processes of interest with the real time of each of the processes of interest; and, optionally, VI. Adjusting the predicted execution times of each of the processes of interest based on the real time of each of the processes of interest. The present invention further relates to a method for management and prediction of operation times in an environment, to a computer-readable memory and to a system (100) for management and prediction of operation times in an environment.

Description

METHOD FOR TRAINING A NEURAL NETWORK, METHOD AND SYSTEM FOR MANAGING AND PREDICTING OPERATIONS IN AN ENVIRONMENT AND MEMORY READ BY A CORRESPONDING COMPUTER. Technical Field

[0001] The present invention belongs to the field of processing operations, more specifically, to the field of storing articles, individually or in an orderly arrangement, in warehouses or depots. The present invention also belongs to the field of data processing systems or methods, especially adapted for administrative, management, supervision or predictive purposes. Introduction

[0002] The present invention relates to a method for training a neural network, a method and system for managing and predicting the timing of operations in an environment, and corresponding computer-readable memory. More particularly, the present invention relates to a method for training a neural network capable of managing and predicting the timing of operations in an environment, and to a method and system for managing and predicting the timing of operations in an environment and corresponding computer-readable memory, taking into account a real-world environment, such as a logistics warehouse. Fundamentals

[0003] Logistics warehouses are environments with intense movement of people and equipment transporting cargo, essential for maintaining the flow of delivery and receipt of materials within the established times and according to the production or storage order.

[0004] The growing demand for a wide variety of products and the need to meet those demands with greater speed and efficiency are making the supply chain increasingly complex. One of the stages in the logistics chain is the storage of inventory items. This stage presents numerous challenges, such as optimizing physical storage space and ensuring speed in locating and dispatching stored items.

[0005] To address these challenges, warehouse managers need accurate information about each of the processes involved in warehouse management, such as receiving, picking, packing, inventory, quality control, reduction, and shipping of stored items. This information is used to control processes and optimize the time spent on warehouse operations.

[0006] This is because, during the execution of each of the processes involved in warehouse management, it is necessary to have employees and equipment so that each specific process is carried out correctly. Thus, it is to be expected that any errors resulting from poorly calculated estimates for carrying out any of the processes involved in warehouse management can result in a series of operational and financial problems, since delays in these stages affect the entire workflow of the logistics chain, generating losses due to additional costs, rework, increased service time, and a greater possibility of errors.

[0007] In view of this and the need for digitalization of solutions for the logistics chain, some warehouse management systems, or "Warehouse Management Systems" (WMS), are already available. These systems aim to monitor the flow of stored items to allow for the relocation of items within the warehouse if one or more items are moved. Such systems use technologies such as barcodes or radio frequency identification (RFID) to collect data. This data is stored in databases and subsequently used to generate reports or maps containing information such as position and handling frequency, which assist in the relocation of items. This data collection can be highly dependent on human interaction, which can also lead to errors.

[0008] However, despite being fundamental for monitoring the flow of stored items, these warehouse management systems do not aim to predict or estimate the time it takes for each of the processes involved in managing a warehouse, which can lead to the operational and financial problems mentioned above.

[0009] Due to the high number of processes involved in warehouse management and the increasing demand generated by the large volume of items traded today, it is essential to implement a system capable of estimating, with a high degree of certainty, the time required to carry out each of these processes, in order to reduce worker idle time and increase productivity, as well as predict the need for shift extensions in advance. State of the art

[0010] Known state-of-the-art solutions for weather forecasting systems of the nature discussed here can be verified in state-of-the-art documents such as JP 2008168989, entitled "Automated Warehouse Work Completion Time Forecasting Device," which refers to a management controller that extracts current position data, which extracts a plurality of data relating to the transport of objects in a warehouse, where this data includes the current position data of these objects, transport destination data, transport performance data including transport origin data and transport destination data, from a transport performance database. After data collection, the management controller calculates the average work time by averaging the transport work time included in the extracted transport performance data.The management controller then reads an increase / decrease parameter corresponding to the transport condition when the transport job instruction data is executed and calculates the job completion forecast time by correcting the average job time by the read increase / decrease parameter.

[0011] Although document JP 2008168989 teaches about a controller capable of providing a job completion time forecast focused on the transportation of goods in the warehouse, this solution is limited to managing and predicting the transportation time of cargo from one point to another, without addressing the time spent on other logistical processes in a warehouse.

[0012] Another solution can be found in the prior art document JP 2023005770, entitled “Picking operation management device”, which presents a warehouse management system, a picking operation management device, an internal warehouse terminal and a display device that can communicate with each other via a network, wherein the picking operation management device comprises a control device, which acquires information showing an operating instruction, predicts the transit time when the picking operation is performed from the information showing the operating instruction,It predicts the post-transit operation time required for a post-transit operation to be performed by an operator after transit to a location and calculates the target time for the picking operation based on the transit time when the picking operation is performed and the post-transit operation time required for the post-transit operation to be performed by the operator.

[0013] Although document JP 2023005770 teaches about a system for controlling and managing the average time of picking operations in the warehouse, the average time controlled and managed by this solution does not take into account the other steps involved in the logistics process of a warehouse, such as receiving loads, lowering loads from warehouse shelves, and other processes.

[0014] Thus, the state of the art does not offer reliable, versatile, and low-cost solutions for a weather forecasting system capable of predicting the operating time of each operation performed in a given environment. Objectives of the invention

[0015] The objective of the invention is, therefore, to provide a method for training a neural network capable of managing and predicting the timing of operations in an environment, according to the characteristics of claim 1 of the attached claims.

[0016] Another objective of the present invention is, therefore, to provide a method for managing and forecasting the timing of operations in an environment, in accordance with the characteristics of claim 10 of the attached claims.

[0017] Another objective of the present invention is, therefore, to provide a computer-readable memory, in accordance with the characteristics of claim 11 of the attached claims.

[0018] Yet another objective of the present invention is, therefore, to provide a system for managing and forecasting the timing of operations in an environment, in accordance with the characteristics of claim 12 of the attached claims.

[0019] Other features and details of the features are represented by the dependent claims. Description of the figures

[0020] For a better understanding and visualization of the object of the present invention, it will now be described with reference to the attached figures, representing the technical effect obtained through an exemplary embodiment that is not limiting the scope of the present invention, in which, schematically:

[0021] [Fig. 1]: presents a diagram of a system for managing and forecasting the timing of operations in an environment, according to the present invention. Detailed description of the invention

[0022] The following detailed description refers to the accompanying drawings in which embodiments of the present invention are represented, by way of non-limiting illustration. These embodiments are described in such a way as to allow a person skilled in the art to reproduce their results. Other embodiments resulting from structural, mechanical, logical, electrical and electronic changes are possible and can be carried out without departing from the spirit and scope of the present invention. The following detailed description should therefore not be understood in a restrictive or limiting manner.

[0023] The present invention relates to a method and a system (100) for managing and predicting the timing of operations in an environment and to a corresponding computer read memory. In addition, the present invention relates to a method for training a neural network (420) for managing and predicting the timing of operations in an environment.

[0024] In the context of the present invention, the term "operations" refers to any and all operations that can be performed in a warehouse, including, but not limited to: receiving, picking, packing, inventory, quality control, reduction, and shipping of stored items.

[0025] In the context of the present invention, the term "environment" refers to any and all environments capable of being used as a storage facility for items, including, but not limited to, warehouses, sheds, distribution centers, self-storage facilities, bonded warehouses, industrial depots, and backup stocks.

[0026] In the context of the present invention, the term “neural network” refers to a machine learning model that may be a convolutional neural network, wherein the term “train” refers to adjusting the parameters of the machine learning model so that, from a number of reference images associated with the metadata of a plurality of cameras (210) installed to observe the processes of an environment, it is able to predict the execution time of each of the operations of interest of an environment.

[0027] In this same sense, preferably, the neural network (420) of the present invention has a recurrent topology (“Recurrent Neural Network” – “RNN”).

[0028] Method for training a neural network (420) capable of managing and predicting the timing of operations in an environment

[0029] The method for training a neural network (420) capable of managing and predicting the time of operations in an environment basically comprises the following steps: Importing data from a WMS (220) relating to a time period predetermined by a user; Importing metadata (211) from a plurality of cameras (210) relating to the time period predetermined by the user; Labeling the metadata (211) imported from the cameras (220) in the previous step; Repeating the previous steps for a given time period, in order to create a prediction of the execution time of each of the processes of interest; optionally Monitoring the processes of interest in real time and comparing the predicted execution times of each of the processes of interest with the actual time of each of the processes of interest; and, optionally Adjusting the predicted execution times of each of the processes of interest based on the actual time of each of the processes of interest.

[0030] Regarding stage I, WMS(220) data is imported from a Data Lake(310) to a Data Warehouse(320). Additionally, the data imported from WMS(220) includes information about received items, picking, packing, environment inventory, quality of received items, item demotion, and shipment of stored items within a time interval predetermined by a user.

[0031] Regarding stage II, the metadata (211) from the cameras (210) will be imported into a Data Lake (310). Additionally, the cameras (210) must be positioned in such a way as to allow unambiguous monitoring of each of the user's tasks of interest. Furthermore, the time period pre-determined by the user must encompass the beginning and end of a given process of interest.

[0032] It is worth noting that, since the number of occurrences of each object is variable, the time period predetermined by the user defines a fixed-size window of time instants to be considered by the neural network (420). In this way, a mask is created to handle objects that have a number of occurrences smaller than the defined size, which allows the model to ignore time instants in which the object did not exist.

[0033] In other words, the time window size can be defined by analyzing the distribution of the number of frames per object, where all objects present in a frame are identified over a period of time and, after analyzing where the distribution of these occurrences is concentrated, the time window size is defined based on the time range in which the highest percentage of objects per frame are concentrated.

[0034] In a non-limiting example of the present invention, an exemplary database provides objects ranging from 1 to 2,624 occurrences over a total time window. However, in this exemplary situation, 75% of the data are between 10 and 200 occurrences. Therefore, the ideal time window considering this example is between 10 and 200 occurrences, since it concentrates the highest percentage of data from the objects of interest to the neural network (420). Furthermore, objects that presented a number of occurrences outside the range between 10 and 200 occurrences were considered outliers and were removed from the database.

[0035] In the context of the present invention, the start of the timing of the receiving process is counted by the neural network (420) from the moment the trailer / box / grain carrier / sider of the transport vehicle is opened and ends when the trailer / box / grain carrier / sider of the transport vehicle is closed.

[0036] After accounting for the start and end times of the receiving process, the neural network (420) calculates the average unloading time based on the type of transport vehicle at the dock. That is, the transport vehicle data is grouped and categorized based on its size, ensuring that the receiving process is accurately predicted based on the expected load size that a transport vehicle is capable of carrying.

[0037] It is worth noting that it is also possible to calculate the average time for checking items received by the truck from the WMS records.

[0038] In the context of the present invention, the start of the time-taking of the picking process is accounted for by the neural network (420) at the moment of task acceptance and is finalized at the moment of delivery of the item of interest to the location of interest. Preferably, the picking process is automatically finalized at the moment of delivery, when an item is deposited on the depacking bench.

[0039] In the context of the present invention, the start of the depacking process time is accounted for by the neural network (420) at the moment the item is deposited on the depacking bench, and is finalized at the moment the item is packaged.

[0040] More specifically, one or more depacking orders are automatically created based on the creation of a depicking order. That is, depicking orders are identified in the database and, after being identified, one or more depacking orders are created in response to each depicking order.

[0041] Therefore, to calculate the duration of the depacking processes for items from these identified picking orders, the durations of the individual packing orders are added together to obtain the linear duration of this process. In other words, the time that a single operator would take to perform the sequential packing of all products from the same picking order is calculated.

[0042] Additionally, besides the execution time of the depacking process, the neural network (420) also takes into account information such as item volume, item mass, number of items and number of tasks.

[0043] It should be noted that information regarding item volume, item mass, item quantity, and task quantity is recorded only during the execution of the depacking process. In other words, this information is not available in the system (100) for management and forecasting of operation times in an environment in advance, making it impossible to use it for calculating the predicted duration of the depacking process.

[0044] Thus, in order to account for information about item volume, item mass, number of items and number of tasks, the neural network (420) was configured to map all item movement between the picking and packing process, as well as to identify the complete breakdown of a picking order into its respective packings.

[0045] In the context of the present invention, inventory and quality processes can be managed by the system (100) for time management and forecasting of operations in an environment, through inventory and quality data sent by the WMS.

[0046] In the context of the present invention, the downgrading process is accounted for by the neural network (420) by calculating the weighted average of the date and time information at which the downgrading processes were initiated and completed.

[0047] The system (100) for managing and forecasting operation times in an environment identifies that the process of downgrading an item has been initiated upon receiving information that it is that item’s turn to be downgraded in the task “queue”.

[0048] After that, an available resource will be assigned to that task, and the operator will start the task. Then, this task will have a status of “D” in the system.

[0049] Finally, the operator will scan the origin of the reduction, the product code, and then scan the destination position. After completing this last step, the task will have the status “C”, for confirmed.

[0050] It should be noted that each stage of this downgrading process is recorded and monitored by the system (100) for management and forecasting of operation times in an environment.

[0051] In the context of the present invention, the start of the time-taking of the shipping process of the stored items is counted by the neural network (420) at the moment the process of loading the items into a transport vehicle begins and ends at the moment the process of loading the items into a transport vehicle ends.

[0052] It is worth noting that the shipping process of the stored items is accounted for by the neural network (420) through the calculation of the weighted average of the date and time information in which the shipping processes of the stored items were started and finished.

[0053] Regarding stage III, the metadata (211) imported from the cameras (210) in the previous stage is accessed by a user, who groups and labels this metadata (211) manually, as is usually done in the state of the art, in order to form samples that characterize the useful life of an object, as well as its characteristics over time.

[0054] In the context of the present invention, the term “samples” refers to a sequence of frames of an object of interest captured by the camera (210).

[0055] Still regarding stage III, these metadata (211) imported by the cameras (210) are primarily separated into objects of interest or processes of interest or others, wherein: the class objects of interest comprises forklifts, pallet trucks, people, pallets, or any object acting in any stage of the process of interest of the invention; the class processes of interest comprises receiving, picking, packing, inventory, quality, lowering and shipping of the stored items; and the class others comprises objects that do not fall into the classes of objects of interest and / or processes of interest.

[0056] It is worth noting that the neural network (420) is able to identify an object and any processes in the environment through the area, speed and number of points that form the outline of that object.

[0057] Subsequently, the information about each of the items allocated to each of the classes is normalized using the min-max normalization technique.

[0058] It is worth noting that it is possible to use the information about the center of gravity of the metadata (211) to represent the metadata (211) in the environment reference frame, in order to represent the position of the center of the object in relation to the camera (210) that generated it.

[0059] However, the different cameras (210) end up generating metadata (211) on a scale relative to their position and, therefore, it is necessary to first apply transformations to the metadata (211) received by the cameras (210), since they are positioned in different areas from each other.

[0060] Thus, in order to make all cameras (210) have the same orientation, it is necessary to rotate the metadata (211) received by the cameras (210) around the center of their respective camera (210), in order to standardize the orientation of all cameras (210).

[0061] After this step, the metadata (211) can be flattened to remove distortions generated by the camera lens (210), so as to match the scale of the floor plan of the environment, and can then be scaled using the min-max normalization technique, preferably considering the range between 0 (zero) and 1 (one).

[0062] Regarding stage IV, the previous stages are repeated for a period of time determined by the user. For example, a user could repeat these stages for a period of months, or years, depending on the complexity and time variation observed in the stages of the processes analyzed by the neural network (420).

[0063] Regarding stage V, the monitoring of the processes of interest in real time is carried out using the same cameras (210) as in the previous stages.

[0064] Regarding step VI, the step of adjusting the predicted execution times can be done manually by the user or can be done automatically by the neural network itself (420), depending on the user's preferences.

[0065] Clearly, the predicted execution time for each of the processes of interest will more closely approximate the actual execution time of those processes the longer the time specified by the user in step IV.

[0066] Preferably, the method for training a neural network (420) capable of managing and predicting the time of operations in an environment should be performed using sparse cross-entropy as the cost function and an adaptive Adam-type optimizer.

[0067] Preferably, the trained (420) neural network comprises two recurrent LSTM layers, preferably with 64 and 32 neurons, respectively, both with ReLU (Rectified Linear Unit) activation function. The trained (420) neural network is divided into two parts, the first comprising two dense hidden layers for object prediction, with 32 and 16 neurons, and the second comprising four dense hidden layers for process prediction, with 64, 32, 16 and 8 neurons, respectively.

[0068] It is worth noting that, in this case, the output layers comprise five neurons for the object and six neurons for the process, with softmax activation function, one neuron for each possible class.

[0069] Even more preferably, the maximum number of iterations can be set to 5000, the stopping criterion can be set if 10 consecutive epochs do not show improvement in the model metrics. In addition, it is also possible to define a dropout rate to randomly turn off 30% of the neurons during training, in order to promote data regularization and prevent overfitting of the neural network (420).

[0070] Furthermore, to evaluate the trained (420) neural network, it is possible to use the sparse categorical hit rate as an evaluation metric, in order to maintain consistency with the chosen coding for the labels and the chosen cost function. It is also possible to evaluate the trained (420) neural network using a confusion matrix for object classification and one for process classification, so that it is possible to assess the distribution of the model's predictions.

[0071] It is worth noting that, in addition to the possibility of predicting the time of operations in an environment in real time, the trained neural network (420) is also able to show the number of items a unit will receive and the prediction for them to be dispatched to the next stage.

[0072] It is also worth noting that the neural network (420) is also able to report, based on the information received from the WMS(220), the quantity of each resource, e.g., people, forklifts, pallet trucks / carts, workstations, among others, in each area of ​​the process for carrying out a given task.

[0073] In this way, the trained neural network (420) is able to discern the entire path of an operation in an environment, being fully capable of estimating the time required for the execution of each of the processes involved in warehouse management.

[0074] Method for managing and forecasting the timing of operations in an environment.

[0075] The method for managing and forecasting operation time in an environment basically comprises the following steps: receiving raw data from a WMS(220) of an environment; storing the raw data from the WMS(220) of the previous step in a Data Lake(310); performing an exploratory data analysis (EDA) of the data stored in the Data Lake(310) of the previous step; sending the useful and necessary information to the database of a Data Warehouse(320); establishing a connection between the data saved in the database of the Data Warehouse(320) of the previous step and a trained neural network(420); calculating the time required to perform each of the operations of interest through the trained neural network(420); and, optionally incorporating the useful and necessary data from the Data Warehouse(320) into the trained neural network(420) model.

[0076] In more detail, the method for managing and predicting operation times in an environment comprises the following steps: receiving, in an API(221), the raw data from a WMS(220) of an environment through a data acquisition subsystem; storing the raw data from the WMS(220) of the previous step in a Data Lake(310); performing an exploratory data analysis “EDA” of the data stored in the Data Lake(310) of the previous step; sending, through a data acquisition subsystem, the useful and necessary information to the Data Warehouse(320) database; establishing, through a data acquisition subsystem, a connection between the data saved in the Data Warehouse(320) database of the previous step and a neural network (420) trained to predict operation times in an environment; calculating the time required to perform each of the operations of interest through the trained neural network (420);and optionally incorporate useful and necessary data from the Data Warehouse(320) into the trained neural network model (420).;

[0077] Regarding step A, raw data from the WMS(220) of an environment is received by an API(221) and feeds into a Data Lake(310). Raw data refers to any and all types of data that can be extracted from the WMS(220), and includes: customer delivery data, task orders and the execution of these tasks, with their respective times, quantity of items, weight and volume.

[0078] Regarding stage C, exploratory data analysis (EDA) is performed using an exploratory analysis system (410) and aims to filter information from both the Data Warehouse (320) and the Data Lake (310) in order to organize it for neural network analysis (420).

[0079] Regarding step D, the useful and necessary information sent to the Data Warehouse database (320) is any and all information analyzed in the previous step that can be used by the neural network (420) in the subsequent step to predict the time of operations in an environment. More specifically, the useful and necessary information includes: number of pending operations in the queue, types of pending operations in the queue, product code, number of items for each product, task creation date, among others.

[0080] Regarding stage E, the data acquisition subsystem is associated, that is, it is in data communication with image capture devices, such as cameras, installed in the environment in question, for example, a logistics warehouse, and may also include a communication module with other systems, for example, a WMS.

[0081] The aforementioned data acquisition subsystem is responsible for integrating the data sources, such as image capture devices, with the data storage subsystem.

[0082] It should be noted that it is possible to send the raw data from WMS(220) to the neural network (420) along with the data relating to useful and necessary information. However, even in this case, the neural network (420) would continue to take into account only the data relating to useful and necessary information.

[0083] Regarding step F, the calculation of the time required to perform each of the operations of interest is carried out using the trained neural network (420), which receives the useful and necessary information to predict the time required to perform each of the operations of interest and prints to a user an accurate estimate of the time required to perform each of the operations of interest and the total time required to perform all operations in an environment.

[0084] Regarding step G, if the prediction calculated by the neural network (420) is not close enough to the observed real-time of at least one of the operations of interest, a user can update the neural network (420) database with new data relating to the operation of interest that was not close enough to real-time. In this way, the user can help to continuously improve the neural network (420) with new data, until the prediction calculated by the neural network (420) satisfactorily approximates the real-time of each of the operations of interest. Memory read by computer

[0085] In the context of the present invention, a computer-readable memory is any memory or storage device, remote or local, volatile or non-volatile, transient or non-transient (permanent), that stores information and instructions, and in particular, a computer-readable memory that stores instructions capable of executing a method according to the invention.

[0086] System (100) for managing and forecasting the time of operations in an environment

[0087] In addition to at least one trained neural network (420) and at least one processor, the system (100) for managing and predicting the timing of operations in an environment of the invention also basically comprises:

[0088] - At least one data source system (200);

[0089] - At least one data storage system (300);

[0090] - At least one data analysis system (400); and

[0091] - At least one output data system (500). Data source system (200)

[0092] A data source system (200) is, in the context of the present invention, a system responsible for providing data to the system (100) for management and time forecasting of operations in an environment and comprises a plurality of cameras (210), at least one set of metadata (211) of the cameras (210), at least one WMS system (220), at least one API (221) of the WMS (220).

[0093] The data source system (200) may additionally comprise, optionally and preferably, at least one central server, at least one computer read memory, at least one processor, at least one communication and data network (when associated with data input into the system), one or more sets of information (collected or generated by the data sources).

[0094] It should be noted that the data source system (200) may optionally also comprise sensors, local and / or remote databases, data collection devices, IoT platforms, integration mechanisms with communication networks for data transfer, in real time and / or asynchronously, to the system (100) for management and time forecasting of operations in an environment, and configured to support multiple communication protocols and compatibility standards common in the state of the art. Cameras (210)

[0095] A camera (210) of the invention is a data and information acquisition device capable of processing, storing, and transmitting data related to environmental monitoring. The camera (210) may comprise digital and analog optical sensors for capturing images and videos, integrated into data network communication systems, allowing interaction with external storage and processing platforms. Optionally, the camera (210) may incorporate additional sensors, such as light or motion sensors, to enhance its functionality. This configuration enables the detection and recording of events in the monitored environment. Metadata set (211)

[0096] A set of metadata (211) are the metadata (211) imported from the cameras (210) in real time and are automatically separated by the neural network (420) trained on objects of interest or processes of interest or others, as performed in the neural network training step (420).

[0097] It is worth noting that, as in the neural network training stage (420), it is possible to use information about the center of gravity of the metadata (211) to represent the metadata (211) in the environment's reference frame, in order to represent the position of the center of the object in relation to the camera (210) that generated it.

[0098] Furthermore, the metadata (211) can be flattened to remove distortions generated by the camera lens (210), so as to match the scale of the environment floor plan, and can thus be scaled using the min-max normalization technique, preferably considering the range between 0 (zero) and 1 (one). API (221)

[0099] An API “Application Programming Interface” system is, in the context of the invention, a component responsible for enabling integration and communication between different systems of the system (100) for managing and forecasting the timing of operations in an environment, such as the WMS system (220), data source systems (200), data storage (300) and data analysis (400). Central server

[0100] In the context of the present invention, a central server is a computer or computing system or computing circuit, with one or more centralized computing systems or one or more data processing centers, that makes services and resources available or stores them on and through a communication and data network.It comprises one or more electronic processors capable of executing tasks based on a computer program or set of instructions stored in a computer-readable medium, preferably a computer equipped with a processor, memory for data storage, connection to one or more communication and data networks and to one or more remote databases and / or a local and / or centralized and / or decentralized and / or cloud-based information storage and retrieval environment, and also equipped with all the usual peripherals of the state of the art, being capable of exchanging information with the electronic and physical environment, interfaces, applications, mobile equipment, other memory devices, etc.

[0101] A server can be at least a web server that delivers or serves web pages, an application server that handles application operations between users and applications or databases, a cloud server, a database server, a file server, a service server, a game server implementing games or services for a game, and a media server providing media such as streaming video or audio.

[0102] The system (100) for managing and forecasting the time of operations in an environment of the invention may include one or more processors for processing commands and information according to the instructions of computer programs and one or more memories that store information in one or more data structures, and may be a centralized warehouse management system or network, for example, but not limited to a server that may be run privately by a third-party entity or the same entity that is running the server.

[0103] A computer program according to the invention is a program that runs on a processor of the invention and, thus, on a processor of the equipment of the invention, for example, in the form of an application.

[0104] The system (100) for managing and forecasting the timing of operations in an environment can also be imagined as being a publicly accessible network system, e.g., a distributed decentralized computing system. Each processor comprises memory that stores information and data and can execute one or more sets of instructions to perform various functions associated with the system (100) and methods for managing and forecasting the timing of operations in an environment according to the invention.

[0105] The central server of the invention is, in particular, the server that hosts and manages the database of one or more environments of interest, for example, but not limited to the scope of the present invention, of one or more warehouses of interest, comprising at least one remote central processor.

[0106] It should be noted that the central system server (100) for managing and forecasting the timing of operations in an environment of the invention has at least one processor or computer system or processing circuit, these preferably being independent of each other and / or autonomous.

[0107] Processor or computer system or processing circuit

[0108] In the context of the present invention, a processor is a central processing unit or CPU, or a computer system, or a processing circuit that executes the instructions of a computer program or application, processing and executing arithmetic and logical operations, and data input and output. The computer program is stored on a computer-readable medium with memory for data storage, connection to one or more communication and data networks, and to one or more remote databases and / or a local and / or centralized and / or decentralized and / or cloud-based information storage and retrieval environment. It is also equipped with all the usual peripherals of the prior art and is capable of exchanging information with electronic and physical media, interfaces, applications, mobile equipment, other memory devices, etc.

[0109] A processor according to the invention may be, form part of, or be divided into one or more modules. The term module, according to the invention, refers to an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group of processors), and a memory that executes one or more software programs or firmware. It also refers to a combinational logic circuit and / or other suitable components capable of providing the functionalities in question.

[0110] A processing circuit according to the invention is configured to determine a neural network (420) trained according to the invention.

[0111] This processing circuit may therefore include a processor, such as a central processing unit (CPU), a microcontroller, a microprocessor, a field programmable gate array (FPGA), a graphics card or special hardware for neural networks such as the trained neural network (420) of the invention. Communication and data network

[0112] A communication and data network, in the context of the present invention, is a centralized or decentralized network that interconnects one or more active or passive components of the system (100) for management and time prediction of operations in an environment according to the invention. Servers, processors, data processing center, database and cameras (210) may be connected to one or more communication and data networks, physical memories, cloud and similar, the internet, one or more data and / or program clouds, computer terminals, mobile devices, telephone devices, barcode readers, QR codes, DataMatrix codes and the like.

[0113] The communication and data network may comprise any wired or wireless connection, the internet, or any other form of communication, and may include any number of different communication and data networks between any server, device, resource, and system, and / or other servers, devices, resources, and systems described in this document. The communication and data network may enable communication between various computing resources or devices, servers, and systems, and may employ different types of networks, for example, but not limited to, computer networks, telecommunications networks (e.g., cellular), mobile wireless data networks, cable, radio, and similar networks, and any combination of these and / or other networks. Set of information

[0114] A set of information according to the invention is a set of data acquired and / or transmitted and / or stored by and in one or more of the system components (100) for managing and forecasting the timing of operations in an environment of the invention and comprises information about items received, picking, packing, inventory of the environment, quality of items received, stock reorganization, item reduction and shipment of stored items within a time interval predetermined by a user.

[0115] Non-limiting examples of information of this nature are: Unique identification data of the items to be received by the environment; Individual data relating to the items received for storage; Data relating to the space available for receiving items in the environment; Data relating to the objects available to act in the processes related to the storage of the items; Training image datasets; Input image datasets; and Resulting output datasets.

[0116] The unique identification data for items to be received by the environment includes, but is not limited to, QR codes, barcodes, labels, RFID (Radio Frequency Identification), Bluetooth Beacons, permanent markers, data from the WMS(220), any identifications through specific sizes and formats, color-coded stickers or any other means of identification capable of identifying the items in a shipment batch.

[0117] Individual data relating to items received for storage includes, but is not limited to, product code, item description, batch or serial number, quantity, weight, dimensions, date of receipt, place of origin, storage location, item condition, validity or expiration date, certifications or documents, unit value, invoice or transport document, tax classification, storage requirements, product status, priority of use or shipment, or any other relevant information regarding the items to be stored in an environment.

[0118] The data relating to the space available for receiving items in the environment includes, but is not limited to, the number of available shelves / pallets, the dimensions of each shelf / pallet, the maximum weight capacity per shelf / pallet, the location of the shelves / pallets in the environment, the type of storage structure, e.g., static, dynamic, environmental conditions, e.g., temperature, humidity, total available space in the warehouse, usable height of the environment, accessibility for equipment, e.g., forklifts, carts, specific storage requirements, e.g., refrigerated areas, safety zones, or any other relevant information regarding the environment and / or the internal and / or external area of ​​the environment.

[0119] The data relating to the objects available to work in the processes related to the storage of items includes, but is not limited to, the type and quantity of equipment available, e.g., forklifts, pallet jacks, conveyors, load capacity of the equipment, maintenance status of the equipment, number of operators available, level of qualification and / or training of the operators, work shifts of the team, auxiliary tools available, e.g., barcode readers, scales, hourly handling capacity, safety equipment available, e.g., PPE, protection systems, software used in the process, e.g., WMS(220), ERP, or any other relevant information regarding the equipment and operators available to work in the processes related to the storage of items.

[0120] The training image datasets comprise, but are not limited to, the images used to train a neural network (420) in a machine learning model, containing images captured of monitored items and objects in an environment, where each of the training images was named and normalized using the min-max normalization technique, in order to represent the position of the object's center relative to the camera (210) that generated it. The images were obtained using cameras (210) and / or any image capture devices available in the environment and captured at various angles, lighting conditions, shadows, etc., seeking to reproduce as closely as possible the real-life conditions of an environment.

[0121] The sets of images obtained as input comprise, but are not limited to, images of the regions of interest in the environments of interest, these images being obtained by the cameras (210) previously installed in the environments, and which will serve as input data for the trained neural network (420), which will apply the machine learning model to these input images.

[0122] The resulting sets of output data include, but are not limited to, the predicted execution time of each of the operations of interest in an environment, the predicted total execution time of the operations of interest in an environment, the number of objects available for performing each of the operations of interest in an environment, the number of objects available for performing all operations of interest in an environment, the percentage of shelves / pallets available for receiving items, or any other information relevant to the management of an environment intended to receive and store items. Data storage system (300)

[0123] A data storage system (300) is, in the context of the present invention, a system responsible for storing the system data (100) for management and time forecasting of operations in an environment and comprises at least one Data Lake (310), at least one Data Warehouse (320), at least one Data Mart (330).

[0124] The data storage system (300) may optionally and preferably comprise at least one database and at least one communication and / or data network.

[0125] It should be noted that the data storage system (300) may optionally include redundancy, scalability and information protection features, ensuring the integrity, security and accessibility of stored data, computer-readable media such as relational and non-relational databases, distributed systems, local and / or cloud-based servers, capable of operating in centralized, decentralized and hybrid environments. Data Lake (310)

[0126] A Data Lake(310) is, in the context of the invention, a centralized repository that stores large volumes of data in its raw, structured, semi-structured or unstructured state. Data Warehouse (320)

[0127] A Data Warehouse(320) is, in the context of the invention, a centralized and structured repository that stores structured data for queries and analyses. Data Mart (330)

[0128] A Data Mart (330) is, in the context of the present invention, a system responsible for storing specific subsets of data originating from the data storage system (300), being designed to meet the needs of a specific group of users or processes within an organization.

[0129] Additionally, the Data Mart(330) operates as a repository of structured, organized and optimized data for fast queries and analyses, and can be deployed in local, centralized, distributed and / or cloud-based environments.

[0130] Preferably, the Data Mart(330) comprises integration mechanisms with internal and external data sources, supporting multiple protocols and formats, and equipped with security, scalability, access control and redundancy features typical of the state of the art, enabling specific and customized analyses through integration with the data analysis system (400) and / or the output data system (500). Databases

[0131] A database or data bank according to the invention is any and all set of data, files, information, instructions and records that form organized collections of data that relate to each other, hosted in one or more memories or storage devices of the system (100) for management and time forecasting of operations in an environment of the invention and that can be accessed, fed and managed by the data processing centers of the invention.

[0132] Both the servers and the processors and other equipment and devices of the system (100) for managing and forecasting the timing of operations in an environment of the invention may comprise one or more databases, independent and / or interconnected with each other. Data analysis system (400)

[0133] A data analysis system (400) is, in the context of the present invention, a system responsible for analyzing the data from the system (100) for management and forecasting of operation times in an environment and comprises at least one exploratory analysis system (410), at least one neural network (420), and can operate locally and / or in cloud computing environments, with support for integration with the data source system (200) and the data storage system (300). Exploratory analysis system (410)

[0134] The exploratory analysis system (410) aims to filter information from both the Data Warehouse (320) and the Data Lake (310) in order to organize it for neural network analysis (420)

[0135] An exploratory analysis system (410) is, in the context of the invention, a system that aims to filter information from both the Data Warehouse (320) and the Data Lake (310) in order to organize it for analysis by the neural network (420), so as to identify patterns, trends and insights before submitting this data to the neural network (420). Neural network (420)

[0136] A neural network (420) is, in the context of the invention, a neural network (420) trained using the “Method for training a neural network (420) capable of managing and predicting the timing of operations in an environment” taught by the present invention.

[0137] The data analysis system (400) may additionally comprise, optionally and preferably, one or more sets of instructions relating to algorithms and / or analysis models used.

[0138] It is worth noting that the data analysis system (400) uses computational algorithms and / or statistical techniques in general to perform structured and unstructured data processing and report generation. Output data system (500)

[0139] An output data system (500) is, in the context of the present invention, a system responsible for presenting the data analyzed by the system (100) for management and forecasting of operation times in an environment in visual and interactive forms, such as, for example, a process forecast map (510) using adaptable user interfaces, output devices such as screens, printers and mobile devices, and connectivity with communication networks, being compatible with the usual standards and technologies of the state of the art, to offer accessible and customizable reports.

[0140] Additionally, the output data system (500) can also present the data analyzed by the system (100) for management and forecasting of operation times in an environment through graphs, maps, pivot tables, dashboards and other visualization elements.

[0141] The output data system (500) comprises at least one communication and data network, when related to the presentation or transmission of data, and one or more sets of information, prepared for presentation and / or use by the neural network (420).

[0142] Thus, as illustrated in the present invention, the system (100) for managing and forecasting the time of operations in an environment of the invention essentially comprises:

[0143] - At least one memory location readable by a computer;

[0144] - At least one processor;

[0145] - A plurality of cameras (210);

[0146] - At least one set of metadata (211) from the cameras (210);

[0147] - At least one WMS system (220);

[0148] - At least one API (221) from WMS(220);

[0149] - At least one Data Lake (310);

[0150] - At least one Data Warehouse (320);

[0151] - At least one Data Mart(330);

[0152] - At least one exploratory analysis system (410);

[0153] - At least one neural network (420); and

[0154] - An output data system (500).

[0155] Preferably, the system (100) for managing and forecasting the time of operations in an environment of the invention comprises:

[0156] - At least one central server;

[0157] - At least one memory location readable by a computer;

[0158] - At least one processor;

[0159] - A plurality of cameras (210);

[0160] - At least one set of metadata (211) from the cameras (210);

[0161] - At least one WMS system (220);

[0162] - At least one API (221) from WMS(220);

[0163] - At least one communication and data network (when associated with data entry into the system);

[0164] - At least one set of information collected or generated by the data sources;

[0165] - At least one database;

[0166] - At least one Data Lake (310);

[0167] - At least one Data Warehouse (320);

[0168] - At least one Data Mart(330);

[0169] - At least one communication network;

[0170] - At least one set of information stored in a structured and / or unstructured way;

[0171] - At least one exploratory analysis system (410);

[0172] - At least one neural network (420);

[0173] - At least one set of instructions related to algorithms and / or analysis models used;

[0174] - At least one output data system (500);

[0175] - At least one communication and data network; and

[0176] - At least one set of information for presentation and / or use by the neural network (420).

[0177] The memory, the trained neural network (420), the processors, servers, databases, communication and data networks and other devices and / or equipment eventually and additionally included in the system (100) for managing and predicting the timing of operations in an environment, are all interconnected by one or more communication and data networks. Images and data are stored as one or more electrical signals and the processing of these signals is done by one or more components of the system (100) for managing and predicting the timing of operations in an environment of the invention. Conclusion

[0178] It will be readily understood by those skilled in the art that modifications can be made to the present invention without departing from the concepts set forth in the description above. These modifications should be considered as falling within the scope of the present invention. Consequently, the particular embodiments described in detail above are merely illustrative and exemplary and not limiting as to the scope of the present invention, to which the full extent of the appended claims and any and all equivalents thereof should be given.

Claims

Method for training a neural network (420) capable of managing and predicting the time of operations in an environment, characterized by the fact that it comprises the steps of: Importing data from a WMS (220) relating to a time period predetermined by a user; Importing metadata (211) from a plurality of cameras (210) relating to the time period predetermined by the user; Labeling the metadata (211) imported from the cameras (220) in the previous step; Repeating the previous steps for a given time period, in order to create a prediction of the execution time of each of the processes of interest; optionally Monitoring the processes of interest in real time and comparing the predicted execution times of each of the processes of interest with the actual time of each of the processes of interest; and, optionally Adjusting the predicted execution times of each of the processes of interest based on the actual time of each of the processes of interest. Method for training a neural network (420), according to claim 1, characterized in that, in step I, WMS(220) data are imported from a Data Lake(310) to a Data Warehouse(320); wherein the data imported from WMS(220) comprise information about items received, picking, packing, environment inventory, quality of items received, item downgrading and shipment of items stored in a time interval predetermined by a user. Method for training a neural network (420), according to claim 1 or 2, characterized in that, in step II, the metadata (211) of the cameras (210) will be imported into a Data Lake (310); wherein the cameras (210) must be positioned so as to allow unambiguous monitoring of each of the user's tasks of interest; wherein the time period predetermined by the user encompasses the beginning and end of a given process of interest; wherein the time period predetermined by the user defines a fixed-size window of time instants to be considered by the neural network (420); and wherein, additionally, the item volume, item mass, number of items and number of tasks are also considered by the neural network (420). Method for training a neural network (420), according to claim 3, characterized in that the time window size is defined from the analysis of the distribution of the number of frames per object; wherein all objects present in a frame are identified over a time range; and wherein the time window size is defined based on the time range in which the largest percentage of objects per frame are concentrated. Method for training a neural network (420), according to any one of claims 1 to 4, characterized in that the start of the time-taking of the picking process is accounted for by the neural network (420) at the moment of task acceptance; wherein the end of the time-taking of the picking process is accounted for by the neural network (420) at the moment of delivery of the item of interest to the location of interest. Method for training a neural network (420), according to claim 5, characterized in that the start of the timing of the depacking process is accounted for by the neural network (420) at the moment the item is deposited on the depacking bench; wherein the end of the timing of the depacking process is accounted for by the neural network (420) at the moment the item is packed; and wherein one or more depacking orders are automatically created based on the creation of a picking order. Method for training a neural network (420), according to any one of claims 1 to 6, characterized in that, in step III, the metadata (211) imported by the cameras (210) are separated primarily into objects of interest or processes of interest or others and in which: the class objects of interest comprises forklifts, pallet trucks, people, pallets; the class processes of interest comprises receiving, picking, packing, inventory, quality, downgrading and shipping of the stored items; and the class others comprises objects that do not fall into the classes of objects of interest and / or processes of interest. Method for training a neural network (420), according to any one of claims 1 to 7, characterized in that the metadata (211) received by the cameras (210) are rotated around the center of their respective camera (210); and in that, after being rotated around the center of their respective camera (210), the metadata (211) are flattened to remove distortions generated by the camera lens (210). A method for training a neural network (420), according to any one of claims 1 to 8, characterized in that, in step VI, the step of adjusting the predicted execution times is performed manually by the user or is performed automatically by the neural network (420) itself; wherein the method for training a neural network (420) is performed using sparse crossed categorical entropy as the cost function and an adaptive Adam-type optimizer; and wherein the output layers of the neural network (420) comprise five neurons for the object and six neurons for the process, with a softmax activation function, one neuron for each possible class. A method for managing and forecasting the time of operations in an environment, characterized by the fact that it comprises the steps of: receiving raw data from a WMS(220) of an environment; storing the raw data from the WMS(220) of the previous step in a Data Lake(310); performing an exploratory data analysis “EDA” of the data stored in the Data Lake(310) of the previous step; sending the useful and necessary information to the database of a Data Warehouse(320); establishing a connection between the data saved in the database of the Data Warehouse(320) of the previous step and a trained neural network (420), defined in any of claims 1 to 9; calculating the time required to perform each of the operations of interest through the trained neural network (420); and, optionally incorporating the useful and necessary data from the Data Warehouse(320) into the trained neural network (420) model. Computer-readable memory, characterized by the fact that it comprises instructions which, when executed, perform the steps of the method for managing and predicting the timing of operations in an environment defined in claim 10. System (100) for managing and forecasting the timing of operations in an environment, characterized in that it comprises: - At least one trained neural network (420), defined in any one of claims 1 to 9; - At least one processor; - At least one data source system (200); - At least one data storage system (300); - At least one data analysis system (400); and - At least one output data system (500). System (100) for managing and forecasting the timing of operations in an environment, according to claim 12, characterized in that the data source system (200) comprises a plurality of cameras (210), at least one set of metadata (211) for the cameras (210), at least one WMS system (220), at least one API (221) for the WMS (220). System (100) for managing and forecasting the timing of operations in an environment, according to claim 12 or 13, characterized in that the data storage system (300) comprises at least one Data Lake (310), at least one Data Warehouse (320), at least one Data Mart (330). System (100) for managing and forecasting the timing of operations in an environment, according to any one of claims 12 to 14, characterized in that the data analysis system (400) comprises at least one exploratory analysis system (410), at least one neural network (420), capable of operating locally and / or in cloud computing environments, with support for integration with the data source system (200) and the data storage system (300).

Images